Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improves accuracy of vector division approximation example.
The previous implementation was unnecessarily inaccurate: Measured max error of 2.465257280 ulp @ (1.2449559e-01, 5.2960633e+36) func (0x1.fdef16p-4, 0x1.fdfe0ap+121) = 0x1.fff0f8p-126 func_ref (0x1.fdef16p-4, 0x1.fdfe0ap+121) = 0x1.fff0fcee3633cp-126 4'550'824'890 tested in [0 ; inf] x [0x1.000008p-128 ; 0x1p126] 68.8976110218998059% correctly rounded ( 3135409631) 92.7973265084255985% faithfully rounded (+ 1087634201) 7.2026734915744033% >= 1.0 ulp (+ 327781058) Which is 21.7 bits of precision rather than the claimed 23. The new implementation has lower maximum error and higher correctly- and faithfully-rounded rates: Measured max error of 1.493816005 ulp @ (5.0745111e+02, 4.9807361e-01) func (0x1.fb737cp+8, 0x1.fe0702p-2) = 0x1.fd69eap+9 func_ref (0x1.fb737cp+8, 0x1.fe0702p-2) = 0x1.fd69ecfcd5739p+9 4'550'824'890 tested in [0 ; inf] x [0x1.000008p-128 ; 0x1p126] 79.6106155822664476% correctly rounded ( 3622939709) 99.0013070355691127% faithfully rounded (+ 882436413) 0.9986929644308947% >= 1.0 ulp (+ 45448768) Which is 22.4 bits of precision. Max error is greater when the denominator's reciprocal underflows: previously about 5.26 ulp and now about 4.45 ulp. The algorithm does not handle infinite or zero denominators (returns NaN) or subnormal denominators whose reciprocal overflows (returns ±∞). This commit does not document or change this behavior. * src/vector-examples.adoc (Division approximation example): Use `r + r (1 - y r)` formulation for 1/y estimate refinement.
- Loading branch information