The first link in the post doesn’t seem to be valid (on my iPhone at least). Maybe it is an issue of mobile safari not wanting to open… a vhdl blob, though? Maybe double check that, a link to their main page could be more useful.
[edit, it seemed to work the second time, I dunno, maybe it was just a temporary issue, sorry!]
Also, it looks like in the last six minutes (as of me writing this post) the README was updated with an intro that starts thing off with a better high-level description of what’s going on. I think the write-up could still use some work. But it isn’t entirely useless, really, this sort of library should be fully correct, down to the ULP, it is quite foundational after all.
Yeah, it's horrible. Humans who know even a little bit about this want to know how many ULPs the errors are: less than 0.5 is presumably "right", but error >1 ULP is much more newsworthy than if it's 0.6 ULP or even 0.9 ULP.
We measured how far off the float_pkg results are, by comparing the returned float32 and float64 values with exact arithmetic and expressing the distance in units of the last place. The cut is 0.5 of a unit for correct rounding, and sqrt(2.0) in double falls at 0.565, with the correctly rounded neighbour at 0.435. So even that one-unit error is beyond the rounding boundary.
But the misses extend much further than that rounding boundary. Across the same 10,000 calls per operation we measured 19,147 wrong results; 7,213 of those are within one unit, while 3,775 are more than 1,000 away. 4,063 have the wrong sign and 3,370 confuse a number with infinity or NaN. The reciprocal result is useful here: none of its 384 wrong float32 results is a near miss, with 294 returning infinity or NaN for a number, 66 the wrong infinity or NaN, and 24 getting the sign of zero wrong.
The table is in "How far off"; python3 exact.py sizes vectors.txt results.txt prints it.
> For a normal argument it ends on the root or one unit beside it: in the vectors above it is one unit off for 607 of 9,156 normal float32 arguments and 1,146 of 9,236 normal float64 ones, and never further. For a denormal argument the start is too far away for that many steps: 128 of 359 float32 denormals are wrong, 104 of them by more than two units, and sqrt (2.0*-149) returns 11 times the root.
Also, it looks like in the last six minutes (as of me writing this post) the README was updated with an intro that starts thing off with a better high-level description of what’s going on. I think the write-up could still use some work. But it isn’t entirely useless, really, this sort of library should be fully correct, down to the ULP, it is quite foundational after all.
But the misses extend much further than that rounding boundary. Across the same 10,000 calls per operation we measured 19,147 wrong results; 7,213 of those are within one unit, while 3,775 are more than 1,000 away. 4,063 have the wrong sign and 3,370 confuse a number with infinity or NaN. The reciprocal result is useful here: none of its 384 wrong float32 results is a near miss, with 294 returning infinity or NaN for a number, 66 the wrong infinity or NaN, and 24 getting the sign of zero wrong.
The table is in "How far off"; python3 exact.py sizes vectors.txt results.txt prints it.
feel free to crunch the numbers tho, prose aside