HN Reader

This remains one of the best explanations on the topic: https://fabiensanglard.net/floating_point_visually_explained... Saw this when I just started using HN and such posts only inspired me to stick to it: https://news.ycombinator.com/item?id=29368529

This is cool, looks visually a lot like a CIDR range calculator [1] I built a few years ago to help me understand network ranges better.

These types of visualizations are super useful.

[1] - https://cidr.xyz

My favorite FP Fun Fact is that float comparisons can (almost) use integer comparisons. To determine if a > b, reinterpret a and b as signed ints and just compare those like any old ints. It (almost) works!

The implication is that the next biggest float is (almost) always what you get when you reinterpret its bits as an integer, and add one. For example, start with the zero float: all bits zero. Add one using integer arithmetic. In int-speak it's just one; in float-speak it's a tiny-mantissa denormal. But that's the next float; and `nextafter` is implemented using integer arithmetic.

Learning that floats are ordered according to integer comparisons makes it feel way more natural. But of course there's the usual asterisks: this fails with NaNs, infinities, and negative zero. We get a few nice things, but only a few.

Previously I was using this site to explore floating point representations:

https://www.h-schmidt.net/FloatConverter/IEEE754.html

This one has the extra feature of showing the conversion error, but it doesn't support double precision.

Would be cool if this supported the various fp8 formats that have been shipped on GPUs recently!

It doesn't seem to have been shared in this thread but my favourite site on this topic is https://0.30000000000000004.com/

Far too superficial because it lacks explanation of non-normal conditions (denormals, zeroes, infinities, sNaNs, and qNaNs) and the complete mapping of values. This isn't properly educational.

One problem I was struggling with: What's the shortest, unambiguous decimal representation of a float?

For example, if you use single precision floats, then you need up to 9 digits of decimal precision to uniquely identify a float. So you would need to use a printf pattern like %.9g to print it. But then 0.1 would be output as 0.100000001, which is ugly. So a common approach is to round to 6 decimal digits: If you use %.6g, you are guaranteed that any decimal input up to 6 significant digits will be printed just like you stored it.

But you would no longer be round-trip safe when the number is the result of a calculation. This is important when you do exact comparisons between floats (eg. to check if data has changed).

So one idea I had was to try printing the float with 6 digits, then scanning it and seeing if it resulted in the same binary representation. If not, try using 7 digits, and so on, up to 9 digits. Then I would have the shortest decimal representation of a float.

This is my algorithm:

    int out_length;
    char buffer[32];
    for (int prec = 6; prec<=9; prec++) {
        out_length = sprintf(buffer, "%.*g", prec, floatValue);
        if (prec == 9) {
            break;
        }
        float checked_number;
        sscanf(buffer, "%g", &checked_number);
        if (checked_number == floatValue) {
            break;
        }
    }

I wonder if there is a more efficient way to determine that shortest representation rather than running printf/scanf in a loop?

fantastic explanation and very nice to see both the decimal and binary representations together

https://raku.org defaults to Rationals (Rats) and provides FatRat for arbitrary precision

otherwise even relatively normal calculations (eg what’s the mass of electron in quectogram) fail

See also https://integer.exposed/

Why tf is there a .exposed TLD?

This came during my OMSCS Game AI course as an example of the dangers of using floats to represent game object location. If you get further from the origin point (or another game element you are referencing from) you are losing precision as the float needs to use more of the significand to store the larger value.

I'm glad this exists, except for the fact that IEEE754 is the devil, and posits are better (not assuming hardware support). (Even better than both are bignum rationals, but those are the slowest.)

It would be nice if the site had an explanation why the decimal 0.1 has no finite representation in base-2 system. In short it's about factorization of the base of the system - base-2 lacks 5 that is present in base-10. It is analogous to 1/3 or 1/7 not having a finite representation in base-10 dot notation of a fraction.

Good job on showing the hexadecimal version. I recently had a challenging C bug where I was using printf("%a") to show the underlying representation, which uses hexadecimal, and it was a little confusing at first. This site would have been helpful.

The best explanation on floats - https://dennisforbes.ca/blog/features/floating_point/underst...

Fantastic. A visual interactive explanation of how floating-point representation works.

Good stuff. Having fp8/fp4 would be great, too!

The award for "most fun integer" in 32 bit float is 16777217 (and 9007199254740992 for 64bit).

It's sometimes fun to have these kinds of edge cases up your sleeve when testing things.

For a .exposed domain, it's not really shocking.

The real shocking fact about floating point is that they are even used at all.

It's throwing out of the window the most basic property operations on number should have : "associativity" and all that for a gain in dynamic range which is not necessary most of the time.

The associativity we expect to hold is (a+b)+c == a+(b+c) and (ab)c == a(bc) and these don't hold for floats even though most math formulas and compiler optimizations rely on these to hold. It's a sad miracle that everything somehow still works out OK most of the time.

You lose determinism most of the time with respect to compiler optimizations, and platform reproducibility if processor don't exactly respect IEE-754 (or is it IEE-854).

The real problem comes when you want to use parallelism. With things like atomic operations and multiple processor doing things out of order, you lose determinism and reproducibility, or add a need for synchronisation or casting operations everywhere.

Even more problematic, is that because number operations are used so often, they are set in "stone", and are implemented at the hardware level. And they use much more transistor because they are more complex than integer arithmetic.

Real programmers don't use floating points, only sloppy lazy ones do.

Real programmers use fixed point representation and make sure the bounds don't overflow/underflow unexpectedly.

Let's ban all hardware floating-point implementation : Just imagine future alien archeologists having a laugh at us when they look at our chips and think "no wonder they were doomed they can't even do a+b right : its foundations were built on sand".

not obvious you need to press enter to change the value

assuming this is IEEE 754