sdlc.dev

Computer science fundamentals for working engineers path · How data is stored · Topic 4 of 13

Floating point: why 0.1 + 0.2 isn't 0.3

Not started

Almost every language stores non-integer numbers in the IEEE 754 binary floating-point format. It is scientific notation in base 2: a value is stored as a sign, an exponent, and a fixed number of significant binary digits. That gives an enormous range, from about 10⁻³⁰⁸ to 10³⁰⁸ for a 64-bit double, with about 15 to 17 significant decimal digits everywhere in it. The price is that most decimal fractions can't be stored exactly, and every operation rounds its result.

Why it matters: Floating-point surprises turn up in money calculations, equality checks, loop counters and aggregates. They are predictable once you know what a double actually stores.

  1. 1Scientific notation in binary

    IEEE 754 stores a sign, an exponent and a fraction. A double has 53 significant bits, about 15 to 17 decimal digits.

  2. 2Most decimals can't be stored exactly

    0.1 in binary repeats forever, like 1/3 in decimal, so it is rounded to the nearest double. Each operation rounds again.

  3. 3Compare with tolerance

    Test whether two results are close enough for your purpose, not whether they are identical.

  4. 4Money is not a float

    Store currency as an integer number of the smallest unit (cents) or use a decimal type.

1

How it works

The explanation, with examples you can run.

What a double holds

A 64-bit double has 1 sign bit, 11 exponent bits and 52 fraction bits (plus an implicit leading 1, so 53 significant bits). The value is ±1.fraction × 2^exponent. A 32-bit float has 24 significant bits, about 7 decimal digits.

Fractions that are sums of powers of two (0.5, 0.25, 0.375) are exact. But 0.1 is 1/10, and 10 has a factor of 5, so in binary it repeats forever: 0.0001100110011… The double nearest to it is slightly more than 0.1:

⌘/Ctrl + Enter
OutputNot run yet

Press Run to see what your code prints, in order.

Runs in this browser, in an isolated frame with no access to this page. console.log, info, warn and error are shown; code runs as a classic script, so use an async function for await.

Node printed 0.30000000000000004, false, 0.10000000000000000555, true, 0.9999999999999999, then 9007199254740992 9007199254740992 (above 2⁵³ the gap between doubles is 2, so the +1 is lost), Infinity -Infinity NaN false, and false true.

Special values

IEEE 754 defines more than numbers. Dividing a non-zero number by zero gives ±Infinity rather than an error. Invalid operations such as 0/0 or Math.sqrt(-1) give NaN (not a number), which is unequal to everything including itself, and which spreads through every calculation it touches. There are two zeros, +0 and −0, which compare equal but behave differently (1/−0 is −Infinity). Very small numbers near zero lose precision gradually ("subnormals") instead of dropping straight to zero.

Living with rounding

  • Don't test for exact equality of computed results. Check whether the difference is small relative to the magnitude, for example Math.abs(a - b) <= 1e-9 * Math.max(Math.abs(a), Math.abs(b)). Number.EPSILON (2⁻⁵², the gap between 1 and the next double) is only a sensible tolerance for values near 1.
  • Order matters. Floating-point addition isn't associative: (a + b) + c can differ from a + (b + c). Summing many values of mixed size is more accurate smallest first, or with compensated (Kahan) summation. Parallel sums can give slightly different answers from run to run because the order changes.
  • Use integers for counting. Loop on an integer and compute i * 0.1, rather than adding 0.1 repeatedly.
  • Money needs exact decimals. Store amounts as integer cents (or the currency's smallest unit), or use a decimal type: decimal in C#, BigDecimal in Java, Decimal in Python, NUMERIC in SQL. Decide rounding rules (for example banker's rounding) explicitly.
  • Display is not storage. toFixed(2) and format strings round for display only; the stored value is unchanged.

These rules aren't JavaScript quirks. C, Java, Python, C# and Go give the same answers for the same operations on doubles, because they all use the same hardware format.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Why is `0.1 + 0.2 === 0.3` false in JavaScript (and in Python, Java, C and C#)?
  2. 2.What does `NaN === NaN` evaluate to?
  3. 3.Adding 0.1 ten times starting from 0 gives what in this topic's example?

0 of 3 right so far. Get all three to finish this topic.

Go deeper