IEEE 754 Floating-Point Standard

1. Overview

IEEE 754 (formally: IEEE Standard for Floating-Point Arithmetic) is the international standard for floating-point arithmetic. It was first published in 1985 (IEEE 754-1985), revised in 2008 (IEEE 754-2008), and the latest edition was published in 2019 (IEEE 754-2019).

Virtually all modern processors (CPUs, GPUs) implement the formats and operations of this standard, and programming language types like float (single precision) and double (double precision) are based on it. Some environments, however, have operating modes that do not follow the standard, such as a setting that replaces subnormal numbers with 0 for speed (flush-to-zero); the property in §3 does not hold in that mode.

IEEE 754-2008 specifies quadruple precision (128-bit), together with single (32-bit) and double (64-bit) precision, as basic formats for arithmetic, and half precision (16-bit) mainly as a format for data interchange.

2. Special Values in Detail

The basics of special values (combinations of the exponent and significand fields) were covered in Chapter 1 §4. Here we add details that matter in practice.

Two Kinds of NaN

There are two kinds of NaN: quiet NaN (qNaN) and signaling NaN (sNaN). A qNaN used as an input to an arithmetic operation raises no exception and simply propagates as the NaN result. Ordered comparisons ($\lt$, $\le$, $\gt$, $\ge$), however, raise the invalid-operation exception even for a qNaN (the standard also provides non-signaling versions of these comparisons). An sNaN raises the invalid-operation exception as soon as it is used as an input to most operations.

To test for NaN, the reliable way is the isnan-style function of each language (isnan in C, math.isnan in Python, etc.). The equality test x == x, which is false only for NaN, also works, but compiler speed options (such as -ffast-math) optimize under the assumption that NaN never appears and may remove this test:

x = 0.0 / 0.0  // NaN
x == x         // false (can detect NaN this way)
x != x         // true

Signed Zero ($\pm 0$)

$+0$ and $-0$ compare as equal ($+0 == -0$ is true), but $1/(+0) = +\infty$ while $1/(-0) = -\infty$, so they are distinguished.

3. Gradual Underflow

Subnormal (denormalized) numbers fill the gap between the smallest normalized number and zero, enabling gradual underflow.

Subnormal numbers have fewer significant digits than normalized numbers, but they guarantee the important property $\text{fl}(x - y) = 0 \Rightarrow x = y$ (where $\text{fl}$ denotes rounding to a floating-point number). Without subnormal numbers, there would be cases where $\text{fl}(x - y) = 0$ for distinct values $x \neq y$, and a routine such as "check that $x \neq y$, then compute $1/(x - y)$" would divide by zero. This property does not hold under a setting that replaces subnormal numbers with 0 (flush-to-zero).

4. The Correct-Rounding Guarantee

IEEE 754 guarantees that the basic operations ($+, -, \times, \div, \sqrt{\phantom{x}}$) are correctly rounded: the result is identical to performing the operation with infinite precision and then rounding.

The rounding modes are listed in Chapter 1 §6. The default is round to nearest, ties to even.

Thanks to correct rounding, the result of a basic operation $\circ \in \{+, -, \times, \div\}$ above satisfies, under the default round-to-nearest-even mode and as long as neither overflow nor underflow occurs (i.e., the absolute value of the exact result lies in the range of normalized numbers),

Here $u$ is the unit roundoff treated in Chapter 3; in double precision $u = 2^{-53} \approx 1.11 \times 10^{-16}$, while machine epsilon is $\varepsilon_{\text{mach}} = 2^{-52}$. Outside that range the model fails (for example, $2^{-1060} \times 2^{-1060}$ rounds to 0, giving $\delta = -1$). This property is the foundation of numerical stability analysis and is used repeatedly in error analysis.

5. Frequently Asked Questions

6. References

• Wikipedia, "IEEE 754"
• Wikipedia, "Double-precision floating-point format"
• D. Goldberg, "What Every Computer Scientist Should Know About Floating-Point Arithmetic," ACM Computing Surveys, 23(1), 5–48, 1991.
• IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic, IEEE, 2019.
• M. L. Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001.