Floating-Point Numbers

Goals

Understand the structure of IEEE 754 floating-point numbers (sign, exponent, significand), the differences between normalized numbers, subnormals, and special values, and learn about machine epsilon and rounding modes.

Prerequisites

  • Basics of binary numbers (bits, bytes)
  • The concept of exponential notation
Contents

§1 Representing Numbers in Computers

Computers represent numbers using a finite number of bits. Floating-point numbers are the standard way to approximately represent real numbers.

Floating-Point Representation

A number $x$ is expressed in the form:

$$x = \pm m \times b^e$$

where:

  • $m$: significand (mantissa)
  • $b$: base (radix), usually 2
  • $e$: exponent

Since the number of digits of $m$ (the precision) and the range of $e$ are fixed, only finitely many numbers can be represented.

Example: Floating-point in decimal

$12345 = 1.2345 \times 10^4$

$0.00678 = 6.78 \times 10^{-3}$

The decimal point "floats" to express numbers of vastly different magnitudes.

§2 The IEEE 754 Standard

IEEE 754 is the international standard for floating-point representation and arithmetic.

Main Formats

FormatBitsSignExponentSignificandDecimal digits
Single precision (float)321823~7
Double precision (double)6411152~16
Single Precision (32-bit) Sign S 1 bit Exponent E 8 bits Significand M 23 bits Normalized: value = (-1)S × 1.M × 2(E−127)
Figure 1. Single-precision (float, 32-bit) bit layout. Bias = 127. The formula applies to normalized numbers (§3).
Double Precision (64-bit) Sign S 1 bit Exponent E 11 bits Significand M 52 bits Normalized: value = (-1)S × 1.M × 2(E−1023)
Figure 2. Double-precision (double, 64-bit) bit layout. Bias = 1023. The wider significand provides ~16 decimal digits of precision. The formula applies to normalized numbers (§3).

This section covers the bit layout and basic format only. Rounding modes are detailed in §6; deeper coverage of special values, gradual underflow, and correctly-rounded operations is given in the next chapter, “IEEE 754 Floating-Point”.

§3 Normalized Numbers

A normalized number has its exponent adjusted so that the leading digit of the significand $1.M$ is always 1 (this leading 1 is not stored, so the most significant bit of the stored field $M$ may be 0).

Normalized Representation (double precision)

$$x = (-1)^S \times 1.M \times 2^{E-1023}$$

The leading "1." is implicit (the hidden bit) and is not stored. This saves one bit, effectively gaining an extra bit of precision.

The implicit "1." applies only when the exponent field E is 1 or greater (normalized numbers). When E = 0, the implicit bit switches to "0.", representing subnormal numbers and zero (§4).

Example: Representing $-6.5$

$-6.5 = -1.1010_2 \times 2^2$

  • Sign S = 1 (negative)
  • Exponent E = 2 + 1023 = 1025 = $10000000001_2$
  • Significand M = $1010000\ldots0_2$ (the part after "1.")

Double-Precision Range

  • Smallest normalized number: $\approx 2.2 \times 10^{-308}$
  • Largest finite value: $\approx 1.8 \times 10^{308}$

§4 Special Values

IEEE 754 defines several special values based on the combination of the exponent E and the significand M. The implicit leading bit (hidden bit) changes depending on E.

Exponent ESignificand MHidden bitInterpretationValue (double precision)
00—Zero$\pm 0$
0$\neq 0$0.Subnormal$(-1)^S \times 0.M \times 2^{-1022}$
1–2046any1.Normalized$(-1)^S \times 1.M \times 2^{E-1023}$
20470—Infinity$\pm\infty$
2047$\neq 0$—NaNnot a number (NaN)

Infinity

Exponent all 1s, significand all 0s. $1/(+0) = +\infty$, $1/(-0) = -\infty$.

NaN (Not a Number)

Exponent all 1s, significand nonzero. Results from $0/0$, $\infty - \infty$, etc.

Subnormal Numbers

Exponent all 0s, significand nonzero. Represent very small numbers close to zero.

Zero

Exponent and significand all 0s. Two variants: $+0$ and $-0$.

Example: Smallest Subnormal (Double Precision)

With $E = 0$ and $M$ having only its least significant bit set ($M = 0\ldots001$):

$$x = 1 \times 2^{-52} \times 2^{-1022} = 2^{-1074} \approx 4.9 \times 10^{-324}$$

This is $2^{52} \approx 4.5 \times 10^{15}$ times smaller than the smallest normalized number ($\approx 2.2 \times 10^{-308}$). In the subnormal range, precision (the number of effective significand bits) gradually degrades, but in exchange the drop to zero on underflow is softened — this is gradual underflow.

§5 Machine Epsilon

Machine Epsilon $\varepsilon_{\mathrm{mach}}$

The difference $\varepsilon_{\mathrm{mach}}$ between 1 and the smallest floating-point number greater than 1 ($2^{-52}$ in double precision). With round-to-nearest, the unit roundoff is $u = \varepsilon_{\mathrm{mach}}/2$.

Note: with round-to-nearest, the smallest positive floating-point number $\varepsilon$ such that $1 + \varepsilon \neq 1$ is about half of this (a number slightly above $2^{-53}$), so it does not coincide with $\varepsilon_{\mathrm{mach}}$.

It represents the "relative precision" of floating-point numbers.

Values

  • Single precision: $\varepsilon_{\mathrm{mach}} \approx 1.2 \times 10^{-7}$ ($2^{-23}$)
  • Double precision: $\varepsilon_{\mathrm{mach}} \approx 2.2 \times 10^{-16}$ ($2^{-52}$)

Meaning of Machine Epsilon

When a real number $x$ in the normalized range is rounded to a floating-point number $\mathrm{fl}(x)$ with round-to-nearest:

$$\dfrac{|\mathrm{fl}(x) - x|}{|x|} \leq \dfrac{\varepsilon_{\mathrm{mach}}}{2}$$

That is, the relative error is at most $\varepsilon_{\mathrm{mach}}/2$.

§6 Rounding Modes

IEEE 754 defines five rounding modes (roundTiesToAway is required only for decimal formats and is often not implemented for the binary single and double formats):

  • Round to nearest, ties to even (roundTiesToEven, default): round to the nearest representable value; break ties by choosing the value whose significand is even
  • Round to nearest, ties away from zero (roundTiesToAway): round to the nearest; break ties by choosing the value farther from zero
  • Round toward zero (roundTowardZero, truncation)
  • Round toward $+\infty$ (roundTowardPositive, ceiling)
  • Round toward $-\infty$ (roundTowardNegative, floor)
Rounding positive numbers (a < x < b, adjacent FP numbers) 0 a (even signif.) midpoint b (odd signif.) x₁ x₂ x₁ (near a) x₂ (near b) 1 TiesToEven (default) midpoint→a (even signif.) 2 TiesToAway midpoint→b (away from 0) 3 TowardZero (truncation) always → a 4 Toward +∞ (ceiling) always → b 5 Toward −∞ (floor) always → a
Figure 3. Five rounding modes (positive numbers). x₁ (near a) and x₂ (near b) show both cases. Modes 1, 2 always round to nearest; 3, 5 always round toward a (toward 0 / −∞); 4 always rounds toward b (+∞). Only at the midpoint do modes 1 and 2 differ.
Rounding negative numbers (a < x < b, adjacent FP numbers) 0 a (odd signif.) midpoint b (even signif.) x₁ x₂ x₁ (near a) x₂ (near b) 1 TiesToEven (default) midpoint→b (even signif.) 2 TiesToAway midpoint→a (away from 0) 3 TowardZero (truncation) always → b (toward 0) 4 Toward +∞ (ceiling) always → b 5 Toward −∞ (floor) always → a
Figure 4. Five rounding modes (negative numbers). Compared to the positive case, modes 3 (TowardZero) and 4 (Toward +∞) now both point toward b (closer to zero), while mode 5 (Toward −∞) points toward a (larger absolute value).

Quick Reference: When to Use Each Rounding Mode

ModeGood forNot recommended for
TiesToEven General scientific computing (default). Avoids systematic bias at ties When directional guarantees are needed
TiesToAway Financial/accounting computation in decimal formats (usually unavailable for binary single/double) Large-scale summation (slight upward bias)
TowardZero Sign-symmetric truncation (C's (int)x always truncates toward zero regardless of the rounding mode; converting out-of-range values or NaN is undefined behavior) Precision-sensitive iterative computation
Toward $+\infty$ Upper bound in interval arithmetic Standalone computation (always biased upward)
Toward $-\infty$ Lower bound in interval arithmetic Standalone computation (always biased downward)

In interval arithmetic, Toward $+\infty$ and Toward $-\infty$ are used together to compute an interval $[\text{lower}, \text{upper}]$ that is guaranteed to contain the exact result.

§7 Summary

  • Floating-point numbers: approximate real numbers in the form $\pm m \times 2^e$
  • IEEE 754: international standard. Single precision (32-bit), double precision (64-bit)
  • Normalized numbers: the leading "1." of the significand $1.M$ is implicit (hidden bit); switches to "0." for subnormals
  • Special values: $\pm\infty$, NaN, $\pm 0$, subnormals
  • Machine epsilon: the difference between 1 and the smallest floating-point number greater than 1 ($2^{-52}$ in double precision)
  • Single precision: ~7 decimal digits; double precision: ~16 decimal digits

Frequently Asked Questions

Q1. What is a floating-point number?

A floating-point number is a format for approximately representing real numbers in a computer, expressed as $\pm m \times 2^e$. Standardized by IEEE 754, single precision (32-bit, about 7 digits) and double precision (64-bit, about 16 digits) are widely used.

Q2. What is machine epsilon?

Machine epsilon $\varepsilon_{\mathrm{mach}}$ is the difference between 1 and the smallest floating-point number greater than 1, and it represents the relative precision of floating-point numbers. In double precision it is $2^{-52} \approx 2.2 \times 10^{-16}$. When a real number in the normalized range is rounded to floating point with round-to-nearest, the relative error is at most $\varepsilon_{\mathrm{mach}}/2$.

Q3. What is NaN?

NaN (Not a Number) is a special value defined in IEEE 754 that represents the result of operations such as $0/0$ or $\infty - \infty$. It occurs when the exponent field is all 1s and the significand is nonzero. NaN compares "unequal" to every value, including itself: comparisons such as == and < are all false, and only != is true (which is why x != x detects NaN).