Machine Epsilon

1. Definition

Machine epsilon $\varepsilon_{\mathrm{mach}}$ is the fundamental quantity indicating the precision limit of a floating-point number system. Two common definitions:

For a floating-point system with base $\beta$ and precision $p$ (significand digits including the hidden bit):

The binary formats of IEEE 754 (single, double, and so on) use $\beta = 2$, so $\varepsilon_{\mathrm{mach}} = 2^{1-p}$.

2. IEEE 754 Values

Available as constants in most languages: C/C++ DBL_EPSILON, Python sys.float_info.epsilon, MATLAB eps.

3. Unit Roundoff

The unit roundoff $u$ is central to floating-point error analysis:

For IEEE 754 double precision, $u = 2^{-53} \approx 1.11 \times 10^{-16}$.

The fundamental theorem of floating-point arithmetic: for $\circ \in \{+, -, \times, /\}$,

(as long as the result neither overflows nor underflows), where $\mathrm{fl}(\cdot)$ denotes round-to-nearest. This property is the starting point for all error analysis.

4. Measurement Algorithm

Machine epsilon can be measured with a simple algorithm:

function computeMachineEpsilon():
    eps = 1.0
    while (1.0 + eps) > 1.0:
        eps = eps / 2.0
    return 2.0 * eps

Note: Compiler optimizations (especially extended precision registers) may affect the result. In C/C++, use the volatile qualifier or control optimization levels.

5. Applications

Convergence Tolerance

Iterative method stopping criteria are often based on $\varepsilon_{\mathrm{mach}}$:

• Example stopping criterion for Newton's method: the relative change $|x_{n+1} - x_n| \le \tau |x_{n+1}|$ ($\tau$ from a few times $\varepsilon_{\mathrm{mach}}$ up to $\sqrt{\varepsilon_{\mathrm{mach}}}$; if the residual $|f(x_n)|$ is used, scale it to $f$)
• Eigenvalue error of a symmetric (Hermitian) matrix: $O(\varepsilon_{\mathrm{mach}} \|A\|)$ (for nonsymmetric matrices it depends on the conditioning and can be larger)

Approximate Floating-Point Comparison

Since equality comparison of floating-point numbers is unreliable due to rounding, use:

The relative tolerance $\tau_{\mathrm{rel}}$ is taken as several times $\varepsilon_{\mathrm{mach}}$ or more, depending on how errors accumulate in the computation, and an absolute tolerance $\tau_{\mathrm{abs}}$ is used together near 0.

Condition Number

For a backward-stable algorithm, the relative error of the numerical solution is roughly $\kappa \cdot u$ (with $u = \varepsilon_{\mathrm{mach}}/2$), where $\kappa$ is the condition number. When $\kappa \cdot u$ approaches 1, hardly any significant digits remain.

6. Worked Examples

7. Pitfalls

• Inconsistent definitions: Some references use $\varepsilon_{\mathrm{mach}}$ to mean unit roundoff $u = \varepsilon_{\mathrm{mach}}/2$. Be aware of potential confusion.
• Not the smallest positive float: Machine epsilon is entirely different from the smallest positive floating-point number ($2^{-1074} \approx 4.9 \times 10^{-324}$ for double; the smallest positive normalized number is $2^{-1022}$).
• Extended precision: On x86, the FPU may use 80-bit extended precision, causing measured values to differ from expected.

8. Frequently Asked Questions

9. References

• Wikipedia, "Machine epsilon"
• Wikipedia, "IEEE 754"
• D. Goldberg, "What Every Computer Scientist Should Know About Floating-Point Arithmetic," ACM Computing Surveys, 23(1), 1991.
• N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, 2002.