What is numerical computation (numerical analysis)?

Numerical computation is a general term for methods that approximately solve mathematical problems as concrete numerical values that can be handled by computers. It is used to solve problems such as differential equations, integrals, and systems of equations to practical accuracy when exact analytical solutions cannot be obtained.

What is the difference between analytical solutions and numerical solutions?

An analytical solution is an exact solution expressed in formulas and symbols, such as √2 or π. A numerical solution is an approximate solution expressed as concrete numbers like 1.41421356..., which contains error but can be directly used in practical calculations.

How much does computation speed differ depending on the algorithm?

When finding √2, the bisection method achieves an error of about 10⁻⁴ after 10 steps, while Newton's method reaches an error of 10⁻¹² in only 4 steps. The choice of algorithm has an order-of-magnitude impact on computational efficiency.

Chapter 1: What is Numerical Computation?

What is Numerical Computation?

Numerical computation is the method of solving mathematical problems numerically using computers.

In mathematics, it is sometimes possible to express the solution of an equation using a formula or to compute an integral symbolically. However, in many real-world problems, such "clean answers" are rarely obtainable. Numerical computation provides methods for finding "approximate numerical answers" to such problems.

Example: Quadratic vs. Quintic Equations

The solutions of a quadratic equation $ax^2 + bx + c = 0$ can be found using the quadratic formula:

$$x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$$

However, there is no general formula for solving equations of degree five or higher (Abel-Ruffini theorem). Numerical computation is required to solve such equations.

Analytical Solutions vs Numerical Solutions

Example: Solving $x^2 = 2$

Analytical solution (exact solution): $x = \sqrt{2}$ — an exact solution expressed as a formula

Numerical solution (approximate solution): $x \approx 1.41421356237...$ — a concrete number

Analytical solutions are exact, but in practice there are many situations where concrete numerical values are needed. Numerical analysis is the discipline of efficiently finding such numerical solutions.

Comparison of Analytical and Numerical Solutions
Aspect	Analytical Solution	Numerical Solution
Form	Formulas / Symbols	Concrete numbers
Precision	Exact	Approximate (with error)
Applicability	Limited	Broad
Advantage	Reveals general properties	Provides practical values

The Concept of Approximation

The core of numerical computation is "approximation." Rather than seeking a perfect answer, the goal is to efficiently find a "sufficiently good answer."

Why Approximation is Necessary

Many problems cannot be solved analytically
Computers can only handle a finite number of digits
In practice, a certain level of precision is sufficient

Example: Approximation of Pi

Pi $\pi = 3.14159265358979...$ is an infinite decimal. However, in practice:

Elementary school: $\pi \approx 3.14$
Engineering calculations: $\pi \approx 3.14159$
High-precision calculations: $\pi \approx 3.141592653589793$ (double precision)

Approximate values are used at a level of precision appropriate to the application.

Applications of Numerical Computation

Solving Equations

Finding $x$ such that $f(x) = 0$. Example: solving $x^5 - x - 1 = 0$

Numerical Integration

Computing the value of $\displaystyle\int_a^b f(x)\,dx$. Example: $\displaystyle\int_0^1 e^{-x^2}\,dx$

Differential Equations

Weather forecasting, fluid simulations, etc.

Matrix Computation

Systems of linear equations, data analysis, machine learning

Importance of Algorithms

In numerical computation, "how the computation is carried out" is extremely important. Even for the same problem, the choice of algorithm affects:

Speed can vary dramatically
Accuracy can differ
Stability (accumulation of errors) can be different

Example: Finding $\sqrt{2}$ with Two Algorithms

The positive solution of $x^2 = 2$ ($\sqrt{2} = 1.41421356237...$) is computed using a slow algorithm and a fast algorithm for comparison.

Method 1: Bisection Method (slow)

Since $1^2 = 1 < 2$ and $2^2 = 4 > 2$, the solution lies in the interval $[1, 2]$. The midpoint of the interval is evaluated, and the interval is narrowed to the half containing the solution. This process is repeated.

Step	Approximation	Error
1	1.5	$8.6 \times 10^{-2}$
5	1.40625	$8.0 \times 10^{-3}$
10	1.4150390625	$8.3 \times 10^{-4}$

Even after 10 steps, the error is on the order of $10^{-4}$. Precision improves by about 1 bit per step.

Method 2: Newton's Method (fast)

Starting with initial value $x_0 = 1$, the following formula is iterated:

$$x_{n+1} = \frac{1}{2}\left(x_n + \frac{2}{x_n}\right)$$

Step	Approximation	Error
1	1.5	$8.6 \times 10^{-2}$
2	1.41666...	$2.5 \times 10^{-3}$
3	1.41421568...	$2.1 \times 10^{-6}$
4	1.41421356237...	$1.6 \times 10^{-12}$

Only 4 steps to reach error $10^{-12}$. The number of correct digits roughly doubles with each step.

Even for the same problem, the convergence speed differs dramatically depending on the choice of algorithm.

Summary

Numerical computation is the method of solving mathematical problems numerically
Analytical solutions are formulas; numerical solutions are concrete numbers
Many problems cannot be solved analytically, so approximation is necessary
The choice of algorithm affects both accuracy and speed
Numerical computation is used in various fields of science and engineering