Maths for AI
Topic 16 · Information & computation · ⏱ about 2.5 h

Floating Point & Numerical Stability

How computers store real numbers, why 0.1 + 0.2 is not 0.3, overflow and underflow, the stable softmax and log-sum-exp tricks, working with log-probabilities, and what float16 and bfloat16 change when training large models.

Show:
In this topic
Beginner

Computers approximate real numbers

A floating-point number stores a sign, a set of significant digits (the mantissa) and an exponent, like scientific notation in binary: 1.0110×251.0110 \times 2^{5}. Because there are only so many digits, most real numbers are rounded. 0.1 has no exact binary form, which is why 0.1 + 0.2 gives 0.30000000000000004.

TypeBitsSignificant decimal digitsLargest valueUsed for
float6464about 16about 1.8 × 10308NumPy and pandas by default, science
float3232about 7about 3.4 × 1038deep learning by default
float1616about 365,504mixed-precision training, inference
bfloat1616about 2–3about 3.4 × 1038training large models on modern GPUs and TPUs

The gap between 1 and the next representable number is machine epsilon: about 2.2×10−162.2\times10^{-16} for float64 and 1.2×10−71.2\times10^{-7} for float32.

Beginner

Overflow, underflow and cancellation

  • Overflow: a result too large to store becomes inf. e1000e^{1000} overflows even in float64; e89e^{89} overflows float32. Then inf / inf gives nan (not a number), and one nan spreads through every calculation it touches: a classic cause of a training loss suddenly showing nan.
  • Underflow: a result too small becomes 0. Multiplying many probabilities (a sentence of 1,000 tokens) underflows to 0, and then log(0) is -inf.
  • Cancellation: subtracting two nearly equal numbers loses most significant digits. Computing a variance as E[x2]−E[x]2E[x^2] - E[x]^2 on large values can even come out negative.
Beginner

The stable softmax and log-sum-exp

Softmax turns scores (logits) into probabilities: softmax(z)i=ezi∑jezj\text{softmax}(z)_i = \dfrac{e^{z_i}}{\sum_j e^{z_j}}. Computed directly, a logit of 1000 overflows. The fix uses one identity: subtracting the same number from every logit does not change the result, because the factor e−ce^{-c} cancels from top and bottom. Subtract the maximum:

softmax(z)i=ezi−m∑jezj−m,m=max⁡jzj\text{softmax}(z)_i = \frac{e^{z_i - m}}{\sum_j e^{z_j - m}}, \quad m = \max_j z_j

Now the largest exponent is e0=1e^0 = 1: no overflow, and at least one term in the sum is 1, so no division by zero. The same trick gives the log-sum-exp: log⁡∑jezj=m+log⁡∑jezj−m\log\sum_j e^{z_j} = m + \log\sum_j e^{z_j - m}.

That is why libraries offer combined functions such as PyTorch's cross_entropy, which takes raw logits and applies log_softmax internally with this trick. Passing probabilities you have already softmaxed, or taking the log of a softmax yourself, throws that protection away.

Try it

Try it: naive versus stable softmax

Raise the logits. In float32, the naive formula overflows to inf and produces nan; the stable formula gives the same, correct probabilities at any scale.

Interactive: this part runs in your browser.

Beginner

Habits that prevent numerical bugs

  • Work with log-probabilities: add logs instead of multiplying probabilities.
  • Use the library's fused functions: logsumexp, log_softmax, log1p (for log⁡(1+x)\log(1+x) with tiny xx), and losses that take logits.
  • Scale your inputs: standardised features keep values in a comfortable range (see feature scaling).
  • Never compare floats with ==: use np.isclose or np.allclose with a tolerance.
  • Watch for nan early: check the loss every few steps; np.isfinite(x).all() is a cheap assertion.
  • Mixed precision: with float16 training, a loss scaler multiplies the loss so small gradients do not underflow to zero; bfloat16 has float32's range and usually needs no scaling (see efficient deep learning).
Beginner

Code: see the problems and the fixes

Run it to see float precision, overflow, underflow and the stable softmax in action.

Python
import numpy as np
np.seterr(all="ignore")                 # show inf/nan instead of warnings

print(0.1 + 0.2, np.isclose(0.1 + 0.2, 0.3))
print("eps float64:", np.finfo(np.float64).eps, " eps float32:", np.finfo(np.float32).eps)
print("float16 max:", np.finfo(np.float16).max, " 70000 as float16:", np.float16(70000))

def softmax_naive(z):
    e = np.exp(z)
    return e / e.sum()

def softmax_stable(z):
    e = np.exp(z - z.max())
    return e / e.sum()

z = np.array([1000.0, 999.0, 990.0], dtype=np.float32)
print("naive: ", softmax_naive(z))
print("stable:", softmax_stable(z).round(5))

# underflow: multiplying 2,000 probabilities of 0.6
probs = np.full(2000, 0.6)
print("product:", np.prod(probs), " sum of logs:", np.log(probs).sum().round(2))

def logsumexp(z):
    m = z.max()
    return m + np.log(np.exp(z - m).sum())
print("log-sum-exp:", logsumexp(np.array([1000.0, 999.0, 990.0])).round(4))

# cancellation in the variance formula
x = 1e9 + np.array([1.0, 2.0, 3.0])
print("E[x^2]-E[x]^2:", (x**2).mean() - x.mean()**2, " np.var:", x.var())
Open in playground ↗
Beginner

Practice: check yourself

Work each question out (on paper, in your head, or in Python), type the answer and press ✔ Check. Fractions like 1/4 are fine. Hints and a worked answer are there when you need them.

Question 1.
Stable softmax subtracts the largest input first. For inputs (1000,1001)(1000, 1001), what is the softmax probability of the second? Give 3 decimal places.
Question 2.
Float32 has about 7 significant digits. Roughly how many significant digits does float64 have?
Check yourself

Quiz

1. Why does subtracting the maximum logit not change the softmax result?

2. Your training loss suddenly shows nan. A likely cause is…

3. Why work with log-probabilities instead of multiplying probabilities?

4. What is the main advantage of bfloat16 over float16?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready