Why does my FP8 training run diverge when the same model trains fine in bf16?

First establish that it is numerics at all¶

A diverging run is not evidence of a numerics bug. It is evidence of a diverging run, and the two most common causes — a learning rate that was always marginal, and a difference in data order or initialization — have nothing to do with precision.

The test that separates them is repetition under a fixed seed. Run twice at the same precision, same seed, same data order. If the two runs blow up at different steps, you have a nondeterminism problem and any conclusion about precision drawn from a single run is a coin flip. If they blow up at the same step, you have something reproducible and you can bisect it. Then run that same seed in bf16: if bf16 survives what FP8 does not, on the same batch at the same step, the difference is in the arithmetic. If bf16 dies too, you have a training problem that low precision merely arrived at sooner.

The two failures that both get called “precision”¶

Resolution. E4M3 carries three mantissa bits and E5M2 carries two. One round-to-nearest step in E4M3 can therefore move a value by up to about six percent of itself, and in E5M2 by up to about twelve. That is per operation, on every operand. Nothing about it is subtle — the design bet of low-precision training is that gradient descent averages it away, and for the bulk of the tensors it does.

Range. E4M3’s largest finite value is 448. A tensor whose scale is set by a handful of outliers pushes everything else down the exponent range, where values crowd into the same few representable numbers or flush to zero. This is what per-tensor and per-block scaling exist to prevent, and it is the failure that actually kills runs. Not the rounding in the middle of the distribution: the saturation at one end and the collapse to zero at the other.

They present differently. Resolution loss is a small, consistent bias everywhere. Range loss is one tensor, at one step, at the moment its distribution moved — which is why it survives every smoke test run on the first thousand steps.

Where accumulation error comes from¶

Summing n numbers in sequence in a format with unit roundoff u admits an error proportional to n·u times the sum of the magnitudes. That classical recursive-summation bound has two consequences people are repeatedly surprised by. It grows with the length of the reduction, so a longer sequence, a larger batch or a wider hidden dimension makes the arithmetic worse with no other change to the model. And when the running total is much larger than the next addend, the addend can fall below half a unit in the last place of the total and be discarded outright — add enough small numbers to a large one and you add nothing at all.

On current tensor cores the FP8 matrix multiply accumulates into a wider format, so the matmul is usually not where this bites. Verify that for your kernel rather than assuming it, then look at the reductions nobody thinks of as arithmetic: the gradient all-reduce across data-parallel ranks, the loss reduction over a long sequence, the optimizer’s moment accumulation, and any fused kernel that changed the reduction order relative to the reference implementation you validated against. That last one is the quiet one: reduction order is not part of most kernels’ contracts, so a drop-in replacement can be numerically different while being functionally identical, and nothing in the interface says so.

Why a healthy-looking loss curve is weak evidence¶

A training loss is a mean over a great many tokens, and a mean is a low-pass filter. A defect that is severe on a small fraction of tokens and absent on the rest sits below the resolution the curve is plotted at, right up until it does not.

So an FP8 curve that tracks the bf16 curve is compatible with three situations: the numerics are fine, the numerics are uniformly degraded by less than the curve’s own noise, or the numerics are badly wrong somewhere an average does not reach. Telling them apart needs a quantity with a tail rather than a mean:

• the per-step maximum gradient norm, not the average;
• the fraction of each tensor that saturated or flushed to zero, per step, per tensor class;
• the relative deviation between the FP8 forward pass and a bf16 forward pass on the same batch and the same weights, reported as a maximum over elements.

The third is the most informative and the least often instrumented. It costs one extra forward pass on a sampled step and it converts “the curve looks fine” into a number that either moves or does not.

The bisection¶

Once you have a reproducible divergence, cast tensor classes back to bf16 one at a time and rerun to the same step. Cheapest first:

1. the gradient all-reduce and any cross-rank collective;
2. the optimizer states;
3. the backward pass — activation gradients;
4. the forward pass.

Whichever cast stops the divergence names the class. From there it is usually localizable further, to one layer whose activation or gradient distribution moved enough to break its scaling — which is a concrete bug with a concrete fix, and a different thing entirely from “FP8 is unstable”.

One boundary case is worth stating explicitly, because it is the one most often misread as success. If lowering the learning rate makes the FP8 run survive, that is not a passing verdict. Lee et al. put the standard plainly: reduced-precision training schemes “must have similar training stability and hyperparameter sensitivities to their higher-precision counterparts in order to be cost-effective”. A scheme that only trains inside a narrower hyperparameter window has already told you it lost something; you have measured the loss rather than repaired it.

It is also worth knowing that the fix is often structural rather than a matter of tuning. The FP4 work above traced instability to a scale inconsistency introduced by tensor transposition — forward and backward computations disagreeing about the scaling of the same values — and fixed it by making the quantization transposition-invariant. That is a bug with a cause, found by looking, not a property of four-bit arithmetic.

What can be bounded, and what can only be measured¶

Bounded. The forward error of one kernel, given ranges on its inputs. That is a theorem about real arithmetic, provable against a formal model of the floating-point operations, and it holds for every input in range including the ones you never ran — the only kind of statement here that covers inputs you have not seen.

Not bounded. Whether the run converges. That depends on the data, the optimizer trajectory and the loss surface, and no floating-point analysis reaches it. Anyone offering a proof that your training will converge in FP8 is selling something.

In between, and this is the useful part. A bound on a per-step quantity becomes a runtime check. If a kernel’s relative forward error provably cannot exceed some expression, you can evaluate that expression each step and compare it against the measured deviation from a wider-precision recomputation of the same step. Zero violations across a real run is a claim someone else can re-run. A violation is a bug localized to one kernel, at one step, on known inputs — which is a much better place to stand than in front of a diverging loss curve.

The method, and exactly how far I have carried it¶

The method is three steps. State the bound as a theorem about real arithmetic. Prove it in a proof assistant against a formal model of the arithmetic, so that nothing rests on the author’s own algebra. Then run it against real data and count the violations.

Being explicit about the limits of that here, because this page is worth nothing otherwise: the instance I have shipped is not FP8 and not transformer training. It is driftbound — forward-error bounds for the matrix profile’s streaming inner product, where five of the six headline results carry machine-checked proof obligations across Lean 4/Mathlib, Coq/Flocq and Gappa, with the Lean development pinned to the three classical axioms and no sorry; the sixth is stated as an open conjecture; and the bound was run against a full Parkfield seismic record with zero violations. A streaming inner product is a far smaller object than a training step, and I am not claiming the second on the strength of the first.

What transfers is the shape: an exact statement, a mechanically checked proof of it, and an empirical count of violations on data nobody chose to flatter it. And the diagnostic half of this page — the seed bisection, the precision bisection, the tail statistics, the wider-precision recomputation — needs none of the proof machinery at all. That half is where a diverging run should start, and most of the time it is where it ends.