Bias vs variance: which one is breaking your model?

Two models fail with the same headline score, and the same "fix" makes both worse:

model A   train 45%   val 47%    high bias
model B   train 2%    val 31%    high variance
fix applied to both: more layers

Model A stayed blind, model B memorized in higher definition. One diagnosis, two diseases, opposite medicine.

The two failure modes

High bias is a model too simple for the pattern. It underfits: train error is high, validation error is high, and the two are close. It has not learned enough from the data it saw, so seeing more changes little.

High variance is a model flexible enough to memorize the noise. It overfits: train error is near zero, validation error is high, and the gap between them is the disease. It learned the training set instead of the pattern.

Capacity moves you along the line between them. That is the tradeoff: every notch of flexibility you add to fight bias is a notch available for memorizing noise.

Read the diagnosis off two numbers

The fourth row looks impossible and therefore matters most: validation beating training usually means leakage or a broken split, and no amount of model work fixes a measurement problem.

The fixes are opposites

High bias: more features, a bigger model, less regularization, longer training. Give the model the capacity it lacks.

High variance: more data first (the single most reliable cure), then regularization (L1/L2, dropout, early stopping), or a simpler model. Take capacity away or drown the noise.

Cross-validation belongs to neither list: it is not a fix but an honest measurement, the thing that makes the diagnosis table trustworthy in the first place.

The one-fix-for-both trap

Half of "we tried everything" stories are one disease treated with the other's medicine: more data poured into a high-bias model (same underfit, slower), or more layers added to a high-variance one (memorizes the noise in HD). Two numbers first. Then exactly one fix. Then measure again.

The takeaway

Bias and variance are opposite failures with opposite cures, and your train and validation errors already tell you which one you have. The print-ready PDF above has the diagnosis table and both fix lists on one page.