Overfitting vs underfitting, explained in three curves
The difference between learning the pattern and memorizing the noise, how to diagnose which one your model did, and the five fixes.
Get the free PDF
One page, print-ready, free to share. No signup needed.
Show the same scatter of points to three models. The too-simple one draws a straight line through a curve. The right one draws the trend. The too-flexible one threads through every single point, noise included. That third model looks perfect on paper and fails in production: that is overfitting.
Three fits, same data
- Underfit: too simple to see the shape. High error on training data, high error on test data.
- Good fit: learned the trend, ignores the wiggles. Low error on both.
- Overfit: memorized every wiggle of noise. Near-zero training error, high test error.
The classic analogy: a student who memorized past exams aces every practice test and fails the real one.
Diagnose yours
| Underfitting | Overfitting | |
|---|---|---|
| train error | high | near zero |
| test error | high too | high |
| cause | model too simple | model memorized noise |
| fix | more features, capacity | simplify, regularize |
Compare train error vs test error. The gap is the diagnosis.
Five fixes that work
In rough order of how often they are the right answer:
- More data: noise averages out with volume.
- Simpler model: fewer parameters to memorize with.
- Regularization: L1/L2 penalties on extreme weights.
- Early stopping: quit training while validation still improves.
- Cross-validation: five splits for an honest estimate, so you catch the problem before production does.
The trap: test-set leakage
Scale, encode, or select features BEFORE the train/test split and your test score is fiction: the preprocessing saw the test rows, so information leaked.
- The symptom: great offline metrics, embarrassing production performance.
- The fix: split first; fit every transform on the training set only.
This is exactly why sklearn's Pipeline exists. Use it and the leak becomes structurally impossible.
Frequently asked questions
- How do I detect overfitting?
- Compare training error with test (or validation) error. Near-zero training error with clearly worse test error is the signature of overfitting. Similar but high errors on both sides point to underfitting instead.
- What is the difference between overfitting and underfitting?
- An underfit model is too simple to capture the real pattern: it is wrong everywhere, train and test alike. An overfit model is too flexible: it memorized the training data, noise included, and falls apart on anything new.
- Does more data always fix overfitting?
- It usually helps, because noise averages out with volume, but it is not always available or sufficient. Regularization, simpler models, early stopping and cross-validation attack the same problem from the model side.
- What is regularization in one sentence?
- A penalty added to the loss for extreme weights, which discourages the model from bending itself around individual noisy points: L2 shrinks weights smoothly, L1 pushes some to exactly zero.
Get the free PDF
One page, print-ready, free to share. No signup needed.