Polynomial of degree d fit to noisy data. Too simple (d=1) → high bias, underfit. Too flexible (d=12) → high variance, overfit. Test error has a sweet spot in the middle.
Polynomial degree3
Train MSE—
Test MSE—
Regime—
Fit on training data (true curve = green dashed)
Error vs degree
3
Total error = bias² + variance + noise. Increasing model complexity reduces bias but inflates variance. The "best" model is the one that minimizes test error — not training error (which always keeps dropping).