Bias-Variance Tradeoff

Understanding the four fundamental scenarios in machine learning

High Bias, High Variance

Worst Case: Shots are both far from target (biased) and scattered everywhere (high variance). The model is both systematically wrong and inconsistent.

High Bias, Low Variance

Underfitting: Shots are consistently off-target but grouped together. The model is predictable but systematically wrong.

Low Bias, High Variance

Overfitting: Shots are centered around the target but widely scattered. The model can be accurate on average but inconsistent.

Low Bias, Low Variance

Ideal Case: Shots are tightly grouped around the target. The model is both accurate and consistent - perfect generalization!

Key Concepts:

Target (×): True value we want to predict
Shots (●): Model predictions from different training sets

Bias: How far the average prediction is from the target

Variance: How much predictions vary between different training sets

Goal: Minimize total error = Bias² + Variance + Noise