A decision tree picks the split that most reduces impurity (Gini), then recurses. Each split is a vertical or horizontal cut in feature space. Crank up max depth and watch it overfit.
Tree depth0
Leaves0
Train accuracy—
The bias-variance trade: depth 1–2 is too rigid (underfit), depth 6 carves out every single point (overfit). Random forests build many such trees on resampled data, then vote — averaging out the noise.