Train many decision trees on bootstrap samples of the data (and random feature subsets), then vote. Each tree is mediocre; the forest is robust.
Number of trees15
Avg single-tree acc—
Forest acc (majority vote)—
Boundary of ONE tree (jagged)
Boundary of the WHOLE forest (smooth)
15
4
Why averaging helps: each tree is high-variance — small data wiggles change its splits a lot. Bootstrap + feature randomness makes the trees disagree in different places, so when you average them, the noise cancels but the signal remains. This is the "wisdom of crowds" for models.