28 — Confusion Matrix & Metrics

Every binary prediction lands in one of four boxes. Adjust the model threshold and watch metrics trade off — better recall usually costs precision.

Confusion matrix (positive class = "buyer")

	Predicted
	Buyer (1)	Non-buyer (0)
Actually Buyer	TP—	FN (Type II)—
Actually Non-buyer	FP (Type I)—	TN—

Metrics derived from the matrix

Accuracy

—

(TP+TN) / N

Precision

—

TP / (TP+FP)

Recall (Sensitivity)

—

TP / (TP+FN)

Specificity

—

TN / (TN+FP)

F1-score

—

2·P·R / (P+R)

FPR (1−Spec)

—

FP / (FP+TN)

Score distribution by class (slide threshold to cut)

Classification threshold: 0.50

Pick metrics for the problem: medical screening cares about recall (catch every sick patient). Spam filters care about precision (don't junk real mail). F1 balances both. Accuracy lies on imbalanced data — a 99% accurate fraud model could just be "predict no fraud."