Precision vs recall vs F1: which metric should you use?

Every argument about classifier quality is secretly one of two questions. Precision: when the model raised an alarm, was it right? Recall: of all the real cases, how many did it catch? Get these two straight and the rest of the metric zoo falls into place.

The four outcomes

Precision

TP / (TP + FP): of my alarms, how many were real?

Optimize precision when false alarms are the expensive error. A spam filter with low precision sends real mail to the junk folder. Fraud review queues drown analysts when precision drops.

Recall

TP / (TP + FN): of the real ones, how many did I catch?

Optimize recall when a miss is the expensive error. Cancer screening, safety inspections, search: the false negative is what hurts.

F1

2PR / (P + R): the harmonic mean. It punishes imbalance between the two, so you cannot buy a good F1 by maxing one and ignoring the other. Use it when you need a single number and classes are skewed.

The threshold dial

Every classifier outputs a score; the threshold turns it into a yes/no.

• Raise it: precision up, recall down. Fewer, surer alarms.
• Lower it: recall up, precision down. Catch more, cry wolf more.

You rarely get both. The threshold decides which one you buy with the other.

The trap: 99% accurate, 100% useless

On data with 1% fraud, a model that always predicts "legit" scores 99% accuracy and catches zero fraud. Accuracy is blind to imbalance because the giant true-negative cell dominates it.

The fix: report precision and recall (or F1), never accuracy alone. And make "what is the class balance?" your first question about any classifier.