Linear or logistic regression: a number, or a yes/no

Which model do you build first? It depends on the question. If the answer is a number, you want a linear regression. If the answer is a yes or a no, you want a logistic one. This page covers what each predicts, why logistic outputs a probability, the threshold that is a business choice, and the one thing both models share. The print-ready A4 PDF is at the bottom.

Linear

• Predicts a number: price, demand, minutes.
• The output can be any value, from minus to plus.
• The loss is squared error.

Logistic

• Predicts a probability: churn, fraud, click.
• The output is between 0 and 1, through the sigmoid.
• The threshold turns the probability into a yes or a no. 0.5 is a choice, not a law.

Same inputs, two outputs

Both models draw a line through the features. One reads the line as a number, the other pushes it through a sigmoid and reads a probability.

Remember

• Both are linear in the inputs. The difference is the output.
• With imbalanced classes, move the threshold and check recall.

Imbalance matters because a model on rare events, such as fraud, can look accurate by saying no every time. Recall tells you how many of the real cases you caught.

The trap: the threshold is a decision

A logistic model gives a probability. The yes/no comes from a threshold you choose.

A lower threshold flags more cases as yes, so you catch more real ones and accept more false alarms. A higher threshold does the opposite. There is no neutral setting: each value is a trade between the two kinds of error.

Interview phrasing worth memorizing: logistic regression gives me a probability; I pick the threshold with the business, from the cost of each kind of error.