Which machine learning algorithm should I use for my problem?
Nine problems, nine first picks, chosen by three questions: what output, how much data, who needs to understand it. Plus the baseline trap that invalidates most first models.
Get the free PDF
One page, print-ready, free to share. No signup needed.
Stop defaulting to XGBoost. Model choice is not a leaderboard question, it is three ordinary questions: what output do you need, how much data do you have, and who needs to understand the result. Answer those and the first model picks itself.
The coarse routing is a three-node tree: predicting a number leads to the regression family; not a number but labels available leads to classification; no labels at all leads to clustering. The grid below refines that into nine first picks.
Ask the problem, read the pick
| The problem | First pick | The note |
|---|---|---|
| Predict a number, explain the drivers | Linear regression | coefficients are the explanation |
| Yes/no, and you must justify it | Logistic regression | the baseline in banking for a reason |
| Yes/no, accuracy is all that matters | Gradient boosting | XGBoost/LightGBM on tabular data |
| Find groups, no labels exist | k-means | then LOOK at the clusters |
| Too many columns to see anything | PCA | compress first, model second |
| Forecast with trend and seasons | Seasonal model | SARIMA or Prophet before deep nets |
| Text, images, audio | Neural network | or a pretrained model + fine-tune |
| Tiny dataset, must stay readable | Decision tree | print it, show the stakeholder |
| Any of the above, day one | A dumb baseline | mean, majority class, last value |
The last row is not a joke. It is the most important one, and it applies to every other row on the grid: whatever the first pick is, day one belongs to the baseline it has to beat.
Two of the picks deserve a note. k-means comes with a mandatory second step: then LOOK at the clusters, because unlabeled groups mean nothing until a human reads them. And PCA is not a model at all, it is the move you make when there are too many columns to see anything: compress first, model second.
The trap: skipping the baseline
A model is only good compared to something.
- Predict the mean: your regression must beat it. Sometimes it barely does.
- Majority class: 95% accuracy on 95/5 data is worse than doing nothing.
- Last value: brutally hard to beat on most time series.
If the fancy model beats the dumb baseline by 1%, ship the baseline. It is cheaper to run, easier to explain, and it will not wake anyone up at night.
Boosting or neural net?
The question that actually splits modern projects.
Gradient boosting when:
- Data is tabular: rows and columns
- Hundreds to millions of rows
- Features already mean something
- You want feature importance
Neural nets when:
- Text, images, audio, video
- Raw signal, features unclear
- A pretrained model exists
- You can pay the training bill
On tabular business data, boosting wins this comparison far more often than the hype suggests. On raw signal, it is not a contest.
Interpretability is a requirement, not a preference
Two rows of the grid exist purely because someone has to understand the output. Logistic regression over boosting when a prediction must be justified; a decision tree over almost anything when the dataset is tiny and the stakeholder wants to see the logic, because you can print a tree and walk through it in a meeting. Choosing the readable model is not settling. It is reading the requirements.
The takeaway
Three questions pick your first model: what output, how much data, who needs to understand it. Start with the dumb baseline, reach for the grid's first pick, and only escalate when the numbers say so. The full decision grid is a print-ready PDF above. Pin it above your desk and skip the two-hour model debate.
Frequently asked questions
How do I choose a machine learning algorithm?
When should I use logistic regression instead of XGBoost?
What is a baseline model and why do I need one?
Should I use gradient boosting or a neural network?
Get the free PDF
One page, print-ready, free to share. No signup needed.