Data Scientist · #055 · September 24, 2026 · 2 min read
One-hot vs label encoding: which one is silently breaking your model?
Why integer-encoded categories poison linear and distance models, the three encoders and when each is right, the high-cardinality escape hatches, and the leakage rule: one page.
Get the free PDF
One page, print-ready, free to share. No signup needed.
You just told your model blue is less than red. Label encoding is the most common silent bug in tabular ML: nothing crashes, the scores just quietly make no sense. One page on encoding categories without lying. The print-ready A4 PDF is at the bottom.
The bug
{'red': 0, 'green': 1, 'blue': 2}invents an order.- Linear models read blue as 2x green.
- KNN and k-means read it as distance.
# the bug: color has no order
df['color'] = df['color'].map({'red': 0, 'green': 1, 'blue': 2})
# the fix: one column per color
X = pd.get_dummies(df, columns=['color'], drop_first=True)
The three tools
- One-hot: a 0/1 column per category. The default for nominal data.
- Ordinal: integers, only when the order is real (S < M < L).
- Target encoding: category becomes the mean of y. High cardinality only, and fit on train folds only.
In code
pd.get_dummies(df): quick, fine in notebooks.OneHotEncoder(handle_unknown='ignore'): pipelines and production. Theignoresaves you when a category shows up at predict time that training never saw.OrdinalEncoder(categories=[order]): you pass the order yourself, never trust the alphabet.
Too many categories
- 10k zip codes must not become 10k columns.
- Group the rare tail into
"other". - Target encode inside CV folds, on training data only.
The trap: which encoder for which column
| Column | Use | Because |
|---|---|---|
| color, city, browser | one-hot | no order exists |
| S / M / L, satisfaction | ordinal with explicit order | the order is real |
| zip code, merchant id | target encoding or "other" bucket | one-hot explodes |
Gotchas
- Trees forgive label encoding, linear and distance models do not.
drop_first=Truefor linear regression (the dummy trap: k dummies plus an intercept are collinear).- Fit every encoder on the training set only, then transform the test set. Encoders leak like any other fitted transformer.
Interview phrasing worth memorizing: encoding is a modeling decision, not preprocessing trivia. The encoder decides what geometry the model sees.
Frequently asked questions
What is wrong with label encoding categories like colors?
Mapping red=0, green=1, blue=2 invents an order and a distance that do not exist. A linear model learns one coefficient for the column, so blue must pull exactly twice as hard as green. Distance-based models (KNN, k-means, SVM) read blue as closer to green than to red. The categories were names; the encoding turned them into math.
When is label (ordinal) encoding actually correct?
When the order is real: sizes S < M < L < XL, satisfaction scales, education levels. Use OrdinalEncoder and pass the category order explicitly, because the default is alphabetical, which is another invented order. Ratings that are already numeric can usually stay numeric.
Does label encoding hurt tree-based models like random forests and XGBoost?
Much less. Tree splits only ask greater-or-less, so a tree can carve any subset of labels out of an integer encoding with enough splits. It costs some depth but rarely much accuracy. The moment a linear head, a neural embedding-free input, or a distance metric appears, the fake order becomes signal and the damage is real.
How do you encode a column with thousands of categories, like zip codes?
Do not one-hot it into thousands of columns. Group the rare tail into an 'other' bucket, or use target encoding (replace each category with the mean of y for that category), which must be fit inside cross-validation folds on training data only, otherwise the target leaks and your scores inflate.
Get the free PDF
One page, print-ready, free to share. No signup needed.