← All cheatsheets
Data Scientist · #094 · November 6, 2026 · 2 min read

Confidence intervals in plain English

What a 95% interval says and what it does not, how n sets the width (four times the users, half the width), why two overlapping intervals can still differ, and how to report a range instead of a point: one page.

Get the free PDF

One page, print-ready, free to share. No signup needed.

Download the PDF

Readout, slide 9: conversion is 3.1%. Plus or minus what? The bracket is the part people argue with, and the part most slides leave off. This page covers what a 95% interval says and what it does not, how the sample size sets the width, why two overlapping brackets can still differ, and how to report a range instead of a point. The print-ready A4 PDF is at the bottom.

What 95% means

  • [2.7%, 3.5%]: the bracket from 248 conversions out of 8,000 visitors. The point is 3.1%.
  • 95%: of such brackets, computed the same way on repeated samples, catch the true rate.
  • This bracket: holds or does not. There is no 95% about this one; the 95% belongs to the method.

Width

  • n = 8,000: plus or minus 0.4 points, [2.7, 3.5].
  • n = 500: plus or minus 1.5 points, [1.6, 4.6].
  • Four times the n halves the width. Not a quarter: the width goes with the square root of n.

The bracket, six lines

Normal approximation. Fine when the successes and the failures are both above 10.

import math
conv, n = 248, 8_000
p = conv / n                        # 0.031
se = math.sqrt(p * (1 - p) / n)     # 0.0019
lo, hi = p - 1.96 * se, p + 1.96 * se
print(f'{p:.1%} [{lo:.1%}, {hi:.1%}]')
# 3.1% [2.7%, 3.5%]

Report the point, the bracket and the n, in that order, every time: 3.1% [2.7, 3.5], n = 8,000. For small counts (under 10 either way) use the Wilson interval instead; statsmodels.stats.proportion.proportion_confint(conv, n, method='wilson') does it.

Two of them

  • A and B overlap: they can still differ.
  • B minus A: one bracket on the difference.

With A at 3.1% and B at 3.8%, both on 8,000 visitors:

pa, pb, n = 248 / 8_000, 304 / 8_000, 8_000
se = math.sqrt(pa * (1 - pa) / n + pb * (1 - pb) / n)
d = pb - pa
print(f'{d:+.1%} [{d - 1.96 * se:+.1%}, {d + 1.96 * se:+.1%}]')
# +0.7% [+0.1%, +1.3%]

The brackets [2.7, 3.5] and [3.4, 4.2] overlap. The difference excludes zero. B is ahead.

The trap: three readings of a bracket

You seeYou saySay instead
A [2.7, 3.5], B [3.4, 4.2]they overlap, no differenceB minus A: +0.7 [0.1, 1.3]
[2.7, 3.5]95% chance the rate is in therethe method is right 95 times in 100
n = 500: [1.6, 4.6]3.1% conversion3.1%, between 1.6 and 4.6

Overlap is not a test. Compute the bracket on the difference: if it excludes zero, B is ahead, overlap or not.

What a p-value adds to this, and what it does not, is on the p-value sheet. This page is the bracket only.

Frequently asked questions

What does a 95% confidence interval mean?
If you repeated the measurement many times and computed the interval each time, about 95 of every 100 intervals would contain the true rate. The 95% is a property of the method. Any single interval either contains the true rate or it does not.
Is there a 95% chance the true value is inside my interval?
Not in the way the sentence suggests. The true rate is a fixed number; the interval is the thing that moves from sample to sample. Saying the method catches the truth 95 times in 100 is right. Saying this one bracket has a 95% chance is the common shortcut, and it is not what the interval claims.
How does the sample size change the width?
The width shrinks with the square root of n. Four times the users halves the width, not quarters it. At 3.1% conversion, n = 8,000 gives plus or minus 0.4 points, n = 500 gives plus or minus 1.5 points.
If two confidence intervals overlap, is there no difference?
They can still differ. A at [2.7, 3.5] and B at [3.4, 4.2] overlap, yet the interval on the difference, B minus A, is [0.1, 1.3] and excludes zero. Overlap is not a test. Compute the bracket on the difference.

Get the free PDF

One page, print-ready, free to share. No signup needed.

Download the PDF

More cheatsheets