Simpson’s paradox
A trend that appears in every subgroup but reverses when the groups are combined, because the groups differ in size or mix.
Formulaoverall rate = mix of group sizes × group rates
How we work it through
- 1
Break the total into segments before comparing.
- 2
Check whether the mix of segments changed between the two groups.
- 3
Compare within segments, then explain the mix.
The common trap
Comparing overall rates between segments with a different mix, such as new and returning customers.
When an AI answers
An AI assistant
“Team B converts worse overall, so its process is worse.”
- What it didn’t know
- It did not know team B handles far more first-time visitors, who convert less in every team. (Illustrative example.)
- Ask back
- “Compare the teams within new and returning visitors. Did the mix differ?”
Read next
Examples use Meridian, Atterna’s fictional practice marketplace.