Averages across all customers mix people who arrived under different conditions. Cohorts group by when somebody joined, which is usually the variable that explains everything.
Every term used above is defined in our experimentation glossary, and three experiments taken apart step by step sit in the case studies. Longer pieces with the workings attached are in the field notes.
Each row is everybody who arrived in one month; each column is how many remained after one month, two, three. Reading down a column compares like with like, which the average never does.
Do this: Build the table for the last twelve months. Look down columns, not across rows.
If the product improved while acquisition got worse, retention per cohort rises while the blended number stays flat. You would conclude nothing changed, and two things did.
Do this: Whenever a blended metric looks stable, check the cohorts before believing it.
Send traffic and baseline rate. We reply with the smallest effect your test can actually detect. One email, no call.
Most loss happens in the first period after joining. A change that improves month six and not month one moves almost nothing, because most people already left.
Do this: Judge onboarding changes on first period retention, not on lifetime value.
A month with forty customers produces a retention percentage that swings wildly for reasons unrelated to anything you did.
Do this: Group into quarters when monthly cohorts fall below a few hundred people.
This is teaching material and our own reading of standard practice, not advice for your specific site. Check anything important with your own specialist before you act on it.
Traffic, baseline rate and what you changed. We reply with the minimum detectable effect and whether the result meant anything. No cost, no call.