A/B Test Sample Size Calculator
Most failed tests were lost before launch: too little traffic for the lift anyone could realistically expect. Enter your conversion rate, the smallest lift that would change a decision, and your daily sessions to see what the test really requires.
Detects 2.4% → 2.64%. Plan for 7 full weeks.
Talk to us about your measurementAt a 2.4% conversion rate, detecting a 10% relative lift (2.4% to 2.64%) at 95% significance and 80% power takes about 67,000 sessions per variant. At 3,000 sessions a day across both versions, that is roughly 45 days.
Choosing the lift to detect
Pick the smallest lift that would change what you do, not the lift you hope for. Halving it roughly quadruples the sample. If the answer comes back at four months, test a bolder change, move the test to a higher-traffic page such as the product page rather than a single collection, or test on add-to-cart as a validated leading metric.
Run in whole weeks
E-commerce traffic converts differently on weekdays, weekends and paydays. Even if you reach the sample in nine days, run to fourteen. Avoid launching tests across major promotions such as Black Friday, where shopper intent shifts and results rarely transfer to normal weeks.
Frequently asked questions
How long should an A/B test run on a Shopify store?
Long enough to reach the calculated sample, rounded up to whole weeks, with a minimum of two weeks for most stores. Low-traffic stores often find the honest answer is that small tweaks cannot be tested; bolder changes can.
Does testing more variants need more traffic?
Yes. Each extra variant adds an arm and also tightens the significance level each comparison must meet, so the sample per arm rises too.
Should I use relative or absolute lift?
Relative lift (+10% of the baseline) compares easily across pages. Absolute lift (+0.24 points) is clearer when you are estimating revenue. The calculator supports both.
More free tools
- Break-Even ROASThe ROAS and CPA your paid media must clear before a single order makes money.
- A/B SignificanceWhether your variant really won, how big the lift plausibly is, and whether you had the traffic to know.
- Stop or ContinueA straight answer on whether your test can end, and what checking it daily has cost you.
- SRM CheckerCatch a broken traffic split before a corrupted test drives a real decision.
Ready to prove your marketing ROI?
Book a free 30-minute consultation. Bring your ad account numbers; leave knowing which channels earn their budget.
Free. No commitment.