How many visitors does your A/B test actually need?
Enter your baseline conversion rate and the improvement you want to detect. Get the exact sample size per variant and estimated test duration.
Test parameters
Enter your baseline rate and the lift you want to detect.
Enter your baseline rate and MDE
Required sample size will appear here.
Why pre-calculating sample size matters
Most A/B test errors happen before the test even starts.
Too few visitors - false conclusions
With insufficient sample size, random variation can look like a real winner. You might ship a change that performs no better - or actively worse - than what you had before.
Too many visitors - wasted time
Running a test longer than necessary delays decisions and ties up engineering resources. With a proper pre-calculation, you know exactly when to stop.
No pre-calculation - peeking bias
Stopping a test the moment it hits 95% confidence - without a pre-set sample size - inflates your false positive rate to well above 5%. The math only works if you commit to the sample size in advance.
How to choose your minimum detectable effect
The MDE is the most misunderstood input. Here's a practical guide.
A/B testing best practices
Getting the sample size right is step one. Here's the rest.
Set sample size before you start
Calculate the required visitors per variant before launching the test. Write it down. Don't look at results until you've hit that number - this is the single most important discipline in A/B testing.
Choose MDE based on traffic, not wishful thinking
If your page gets 1,000 monthly visitors, you can't detect a 5% relative improvement in a reasonable time. Be realistic: set an MDE you can actually detect in 2-4 weeks.
Run tests for at least a full week
Visitor behavior varies by day of week. A test that runs Thursday-Sunday will capture a different audience mix than Monday-Wednesday. Always run for at least 7 days, ideally 14.
Test one thing at a time
Each additional change you make to the variant adds noise. If the headline, image, and CTA all change, you can't know what caused the result. Test the biggest hypothesized lever first.
Use surveys to form better hypotheses
The tests most likely to produce big results are the ones rooted in specific visitor feedback. Ask visitors why they're not converting - then test the most common answer.
Frequently asked questions
Common questions about A/B test sample sizes, MDE, statistical power, and test duration.
How is the sample size calculated?
This calculator uses the Evans, Peacock & Hastings formula for two-proportion z-tests. It takes your baseline conversion rate, the variant rate implied by your MDE, and the z-scores corresponding to your chosen confidence level and statistical power. The result is the minimum number of visitors per variant needed to reliably detect an effect of that size.
What is the minimum detectable effect (MDE)?
The MDE is the smallest relative improvement in conversion rate you want your test to be able to detect. It's expressed as a percentage of the baseline. If your baseline is 2% and you set an MDE of 10%, you're asking the test to detect a change from 2% to 2.2%. A smaller MDE means you can detect subtler effects - but requires significantly more data. Choose an MDE based on the traffic you have, not the improvement you hope to see.
What's a good MDE to use?
10-20% relative is the most common range. Use 10% if you have high traffic and are optimizing a mature, high-volume page. Use 20% if traffic is moderate and you're testing significant changes. Use 50%+ only when traffic is very low - but at that point, supplement with qualitative research (surveys, user interviews) since A/B tests alone will miss most real effects. Avoid setting MDE below 5% unless you have massive scale - the sample sizes become impractical.
What statistical power should I use?
80% is the standard - it means your test will detect a real effect 80% of the time when it exists (and miss it 20% of the time). 80% is the right choice for most tests. Use 90% when missing a real improvement would be costly (for example, if you're testing a major site redesign and a false negative would delay an important rollout). Higher power requires larger sample sizes, so there's always a tradeoff.
How long should an A/B test run?
Long enough to collect the pre-calculated sample size, and at least 7-14 days regardless of when you hit the sample size. The minimum duration matters because visitor behavior has weekly patterns - people visiting on Monday convert differently than people visiting on Saturday in many industries. A test that completes in 2 days will over-represent a specific day's audience. Aim for 2-4 weeks as a practical guideline, and never exceed 6-8 weeks (at which point you're accumulating seasonal noise).
What if I don't have enough traffic to run a valid A/B test?
Three options: (1) Increase your MDE - you can only detect larger effects, but you can still run valid tests. (2) Test on a higher-traffic page even if the conversion goal is less direct. (3) Switch to qualitative methods. On-site surveys, user interviews, and session recordings don't require statistical significance and can surface the same insights faster on low-traffic sites. For most startups under 10,000 monthly visitors, qualitative research produces better ROI than A/B testing.
Can I use this for multivariate tests (MVT)?
This calculator is for standard two-variant A/B tests. For multivariate tests, you need to account for the number of combinations being tested - the sample size requirement grows substantially. As a rough guide: multiply this calculator's per-variant number by the number of variant combinations in your MVT. For most teams, true MVT is only practical on very high-traffic pages.
Why does a lower baseline conversion rate require a larger sample?
Because rare events have higher variance. If your baseline is 0.5%, the statistical noise around that estimate is proportionally larger than if your baseline is 5%. To reliably distinguish signal from noise at a low baseline rate, you need many more observations. This is why A/B testing is difficult for low-conversion goals like enterprise demo requests - and why qualitative research methods are often more appropriate.