A/B Test Sample Size Calculator

Calculate how many visitors you need for statistically significant results

%
%
Relative improvement (20% means 3% → 3.6%)
To estimate test duration

What is A/B Test Sample Size?

Sample size is the number of visitors needed in each test variation to detect a meaningful difference with statistical confidence. Too small a sample leads to unreliable results.

Key Concepts:

  • Baseline rate: Your current conversion rate
  • MDE: Smallest improvement worth detecting
  • Significance (α): Risk of false positive (5% = 95% confidence)
  • Power (1-β): Chance of detecting real effects (80% standard)

Quick Tips:

  • Lower MDE = more samples needed
  • Lower baseline rate = more samples needed
  • Don't stop tests early just because you see a winner
  • Run tests for full weeks to avoid day-of-week bias

Related Tools

Product-Market Fit NPS Calculator Churn Rate
View all tools →

Frequently Asked Questions

What MDE should I use? ▼
Choose the smallest improvement that would be worth implementing. A 5% improvement might not be worth the engineering effort, but 20% would be. Smaller MDE requires more samples. Be realistic - most A/B tests don't show >20% improvements.
Why 95% significance and 80% power? ▼
These are industry standards balancing accuracy with practicality. 95% significance means 5% chance of a false positive (seeing a winner when there isn't one). 80% power means 20% chance of missing a real winner. Higher values need more samples.
Can I stop a test early if I see a clear winner? ▼
Generally no! "Peeking" at results and stopping early inflates false positive rates dramatically. Run tests to the calculated sample size. Use sequential testing methods if you need valid early stopping.
How long should I run my test? ▼
Until you reach the required sample size, but at least 1-2 full weeks to account for day-of-week variations. Don't run tests longer than 4-6 weeks as user behavior may change, invalidating results.
What if my sample size is huge? ▼
If you need millions of visitors, consider: (1) Increasing MDE - test bigger changes, (2) Testing on higher-traffic pages, (3) Focusing on macro-conversions with higher rates, or (4) Accepting lower confidence for directional insights.

The faster, easier way to capture user feedback at scale

Join over 5,204 product managers and see how easy it is to build products people love.

from 124+ reviews
ProductLift dashboard preview

Did you know 80% of software features are rarely or never used? That's a lot of wasted effort.

SaaS software companies spend billions on unused features. In 2025, it was $29.5 billion.

We saw this problem and decided to do something about it. Product teams needed a better way to decide what to build.

That's why we created ProductLift - to put all feedback in one place, helping teams easily see what features matter most.

In the last five years, we've helped over 5,204 product teams (like yours) double feature adoption and halve the costs. I'd love for you to give it a try.

Ruben Buijs, Founder
Ruben Buijs

Founder & Digital Consultant