Free CRO tool
A/B test calculator
Plan an experiment before you launch it, and read it properly once it ends. Work out the sample size each variant needs, how long that takes at your traffic, and whether a finished test is statistically significant.
Sample size per variant
How many visitors each variant needs before the test can detect the effect you care about.
Your current rate on the page you are testing.
The smallest relative change worth detecting.
Chance of detecting the effect if it is real.
Confidence required before calling a winner.
Two-sided also detects a drop.
Test duration
How long the sample size above takes to collect at your current traffic.
Visitors the tested page receives in a week.
Include the control. Two means control plus one variant.
Share of eligible visitors you send into the experiment.
Significance of a finished test
Enter the totals from a test that has finished and see whether the difference holds up.
Control
Variant
How to read these numbers
Sample size is a plan, not a target to chase
Decide the sample size before the test goes live, then let it run to the end. Checking every morning and stopping the first time a variant looks good is the quickest way to ship a change that does nothing, because early data swings widely before it settles.
A smaller effect costs a lot more traffic
Required sample size grows with the square of the effect you want to detect, so halving the minimum detectable effect multiplies the traffic you need by about four. If the number that comes back is impossible at your traffic, test a bolder change rather than a longer version of a small one.
Significance is not the same as impact
A p-value answers one narrow question: how likely a difference this large would be if the two variants really performed identically. It says nothing about whether the difference is big enough to be worth shipping. Judge the size of the measured difference and the revenue behind it separately from the verdict.
Run at least one full business cycle
Traffic behaves differently on a Tuesday than on a Sunday, and paid, organic and email visitors rarely convert at the same rate. Two full weeks is the usual minimum, so every day of the week lands in both variants.
One test, one primary metric, decided up front
Every extra metric you check adds another chance of a false positive. Name the primary metric before launch, treat the rest as diagnostics, and re-run anything surprising before you build a roadmap on it.
A/B test calculator FAQ
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and the size of the change you want to detect. Lower baseline rates and smaller effects both need more traffic. Enter your current conversion rate and the smallest relative improvement worth acting on in the sample size calculator above, and it returns the visitors needed per variant.
What is a minimum detectable effect?
The minimum detectable effect is the smallest relative change in conversion rate the test is powered to detect. A 10% minimum detectable effect on a 3% baseline means the test is set up to detect a move to 3.3%. Smaller changes can still happen; the test just may not be able to separate them from noise.
What does statistical significance actually mean?
It means the difference you measured would be unlikely to appear if the two variants performed identically. At 95% significance you are accepting roughly a one in twenty chance of calling a difference that is not there. It is a threshold you choose, not proof.
How long should an A/B test run?
Long enough to collect the planned sample size, and long enough to cover at least one full business cycle, which usually means a minimum of two weeks. Stopping as soon as a result looks good inflates false positives, while running for months exposes the test to seasonality and unrelated site changes.
Can I stop a test early if one variant is clearly winning?
Not with the formulas used here. Fixed-horizon tests assume you look once, at the end. Peeking and stopping on the first significant reading pushes the false positive rate well above the level you selected. Stopping early safely requires a sequential testing method designed for it.
Should I use a one-sided or two-sided test?
Two-sided is the safer default, because it detects a drop as well as a gain and a losing variant is useful information. A one-sided test is only appropriate when a negative result would lead to exactly the same decision as no difference at all.