At 2% conversion, an A/B test needs 42,000 visitors to prove a 20% lift
Sample sizes for A/B tests at 1–5% conversion rates, what a 10,000-visitor site can prove, and the free tools left after Google Optimize, as of September 2026.
A site converting 2% of its visitors needs about 42,000 visitors in an A/B test before it can tell a 20% improvement from luck. At 10,000 visitors a month, that is four months for one answer. Conversion rate optimisation (CRO) is usually sold as design advice: cleaner pages, louder buttons, more reviews. For most Indian SMB and mid-market sites the binding constraint is arithmetic, and it decides how you should get started.
01The arithmetic most CRO advice skips
Conversion rate is conversions divided by visitors. CRO is the practice of changing a page and proving the change helped. The proof is the expensive part: a test compares two versions, and the smaller the baseline rate and the smaller the lift you hope for, the more visitors each version needs before the difference is real.
At a 2% conversion rate, proving a 20% lift takes 21,109 visitors per version; a 10% lift takes 80,682
Visitors needed in each version of an A/B test, by baseline conversion rate and the relative lift to detect
- Detect a 10% lift
- Detect a 20% lift
Visitors per version
Table view
| Detect a 10% lift | Detect a 20% lift | |
|---|---|---|
| 1% baseline | 1,63,095 | 42,693 |
| 2% baseline | 80,682 | 21,109 |
| 3% baseline | 53,211 | 13,914 |
| 5% baseline | 31,234 | 8,158 |
Source: TechBag calculation, two-proportion test, two-sided 5% significance, 80% power · as of 30 September 2026 · double the figure for the total across both versions
02What a small site should test, and what it should simply fix
The 2023 version of this piece used the example of moving a 2% site to 3%. That is a 50% relative lift, and it is cheap to prove: 3,826 visitors per version, about 7,700 in all, under a month at 10,000 visitors. The lesson is to test only changes big enough to move the number that much: a new offer, a shorter checkout, a price shown up front. Button colours belong to sites with millions of sessions.
03The tool bill since Google Optimize closed
Google Optimize, the free tester many small sites started on, stopped working on 30 September 2023. Google pointed users to Google Analytics integrations with AB Tasty, Optimizely and VWO.2 The free half of the kit survived: Microsoft Clarity describes itself as “free forever” with heatmaps and session recordings and “no limits on traffic”.3 For product teams, Mixpanel runs experiments inside its analytics and offers India as a data-residency region alongside the US and EU, as we recorded on its intel page.4
In India, the starting kit costs ₹0: Google Analytics plus Clarity. Paid testing platforms list in US dollars; bought through a reseller they are invoiced in INR with 18% GST. The regulation that bites is the Digital Personal Data Protection Act, 2023.5 A recording of a logged-in visitor typing into a form is personal data, and the DPDP Rules, notified on 13 November 2025, bring the security-safeguard and breach-notice duties into force on 13 May 2027.6 Mask form fields in any replay tool now, and keep recordings no longer than the test needs.
Sources
- Evan Miller, Sample Size Calculator (A/B testing) — undated A public calculator to check the inputs used here.
- Google, Optimize Help: the sunset of Google Optimize — Optimize closed 30 September 2023
- Microsoft Clarity, product home page — vendor page, undated
- TechBag, Mixpanel Experiments intel page (India data residency) — as committed 30 September 2026
- Digital Personal Data Protection Act, 2023 — 11 August 2023
- DPDP Rules 2025, commencement timeline — notified 13 November 2025
Related on TechBag: the observability guide covers session replay alongside monitoring; the Mixpanel Experiments page covers testing inside product analytics.
Disclosure. TechBag is a software reseller: it earns a margin when a reader buys through it, including from vendors named here. The analysis is TechBag’s own; no vendor paid for or saw this piece before publication. Facts are labelled reported, TechBag count or our view.
Keep reading
