--- title: "At 2% conversion, an A/B test needs 42,000 visitors to prove a 20% lift" summary: "Sample sizes for A/B tests at 1–5% conversion rates, what a 10,000-visitor site can prove, and the free tools left after Google Optimize, as of September 2026." url: https://www.thetechbag.com/blog/conversion-rate-optimization-it-get-started format: Note desk: "SaaS & business software" published: 2025-09-10 facts_checked: 2026-09-30 author: TechBag (The TechBag Journal) tags: ["CRO","conversion rate optimisation","A/B testing","sample size","Microsoft Clarity","Google Optimize","Mixpanel","DPDP"] cover: https://www.thetechbag.com/blog/covers/conversion-rate-optimization-it-get-started.png --- # At 2% conversion, an A/B test needs 42,000 visitors to prove a 20% lift *Sample sizes for A/B tests at 1–5% conversion rates, what a 10,000-visitor site can prove, and the free tools left after Google Optimize, as of September 2026.* Note · SaaS & business software · By TechBag · Published 2025-09-10 · Facts checked 2026-09-30 · Canonical: https://www.thetechbag.com/blog/conversion-rate-optimization-it-get-started A site converting 2% of its visitors needs about 42,000 visitors in an A/B test before it can tell a 20% improvement from luck. At 10,000 visitors a month, that is four months for one answer. Conversion rate optimisation (CRO) is usually sold as design advice: cleaner pages, louder buttons, more reviews. For most Indian SMB and mid-market sites the binding constraint is arithmetic, and it decides how you should get started. ## The arithmetic most CRO advice skips Conversion rate is conversions divided by visitors. CRO is the practice of changing a page and proving the change helped. The proof is the expensive part: a test compares two versions, and the smaller the baseline rate and the smaller the lift you hope for, the more visitors each version needs before the difference is real. **Figure: At a 2% conversion rate, proving a 20% lift takes 21,109 visitors per version; a 10% lift takes 80,682** *Visitors needed in each version of an A/B test, by baseline conversion rate and the relative lift to detect* | | Detect a 10% lift | Detect a 20% lift | | --- | --- | --- | | 1% baseline | 1,63,095 | 42,693 | | 2% baseline | 80,682 | 21,109 | | 3% baseline | 53,211 | 13,914 | | 5% baseline | 31,234 | 8,158 | Source: TechBag calculation, two-proportion test, two-sided 5% significance, 80% power · as of 30 September 2026 · double the figure for the total across both versions TechBag countTechBag count We computed every column with the standard two-proportion sample-size formula. A 1% site that wants to detect a 10% lift needs 163,095 visitors per version, or 326,190 in all. At 10,000 visitors a month that is 32.6 months, which is not a test; it is a hope. You can check any figure here with a public calculator.[1] ## What a small site should test, and what it should simply fix The 2023 version of this piece used the example of moving a 2% site to 3%. That is a 50% relative lift, and it is cheap to prove: 3,826 visitors per version, about 7,700 in all, under a month at 10,000 visitors. The lesson is to test only changes big enough to move the number that much: a new offer, a shorter checkout, a price shown up front. Button colours belong to sites with millions of sessions. Our view Our view: below roughly 50,000 visitors a month, run fewer tests and more fixes. Find the step where people leave in your analytics, watch a dozen recordings of it, repair what is plainly broken, and keep A/B tests for the two or three bold changes a quarter that could plausibly move conversion by a third or more. ## The tool bill since Google Optimize closed Google Optimize, the free tester many small sites started on, stopped working on 30 September 2023. Google pointed users to Google Analytics integrations with AB Tasty, Optimizely and VWO.[2] The free half of the kit survived: Microsoft Clarity describes itself as “free forever” with heatmaps and session recordings and “no limits on traffic”.[3] For product teams, Mixpanel runs experiments inside its analytics and offers India as a data-residency region alongside the US and EU, as we recorded on its intel page.[4] **In India**, the starting kit costs ₹0: Google Analytics plus Clarity. Paid testing platforms list in US dollars; bought through a reseller they are invoiced in INR with 18% GST. The regulation that bites is the Digital Personal Data Protection Act, 2023.[5] A recording of a logged-in visitor typing into a form is personal data, and the DPDP Rules, notified on 13 November 2025, bring the security-safeguard and breach-notice duties into force on 13 May 2027.[6] Mask form fields in any replay tool now, and keep recordings no longer than the test needs. How we did this Figures as of 30 September 2026 Sample sizes use the two-proportion formula with a two-sided 5% significance level and 80% power, rounded up to whole visitors; months assume 10,000 visitors split evenly across two versions. Our earlier version cited an Aberdeen Group figure (a 30% average lift) and an Econsultancy ratio (US$92 on acquisition per US$1 on conversion); we could not trace either to a dated primary source and left both out. Tool facts come from the vendors’ own pages and TechBag’s Mixpanel intel page as committed on 30 September 2026. Reading the DPDP Act onto session recordings is our view, not legal advice. Spotted an error? Write to us; corrections are published at the foot of the piece. Sources 1. [Evan Miller, Sample Size Calculator (A/B testing)](https://www.evanmiller.org/ab-testing/sample-size.html) — undated A public calculator to check the inputs used here. 2. [Google, Optimize Help: the sunset of Google Optimize](https://support.google.com/optimize/answer/12979939) — Optimize closed 30 September 2023 3. [Microsoft Clarity, product home page](https://clarity.microsoft.com/) — vendor page, undated 4. [TechBag, Mixpanel Experiments intel page (India data residency)](https://www.thetechbag.com/mixpanel/mixpanel-experiments) — as committed 30 September 2026 5. [Digital Personal Data Protection Act, 2023](https://www.meity.gov.in/data-protection-framework) — 11 August 2023 6. [DPDP Rules 2025, commencement timeline](https://legismith.com/dpdp-rules-2025-timeline/) — notified 13 November 2025 Related on TechBag: the [observability guide](https://www.thetechbag.com/browse/devops/observability-apm) covers session replay alongside monitoring; the [Mixpanel Experiments page](https://www.thetechbag.com/mixpanel/mixpanel-experiments) covers testing inside product analytics. **Disclosure.** TechBag is a software reseller: it earns a margin when a reader buys through it, including from vendors named here. The analysis is TechBag’s own; no vendor paid for or saw this piece before publication. Facts are labelled reported, TechBag count or our view. Go deeper on TechBag - [Observability & APM guide](https://www.thetechbag.com/browse/devops/observability-apm) — session replay and product analytics alongside monitoring - [Mixpanel Experiments](https://www.thetechbag.com/mixpanel/mixpanel-experiments) — A/B tests inside product analytics, with an India data-residency option