--- title: "A 3×3×3 multivariate test needs 14.4 lakh visitors; an A/B test, 1.1 lakh" summary: "Visitors a full-factorial test needs at a 3% conversion rate, from a plain A/B test to Optimizely's 64-combination cap, worked out as of September 2026." url: https://www.thetechbag.com/blog/concept-multivariate-testing format: Note desk: "Cloud & DevOps" published: 2025-12-10 facts_checked: 2026-09-30 author: TechBag (The TechBag Journal) tags: ["multivariate testing","A/B testing","experimentation","sample size","Optimizely","Mixpanel","conversion rate","DPDP"] cover: https://www.thetechbag.com/blog/covers/concept-multivariate-testing.png --- # A 3×3×3 multivariate test needs 14.4 lakh visitors; an A/B test, 1.1 lakh *Visitors a full-factorial test needs at a 3% conversion rate, from a plain A/B test to Optimizely's 64-combination cap, worked out as of September 2026.* Note · Cloud & DevOps · By TechBag · Published 2025-12-10 · Facts checked 2026-09-30 · Canonical: https://www.thetechbag.com/blog/concept-multivariate-testing A multivariate test multiplies the traffic bill by the number of combinations it shows. On a page converting 3% of visitors, proving a 10% lift takes 53,211 visitors in each version. An A/B test has two versions and needs 1,06,422 visitors; a test of three headlines, three images and three buttons has 27 and needs 14,36,697. The 2023 version of this piece called multivariate testing “more powerful” than A/B testing. The buyer’s real question is whether their site has the traffic to use that power. ## Every element multiplies the bill Multivariate testing (MVT) changes several elements of a page at once and shows every combination to a share of visitors, so it can find the best mix and any interaction between elements, such as a headline that works only with one image. Optimizely, whose platform the 2023 post named, calls the full-factorial method “more rigorous” but says it “requires higher levels of traffic to generate a significant result”, and caps a test at 64 combinations, six elements with two versions each.[1] **Figure: A 3×3×3 test needs 13.5 times the visitors of an A/B test; Optimizely's 64-combination cap needs 32 times** *Total visitors to finish a full-factorial test, 3% baseline conversion, detecting a 10% relative lift* A/B test 2 versions 1,06,422 visitors 2×2 multivariate 4 combinations 2,12,844 visitors 2×2×2 multivariate 8 combinations 4,25,688 visitors 3×3×3 multivariate 27 combinations 14,36,697 visitors Optimizely cap, 2⁶ 64 combinations 34,05,504 visitors | | Value (visitors) | | --- | --- | | A/B test | 1,06,422 | | 2×2 multivariate | 2,12,844 | | 2×2×2 multivariate | 4,25,688 | | 3×3×3 multivariate | 14,36,697 | | Optimizely cap, 2⁶ | 34,05,504 | Source: TechBag calculation, two-proportion test, two-sided 5% significance, 80% power; combination cap from Optimizely documentation · as of 30 September 2026 · 53,211 visitors per version; no correction for multiple comparisons TechBag countTechBag count We computed each bar as 53,211 visitors per version times the number of versions. At one lakh visitors a month, the A/B test takes about 1.1 months, the 3×3×3 test 14.4 months and the 64-combination test 34 months. Compare each of the 27 combinations against the control with a Bonferroni correction, as a careful analyst would, and the 3×3×3 test needs 1,05,427 visitors per version, 28,46,529 in all. Any public sample-size calculator reproduces the per-version figure.[2] ## When it earns the traffic Optimizely also offers partial-factorial tests, which let you send no traffic to combinations you do not believe in, cutting the bill at the cost of missing some interactions.[1] For most Indian SMB and mid-market sites the better sequence is plainer: test the element most likely to matter as an A/B test, ship the winner, then test the next. Our [conversion-rate piece](https://www.thetechbag.com/blog/conversion-rate-optimization-it-get-started) sets out that A/B arithmetic by baseline rate.[6] Our view Our view: if a 2×2 test, about 2.1 lakh visitors at these settings, would take more than a month of your traffic, you do not have a multivariate problem. Keep MVT for pages with that volume and a specific suspicion that two elements interact. One tool the 2023 post recommended no longer exists: Google Optimize, the free option, closed on 30 September 2023.[3] ## Buying it in India Testing platforms are sold on US-dollar list prices or quotes; bought through a reseller they are invoiced in INR with 18% GST, and the cheapest saving is to do the traffic arithmetic before signing. If your product analytics already runs on Mixpanel, its native experiments run A/B/n tests on the same events, with India among its data-residency regions as recorded on our intel page.[4] The regulation that bites is the Digital Personal Data Protection Act, 2023: a test that assigns logged-in customers to versions processes personal data, and the notice and consent duties in the DPDP Rules, notified on 13 November 2025, apply from 13 May 2027.[5] How we did this Figures as of 30 September 2026 Visitor counts use the two-proportion sample-size formula with a two-sided 5% significance level and 80% power, rounded up to whole visitors, at a 3% baseline and a 10% relative lift; totals assume an even split across versions. The Bonferroni figure divides the 5% level by 26 comparisons. Optimizely’s method descriptions and the 64-combination cap are quoted from its documentation; other platforms’ limits were not counted. The 2023 post’s case studies (Amazon +3%, Netflix +20%, Walmart +10%) could not be traced to any dated source and were dropped. Mixpanel facts are as recorded on TechBag’s intel page at the 30 September 2026 commit. Spotted an error? Write to us; corrections are published at the foot of the piece. Sources 1. [Optimizely Documentation, Multivariate tests for Optimizely Web Experimentation](https://docs.optimizely.com/web-experimentation/docs/multivariate-tests-for-optimizely-web-experimentation) — page created 20 July 2026 2. [Evan Miller, Sample Size Calculator (A/B testing)](https://www.evanmiller.org/ab-testing/sample-size.html) — undated A public calculator to check the per-version input. 3. [Google, Optimize Help: the sunset of Google Optimize](https://support.google.com/optimize/answer/12979939) — Optimize closed 30 September 2023 4. [TechBag, Mixpanel Experiments intel page (A/B/n tests, India data residency)](https://www.thetechbag.com/mixpanel/mixpanel-experiments) — as committed 30 September 2026 5. [MeitY, Digital Personal Data Protection Rules, 2025 (G.S.R. 846(E)), Rules 1(4) and 3](https://www.meity.gov.in/static/uploads/2025/11/53450e6e5dc0bfa85ebd78686cadad39.pdf) — 13 November 2025 6. [TechBag blog, At 2% conversion, an A/B test needs 42,000 visitors to prove a 20% lift](https://www.thetechbag.com/blog/conversion-rate-optimization-it-get-started) — published 30 December 2025, facts checked September 2026 **Disclosure.** TechBag is a software reseller: it earns a margin when a reader buys through it, including from vendors named here. The analysis is TechBag’s own; no vendor paid for or saw this piece before publication. Facts are labelled reported, TechBag count or our view. Go deeper on TechBag - [Mixpanel Experiments](https://www.thetechbag.com/mixpanel/mixpanel-experiments) — A/B/n tests on the events you already track, with India data residency - [DevOps guides](https://www.thetechbag.com/browse/devops) — observability, developer tools and the data stack behind product testing