Cloud & DevOpsNote
A 3×3×3 multivariate test needs 14.4 lakh visitors; an A/B test, 1.1 lakh
Visitors a full-factorial test needs at a 3% conversion rate, from a plain A/B test to Optimizely's 64-combination cap, worked out as of September 2026.
A multivariate test multiplies the traffic bill by the number of combinations it shows. On a page converting 3% of visitors, proving a 10% lift takes 53,211 visitors in each version. An A/B test has two versions and needs 1,06,422 visitors; a test of three headlines, three images and three buttons has 27 and needs 14,36,697. The 2023 version of this piece called multivariate testing “more powerful” than A/B testing. The buyer’s real question is whether their site has the traffic to use that power.
01Every element multiplies the bill
Multivariate testing (MVT) changes several elements of a page at once and shows every combination to a share of visitors, so it can find the best mix and any interaction between elements, such as a headline that works only with one image. Optimizely, whose platform the 2023 post named, calls the full-factorial method “more rigorous” but says it “requires higher levels of traffic to generate a significant result”, and caps a test at 64 combinations, six elements with two versions each.1
A 3×3×3 test needs 13.5 times the visitors of an A/B test; Optimizely's 64-combination cap needs 32 times
Total visitors to finish a full-factorial test, 3% baseline conversion, detecting a 10% relative lift
Source: TechBag calculation, two-proportion test, two-sided 5% significance, 80% power; combination cap from Optimizely documentation · as of 30 September 2026 · 53,211 visitors per version; no correction for multiple comparisons
02When it earns the traffic
Optimizely also offers partial-factorial tests, which let you send no traffic to combinations you do not believe in, cutting the bill at the cost of missing some interactions.1 For most Indian SMB and mid-market sites the better sequence is plainer: test the element most likely to matter as an A/B test, ship the winner, then test the next. Our conversion-rate piece sets out that A/B arithmetic by baseline rate.6
One tool the 2023 post recommended no longer exists: Google Optimize, the free option, closed on 30 September 2023.3
03Buying it in India
Testing platforms are sold on US-dollar list prices or quotes; bought through a reseller they are invoiced in INR with 18% GST, and the cheapest saving is to do the traffic arithmetic before signing. If your product analytics already runs on Mixpanel, its native experiments run A/B/n tests on the same events, with India among its data-residency regions as recorded on our intel page.4 The regulation that bites is the Digital Personal Data Protection Act, 2023: a test that assigns logged-in customers to versions processes personal data, and the notice and consent duties in the DPDP Rules, notified on 13 November 2025, apply from 13 May 2027.5
Sources
- Optimizely Documentation, Multivariate tests for Optimizely Web Experimentation — page created 20 July 2026
- Evan Miller, Sample Size Calculator (A/B testing) — undated A public calculator to check the per-version input.
- Google, Optimize Help: the sunset of Google Optimize — Optimize closed 30 September 2023
- TechBag, Mixpanel Experiments intel page (A/B/n tests, India data residency) — as committed 30 September 2026
- MeitY, Digital Personal Data Protection Rules, 2025 (G.S.R. 846(E)), Rules 1(4) and 3 — 13 November 2025
- TechBag blog, At 2% conversion, an A/B test needs 42,000 visitors to prove a 20% lift — published 30 December 2025, facts checked September 2026
Disclosure. TechBag is a software reseller: it earns a margin when a reader buys through it, including from vendors named here. The analysis is TechBag’s own; no vendor paid for or saw this piece before publication. Facts are labelled reported, TechBag count or our view.
Keep reading
