A/B testing a landing page: the guide to testing without skewing the results

Most landing page "A/B tests" don't test anything at all: one version runs for two weeks, gets replaced by another, the second one's conversion rate looks better, and a firm conclusion gets drawn from it. The problem is that traffic, the season, an ongoing ad campaign, or simple chance could just as easily explain the gap. A real A/B test sends both versions at the same time, to the same traffic, split at random — that's the only way to isolate the effect of the change being tested. This guide covers what to test first on a landing page, how long to run a test before drawing a conclusion, and which tools to use in 2026 without tanking load speed.

What an A/B test is (and isn't)

An A/B test compares two versions of the same page sent simultaneously to traffic split at random, changing only one element at a time. Two mix-ups come up constantly. The first, the sequential test — one version in January, the other in February — which confuses the effect of the change with the effect of seasonality, a different campaign, or industry news. The second, the too-many-variables-at-once test — a new headline, a new CTA and a new photo all at the same time — which makes it impossible to know which of the three made the difference. If you want to test several elements together, that's a multivariate test, a different method that needs far more traffic to stay reliable; for a typical sales landing page, a single-variable A/B test is almost always the right call.

The 5 elements worth testing first

Not every element of a landing page is worth testing equally. Here are the five that, in roughly the order of impact usually observed, are worth testing before anything else — the full structure of a high-performing page is covered in our anatomy of a landing page that converts. Short of ideas? The gaps you spot when you analyze your competitors' landing pages make good hypotheses, provided you test them on your own traffic instead of copying them.

1. The headline and core promise — this is what decides whether the visitor keeps reading or leaves. Two different angles (the outcome delivered vs. the pain solved, for example) often produce sharper gaps than any other test.
2. The CTA — label ("Buy now" vs. "View the template"), color, position (above the fold vs. repeated several times). A quick change to ship, with a measurable effect after just a few hundred clicks.
3. Social proof — named testimonials vs. client logos, a headline stat vs. a detailed quote. Social proof reassures, but not every form of it reassures the same audience the same way.
4. The offer and its anchoring — showing a struck-through price or not, leading with the single template (€89) or the bundle (€229), mentioning a guarantee or not.
5. The form — number of fields, single step vs. a two-step form. On capture pages (waitlist, ebook), this is often the lever with the fastest payoff for the least effort.

How long (and how much traffic) a reliable test needs

Three simple rules avoid most false positives.

• Never cut a test based on the dashboard. Decide the duration or the number of conversions to reach before launching the test, and stick to it — checking the numbers every day and stopping as soon as a gap "looks" significant is the most common A/B testing bias ("peeking"), and it artificially inflates the false-positive rate.
• Cover at least one full cycle, generally 1 to 2 full weeks, weekdays and weekend included: buying behavior on a Monday morning isn't the same as on a Sunday evening, and a variant can look like a winner after three days and then flip over the full week.
• Check statistical significance before concluding, with a dedicated calculator (most A/B testing tools include one) rather than eyeballing a raw percentage — a 3-point gap out of 40 visitors means, strictly speaking, nothing at all.

On a low-traffic landing page (under a few hundred visitors a week), a classic A/B test can take months to become reliable. In that case, it's better to run successive changes measured over longer periods, or gather qualitative feedback (5 to 8 users talking through the page out loud) rather than force an underpowered quantitative test — a false "winner" based on 40 visits costs more than no conclusion at all.

Tools for A/B testing a landing page in 2026

Google Optimize shut down in 2023; the landscape has rebuilt itself around tools that are more technical but often lighter. For a static or Next.js landing page like LanderKit's templates, three families of tools dominate today.

• GrowthBook — open source, free to self-host, aimed at technical teams: traffic splitting happens server-side (edge/middleware), which avoids the visual flicker of content swapping on load that's common with browser-side tools.
• PostHog — combines product analytics and experimentation in one tool, handy if you're already tracking visitor behavior beyond just conversion.
• Statsig — geared toward feature flags and experimentation, used by product teams; ships with a built-in statistical significance dashboard.
• The DIY route — for a simple test on a static page, a Next.js middleware that randomly splits visitors between two routes (/a and /b), followed by UTM-based conversion tracking in your analytics tool, works perfectly well, with no external dependency or third-party script to load.

The mistakes that most often skew an A/B test

• Changing several elements at once and then wondering which one made the difference — one test, one variable.
• Mixing very different traffic sources (paid ads and organic SEO, for example) without separating them in the analysis: their baseline conversion rate already differs before any test runs.
• Ignoring the device: a variant can win on desktop and lose on mobile, especially if the test is about form length or the size of a visual.
• Testing without a primary goal set in advance — without deciding upfront whether the only metric that counts is the CTA click, the completed form, or the purchase, you end up picking whichever metric fits the conclusion you wanted after the fact.
• Restarting a test after a tweak without resetting the data already collected, which mixes two non-comparable periods into a single result.

A concrete case with a LanderKit template

Every LanderKit template ships as source code, which makes an A/B test simple to set up: duplicate the Page.tsx component into PageB.tsx, change a single element (the headline, the CTA, or how the offer is framed), deploy both variants on two routes, and split traffic with a middleware or the tool of your choice. On a single-product e-commerce page like the e-commerce product template, a classic test often pits a struck-through price against a plain price; on a SaaS waitlist page, the number of form fields is usually the first thing to test. In both cases, keep the page structure — the section order already has a track record, covered in our conversion levers guide — and touch only the one variable you chose.

Multivariate testing (MVT) vs. A/B testing: which one for your landing page?

Why run three separate A/B tests — headline, then CTA, then image — when you could test all the combinations at once? Multivariate testing (MVT) promises to get there faster. It delivers on that promise under one condition most landing pages never meet: enough traffic.

A classic A/B test isolates a single variable: two versions of a headline, everything else identical. A multivariate test runs several variables at once, automatically generating every combination of them — what's known as a full factorial design. Two headlines × two CTA colors × two hero images give you 2 × 2 × 2 = 8 combinations, each shown to a slice of traffic. The result doesn't just point to the best combination: it also isolates each variable's own effect, and — the real draw — the interaction between them. For instance, whether a given CTA color only works paired with a specific headline, a signal no standalone A/B test could ever surface. The reference guide from Microsoft's experimentation team, published in 2009 in Data Mining and Knowledge Discovery, formalizes this distinction under the name "Multivariable Tests" (Kohavi, Longbotham, Sommerfield & Henne, 2009) and, fifteen years on, remains the most cited reference on the topic.

A/B test vs. multivariate test: the real difference

• A/B test: one variable, two or three versions, a result that's easy to read and easy to set up. It's the default method described in our landing page A/B testing guide.
• Multivariate test: several variables tested at once, with a combination count that explodes with every added variable (3 variables at 2 versions each = 8 combinations, at 3 versions each = 27), yielding a richer result — but only if the traffic keeps up.

Our article on A/B testing without traffic puts a number on what a classic two-arm test needs: several thousand visitors per arm to detect a 20% relative lift on a 3% conversion rate — in the range of 5,000 to 10,000 monthly visitors on the page. Apply that same math to an 8-combination factorial plan and you're looking at 20,000 to 40,000 monthly visitors on that single page — and that's before accounting for interaction effects, which are statistically smaller than main effects and therefore even more expensive to detect with confidence. In their 2020 reference book, the same authors broaden that point: past two or three variables tested together, the number of combinations grows faster than the traffic available on nearly every site, which is why multivariate testing stays the exception rather than the rule even inside large experimentation teams (Kohavi, Tang & Xu, 2020). The full duration math, variable by variable, is in our guide on how long an A/B test should run.

If you're starting from scratch, there's no point testing a structure you're guessing at: the 10 LanderKit templates (€89 each, €229 for the full bundle) ship with a structure that's already proven, so all that's left to test is the real levers — a headline, a CTA, an offer — instead of guessing whether the form should sit on the left or the right.