A/B testing tools
A/B testing tools are software with which two or more variants of a page can be tested against one another under controlled conditions. Some visitors see variant A, others variant B, and the tool measures which version achieves more conversions. Decisions about improvements thus rest on solid data instead of gut feeling. A/B testing tools are therefore the central instrument for statistically sound conversion rate optimisation.
Also known as: A/B testing tools, split testing tools, A/B test software, experimentation tools
What are A/B testing tools?
A/B testing tools enable controlled experiments on your website. Instead of simply making a change and hoping it works, you show two variants in parallel: visitors are split at random and the tool measures which version produces more purchases or enquiries. That creates a genuine, statistically solid comparison.
The great advantage is objectivity. Debates about design or wording are settled by data, not opinion. A variant only wins if the difference is statistically significant and not down to chance. That guards against expensive wrong decisions.
A/B testing is the heart of a methodical optimisation process. Combined with Heatmaps, session recordings and clean Web analytics a cycle of observing, hypothesising, testing and implementing emerges that keeps improving the conversion rate.
Which A/B testing tools are there, compared?
Since Google Optimize was discontinued, the native experiments in Google Analytics 4 in combination with other tools is a common starting point. VWO, short for Visual Website Optimizer, is a popular all-in-one platform combining visual editing of variants, heatmaps and testing, and suits mid-sized companies well.
AB Tasty is a European solution with a strong focus on personalisation and experience optimisation. Optimizely is seen as the enterprise benchmark and offers powerful server-side testing suited to complex applications and feature tests in product development too.
The right choice depends on traffic, budget and technical ambition. For many companies VWO or AB Tasty is the pragmatic middle ground, while Optimizely plays to its strengths with large data volumes, high test frequency and server-side scenarios.
| Tool | Strength | Server-side? | Price range |
|---|---|---|---|
| GA4 experiments | Entry level, integration with Google Analytics 4 | Limited | Free |
| VWO | All-in-one, visual editor, heatmaps | Yes, on higher plans | from ~ a monthly price depending on traffic |
| AB Tasty | European, personalisation | Yes | from ~ a higher monthly price |
| Optimizely | Enterprise, feature and server testing | Yes, pronounced | Enterprise |
How is an A/B test run properly?
A clean A/B test starts with a clear hypothesis derived from data, from heatmaps or Google Analytics 4 for instance. Rather than testing at random, you state a concrete assumption, for example that a shorter form raises the conversion rate. Only then does the result yield usable insight.
You then define an unambiguous target metric and the sample size needed. The test has to run long enough to be statistically solid, generally several weeks depending on traffic. Ended too early, the result may be down to chance.
Only once a difference is statistically significant is the winning variant adopted permanently. Losing tests are not wasted effort but yield valuable insight into your audience. At Elisabit we accompany this process from hypothesis to clean analysis.
- 1Form a data-based hypothesis, for instance from heatmaps or GA4.
- 2Define a clear target metric and conversion.
- 3Define the sample size and run time needed.
- 4Run the test and don’t stop it too early.
- 5Check statistical significance and roll out the winner.
Server-side versus client-side testing
Most A/B tests run client-side: the tool changes the page with JavaScript in the visitor's browser. That is simple to set up but can cause a brief flicker on slow connections, when the original page shows before the variant. For simple tests of copy and layout it still suits well.
Server-side testing renders the variant on the server before the page is delivered. That avoids flicker, performs better and allows deep tests of features, prices or whole flows. It is technically more demanding and mostly reserved for enterprise tools such as Optimizely.
Which variant fits depends on the use case. For classic marketing pages, client-side testing is often enough, while product teams and data-intensive applications benefit from server-side testing. A Performance audit helps assess whether your setup is technically suitable.
A/B testing tools and the GDPR
A/B testing tools assign variants to visitors and measure their behaviour, for which they often set cookies and collect data. In Germany this can require consent via a Consent management platform may be required, especially where data that can identify individuals is processed or passed to third parties.
Anyone wanting to be safe checks where the data is stored and prefers European providers or server-side solutions with EU data storage. GDPR-compliant tracking as the basis also keeps the measurement of test results legally clean.
In the German market especially, a privacy-compliant testing setup is a signal of trust. It shows you think about optimisation and data protection together and at the same time secures a solid data basis for your Conversion rate optimisationwithout taking legal risks.
Frequently asked questions
Which A/B testing tool replaces Google Optimize?
Since Google Optimize was discontinued, there is no direct free one-to-one replacement from Google. The native experiments in Google Analytics 4 combined with other tools serve as a starting point. For full testing, many companies turn to VWO, AB Tasty or Optimizely, which offer considerably more for visual editing, analysis and personalisation.
How long does an A/B test need to run?
An A/B test has to run long enough to give statistically solid results, generally at least two to four weeks. The exact duration depends on traffic and the size of the improvement expected. It matters to cover at least one full week, to even out day-of-week variation, and not to stop the test as soon as one variant is briefly ahead.
What does server-side A/B testing mean?
In server-side A/B testing the variant is rendered on the server before the page is delivered to the browser. That avoids the brief flicker of client-side tests, performs better and allows deep tests on features, prices or whole processes. It takes more technical effort and is available above all in enterprise tools such as Optimizely.
How many visitors do I need for an A/B test?
The number of visitors needed depends on your current conversion rate and the improvement wanted. The smaller the difference expected, the more data is needed. As rough guidance, several thousand visitors and a few hundred conversions per variant are often required. Online calculators help estimate the sample size realistically before the test.
Do I need a consent banner for A/B tests?
That depends on the tool and the data processing. If the A/B testing tool sets cookies or processes personally identifiable data, consent through a consent management platform is generally needed in Germany. Tools with European data storage or server-side testing with anonymised data can reduce the risk. A data protection review of the setup is advisable.
What is the difference between an A/B test and a multivariate test?
An A/B test compares two or a few complete variants of a page against one another. A multivariate test checks several elements at once in different combinations: headline, image and button for instance. Multivariate tests give more detailed insight into how elements work together but need considerably more traffic to reach statistically solid results.
Related terms
Controlled testing of two variants (A vs. B) to optimise content on the basis of data.
A systematic process that increases the share of website visitors who complete a desired action.
The share of visitors who complete a desired action — the central metric in online marketing.
Software that visualises how website visitors click, scroll and move.
Recordings of real user sessions — mouse, clicks and scrolling — that make behaviour on a website traceable.
Collecting, measuring and analysing website data in order to improve content and conversions in a targeted way.
Put AI to work for your business?
We help you integrate artificial intelligence into your processes, your marketing and your website — strategically and securely.
Your contact
Stefan
I look forward to hearing about your project and finding the best solution together.
