2024 A/B Testing Tools Review for Conversion Rate Optimization Experts

1. Optimizely is an enterprise-level experimentation platform. It is the most well-known platform in this space due to a lot of VC funding and advertising. At $1,440 per month (paid annually) it is one of the most expensive of A/B testing tools. They also offer a pay-as-you-go pricing model that costs $49 per 1000 unique monthly visitors. The platform supports testing on the web, mobile web, and mobile apps.
 

2. VWO (Visual Website Optimizer) [Updated 2-14-23] With the sunsetting of Google Optimize VWO has released a free plan to attract the Optimize users. They have also reversed their earlier decision to show pricing on the site. Since 2019 VWO has improved the features across the platform. The code is significantly less buggy than it was when it first launched. 

[Updated 1-31-19] I just finished a meeting with VWO where they provided a sneak peek of VWO’s new interface. It has some new features that make it very easy to see with visual graphs if a test is stable. They also removed some of the confusion when comparing variations and how they are doing against control. The data is presented so a novice can understand it and an expert can dig into the statistics. The data and graphs will also be exportable to a PDF to make reporting easier for clients. The new version should be out in the second quarter of this year. I will update this post when they have a firm date. Regarding my next comment… they are reinvesting in the tool to justify the cost and this will be going beyond just the presentation layer of the system.

[Updated 1-17-19] VWO has completely changed its pricing model to follow Optimizely. There are now only annual subscriptions and the price has doubled. VWO offers other CRO tools like heatmaps and videos. VWO works across mobile, tablet, and desktop websites.

3. Freshmarketer has acquired Zarget. [Updated 1-17-19] The pricing is very good and the support is excellent. 50k users under test for $159 a month and no annual commitment. They also have a number of other conversion tools similar to VWO. The testing system presents nice reports and does conversion rate over time which I feel is a key visual indicator of test stability when comparing variations. Setting up some tests is a bit different but it is clear they are building better functionality. The acquisition included all the developers so they did not lose any talent

4. Convert has received high praise from its users, who tend to be SMB customers. It was the first solid A/B testing tool on the market before Optimizely. Their focus is on being the best in the market for website A/B testing.  [Updated 1-17-19] Price starts at $699/month or $599 a month (paid annually).  It has an intuitive interface and excellent customer support

5. Crazyegg known for its excellent heatmap tools has added A/B testing. Pricing starts at $29 a month for 30k visitors. Note this is not users under test, so it is harder to compare to other testing pricing plans and will have to throttle the traffic on the pages for larger sites. That said I would love some feedback on their A/B tool.

6. Google Optimize [updated 2-14-23] Google has decided to sunset its Optimize product in September of 2023. You can learn more about it here. Firebase testing for apps will still use the code from Optimize and is not sunsetting at this time.  Optimize was an A/B testing tool that connects to Google Analytics. Since you already have all your data in your Google Analytics account. You can’t beat the cost and convenience of this tool since it is free. It allows you to split test up to 10 full versions of a single page, each delivered from a distinct URL. However, goal setting is very cumbersome relying on Google Analytic goals and segments, and in some cases would take Tag Manager to set up goals. Hopefully, in the future, they will build features that take seconds in other tools instead of 10 to 30 minutes. 

[Update 5-7-17] I asked Google about the goal issue and they said  that within a year you will be able to set goals on elements and that next month they will be adding Adwords to Optimize.]

[Update 1-17-19]  Google launched new features today. I have not played with them yet but if they can set goals quickly without google analytics then they are moving in the right direction. 

[1-31-19] The new goals called Custom Objectives get around the biggest issue of only using the Google Analytics goals which were limited. They still don’t have one-click goal setting but it is getting there. So the time it takes to set all of these up compared to clicking a button and calling it a goal can be significant. What is nice is it is easy to set purchase and revenue goals if you have enhanced eCommerce already set up. This can be difficult in other systems

7. Omniconvert provides an A/B testing tool for tablets, mobile, and desktops. Some of the unique features are ready-to-use overlays, multiple overlay formats, OnPage variables, frequency capping, and IP address targeting. Pricing starts at $38/month (paid annually).

8. AB Tasty [Update 2-14-23] Google had listed AB Tasty as one of the tools it recommends once Optimizley sunsets. Pricing is not shown on the site anymore.  AB Tasty is a tool that tests user re-engagement and it is designed for marketing teams. You can modify pages using a visual editor, so you don’t need a technical background to use it. This is a good tool to start A/B testing if you’ve never done it before and you don’t want to invest a lot of money. You must schedule a consultation with one of their experts as pricing is not available on their site

9. Intellimize helps marketers drive conversions and revenue growth by using artificial intelligence to personalize websites to deliver the right experience to each visitor on any device.   Since this is a newer approach, I reached out to them for an interview so that they can clarify how they do what they do.

Pricing? – We are a SaaS-based pricing model with an annual term and monthly pricing.  Cost varies considerably based on the needs of the client and we don’t publish pricing as our work for clients.  We work mostly with middle-market companies. As you noted, it’s common for companies to run their entire site through our system as we can optimize flows for them.

Does this affect SEO? – Great question. Google has established guidelines for companies like us to follow.
1. Google asks personalization companies like ours to serve a random experience to them. We do not “listen” for Google.
2. For keywords that are important to you, we ensure that variations of your page contain those keywords and metadata to ensure that your page will keep ranking.
3. It’s important to remember we are changing elements of the page (headlines, CTAs, images, etc.) rather than setting up new pages so your existing work will remain intact.

How do you measure/define the audiences for each variation? Is it from data points in the site or data found outside of the site or both?

Our system does not require any data integration to get started and can accept data from 1st and 3rd party sources.  The most commonly used data by our system would be contextual data because it’s readily available for each visitor.  This includes features like: IP address, device type, browser, time of day, day of week, and geographic location (city level). We also commonly use firmographic and demographic data as well as behavioral data about how the user interacts on the site.

Finally, our approach to A/B testing is rooted in the principle that A is probably best for some people and B is probably best for others.  Instead of finding a global winner (the best average), our AI-based system learns which version of your site is best for each visitor and automatically allocates more traffic to your best-performing variations.

10. Adobe Target provides an intuitive user interface and works best with sites that already use Adobe Analytics. It is easy to set up and test. Pricing is not available on their website but is around the cost of Optimizely or more

11. SiteSpect is a server-side testing solution and is the #1 choice for more technically savvy users and clients who are sensitive to security. Pricing was not available on the website

12. Oracle Marketing provides simple A/B tests all the way up through complex multivariate tests. You can test on any public or secure page with an easy-to-use visual editor. The tool automatically builds a unique customer profile for each online visitor. Pricing was not available on the website.

13. Qubit + Coveo is a testing platform focused on personalization and advanced segmentation. Like most A/B testing tools, it uses multivariate to show which combination of changes is more valuable than the individual changes. Pricing was not available on the website but they do feature a pro and an enterprise plan.

14. Leanplum offers an integrated A/B testing tool for native mobile apps. Pricing was not available on the website.

15. Taplytics is a visual A/B testing tool for native mobile apps and for web pages. Pricing was not available on the website, though they do have a free trial.

16. Monetate does A/B testing and personalization. Their product is targeted at retailers and e-commerce. Most of their customers are enterprises.

17. Kameleoon is a website optimization solution with A/B testing for desktop and mobile. Their platform is devoted to web personalization. They offer a freemium pricing model.

18. ChangeAgain.me is an A/B testing tool for websites. It integrates with Google Analytics and has a visual editor to update the visual elements on a website. Pricing starts at $14/month. 

19. Nelio A/B Testing is a website optimization solution for publishers and marketers using a WordPress site. Users can A/B test any piece of their WordPress site. Pricing starts at $29/month.

20. Invesp offers an A/B testing tool called Pii testing optimization engine. The website shows it as a free tool.

21. Analytics-toolkit This site has a number of A/B statistics and analytics calculators. I always run my own statistics with tools that I created in Excel to double-check results when using A/B tools. Pricing starts at $15/month.

22. SiteGainer has a drag-n-drop website editor that loads directly on your website and allows you to A/B test any part of your website. Pricing is not available on the website. 

23. TrustRadius collects in-depth user reviews on various A/B testing tools and includes a user ranking for each product. TrustRadius also has a TrustMap comparison tool that allows you to compare products and services based on user satisfaction ratings.

24. Mutiny is an AI-driven tool for B2B that utilizes AI suggestions to target specific audiences, then rubs A/B tests for those audiences. Pricing is not available on the website. 

25. Conductrics is an advanced testing tool that combines machine learning, testing, and surveys to measure customer attitudes and behavioral measures huge conversions, submits, and revenue. Pricing is not available on the website. 

26. Dynamic Yield is a powerful suite that includes A/B testing amongst other tools that you’re able to apply across a variety of platforms. With Dynamic Yield’s Experience OS, you can use machine learning to match content, products, and offers to specific customers. 

If you have an A/B testing tool that you like and recommend, contact us and we will add it to the list.