Visual / WYSIWYG editor
Set up A/B testing campaigns using a WYSIWYG editor to create site versions and preview design changes before testing them. These editors often don’t require coding knowledge in order to operate them.
Cat avg: 7.3
Top Performing Features
Test duration calculator
Automatic calculation of the estimated test duration needed to gain statistically significant results.
Cat avg: 7.4
Client-side tests
Ability to run client-side tests (e.g. A/B, A/B/n, multivariate, and funnel tests) to test out UI changes.
Cat avg: 6.8
Experiment scheduler
Ability to schedule experiments to run, or not run, during specific times (e.g. Not to run during a holiday weekend or while a site-wide promotion is going on).
Cat avg: 7.8
Worst Performing Features
Mobile app testing
Ability to run tests to optimize mobile applications.
Cat avg: 8
Page surveys
Create on-page surveys and select which segment of users are asked survey questions using defined audience segments (e.g. new vs. returning users, mobile users, desktop users, etc).
Cat avg: 7
Server-side tests
Ability to run server-side tests (e.g. A/B, A/B/n, multivariate, and split URL tests) to test out more complex design changes, roll out features to specific audience segments, or split site traffic between different site versions.
Cat avg: 7.8
AB Tasty Features from Reviews
Testing and Experimentation
These features enable companies to plan, set up, and execute different types of tests (e.g. A/B, A/B/n, multivariate, split URL tests).
8.1+8%
a/b experiment testing
Create and test variations of a website, changing site elements such as headlines, CTAs, images, page design and layout, technical SEO changes, and new feature additions and collect statistical results of each variation’s conversion rates or other metrics.
Category average: 8.6
Split URL testing
Test out larger design changes by splitting your site traffic across two different landing pages to identify which site performs the best. It can be used to test the impact and feasibility of things such as new designs, personalization efforts, and new site architecture.
Category average: 8.2
Multivariate testing
Ability to test multiple site design changes at once across one or multiple variations and identify which variation impacts conversion rates, or other predefined goals, the most.
Category average: 8.2
Multi-page/funnel testing
Create an experiment that makes changes across multiple pages, like a funnel or a site-wide experience.
Category average: 7.3
Mobile app testing
Ability to run tests to optimize mobile applications.
Category average: 8
Test significance
Ability to set the statistical significance level, or confidence interval, of a given test, for example at the 90% or 95% level.
Category average: 7.3
Visual / WYSIWYG editor
Set up A/B testing campaigns using a WYSIWYG editor to create site versions and preview design changes before testing them. These editors often don’t require coding knowledge in order to operate them.
Category average: 7.3
Advanced code editor
Allows users to create and edit experiments with HTML, CSS, JS.
Category average: 7.2
Page surveys
Create on-page surveys and select which segment of users are asked survey questions using defined audience segments (e.g. new vs. returning users, mobile users, desktop users, etc).
Category average: 7
Preview mode
Preview your experiment before running it live on your site or app.
Category average: 7
Test duration calculator
Automatic calculation of the estimated test duration needed to gain statistically significant results.
Category average: 7.4
Experiment scheduler
Ability to schedule experiments to run, or not run, during specific times (e.g. Not to run during a holiday weekend or while a site-wide promotion is going on).
Category average: 7.8
Client-side tests
Ability to run client-side tests (e.g. A/B, A/B/n, multivariate, and funnel tests) to test out UI changes.
Category average: 6.8
Server-side tests
Ability to run server-side tests (e.g. A/B, A/B/n, multivariate, and split URL tests) to test out more complex design changes, roll out features to specific audience segments, or split site traffic between different site versions.
Category average: 7.8
Mutually exclusive tests
Ability to make tests mutually exclusive so that a given visitor is only part of one test at a time, this helps prevent tests from interfering with one another.
Category average: 6.9
Audience Segmentation & Targeting
A set of tools used for website optimization experiments (e.g. A/B, A/B/n, funnel, split URL, multivariate tests) that can help users segment their audience in to different groups for the purpose of exposing specific audiences to tests or personalization efforts.
8.5+10%
Standard visitor segmentation
Ability to segment, or target audiences based on criteria you set (e.g. URL, cookies, IP address, custom javascript, traffic source, device, browser, language, ad campaign, geo-targeting, time of day) and enable tests to run for specific visitor segments.
Category average: 7.8
Behavioral visitor segmentation
Ability to segment, or target audiences based on whether or not they have performed certain actions, such as clicking on a CTA, and enable tests to run for specific visitor segments.
Category average: 7
Traffic allocation control
Ability to set what percentage of website traffic receives specific test variants in order to roll out code only to a subset of site visitors.
Category average: 8.4
Website personalization
Ability to optimize user experience for individual site visitors based on certain characteristics and past actions (e.g. past purchases, geolocation, demographics, device type, referral source, etc..). An example of this is product and/or content recommendations based on visitor characteristics.
Category average: 8.2
Results and Analysis
Tools that allow users to evaluate the results of website optimization tests (e.g. A/B, A/B/n, multivariate, and split URL tests), or view visitor interaction with webpages and specific site elements.
8.3+10%
Heatmap tool
A tool that shows which elements of the page generate the most visitor engagement.
Category average: 8.4
Click analytics
Click analytics reports display how many clicks certain page elements receive and provides visitor engagement insights.
Category average: 8.7
Scroll maps
Scroll maps display how far down the page users scroll.
Category average: 8.5
Conversion tracking
Enables users to set up and customize conversion funnels to track site visitors' journeys and determine areas that see the most visitor drop-off.
Category average: 7.4
Goal tracking
Enables users to set up key website/mobile performance metrics on their landing pages and track them.
Category average: 7.5
Test reporting
Provides users with reports for each test that record the performance of each variation tested against selected metrics such as conversion rate. These reports indicate when a given test variation has performed statistically better than the original (control) site version.
Category average: 7.4
Results segmentation
The ability to segment test results by specific criteria (e.g. browser type, device type, source, time of day, campaign).
Category average: 7.2
Experiments results dashboard
Provides users with a dashboard displaying test results for all live tests. Some products may also include test result histories accessible from the dashboard.
Category average: 6.5
AB Tasty Features from the Vendor
Testing and Experimentation
Vendor-contributeda/b experiment testing
Create and test variations of a website, changing site elements such as headlines, CTAs, images, page design and layout, technical SEO changes, and new feature additions and collect statistical results of each variation’s conversion rates or other metrics.
Split URL testing
Test out larger design changes by splitting your site traffic across two different landing pages to identify which site performs the best. It can be used to test the impact and feasibility of things such as new designs, personalization efforts, and new site architecture.
Multivariate testing
Ability to test multiple site design changes at once across one or multiple variations and identify which variation impacts conversion rates, or other predefined goals, the most.
Multi-page/funnel testing
Create an experiment that makes changes across multiple pages, like a funnel or a site-wide experience.
Mobile app testing
Ability to run tests to optimize mobile applications.
Test significance
Ability to set the statistical significance level, or confidence interval, of a given test, for example at the 90% or 95% level.
Visual / WYSIWYG editor
Set up A/B testing campaigns using a WYSIWYG editor to create site versions and preview design changes before testing them. These editors often don’t require coding knowledge in order to operate them.
Advanced code editor
Allows users to create and edit experiments with HTML, CSS, JS.
Page surveys
Create on-page surveys and select which segment of users are asked survey questions using defined audience segments (e.g. new vs. returning users, mobile users, desktop users, etc).
Preview mode
Preview your experiment before running it live on your site or app.
Test duration calculator
Automatic calculation of the estimated test duration needed to gain statistically significant results.
Experiment scheduler
Ability to schedule experiments to run, or not run, during specific times (e.g. Not to run during a holiday weekend or while a site-wide promotion is going on).
Client-side tests
Ability to run client-side tests (e.g. A/B, A/B/n, multivariate, and funnel tests) to test out UI changes.
Server-side tests
Ability to run server-side tests (e.g. A/B, A/B/n, multivariate, and split URL tests) to test out more complex design changes, roll out features to specific audience segments, or split site traffic between different site versions.
Mutually exclusive tests
Ability to make tests mutually exclusive so that a given visitor is only part of one test at a time, this helps prevent tests from interfering with one another.
Audience Segmentation & Targeting
Vendor-contributedStandard visitor segmentation
Ability to segment, or target audiences based on criteria you set (e.g. URL, cookies, IP address, custom javascript, traffic source, device, browser, language, ad campaign, geo-targeting, time of day) and enable tests to run for specific visitor segments.
Behavioral visitor segmentation
Ability to segment, or target audiences based on whether or not they have performed certain actions, such as clicking on a CTA, and enable tests to run for specific visitor segments.
Traffic allocation control
Ability to set what percentage of website traffic receives specific test variants in order to roll out code only to a subset of site visitors.
Website personalization
Ability to optimize user experience for individual site visitors based on certain characteristics and past actions (e.g. past purchases, geolocation, demographics, device type, referral source, etc..). An example of this is product and/or content recommendations based on visitor characteristics.
Results and Analysis
Vendor-contributedHeatmap tool
A tool that shows which elements of the page generate the most visitor engagement.
Click analytics
Click analytics reports display how many clicks certain page elements receive and provides visitor engagement insights.
Scroll maps
Scroll maps display how far down the page users scroll.
Conversion tracking
Enables users to set up and customize conversion funnels to track site visitors' journeys and determine areas that see the most visitor drop-off.
Goal tracking
Enables users to set up key website/mobile performance metrics on their landing pages and track them.
Test reporting
Provides users with reports for each test that record the performance of each variation tested against selected metrics such as conversion rate. These reports indicate when a given test variation has performed statistically better than the original (control) site version.
Results segmentation
The ability to segment test results by specific criteria (e.g. browser type, device type, source, time of day, campaign).
Experiments results dashboard
Provides users with a dashboard displaying test results for all live tests. Some products may also include test result histories accessible from the dashboard.
Platform Integration Features
Vendor-contributedAPI
Access to an API to start, stop, or pause campaigns and integrate test result reports into dashboards.
Web analytics integration
The tool integrates with third-party web analytics tools (e.g. Google Analytics).
Content Management System Integration
The tool integrates with third party content management systems (CMS).
Search Engine Optimization/Marketing Integration
The tool integrates with third party SEO and/or SEM tools.
Integration with CRM or DMP software
The tool integrates with third party CRM and/or DMP software and enables users to use this data for segmentation and personalization.