Optimizely Web Experimentation vs. UserTesting

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Optimizely Web Experimentation
Score 8.7 out of 10
N/A
Whether launching a first test or scaling a sophisticated experimentation program, Optimizely Web Experimentation aims to deliver the insights needed to craft high-performing digital experiences that drive engagement, increase conversions, and accelerate growth.N/A
UserTesting
Score 8.0 out of 10
N/A
UserTesting helps UX researchers, designers, product teams, and marketers gather actionable insights through research, testing, and feedback. With a network of real people ready to share their perspectives, UserTesting enables organizations to make customer-first decisions at scale.N/A
Pricing
Optimizely Web ExperimentationUserTesting
Editions & Modules
No answers on this topic
No answers on this topic
Offerings
Pricing Offerings
Optimizely Web ExperimentationUserTesting
Free Trial
YesYes
Free/Freemium Version
NoNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeOptionalNo setup fee
Additional Details
More Pricing Information
Community Pulse
Optimizely Web ExperimentationUserTesting
Features
Optimizely Web ExperimentationUserTesting
Testing and Experimentation
Comparison of Testing and Experimentation features of Product A and Product B
Optimizely Web Experimentation
8.0
Ratings
3% below category average
UserTesting
-
Ratings
a/b experiment testing9.00 Ratings00 Ratings
Split URL testing8.50 Ratings00 Ratings
Multivariate testing8.40 Ratings00 Ratings
Multi-page/funnel testing7.90 Ratings00 Ratings
Cross-browser testing8.10 Ratings00 Ratings
Mobile app testing8.00 Ratings00 Ratings
Test significance8.40 Ratings00 Ratings
Visual / WYSIWYG editor8.20 Ratings00 Ratings
Advanced code editor8.00 Ratings00 Ratings
Page surveys6.20 Ratings00 Ratings
Visitor recordings8.40 Ratings00 Ratings
Preview mode7.60 Ratings00 Ratings
Test duration calculator7.90 Ratings00 Ratings
Experiment scheduler8.20 Ratings00 Ratings
Experiment workflow and approval7.80 Ratings00 Ratings
Dynamic experiment activation7.50 Ratings00 Ratings
Client-side tests7.80 Ratings00 Ratings
Server-side tests7.20 Ratings00 Ratings
Mutually exclusive tests8.20 Ratings00 Ratings
Audience Segmentation & Targeting
Comparison of Audience Segmentation & Targeting features of Product A and Product B
Optimizely Web Experimentation
8.2
Ratings
4% below category average
UserTesting
-
Ratings
Standard visitor segmentation8.40 Ratings00 Ratings
Behavioral visitor segmentation7.60 Ratings00 Ratings
Traffic allocation control9.10 Ratings00 Ratings
Website personalization7.80 Ratings00 Ratings
Results and Analysis
Comparison of Results and Analysis features of Product A and Product B
Optimizely Web Experimentation
8.3
Ratings
1% below category average
UserTesting
-
Ratings
Heatmap tool9.30 Ratings00 Ratings
Click analytics8.80 Ratings00 Ratings
Scroll maps8.50 Ratings00 Ratings
Form fill analysis8.00 Ratings00 Ratings
Conversion tracking8.70 Ratings00 Ratings
Goal tracking8.20 Ratings00 Ratings
Test reporting7.90 Ratings00 Ratings
Results segmentation7.70 Ratings00 Ratings
CSV export7.90 Ratings00 Ratings
Experiments results dashboard8.00 Ratings00 Ratings
Best Alternatives
Optimizely Web ExperimentationUserTesting
Small Businesses
Convert Experiences
Convert Experiences
Score 9.9 out of 10
Smartlook
Smartlook
Score 8.2 out of 10
Medium-sized Companies
Dynamic Yield
Dynamic Yield
Score 8.3 out of 10
Optimal
Optimal
Score 9.0 out of 10
Enterprises
Dynamic Yield
Dynamic Yield
Score 8.3 out of 10
Optimal
Optimal
Score 9.0 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
Optimizely Web ExperimentationUserTesting
Likelihood to Recommend
8.7
(0 ratings)
8.1
(0 ratings)
Likelihood to Renew
9.5
(0 ratings)
8.3
(0 ratings)
Usability
10.0
(0 ratings)
8.1
(0 ratings)
Availability
10.0
(0 ratings)
9.1
(0 ratings)
Performance
7.3
(0 ratings)
9.1
(0 ratings)
Support Rating
10.0
(0 ratings)
8.3
(0 ratings)
Online Training
3.0
(0 ratings)
-
(0 ratings)
Implementation Rating
8.0
(0 ratings)
4.6
(0 ratings)
Configurability
6.0
(0 ratings)
7.3
(0 ratings)
Product Scalability
8.0
(0 ratings)
7.3
(0 ratings)
Vendor post-sale
-
(0 ratings)
9.1
(0 ratings)
Vendor pre-sale
-
(0 ratings)
9.1
(0 ratings)
User Testimonials
Optimizely Web ExperimentationUserTesting
Likelihood to Recommend
I think it can serve the whole spectrum of experiences from people who are just getting used to web experimentation. It's really easy to pick up and use. If you're more experienced then it works well because it just gets out of the way and lets you really focus on the experimentation side of things. So yeah, strongly recommend. I think it is well suited both to small businesses and large enterprises as well. I think it's got a really low barrier to entry. It's very easy to integrate on your website and get results quickly. Likewise, if you are a big business, it's incrementally adoptable, so you can start out with one component of optimizing and you can build there and start to build in things like data CMS to augment experimentation as well. So it's got a really strong a pathway to grow your MarTech platform if you're a small company or a big company.
Read full review
When working with general market customers. For example, a person between the ages of 30 and 60 who has a checking account and a mortgage with their primary bank, as long as they don't get too specific about a particular bank.
Read full review
Pros
  • The Platform contains drag-and-drop editor options for creating variations, which ease the A/B tests process, as it does not require any coding or development resources.
  • Establishing it is so simple that even a non-technical person can do it perfectly.
  • It provides real-time results and analytics with robust dashboard access through which you can quickly analyze how different variations perform. With this, your team can easily make data-driven decisions Fastly.
Read full review
  • Product Manager who follows up on your UserTesting usage and gives advice/support when you need it.
  • UserTesting University is a great platform to learn how to use UT and general information about research.
  • UserTesting can find participants quickly, so you won't need to wait long before you can start the analysis.
Read full review
Cons
  • The results view is dense and difficult to package easily for leadership, and when filtering by segment it's hard to read comparative outcomes without clearing or swapping filters
  • The organization of experiments and statuses is a cluttered list and the search is limited in use - would love to see that improve with time
  • There are so many other MarTech products out there, would love to see more dedicated integrations so we don't have to invest in something like Zapier or Tray to build hacky automations
Read full review
  • A centralized platform to recommend products and use cases would be handy so that users can suggest what they could use more of.
  • Integration with newer payment methods and gift card services could expand the user base.
  • Some products are business-critical and thus require sophisticated testing. This service could be added.
Read full review
Likelihood to Renew
Because it's an incredible and essential tool for my line of work as a conversion optimization specialist. Really couldn't do my job nearly as effectively without it. It's paid for itself many times over and I feel like I'm only beginning to unlock the tools potential.
Read full review
I'm actually trying to get a license for my current job at my current company (recently moved to a new company). I gave it this rating because I am about to go to our executive to ask that we purchase it! I know that there will be value because we are forecasting a high volume of research next year
Read full review
Usability
Optimizely Web Experimentation's visual editor is handy for non-technical or quick iterative testing. When it comes to content changes it's as easy as going into wordpress, clicking around, and then seeing your changes live--what you see is what you get. The preview and approval process for sharing built experiments is also handy for sharing experiments across teams for QA purposes or otherwise.
Read full review
It can be difficult to organize our tests and go back and find information. I think the AI tools are helping and will help with this, but for now it is time consuming to sort through all of the tests and information and then synthesize it and share it with others. It just takes a lot of time.
Read full review
Reliability and Availability
I would rate Optimizely Web Experimentation's availability as a 10 out of 10. The software is reliable and does not experience any application errors or unplanned outages. Additionally, the customer service and technical support teams are always available to help with any issues or questions.
Read full review
Never encountered any problems
Read full review
Performance
I would rate Optimizely Web Experimentation's performance as a 9 out of 10. Pages load quickly, reports are complete in a reasonable time frame, and the software does not slow down any other software or systems that it integrates with. Additionally, the customer service and technical support teams are always available to help with any issues or questions.
Read full review
Perfectly fine. Never had any problems.
Read full review
Support Rating
They always are quick to respond, and are so friendly and helpful. They always answer the phone right away. And [they are] always willing to not only help you with your problem, but if you need ideas they have suggestions as well.
Read full review
I've used the "power hour," as I think it's called. This really helped me gain a greater understanding of my specific needs and wants from the platform. This excellent service points to the excellent relationship UserTesting provides its clients. I would recommend this service to any of my colleagues.
Read full review
Online Training
The tool itself is not very difficult to use so training was not very useful in my opinion. It did not also account for success events more complex than a click (which my company being ecommerce is looking to examine more than a mere click).
Read full review
No answers on this topic
Implementation Rating
The implementation through the tag management system took a bit of trial and error at first, mostly due to the asynchronous nature of the TMS. We had to manipulate the implementation to assure that the Optimizely code was written to the page at the right time to allow the experiment content load in the browser without showing any of the original content first. We also needed to make some adjustments to the TMS code to get the integration with Site Catalyst timed appropriately.
Read full review
From a technical perspective, the implementation was extremely smooth. Most of the change management / implementation hurdles were clearing use of the tool through our various security, legal, and information privacy teams. Once these concerns were addressed (UserTesting.com was very helpful in providing all the needed documentation), the implementation process was very simple and we were able to get going right away.
Read full review
Alternatives Considered
The ability to do A/B testing in Optimizely along with the associated statistical modelling and audience segmentation means it is a much better solution than using something like Google Analytics were a lot more effort is required to identify and isolate the specific data you need to confidently make changes
Read full review
UserTesting is probably the most polished with the largest tester pool, fastest turnaround, and great tools for both moderated and unmoderated tests. Userlytics is a solid alternative, especially for the budget-conscious. It supports usability testing on lots of devices, and you can do both moderated and unmoderated sessions. The UI isn’t as seamless as UserTesting’s, but it gets the job done and it's pretty flexible in terms of study setup and participant targeting.
Trymata is the most basic, but it’s relatively straightforward and affordable. You still get session recordings and written responses, but lacks dashboard polish or advanced features. It’s great for smaller teams or one-off tests.
Read full review
Scalability
It's incredibly flexible and adapts well to organizations of all sizes, whether you’re running a single site or managing multiple departments and platforms. The ability to deploy experiments seamlessly across different environments is a huge plus, especially for growing businesses. While it’s highly scalable, the last point would depend on the right team leveraging its full potential.
Read full review
The package we have limits the number of people who can set up tests. This prevents us from scaling the use of the platform.
Read full review
Return on Investment
  • we saved money by not implementing certain copy/design
  • we learned that customers from different states react different to a variation
  • we are slowly learning where conversion happens and where to fix the frictions
  • Testing shorter vs longer journeys increased funnel conversion in some states - we avoided implementing this nationwide
Read full review
  • We can build tests and gather the insights from those tests to validate our products at a breakneck pace.
  • By quickly validating our assumptions, we can adjust or refine our designs to ensure the final results are more user-focused.
  • Recently, by testing and comparing two different flows for a form, we were able to create the right solution that gave us better leads compared to the former form. This results in better leads, better data about the user, and better support at the end for this user.
Read full review
ScreenShots

Optimizely Web Experimentation Screenshots

Screenshot of AI-Powered Experimentation with Opal:

- Instant Test Ideas: Generates high-quality A/B test ideas based on any goals and audience insights.
- Smarter Experimentation: The AI can suggest impactful variations, reducing guesswork and increasing test velocity.
- More Than Just Ideas: From hypothesis generation to analyzing results, Opal helps optimize every stage of the experimentation process.Screenshot of the Web Experimentation Visual Editor :

- Tweak experiments using the visual editor or dive into custom code when needed.
- Modify elements, update styling, or add dynamic behaviors.
- Ensure perfect variations while keeping control over every detail of the experiment.Screenshot of AI Content Suggestions:

- Generates copy variations to supercharge experiments.
- The AI suggests high-impact messaging for tests when hovering over a field.
- AI-powered content suggestions help skip the brainstorming process.Screenshot of Advanced Audience Targeting:

- Delivers personalized experiences by targeting users based on behaviors, attributes, and real-time conditions.
- Defines precise audience segments using first-party data, geolocation, and device type.
- Can test and optimize for different audience groups to maximize impact and engagement.Screenshot of Custom Templates in the Visual Editor:

- Offers pre-built templates for common test setups.
- Standardized variations and maintains brand integrity with reusable templates.
- Templates can be customized visually or tweak them with code for full flexibility.Screenshot of the Web Experimentation Results Page:

- Data visualizations help interpret experiment performance.
- Displays which variations are winning with built-in statistical significance calculations.
- Results can be filtered by audience segments, events, and conversions to uncover key trends.

UserTesting Screenshots

Screenshot of UserTesting's several solutions for gathering rich customer experience narratives.Screenshot of Interactive Path Flows
Built on recent research in data mining, the Interactive Path Flow aggregates interaction data across multiple participant sessions to visualize the customer journey, surface unexpected behaviors, and locate key moments in the customer journey.Screenshot of Keyword Mapping
Similar keywords are automatically grouped based on overall sentiment (positive, negative, or neutral) to identify themes. Highlight reels associated with each keyword are grouped together, to discover the why behind each sentiment.Screenshot of Video capture and live streaming
Digital and real-world customer experiences are recorded on desktop and mobile devices or live streamed for in-platform viewing.Screenshot of Audience targeting
Audiences are specified by screening contributors from UserTesting’s global network of contributors or connecting to any preferred network.