Azure DevOps Server vs. TestRail

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Azure DevOps Server
Score 8.3 out of 10
N/A
AzureDevOps Server (formerly Team Foundation Server, or TFS) is a test management and application lifecycle management tool, from Microsoft's Visual Studio offerings. To license Azure DevOps Server an Azure DevOps license and a Windows operating system license (e.g. Windows Server) for each machine running Azure DevOps Server.N/A
TestRail
Score 6.9 out of 10
N/A
TestRail by Gurock, an IDERA company, is presented as a complete web-based test case management solution to manage, track, and organize your software testing efforts.
$35
per month billed annually per user
Pricing
Azure DevOps ServerTestRail
Editions & Modules
No answers on this topic
Professional Cloud
$35
per month billed annually per user
Professional Cloud
$38
per month per user
Enterprise Cloud
$71
per month billed annually per user
Offerings
Pricing Offerings
Azure DevOps ServerTestRail
Free Trial
NoYes
Free/Freemium Version
NoNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeNo setup feeNo setup fee
Additional Details
More Pricing Information
Community Pulse
Azure DevOps ServerTestRail
Features
Azure DevOps ServerTestRail
Test Management
Comparison of Test Management features of Product A and Product B
Azure DevOps Server
-
Ratings
TestRail
7.5
Ratings
7% below category average
Centralized test management00 Ratings7.70 Ratings
Manage test hosts and schedules00 Ratings7.00 Ratings
Map tests to user stories00 Ratings7.70 Ratings
Test execution reporting00 Ratings8.20 Ratings
Defect management00 Ratings7.00 Ratings
Best Alternatives
Azure DevOps ServerTestRail
Small Businesses
GitHub
GitHub
Score 9.0 out of 10
BrowserStack
BrowserStack
Score 8.6 out of 10
Medium-sized Companies
GitHub
GitHub
Score 9.0 out of 10
OpenText ALM/Quality Center
OpenText ALM/Quality Center
Score 8.0 out of 10
Enterprises
Perforce P4
Perforce P4
Score 7.6 out of 10
OpenText ALM/Quality Center
OpenText ALM/Quality Center
Score 8.0 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
Azure DevOps ServerTestRail
Likelihood to Recommend
9.0
(0 ratings)
7.5
(0 ratings)
Likelihood to Renew
10.0
(0 ratings)
-
(0 ratings)
Usability
6.0
(0 ratings)
7.0
(0 ratings)
Support Rating
8.4
(0 ratings)
-
(0 ratings)
Implementation Rating
8.0
(0 ratings)
-
(0 ratings)
Ease of integration
9.0
(0 ratings)
-
(0 ratings)
User Testimonials
Azure DevOps ServerTestRail
Likelihood to Recommend
In our case it was best suited when we started working remotely, we were able to track everything in out projects easily, able to share our codes, give reviews for the codes and also create integration and deployment CI/CD plans for the release and testing. It helped our team members with the productivity, early prototyping and release. Create summarised reports of different aspect of our projects. Even in other scenarios it is one of the best tools to use for collaboration and project management. I haven't found any specific scenario where it is not appropriate
Read full review
Well suited - where you have a set of cases defined say for regression and then you can mark them as Pass/Fail/Skipped, etc. If you wish to present the impact of certain set of new features, bug fixes,etc, you can use this tool Not well suited- where the user is naive and don’t have much understanding of test reporting tools as this tool is slightly difficult to learn at a glance
Read full review
Pros
  • Vast range of Template for Automated Build and Release Pipeline with option to have conditional triggers
  • Code repository mechanism is aligned with GIT which is a well known public repository system
  • Bug Management is also an integral feature of this tool
  • Dashboard creation option of Build and Release pipeline is also available
Read full review
  • Beautiful and productive UI.
  • Nice filtering system allowing me to find what I need quickly.
Read full review
Cons
  • Searching through code can be somewhat cumbersome. It would be nice if there was a way to do general searches in certain areas of the system.
  • Without proper training, the system can be confusing to navigate. This issue can be prevented with good training but it is something to be aware of.
  • Navigation can be clunky at times depending on where you are in the system. For power users, this is not a huge deal but it is a tad bit annoying.
Read full review
  • UI has a lot scope of improvement
  • Uploading of test screenshots and managing them can be more effective
  • User experience can be improved
  • Tracking of tests can be more simplified
Read full review
Likelihood to Renew
Because we are a Microsoft Gold Partner we utilize most of their software and we have so much invested in Team Foundation Server now it would take a catastrophic amount of time and resources to switch to a different product.
Read full review
No answers on this topic
Usability
For standard users the interface is friendly. but if you are a manager some tools are a little confusing to use, like the query system that you always need to create from scratch. Templates should be more helpful for queries and for standard procedures that you need to duplicate PBIs over time. The search history of Work Items is a little painful to use.
Read full review
As I am a tester, for me I found this tool to be new in terms of everything like the management of tests, plans, releases, reporting,etc. It is overall a good tool for test reporting and can be used for reference in the future. I liked several features of on the go placement of screenshots. Also I feel like the UI , the font, the color combination can be improved
Read full review
Support Rating
I have not had to use the support for Azure DevOps Server. There have never been any issues where I was not able to figure it out or quickly resolve. Our Scrum Master has used support before though, and the service has always been prompt and clear with a customer-focus
Read full review
No answers on this topic
Implementation Rating
Do research beforehand and, if possible, do a trial run before implementing into production environment.
Read full review
No answers on this topic
Alternatives Considered
Azure DevOps is a fully integrated solution that solves all of the problems that our separate tools did in a much easier-to-use way. Before we implemented DevOps we had three different solutions that we had to integrate with each other and required a lot of manual intervention to make sure they worked correctly.
Read full review
TestRail definitely saves times. I work in a company that consists of several development teams, all of which have different processes. Some of the teams leverage test cases, some do not. I've noticed that the turn around time it takes for me to pick up a ticket, QA it, and then pass/fail/send feedback is much faster when there is a test case created as I'm not reading through ticket description/comments to figure out what needs to be QA'd.
Read full review
Return on Investment
  • It has streamlined the pipeline and project management for our agile effort.
  • It has helped our agile team get organized since that is a new methodology being leveraged within the Enterprise.
  • The calendar has improved visibility into different OOOs across the project team since we all come from different departments across the larger organization.
Read full review
  • We are able to easily pull reports any time management would like to see them
  • Jira integration makes it easy to attach to stories/defects and save time
  • A quick overview at the system's health as a whole is easy to see from TestRail
Read full review
ScreenShots