Azure DevOps (formerly VSTS, Microsoft Visual Studio Team System) is an agile development product that is an extension of the Microsoft Visual Studio architecture. Azure DevOps includes software development, collaboration, and reporting capabilities.
$2
per GB (first 2GB free)
CucumberStudio
Score 4.9 out of 10
N/A
CucumberStudio, now a SmartBear product since the 2019 acquisition, is a continuous testing platform replacing the former HipTest, supporting agile development with manual and automated code testing, reusable steps, feature history with living documentation, among other features.
N/A
Pricing
Azure DevOps Services
CucumberStudio, from SmartBear
Editions & Modules
Azure Artifacts
$2
per GB (first 2GB free)
Basic Plan
$6
per user per month (first 5 users free)
Azure Pipelines - Self-Hosted
$15
per extra parallel job (1 free parallel job with unlimited minutes)
Azure Pipelines - Microsoft Hosted
$40
per parallel job (1,800 minutes free with 1 free parallel job)
ADO is well suited for the visibility of day-to-day tasks and responsibilities as well as things such as Features, user stories, etc. Off the top of my head, I can't think of any scenario where it might not be well suited, as you can customize ADO to your liking to a degree.
HipTest is good and helpful to create and run functional regressions but it is more complicated to use this tool to create and run API testing scenarios (it does not mean it can not be done). This tool is definitively not appropriate for running performance testing scenarios.
Flexible Requirements Hierarchy Management: AZDO makes it easy to track items such as features or epics as a flat list, or as a hierarchy in which you can track the parent-child relationship.
Fast Data Entry: AZDO was designed to facilitate quick data entry to capture work items quickly, while still enabling detailed capture of acceptance criteria and item properties.
Excel Integration: AZDO stands out for its integration with MS Excel, which enables quick updates for bulk items.
Need to make the changes so that it doesn't occupy most of the CPU utilization and memory
Execution of Bulky SQl Queries leads to either the SQl being out of exception or the VS being unresponsive
Integration with Microsoft products is easy, but with non-Microsoft products it is more difficult, and you have to make a lot of configuration changes to integrate
With every upgrade of the Visual Studio, like from VS 2010 to VS 2013 , we need to upgrade our hardware/machine, as the VS hardware requirement also increases
If code is getting compiled in one visual studio, like in VS 2010, that the same code could possibly give an error when compiled in VS 2013, due to certain changes in keyword, data format, etc., with the VS upgrade
UX is not too obvious and easy (like generating a data table in a scenario or including a scenario in a test run) but you can get used to it relatively easy by playing around with the options, also the documentation is helpful.
To get extra functionalities you have to pay for them separately.
I don't think our organization will stray from using VSTS/TFS as we are now looking to upgrade to the 2012 version. Since our business is software development and we want to meet the requirements of CMMI to deliver consistent and high quality software, this SDLC management tool is here to stay. In addition, our company uses a lot of Microsoft products, such as Office 365, Asp.net, etc, and since VSTS/TFS has proved itself invaluable to our own processes and is within the Microsoft family of products, we will continue to use VSTS/TFS for a long, long time.
Azure DevOps is a powerful, complex cloud application. As such there are a number of things it does great and something where there is room for improvement. One of those areas would be in usability. In my opinion it relies too much on search. There is no easy way to view all projects or to group them in a logical way. You need to search for everything.
When we've had issues, both Microsoft support and the user community have been very responsive. DevOps has an active developer community and frankly, you can find most of your questions already asked and answered there. Microsoft also does a better job than most software vendors I've worked with creating detailed and frequently updated documentation.
The tool includes a basic but complete documentation that can be very helpful to get started using the tool. The UX sometimes does not feel very easy to understand but after some use you can get familiar without reading the documentation. The webinars are not very useful for using the tool, I have seen a couple of them and they are actually focused on promoting their HipTest product to new users instead of being useful and educational for current users.
Jira is fantastic for project management and customer facing portal. It is not good for pure development (no integration with Git, pipeline management, automated testing features). If DevOps were to integrate and adopt the project features of Jira as well as the customer facing interfaces, I feel it would be a complete project management system.
