Katalon

Katalon studio is relatively new to market but its popularity is growing rapidly. We have started exploring this tool recently and found it very useful if you compare it with other open source tools such as Selenium, Appium etc. It works very well for automating Web/Mobile/APIs with features like Record/Run, Object Spy etc. Altogether it is all in one tool for test automation. Easy to integrate with Git and Jenkins. It also has very nice reporting features with Dashboard. Overall Katalon studio is going to be a leader in test automation industry in coming future. Limitations: Katalon studio didn't locate the Android SDK automatically. We need launch Android emulators manually while running mobile automation tests. Katalon studio Debugging feature is not available with free version. Data driven feature at test case level is also not available with Katalon studio free version.

• Record and Run
• Object Spy
• Parallel testing
• Reporting
• Data driven testing

• Katalon studio is going to make very positive impact on ROI
• In one tool, we can have features for Web, Mobile and API automation testing
• I think the licensing cost is a bit on higher side

& ServiceV Pro), LoadUI Pro, ReadyAPI (formerly SoapUI Pro, Selenium and Appium

Selenium, Appium, ReadyAPI (formerly SoapUI Pro, LoadUI Pro, & ServiceV Pro)

• In built framework
• License cost
• Testops and analytics

• 60% savings in efforts
• Easy to maintain test scripts
• Flawless execution

TestComplete

TestComplete, Selenium, SoapUI Open Source, supported by SmartBear

We use Katalon Studio to test our GraphQL based API's using a Data Driven model. Validation of response and its contents for different requests and integration with Azure Devops.

It solved the use case of end to end testing of our applications

• Codeless Automation
• Free Version features
• Lower Pricing relative to competitors

• More Effort spent on investigating Azure pipeline delays and failures when running Katalon test suites
• Licenses not released automatically when an Azure pipeline run is completed. No help received from support team on this

Micro Focus UFT One

Azure Cosmos DB, Azure DevOps Services

There was no proper insight to test reports and reports were not organized well. It was difficult to get information on test flakiness or test case failure history. Our organization had already been using Katalon automation tool for writing test cases and with the Katalon testOps, everything is in one place.

• Single place to analyze test reports
• Provides number of tests automated for each release
• Helps with defect tracking
• Report history for easy triages

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Jenkins and Jira Software

Appium, VMware Fusion, Docker

• Codeless test automation
• Editor to do with and without code
• Flexibility to connect with different data source

• Easy to start solution
• Possibility to have desktop, api, web and mobile application under one tool
• Possibility to integrate with CI-CD tools

TestComplete and Ranorex Studio

Selenium, Appium, TestComplete, Ranorex Studio, SmartBear TestLeft, Micro Focus UFT One (formerly HP UFT)