IBM DevOps Test Performance helps software testing teams test earlier and more frequently by shifting testing left. IBM DevOps Test Performance validates the scalability of web and server applications, identifies the presence and cause of system performance bottlenecks and reduces load testing. Software testing teams can execute performance tests that analyze the impact of load on applications.
N/A
ReadyAPI
Score 7.0 out of 10
N/A
ReadyAPI (formerly SoapUI Pro, LoadUI Pro, and ServiceV Pro) is a REST and SOAP API functional testing tool that enables software developers, QA engineers, and manual testers to work together to create, maintain, and execute complex end-to-end API tests in their CI/CD pipelines without needing to code.
Go for IBM RPT if: 1. You're testing a Java-based Web application with HTTP protocol 2. You wanted to distribute the load across machines easily 3. Your team is in learning phase/not really introduced to a wide range of performance testing tools Do not go for IBM RPT if: 1. You wanted to test REST or any other advanced protocols 2. Your system under test demands a very high user load 3. Your application is written in .NET or any other platform except Java.
As stated, we do a LOT of API testing, the swaggerhub import makes it easy to add APIs. This is very well-suited, as well as easy management of the steps/cases/suites inside of ReadyAPI. The one thing I do wish ReadyAPI was better suited for is changes to data, we have a lot of test cases in ReadyAPI and if we make a change to how the backend data is structured, one-by-one adjustments need to be made to the steps. Less appropriate, UI testing.
SoapUI NG Pro is a prefect tool for setting up complicated test cases with many steps including parameterized Web Service requests, response assertions, data generators, data sources, data sinks (report recorders) and more
You can use Groovy or JavaScript for more complicated automation such as validating the results, extracting data, using external Java libraries or running system processes. Groovy IDE is compatible with Java and you can easily write your code in Java with very few modifications.
SoapUI enables you to run multiple test suites in parallel or in a loop, and provide user friendly reports including all test case results and test case coverage.
SoapUI NG Pro lets you to perform security test against your web services with predefined scenarios such as xml bombing, xml injection, sql injection, buffer overflow tests, monkey testing and so on.
SoapUI NG Pro, offers a simple but practical solution for low to medium load testing scenarios. You can obtain many more Load and Performance testing features by extending your license with purchasing LoadUI Pro.
Memory utilization could have been improved.(Eats up system's RAM)! It may crash if a test is conducted with the heavy load if adequate RAM is not available in the VM/host machine.
Licensing could have been made simpler. IBM's licensing method is difficult to follow.
Support for protocols other than HTTP. Not really up to the current trend.
REST - They have come a long way, but there could still be improvements here. I find the learning curve much higher and not as straight forward using REST vs SOAP.
Composite Projects - I'd really like to see them implement something around saving/refreshing Test Suites when using the Composite project ability in Pro. This is currently an enhancement in feature backlog (see their forum for more details). Functionality around Projects and Test Cases seems pretty sound.
I would definitely renew the ReadyAPI as I was pretty happy while using it. But then I switched my job, and the current workplace is using Postman so we are using it for now. Plus, I don't have any issues with Postman. At the end of the day, it about selecting the tool that gets your work done more efficiently.
SoapUI allows us to combine multiple tests and adhere to the sequence that they need to run in order to complete successfully. It has an excellent GUI design and the reporting mechanism is also very good. It does consume a lot of memory though during concurrent testing
Soap UI has managed to continuously build on it's solid foundation and keep improving by each release. It is by far the most dependable and accurate testing tool out there of its kind. Available via connecting to VM's created as SoapUI test machines give access to it anytime, anywhere practically.
For my purposes, the provided help features have been sufficient. I am sure I would be better off if I were to spend more time studying the app. For now, I load a project, connect, and execute. It just works.
Cost/Licensing: While JMeter is an opensource testing tool from Apache, compared to IBM RPT and HP LoadRunner, RPT is much cheaper than Loadrunner. Functionality:JMeter provides basic functionalities which are adequate for performance testing, however advanced features are not available (such as load testing with GUI, reporting is very basic etc.). But when it comes to Loadrunner, it offers very broad features and supports a variety of protocols. So in this category, Loadrunner is a winner, but RPT is better than JMeter. Ease of operating:JMeter is easy compared to LoadRunner, but it has old GUI and look and feel is not that great to understand. Also, most of the things are to be done in a command line, non-GUI mode. While LoadRunner is very advanced with many options, which also confusing sometimes. But RPT, on the other hand, maintains a balance between simplicity and offering of different features. So winner: RPT.
I have not used other API testing products but I am completely satisfied with the functionality and performance of ReadyAPI. It covers all required API protocols and database connections that are used in our organization. It also allows extending the functionality by adding external DB drivers and wiring custom scripts when the native assertion/data manipulation test steps are not sufficient.
It has an excellent GUI design and the reporting mechanism is also very good. It does consume a lot of memory though during concurrent testing. However, I have read that added monitoring tools have been added, which if so the 7 could possibly go to a 8 or 9.
