ReadyAPI (formerly SoapUI Pro, LoadUI Pro, and ServiceV Pro) is a REST and SOAP API functional testing tool that enables software developers, QA engineers, and manual testers to work together to create, maintain, and execute complex end-to-end API tests in their CI/CD pipelines without needing to code.
N/A
SpecFlow
Score 10.0 out of 10
N/A
SpecFlow is an open source BDD for .NET. that aims to bridge the communication gap between domain experts and developers by binding readable behavior specifications to the underlying implementation.
As stated, we do a LOT of API testing, the swaggerhub import makes it easy to add APIs. This is very well-suited, as well as easy management of the steps/cases/suites inside of ReadyAPI. The one thing I do wish ReadyAPI was better suited for is changes to data, we have a lot of test cases in ReadyAPI and if we make a change to how the backend data is structured, one-by-one adjustments need to be made to the steps. Less appropriate, UI testing.
It is best suited for implementing the automated test cases in a human readable form so it's easy for non-technical members of the team and stakeholders to understand the test cases, features and the functionalities of the application. Automation of Integration tests and End to End tests are good use case. It is less appropriate or situations where the focus is only on the writing and maintenance of unit tests.
SoapUI NG Pro is a prefect tool for setting up complicated test cases with many steps including parameterized Web Service requests, response assertions, data generators, data sources, data sinks (report recorders) and more
You can use Groovy or JavaScript for more complicated automation such as validating the results, extracting data, using external Java libraries or running system processes. Groovy IDE is compatible with Java and you can easily write your code in Java with very few modifications.
SoapUI enables you to run multiple test suites in parallel or in a loop, and provide user friendly reports including all test case results and test case coverage.
SoapUI NG Pro lets you to perform security test against your web services with predefined scenarios such as xml bombing, xml injection, sql injection, buffer overflow tests, monkey testing and so on.
SoapUI NG Pro, offers a simple but practical solution for low to medium load testing scenarios. You can obtain many more Load and Performance testing features by extending your license with purchasing LoadUI Pro.
Versatility to be used in combination with different kinds of automated testing like automated performance testing, API testing, UI testing etc. I use JavaScript, Selenium, C#, email testing libraries, database testing libraries in combination with BDD with SpecFlow. I am able to use all these with SpecFlow to make my automation framework to be able to automate any kind of automated testing.
It provides different widely used runner options like NUnit, XUnit etc. Before I started to work on establishing proper test automation in my workplace, the previous automation framework (non-BDD based) as well as unit tests used NUnit runner. The transition to using BDD was smooth because we could use the same runner and there were no compatibility issues.
The auto-complete feature is good. I use it with Visual Studio as well as Rider and I don't have to recall the entire Gherkin statements. I just type a few words and the entire Gherkin statement implemented in framework is auto-suggested by SpecFlow. It saves time and context switching.
REST - They have come a long way, but there could still be improvements here. I find the learning curve much higher and not as straight forward using REST vs SOAP.
Composite Projects - I'd really like to see them implement something around saving/refreshing Test Suites when using the Composite project ability in Pro. This is currently an enhancement in feature backlog (see their forum for more details). Functionality around Projects and Test Cases seems pretty sound.
SpecFlow does not accepts optional input variables in the methods defined during Gherkin statement implementation. Cucumber supports optional input variables in the methods defined during Gherkin statement implementation.
The tests identified while using SpecFlow with NUnit removes all white spaces in the scenario names. It makes the tests less readable. If the white spaces are not auto-removed, it would be much better for readability as well as their actual identification in the repository.
I would definitely renew the ReadyAPI as I was pretty happy while using it. But then I switched my job, and the current workplace is using Postman so we are using it for now. Plus, I don't have any issues with Postman. At the end of the day, it about selecting the tool that gets your work done more efficiently.
SoapUI allows us to combine multiple tests and adhere to the sequence that they need to run in order to complete successfully. It has an excellent GUI design and the reporting mechanism is also very good. It does consume a lot of memory though during concurrent testing
Soap UI has managed to continuously build on it's solid foundation and keep improving by each release. It is by far the most dependable and accurate testing tool out there of its kind. Available via connecting to VM's created as SoapUI test machines give access to it anytime, anywhere practically.
For my purposes, the provided help features have been sufficient. I am sure I would be better off if I were to spend more time studying the app. For now, I load a project, connect, and execute. It just works.
I have not used other API testing products but I am completely satisfied with the functionality and performance of ReadyAPI. It covers all required API protocols and database connections that are used in our organization. It also allows extending the functionality by adding external DB drivers and wiring custom scripts when the native assertion/data manipulation test steps are not sufficient.
SpecFlow is .Net based which supports C#. Behave is Python based. Cucumber is Java based. Ghost Inspector is no-code based but provides very limited testing features. We wanted to implement BDD so we rued out using Ghost Inspector. Most of the developers in my team are C# experts so it was decided for everyone's comfort to go for SpecFlow rather than Behave or Cucumber. It's import to have technical experts in the language of the automation framework because there are many situations where the solutions to the test automation needs are not straightforward and implementing those requires expertise in the related programming language.
It has an excellent GUI design and the reporting mechanism is also very good. It does consume a lot of memory though during concurrent testing. However, I have read that added monitoring tools have been added, which if so the 7 could possibly go to a 8 or 9.
Everyone stays on the same page regarding the behavior of existing functionalities whether it be technical or non-technical individuals. So there is less need for multiple people to get involved which saves time and thus money.
Reusing the same code through the implemented Gherkin statement saves test automation time and thus reduces cost.
We combine SpecFlow with other opensource testing technologies to make our automation framework more versatile which further saves costs for us.
