TestComplete for Desktop application
Rating: 7 out of 10
IncentivizedUse Cases and Deployment Scope
Used for UI testing of our desktop application. Really helpful to check for regression in the hundreds of small features it has, especially when a Unit Test isn't possible, because a lot of GUI logic is needed before launching the business logic. It is also great to know that each window, tab, and popup was checked before release, especially when the test users mainly focus on the new feature of the software.
Pros
- Repetitive mundane sequence of UI actions
- Checking UI object's properties
- Taking a lot of my time
Cons
- Name mapping is hard to edit: when the hierarchy or the identifiers properties are wrong, it can be quite painful and time-consuming to correct. Especially when it is not clear which item wrongly matches. A tool to correct such items in name mapping would be useful.
- TestComplete isn't multi user friendly, especially when using source control. Even if each user works on different tests, correcting the name mapping file only generates havoc on merge.
- KeywordTests' UX is awful: for some items, editing parameters can be done in two clicks, for others we need to click several times to display the edit button, which would be hidden by the tooltip, and then click blindly on this button.
- And then some properties can be modified in their text boxes, but other need to display the variable editor to be able to change anything.
- Region Checkpoint is mostly useless: such checkpoint would be great for verifying generated images, but then anti-aliasing will never be the same on different rendering and the checkpoint will fail. Increasing the error tolerance will then not detect real issues.
- We have redone the checkpoint by using perceptualdiff (https://github.com/myint/perceptualdiff ) and using only the Region Checkpoint feature to save the reference image (and mask). We also added the possibility to change of reference image, as testing on different graphic cards will give other subtle changes (AMD vs Nvidia).
- TestExecute is too much limited. We needed to write a tool to deploy our software on the test machine, and then change the TC project file to include or not some test. As some tests are quite flaky, we needed a redo of these, so we only select (automatically) the failed tests for doing another run.
- Exception in the test should always mark it failed, and not just discard it as it happens now in some cases.
- We needed a color checkpoint to check the display of some status indicators. Quite strange that it was missing in TC.
Likelihood to Recommend
For UI testing of desktop applications, it does its job: it seems better than concurrent tools. It will automatize most of the UI tests, and allow to catch most regressions and even detect some real bugs. But it isn't free: a lot of time is needed to create all the tests, then these should be updated for each new release of the tested application (especially if the GUI changed). And more time to have a real automatic testing solution, as TestComplete and TestExecute don't have any feature for deploying the tested application.
Vetted Review
6 years of experience