Test Automation is an automated testing process and is the automation of testing activities. Where in this web portal on the area of software quality assurance or the software test is limited, so the IT sector?
This automated testing process tests software products on the market using test automation tools or self-developed test software. This test automation tool addresses and controls the software being tested. After each action, verification points are normally evaluated in order to check whether the result is as planned in the automated test step or in other way anomalies should be logged.
In the end, manually evaluate whether the anomalies are an error or not. For example, the displayed anomaly of the test automation tool could also be based on a new software version with correct and changed software design and "only" the test script was forgotten to update.
There are different levels of interfaces that test automation can use
- GUI level - graphical interface
- API editing - eg. HTTP
- Code-level unit tests - also called module tests or component tests.
When and why is it useful to introduce a test automation?
Automated testing and the introduction of test automation into the QA process are not a " must " to produce high-quality software. However, test automation is useful for complex software with a higher proportion of regression tests (repeatable tests). Basically, a certain degree of regression testing speaks for a test automation.
If, on the other hand, a software (eg in the initial phase of the development process) changes very strongly and constantly, a test automation may not be the best choice, since the cost of maintaining the test automation can then be too high.
Particularly in the course of agile development, test automation can hardly be dispensed with. Most of the best practices and good testimonials from Agile experts agree that automated testing is an extremely useful addition to the shorter iterations. Thus, tests can also run overnight and tests can also run automatically after each code check-in including an automatic build process.
Continuous Integration, Continuous Delivery and Test Levels
Pyramid test stage partitioning and test automation in agile development methods
When we talk about the automatic build process and automatic tests after code check-in, the buzzword is "Continuous Integration," which has just come to the fore with agile methods. In the case of "Continuous Integration", an automatic software build and installation process should run after the code check-in. Afterward, unit tests run through the software to provide developers with feedback on the basic quality of the software in a very short time.
Other levels of testing, such as integration testing and system testing (automated GUI testing), can start automatically and provide quick feedback. It should be noted that the number of automated GUI tests should be much smaller than that of the unit tests. This is to illustrate the diagram of the Agile Test Pyramid.
The target of test automation
A test automation does not have to accelerate the development process, the goal should rather be higher software quality through higher test coverage.
In the end, development time can certainly be shortened, as errors can be found earlier if the automated tests run more often, but this should not be the general focus. What is certain is the higher test quality and thus higher software quality, if a suitable amount of test automation is selected at all test levels according to the software and the goals.
If the software has a very high demand for regression tests, then automated testing is always a time saver as well as a cost saving. Because after the automated tests, thousands of test scripts can be run in a short time, whereas in the manual test you would need a lot of time and manpower.
For large software projects, which are tested for complexity with many regression tests, one quickly reaches such large dimensions in terms of the number of test cases and thus very long test execution times that one could no longer cover it with manual tests, even with multiple testers. In this case, a test automation is of course essential.
Advantages and disadvantages of test automation
Whether a "test script" is a better and more attentive "tester" than a "human being" depends on the exact case. If a manual tester tests regression tests 8 hours a day, if necessary, the same tests every few days, it is certain that a certain degree of operational blindness will be assumed. Because every person sometimes overlooks something, especially with such an approach.
On the other hand, test scripts can also have errors, and test automation tools can have bugs. And it can be forgotten to make any necessary changes to the test automation when software changes, so that the test script still runs through, but does not check everything correctly. Mistakes would be obfuscated and a certain amount of chaos would arise when looking at the results, even though they contain errors that are not found due to the poor quality of the test scripts. A human manual tester might have noticed these errors directly. A test script just checks exactly what you teach it.
Furthermore, a test automation script has just no intelligence. A human tester can always put his experience into play when running manual tests, and look left and right to see if there are any mistakes beyond the exact test steps. Here we come in the space of exploratory and experience-based tests.
Thus, a test automation is certainly never a replacement of the complete manual test. However, it is often a good tool in software quality assurance, and so in many complex and large software projects, a very large proportion of the tests are performed by automated testing methods.
Most errors are found, however, if you use both tools of the software test properly.
Setup of test automation and test tools
There is a large market of commercial and open source test automation tools. A very current and complete list can be found in this article: Test Automation Tools
There are various strategies for setting up a test automation system, from the introduction of existing test tools and customization to the complete in-house development of a complete test automation tool and test automation framework. In general, developing test automation always requires development experience and programming.
With a good test automation, however, only a small part of the test automation has to be able to program. Using methods such as keyword-driven and data-driven test automation can be automated to the outside with the use of "keywords". This may include a graphical user interface (UI) so that testers can easily and efficiently compile the automated tests graphically. It is also important to determine the desired granularity of the keywords by a comprehensive analysis.
The keyword-driven approach also has tremendous benefits in terms of maintenance. Maintenance of test automation is often an underestimated point, as the test automation system usually needs to be continually updated. In general, changes to the software should be made in as few places as possible when the software changes.
A good example of this is the "login" to a software application. This "login" test step will later come in handy in most test scripts and test cases, but no one wants to change hundreds, even thousands of test scripts when changing "login". To do this, a test automation solution must be structured in a reasonably structured way.
Automated testing is a real software project
Unfortunately, automated testing in companies often does not pay attention to what it takes. "Just on the side" is not appropriate here. Test automation quickly becomes so complex that it has to be set up like a software project. This also includes a version control to have the code of the test scripts in a source code repository and to be able to understand changes well.
Expenses should be estimated and test automation included in the project planning of the software being tested. Als,o budget must be planned sufficiently, forthe worker as well as tools and test environment. It will certainly be found errors in the existing test automation solution, these errors should be collected in large test automation projects in bug tracking tools (defect management tools).