The most important aspects to consider when creating an Automation strategy and plan

You have just made your case to upper management regarding test automation and your proposal has been accepted. Surprisingly, they were excited about how your automation plan can move testing much faster. Or, upper management simply asked you to start automating testing, as they are convinced of its benefits. In either case, you now have a blank check and a deadline to meet.

The objective of automation is to simplify as much of the testing work as possible with a minimum effort. Automation value is driven by the coverage it provides compared to the time it takes to automate it versus the time saved by the automation. The assumption is that you do not have infinite resources nor time to produce visible results. This adds weight to the importance of developing a suitable automation strategy for your business case. Aspects like the complexity of the project, its methodologies, technologies, size and geographic locations of the team, make it a challenge for automation engineers to choose a successful approach.

While there is no “bullet proof” strategy for automation, it’s worth mentioning some factors that drive the process.

Methodology Implications for Automation

When using Agile, early discovery of issues is encouraged. Development, on one side, and functional, integration and performance testing on the other, all need to be executed simultaneously. First off, it makes sense to automate functionality that is critical for business and that is most used. As a test case selection criteria, the Low Hanging Fruit pattern can be used to generate momentum – making success visible – or increasing the ramp-up speed of the new automation team members. However, this is regarded as an anti-pattern on the long term. One other thing to consider when dealing with an Agile project, is that automation may be forced to live with the failures, until they are fixed.

With Waterfall, testing is considered a mass effort near the end of the project. This method does not offer predictability on what happens at the end of testing (release pass or fail) and more and more investors tend to opt-out. However, from the automation engineer’s point of view, the Waterfall method offers benefits to automation, including the ability to select test case candidates earlier in the software development lifecycle.

The most important aspects to consider when creating an Automation strategy and plan.jpeg

Technologies

Selecting the right tools for Automation involves evaluating different criteria for each of the development methodologies. In Agile, if a free tool can satisfy the testing needs, then it can be adopted by the whole team. Contrary to that, if considerable money needs to be invested into tools licensing, then probably not everyone in the team will have access to the tools, and in Agile this is a considerable limitation.

As the project matures, engineers will probably realize the limitations of free testing tools and they might see opportunities to enhance the automation outcome by changing just a tiny thing in the tool. Open source tools are relatively easier to customize and could be a better choice, especially when your project is using new technologies that are still in beta and there are no testing tools for them.

Application Stability and Lifecycle

The requirement’s stability and the application’s stability are two factors that can influence the automation effort and rework. As these factors are out of the automation engineer’s control, they can only be mitigated by choosing what and when to automate. An indicator here is the moment when the development team is beginning construction – at this point the requirements are stable enough. The application stability is a tougher factor to control, though. Again, when possible, postponing the automation trigger after having the chance to exercise the application’s features manually, is a good approach. Keeping rework and maintenance effort to the minimum is a key factor in the automation strategy.

Automation decisions can also vary depending on the stage of the application’s lifecycle. It’s helpful to divide selection criteria into 3 primary categories:

Automation targeted for new or start-up efforts
Automation for ongoing efforts
Automation for mature or end-of-life efforts

The checklist might also evolve as your automation efforts mature. Assigning weights to each category can help with test cases selection.

Impact on the Project

One other key factor to consider is complexity. It’s preferable to automate a normally distributed set of test cases, well balanced between very complex, moderately complex and straightforward scenarios. Also, the selection should be created with the goal of increasing speed in the project testing. Identifying the most time-consuming test cases and automating them will have a positive effect on the testing cycle time. Starting automation with the smallest or fastest tests can be considered bad practice.

In the end, not all the factors can be considered when making the automation decisions. Every aspect can have different weights, depending on the context. The primary goal to keep in mind is to create an automation strategy that makes a visible and strong impact on the project.

Looking to upgrade your software testing skills? Check out our trainings.

Cristian Ionita
Consultant in Telecommunications