Test Automation. Busting the myths…

New Thinking 2 July 2012 Jamie McIndoe

In this article, I examine a few common beliefs around automation and how new tools and ideas in automation are turning these on their heads.

You have to be technical to automate

Really? One of the great misconceptions about automation is that you need to have a technical background or education. But while technical skills can give you immediate access to more powerful tools, you can still build automation that will provide great benefit armed with only a good understanding of what to automate and a decent grasp of logic.

I know this from first-hand experience. I came to IT as a manual test analyst with few computer skills, but within a few years was working as a full-time test engineer. I learnt enough of the automation tools and programming languages to get underway and built my technical knowledge along the way. In fact, our team of automation engineers includes several psychology graduates, a qualified pilot and a former saw doctor.

There has been a real convergence of tools and approaches in automation in recent years. It’s increasingly considered as integral to project delivery, rather than something that exists to cover business-as-usual regression testing after project completion. Solutions that allow technical and non-technical people to collaborate and for rapid test development (enabling significant amounts of project test execution to be done via automation) are leading the new wave of automation.

Selenium IDE, for example, has achieved wide adoption by allowing non-technical people to build simple but useful automation without any coding knowledge or technical support. It’s also easy to set up with a large community of users to provide information and develop extensibility. Scripts developed in Selenium IDE can then be exported to more powerful tools such as Selenium WebDriver, where they can be expanded and enhanced.

Other tools, like Concordion and HP’s Business Process Testing, have more powerful functionality and are supported by scripted code to drive test actions. However, they have business-friendly interfaces allowing non-technical people to plan and design comprehensive automation.

Test Automation is only suitable for regression testing

Who says so? This opinion of automation, has persisted from the early days of test automation to today. While there will always be cases where the project environment limits the scope of automation to regression testing only, advances in test automation tools and approaches and changes in development methodologies are enabling automation for a far wider range of work.

Probably the biggest recent change in the automation world is the increasing adoption of Agile development methodologies. The constant development and rapid turnarounds of Agile mean that automation is often required to cover the bulk of test execution as there simply isn’t time for manual-only test execution. The central role that automation plays in Agile is enabled by having highly collaborative teams who focus on developing small chunks of functionality. This allows for the sharing of code, automation work and for automation to develop in parallel with application development.

My biggest personal revelation of my first Agile project was this: it was the first time that my goals as an automator and the project’s goals had ever truly aligned. I was developing and testing for the project to complete the stories we brought in each sprint, rather than coming in to develop automation assets more for use post-implementation than during the project’s lifespan.

I developed automation in parallel with application development, often utilising developer code to minimise my development timeframes. I was able to cover the majority of the project’s functional test execution leaving the test analysts to focus on exploratory testing and test analysis.

My system testing incorporated a regression testing component which grew throughout the project. Automation which had originally been testing new functionality later covered regression testing for the same functionality. However, regression was only one aspect of the project’s automation work and automated testing of new functionality was always a priority.

While automation can offer a lot more than just regression testing within an Agile environment, there is no reason why the same automation approaches cannot be a significant help to any collaborative team which develops iteratively and where automation is involved from early in development.

Test Automation is just automating test execution

Not so. While a lot of what we do in automation is focused on automating test execution, it can also provide benefits in supporting other test or project work.

Consider the efficiency gains you can get from creating macros to save the time and effort of executing repetitive or complex tasks in Microsoft Office. These macros are examples of automation that support you in your work and provide a lot of value even if they only cover a small part of your role.

Some of my most successful automation work – that which has provided immediate benefits in return for minimal development effort – has been in supporting a team’s test efforts. This work has included creating, sourcing and verifying test data, simplifying complex test verification, accelerating manual test execution by automating time-consuming processes and generating reporting data and graphs.

None of the tasks I automated were my responsibility prior to automation, but I was able to identify where my automation skills and experience would be most beneficial to improving the team’s efficiencies.

Automation should be considered wherever there are potential savings in time and effort, regardless of what the task is. This is especially relevant in the Lean/Agile world where even simple and seemingly small savings are actively sought to improve efficiency and positively impact the team’s productivity.

What next for test automation?

Recent advances in automation have enabled it to be used by more people and to cover a wider range of testing and non-testing tasks. But what will the leading trends be in automation in the next few years? Here I introduce some themes that I will expand on in future articles.

Automation that targets system components outside of an application’s GUI has been possible for some time, but it is only really through the rise of Service Oriented Architecture (SOA) testing that this capability has come to people’s attention. GUI automation is easy to understand as it directly emulates end user actions. However, the GUI is often the hardest part of system architecture to build fast, consistent and maintainable automation against – especially as GUIs often undergo far more change during development and post-implementation than any other part of a system.

System architectures are also becoming increasingly complex. Focusing only on automating against the GUI potentially means that important system components are not covered and can only be tested manually. I believe we will see test automation becoming much more about covering a combination of the GUI and other system components to ensure the widest possible coverage and the most maintainable automation.

Technology is moving increasingly towards mobile platforms and mobile applications will become an important focus for test automation.There has been a rapid growth of commercial and open-source automation tools for mobile platforms in the past few years. Many tools run on emulators but an increasing number are capable of working on physical mobile devices.Some of our key government and financial services clients are ramping up their mobile application development capabilities as we speak, so we’re likely to see significant growth of mobile test automation in New Zealand in the near future.

I also believe that test analysts will become increasingly cross-skilled as they become more involved in automation and other types of technical testing. The tools are much more accessible by non-technical people than ever before and the collaboration focus of Agile / Lean projects will ensure their involvement.

Further reading

“Experiences in Test Automation” by Dorothy Graham and Mark Fewster

Published earlier this year, this book provides more than 20 in-depth case studies for automation successes and failures across a wide range of tools and automation approaches. Available from Amazon.com.

Selenium IDE probably the best web tool to play around with when you are starting out with automation.

I will also be writing further articles for this website, including how to  get started in automation, case studies and tool / framework overviews.

Written on a musical diet of: Wire 154, Slint Spiderland, Sebadoh Bakesale, Gang of Four Entertainment. All on vinyl.