We all love test automation, the more the better, right? But, what is test automation? Yes we all know of the classic types of automation like Selenium, which I love and hate with equal amounts. But, what about an Excel macro, what about (quick and dirty) throw away scripts, what about VM (Virtual Machine) setup scripts, and CI (Continuous Integration), and Cucumber, and FitNesse, the list goes on.
Test Automation is NOT Testing:
I need to mention here what test automation isn’t before I’m accused of something else. In essence automation is not testing on the whole it is checking functionality that has been tested and shown to work previously and that it is still working as expected. If you want a more thorough understanding on this topic read this. However, after posting that link for Testing v’s Checking the definition has very recently been redefined, so for an up to date explanation go here.
Don’t use just one Automation Tool.
We all use various tools to assist us day to day with our testing, either to speed up some mundane task or to try and ensure a higher quality of accuracy with complicated error prone tasks. However when we think about our automation strategy what’s the first thing we seem to focus on? We think about which single tool to use rather than the coordination of many varied tools working seamlessly together within an automation framework.
Any automation strategy should not focus on any single tool or types of automation but should be inclusive of every tool that works for you. When you adopt this approach within a coordinated and focused strategy your chances of success are greatly improved.
There are often quoted figures that profess to indicate that some 60% to 70% of all automation strategies fail within the first 6 months. Although these numbers are often quoted there validity isn’t clearly evidenced, perhaps due to the fact that no one wants to admit to failure. However; that’s the only evidence I have, anecdotal, apart from personal experience and other people’s stories and the ratio of success to failure does seem to swing in favour of the negative headline.
Why is it then that the desired nirvana of test automation often fails? I think it’s quite simple; one tool is usually the choice without consideration of the multitude of tools available. The automation strategy isn’t clearly thought through, estimation of how long it’s going to take is often widely of the mark and maintenance is often not taken in to consideration from the start.
It’s very easy to think test automation is the answer to your problems but without thorough planning you are most certainly doomed to failure. Do not think that everyone else is automating therefore we need to automate, or automation will save time as it won’t if it’s not planned properly.
- Referenced Article:
- Test Automation Snake Oil - This article wasn’t used when creating this blog, but was pointed out to me latter.