91. Stop following test scripts and think.
I’m getting this posting “Stop following test scripts and think“ written before Michael Larsen gets to my two testing quotes which were published in the “99 Things you can do to become a better tester” booklet by the Ministry Of Testing recently as it won’t be long, he’s currently at #72 as of this posting.
There have been plenty of postings against testing with scripts from within the Context Driven School of Testing, so was my quote just an obvious trite cliché? It’s obvious to me, yes, but in my experience it’s not obvious to others within the testing field. Why is it that I have often experienced a resistance to changing the way test scripts are written?
Well I’ve been just as resistant and to explain that I need to make an admission. I have been a long term advocate of test scripts, the more detail the better was my philosophy, dam you ISTQB. My basic premise was I wanted test scripts to be written so that anyone who came into QA could immediately start testing against the scripts because they were so detailed and easy to follow.
I didn’t see that as a problem, until I started to understand what we lost by using that tired methodology. Where’s the testing here? There isn’t any, this is checking by following the dots, this kind of testing (in the losses sense of the word) can be done by anybody, but it’s not testing. Very little is gained by using this approach, the only thing it could assist with would be to help plan out a regression automation suite, even then the time spent with this approach really isn’t worth the time or effort, nor is it needed.
Do I use test scripts now? Yes. So what are you talking about then?
It’s not the use of test scripts in general I’m against, it’s the details that’s contained within the test scripts and how the scripts are used by testers that is the problem. The test scripts I currently use are very high level, no expected outcomes; just enough detailed to guide the testing activities around critical functionality prior to each release. The key word in that sentence is guide, not instruct, not to follow a line by line test and expect this approach.
I am not saying this is a perfect approach, but what we do gain is a reasonable understanding of what has been tested from a regression point of view. These are tests, as they allow for the testers individual interpretation of the process flow from start to finish with that particular functional area. They are valuable within my current context within one organisation. The way these scripts are written allows the tester to explore and to think for themselves, they are in essence mini exploratory mission statements, rather than test scripts.
Stop calling them test scripts then!
However in another organisation I work for we don’t have any test scripts, we work from Mind Maps, as that’s a more appropriate approach to use in that context. Why do I have test scripts in one organisation and not in another? It’s not down to not having the authority to change the way either organisation tests, that’s my remit for both. I use exploration test scripts in one organisation because the product we are testing is highly complex and mission critical for our customers and I do not believe in this context Mind Maps would work. The reason being is mainly due to design issues of the Mind Maps themselves, they would become too unwieldy and I don’t believe they would add anything to the types of exploratory testing scripts we current develop.
In the other organisation Mind Maps are the current obvious approach to testing, easy to use as a guide, easy to understand where you are with your testing, small and not unwieldy, it’s a web site after all, not multiple standalone products as with the other organisation.
Exploratory testing isn’t about going straight in to testing without having an idea of what the aim of the exploratory sessions is, and then randomly and aimlessly pressing buttons to see what happens. Exploratory testing has a remit of what you are trying to achieve. The test scripts we use are just the same as an exploratory remit they allow the tester to explore and think. The test script sets up a scenario and what is to be achieved, how they go from the start to the end is down to the testers own imagination.
Yes the title of this blog is somewhat of a lie to pull you in to read it, but just because someone says they use test scripts, I think you should ask what kind of test scripts they use and what they mean when the say test scripts, before you make a judgment.