Date: 20 January 2014
In Part 1, I described what my values are as a tester and how that drives my approach to communication and testing away from massive pre-scribed tests and onerous documentation towards a leaner, more transparent approach. So what is this approach?
I like to employ a style of software testing that emphasises the personal freedom and responsibility of the individual tester to continually optimise the quality of his/her work by treating test-related learning, test design, test execution and test result interpretation as mutually supportive activities that run in parallel throughout the project. When performed by a skilled tester, this approach yields valuable and consistent results.
This approach is comprised of three elements: The Heuristic Test Strategy Model, aspects of Session-based Test Management, presented in a visual model using mind-mapping software. This ensures that testing is performed systematically and intelligently and be auditable, accountable and traceable.
Heuristic Test Strategy Model
The Heuristic Test Strategy Model (HTSM) by James Bach is a set of patterns and heuristics designed to spark ideas around developing a test strategy. It is available to download for free from James’ site. I was fortunate to discover the HTSM early on in my testing career and I can say without any doubt that it is the most useful tool I have ever used. I even carry a copy with me to client sites.
For the purposes of creating a Visual Testing Coverage Model (VTCM), I want to investigate and map out all the different aspects of the product, as well as all the ways in which a stakeholder may value the product. For this reason, I use the Product Elements and Quality Criteria categories for the VTCM.
Session-based Test Management
Session-based Test Management (SBTM) is a way to structure exploratory testing and offers a framework for reporting and measuring testing activities.
The important elements of SBTM are:
Charter: This briefly describes the agenda or mission of the testing session.
Time-boxed session: This is an uninterrupted period of time spent testing or pursuing the Charter. Session lengths are typically 60 to 120 minutes, but you can timebox in half or full-days at the expense of granularity of measurement.
Session Report: This reports what occurred during the test session and can include:
-
Charter or mission
-
Environment
-
Start and end times
-
Tester’s name
-
Notes on how testing was conducted
-
List of any bugs found
-
List of any issues encountered
-
Time spent off charter (called ‘opportunity’ in SBTM)
-
If necessary, percentage of time spent testing, time spent investigating bugs, time spent reporting bugs and time spent setting up data and test environments.
The kind of information recorded can, and should be, tailored for the project. If a standardised reporting format is used, the reports can be parsed to aggregate data that is deemed useful. There are also many tools that support SBTM.
See here for an example of a test session report
The advantage of SBTM is that a tester has the freedom to apply their skills to actively investigate the product, while still being accountable and their work traceable. The advantage over pre-scripted testing is that the tester is reporting on what they actually did, as opposed to describing what they intended to do at some point in the past.
In practice, scripts are often obsolete by the time the software is delivered and, even under scripted conditions, testers will deviate from the script and two different testers may interpret the script two different ways. If important information is discovered under these deviations, this information is usually lost. SBTM allows rapid execution of tests and gives the tester the freedom to perform the next best action based on the information they have just learned.
Find out more about SBTM here:
http://www.satisfice.com/sbtm/
http://www.satisfice.com/articles/sbtm.pdf
http://www.satisfice.com/articles/et-article.pdf
Mind-mapping software
The testing process is not a nice, neat, linear process. Information comes to us non-linearly and often spontaneously. I need a way to capture this information as it arrives without being concerned about how to adjust it to a linear model (such as a test plan document written in MS Word). I also need a way to communicate non-linear, complex information in a way that is easy to understand and process. So far, I have found that mind-mapping software is the best-suited tool for these goals. Along the way, I have discovered a wonderful side effect of using mind-mapping software: unsolicited feedback… which I will go into later.
So, how do we begin?
1. Start planning the framework
I like to begin by arranging the elements of the Structure and Quality criteria around a central node in the mind-mapping software. I then identify the properties I’m interested in, or may be interested in (after all, in these early days, who knows what may be important and the point is to stimulate your thinking in ways you may not have considered) and begin the framework for your map:
2. Keep adjusting the map
Your map is a model. As your model changes, so will your map. Parts that you thought were important will be deleted and parts you didn’t know existed will rise to prominence. The map is a reflection of YOUR understanding of the product at a given time.
It also has other uses. You can use it to manage the testing process by assigning testers to a branch, letting them take ownership of all sub-branches. This lets you quickly assign work without having to go through requirements and specifications one-by-one and manually assigning individual tasks. You could colour branches that you’re currently working on and shade in branches that have been completed. At a glance, you can see what has been done, what is being worked on and what is left to do.
I hope this illustrates a good way to structure, manage and report on testing. The main advantage is that it is quick to do: it is extremely lightweight and can be easily added to and parts moved around with ease. It also doesn’t need to be ‘read’. A multi-page document requires sitting down to read and it is left to the reader to try and interpret what you mean. A map can be glanced at and, almost immediately, the structure, connections, functions, what needs testing and what hasn’t been thought of, are apparent. A map is worth a thousand words.