Testing vs. reporting

Explore Assurity 3 March 2014 Alice Chu

Alice started the night with a presentation about the importance of reporting and discussed the risk of too much focus on reporting prevalent in high-stress projects potentially overriding the work of testing.

The discussion on the priority of reporting was categorised by two main arguments.

The first reason to report is for the benefit of the testers. Reports are often seen as the conventional way to communicate the quality of testing to the business and stakeholders. Solely looking at defect reports provide no correlation on the quality of testing, as defects are dependent on the phase of testing and quality of the system.

Secondly, reporting is absolutely crucial for effective project management to ensure projects run to budget and schedule. Resource management can be drawn from test reports as it helps estimate the amount of resources needed, the time to complete each testing phase and factor in contingencies. Additionally, projects need to be monitored to identify issues and unexpected problems so they can be mitigated in later releases. Reports can help to identify the blocking causes, pinpoint inefficient processes and identify where defects are clustered so that testing can be focused on a particular area.

So the questions are: “When are testing artifacts used for testing and when are they used for the purpose of reporting?” and “At what level of detail are test scripts needed and how many test scripts would be sufficient to satisfy both testing and reporting?” More often than not, meeting the requirements for testing and reporting do not agree and this is where contention is introduced.

A number of topics were discussed on the night, one highlight being that what is perceived as value in testing varies for different people. Some people may view test execution as valued testing. However, test reports actually bring out the value of testing to the business. Participants also discussed different methods of reporting, what can and cannot be reported and, inevitably, the use of visual management reporting tools was raised. Different reporting tools were shared by the group that led to the recurring theme of obsession on metrics.

A great night, enjoyed by all. Thank you to everyone who attended the Workshop and great to see so many people participating in the discussions.