Date: 05 May 2021
“The testing team need a clear communication to the governance body that is separate from the delivery arm of the project, this will ensure quality issues can be raised effectively” 2015 NZ Government Public Sector, ‘Top 10 Lessons Learnt’.
A review of GETS, the Government Electronic Tender System, coupled with market intelligence, indicates there will be a wave of payroll system replacement projects over the coming months, partly due a popular system being no longer supported.
The tsunami has already started with several agencies now in the market for payroll systems and associated system integrators. At the risk of being accused of negativity, it is perhaps timely to review some of the lessons learned from two very high visibility payroll failures, Novapay and Queensland Health (QH), as well another infamous non-payroll specific failure, IT in the NHS. As Robert Kiyosaki writes, “The best way to predict the future is to study the past, or prognosticate”.
Perfect storms
To summarise, Novopay and QH were ‘perfect storms’ or a confluence of failures, given their combination of issues, disruption, delays and money lost. Based on an assessment of the key findings in Novopay and QH, there is significant commonality across the two reports.
This provides supporting evidence for the theory that many IT projects run into difficulties for the same reasons. Clearly, times have changed and it would be disingenuous to ignore the improvements that have been made over the last 10 years, particularly in project management.
In addition, throughout the last decade, frameworks and systems have been developed to help facilitate effective IT governance. However, a considerable number of IT initiatives still fall short or run into significant difficulties. It would appear that, collectively, we often fail to learn from the lessons of the past. Despite the number of prescriptive models and ‘best-practice frameworks’ available and an increased focus on project governance in organisations, achieving key governance outcomes is consistently ranked by Gartner as one of the top concerns of management.
Four contributors to failure
A review of the documentation around Novopay, QH and the UK’s IT in the NHS indicates that there are four main contributors to failure that are common across all three projects – implementation maturity, accountability and clarity of roles, risk and change management and the use of standardised processes and practices.
The maturity of an organisation’s implementation management concerns the approach to such elements as testing, data migration and change management. It is also indicates the level of technical knowhow that resides in the governance function. Given that the raison d’être for governance is effective decision-making, having uninformed participants on your project board is less than ideal.
Clearly, a lack of clarity of role and the associated diffused accountability will have a significant effect on the success of projects within an organisation. Experience shows that there is still a lack of broader governance and executive oversight within certain agencies, despite the existence of guiding documentation, training courses and expert advice.
The latest ‘Treasury Gateway – Lessons Learned’ report identifies that governance is a “significant and growing concern, in part because the increasing trend towards large complex multi-agency and all-of-government projects requires a lift in capability. There are significant concerns that expectations are exceeding capability in this area”.
“During the service design and development phase, the intended pilot and phased rollout of the service were removed from the project plan. Some important areas of functionality were not fully tested prior to Go Live. Some types of testing were not completed to the original scope, on the basis that testing could be completed after Go Live, or that the risks of not doing the testing had been adequately mitigated. Not all System Integration Testing criteria were met” Ministerial Report, Novopay.
Diligent test process
A critical observation from all three projects was the lack of test planning and actual testing undertaken. The project testing team can be the proverbial canary in the development/implementation coal mine. They see indications of success or failure well before anyone else. Therefore, a regular and diligent testing process is essential to ensure delivery of a quality product to the end user.
Recently, IRD completed a very successful ERP implementation; the Project Sponsor attributed that success partly to the inclusion of the Test Manager (TM) on the Project Board. The TM sat next to the Project Manager and worked closely with, but independently of, that function. At the governance level, this arrangement provided the Project Sponsor with another source of data to ascertain ‘what good looks like’. Assurity played a key role in the provision of assurance, test expertise and resource during the project.
There are advantages in having an independent test capability. An informed and independent tester can repeatedly find out more, other, and different defects than a tester working within a programming team – or a tester who is by profession a programmer.
The interdependence of governance and testing, and the level of proficiency demonstrated in both, can have a significant impact on the success of your project. Although often perceived as functions that bookend our projects, they should be integrated into the fabric of the undertaking.
At Assurity, we have a proven track record in delivering effective test strategies and test services. This coupled with our in-depth experience of payroll implementations means that we are uniquely placed to help organisations bridge the governance and testing divide.
Ignore the lessons of the past at your peril.
“Those who cannot learn from history are doomed to repeat it” George Santayana.
