SOLUTION
Assurity, Sinziana says, is instrumental in NZQA’s success and has been for 10 years. “Together we do quite a bit of testing so we can be sure that our systems work as expected before they go into production.”
In practice, Assurity provides performance testing across all NZQA systems with a particular focus on results publication day. This is a significant event for the organisation (and the country) in which the equivalent of months’ worth of logins and user activity ‘hit’ in the minutes and hours when exam results go live.
It may sound simple – just test the maximum number of concurrent users, right? – but nothing could be further from the truth, Sinziana points out. “Probably the biggest challenge is complexity and change. Our systems aren’t constant and every exam cycle is different. The number of systems we have interacting is so substantial that getting someone up to speed on all the dependencies and intricacies can take months or even years. And then, at the same time, there is a major push towards digitisation and digital transformation. These changes are really like changing the engines of a jet in flight.”
This makes people who grasp NZQA’s data flows, exceptions and broader partner ecosystem valuable, she continues.
A further area in which Assurity provides thought and delivery leadership is Targeted Exploratory Testing. This addresses the volume of work necessary for continual quality assurance and is considered a key component of the NZQA delivery model. “Exploratory testing has accelerated the process of evaluating complex intertwined systems with innovative approaches to testing” says Sinziana.
Explaining that Assurity has demonstrated an ability to think outside the box and think big picture, she says examples include planned ‘Just in Time’ testing where team members log in as users and use systems just ahead of go live, with a consistent ability to identify and resolve issues before they have real-world impacts. “With the nature of exams, this is crucial – we simply can’t only test well in advance because come exam day, those tests just aren’t relevant any longer.”
Assurity has also created a dashboarding system which replaces previous disparate reports and spreadsheets tracking performance. By creating a consolidated visual system, NZQA personnel enjoy immediate insight on how exams, results and other essential metrics are tracking. The system is so successful that it is today published on public screens within the company – attesting to the confidence NZQA enjoys that its systems can be relied upon.