Pioneering Test Automation to build a better bus stop for Auckland Transport.
OVERVIEW
Faced with the twin fires of relentless data updates on the routes and operations of its transport assets and the need for manual validation before implementation, Auckland Transport engaged Assurity Consulting to help improve the situation. Together, they embarked on an ambitious plan to automate database construction and testing to expedite changes. As a result, the transport operator now enjoys faster and more accurate data management, transport schedules roll out on time, and the operations team have more capacity to focus on higher-value tasks.
KEY OUTCOMES
ABOUT THE CLIENT
Auckland Transport (AT) is the council-controlled organisation (CCO) of Auckland Council responsible for transport projects and services. AT is responsible for the Auckland region’s transport infrastructure (excluding state highways and railway tracks) and public transport. It designs, builds, and maintains roads, ferry wharves, cycleways and walkways; coordinates road safety and community transport initiatives; and plans, coordinates and funds bus, train and ferry services. It is the largest of the council’s organisations, with over 1700 staff, controlling half of all council rates.
THE CHALLENGE
Getting a single bus to every stop on time on a predictable route and with information fed to every passenger is one thing. Getting thousands of busses to do the same, day in and day out, is quite another. The process depends on a complex interplay of systems and data describing every route, every stop, and every location. That data is used to manage schedules, keep passengers informed and deliver a smooth-running network that hundreds of thousands of people rely upon daily.
The information sitting behind Auckland’s transport system resides in a database known as Equipment Operational Data (EOD) and is used by multiple agencies within AT, including Fuller’s ferries, KiwiRail, Sealink, Transdev, Ritchies and NZBus, explains Hari Emani, AT Test Manager. “Every time a new bus stop is introduced, a route changed or updated, or schedules amended for any reason – such as school and public holidays or special events – the EOD must change to cater for that new reality,” he says.
This process has AT’s line of businesspeople, its AT data team and the EOD testers collaborating using a Kanban board on Azure DevOps. In this process, the businesspeople add stories to the board detailing changes required to the AT HOP system. The data team makes the necessary changes, then hands the EOD data file to the EOD testing team, which verifies the changes, ensuring every update is accounted for and accurate, and then signs off the file as correct and uploads it for reintroduction into operations.
This process is unwieldy and inefficient, with a new EOD data file emailed to each tester. After conducting manual data manipulation processes, including creating multiple individual copies of the EOD data in local databases, the creation of manual test suites, test cases and test steps, test outcomes are updated and manually linked to each user story. Testers perform their work individually, with a manual to-and-fro of multiple versions of the (exceptionally large) file between testers and the data team, before a final, correct version is readied for production. Though changes to the system come thick and fast, lead times of up to two weeks are necessary to get the job done.
Emani says the process is, by necessity, manual, owing to physical, operational realities. “A lot of this work is extraordinarily complex, making automation challenging. When Assurity first came to us, there was real doubt if automation was even possible – but it was desirable, as these manual processes also meant the only way to understand it was through experience, making hiring new testers incredibly difficult.”
OUR SOLUTION
With a test manager on site with AT, Assurity became aware of the challenges facing the transport operator’s data and testing teams. After suggesting a workshop to gain insight into the processes and challenges faced, Assurity recommended a bespoke automation solution in a phased approach which would demonstrate value early on, thereby providing incremental impetus for further improvements.
A resulting recommendation report suggested several approaches for improving data validation (functional) testing, commencing with centralising and automating the test database build. This was successfully delivered through the development of a custom cloud-native application, which has removed the need for every tester to download and create a personal, local version of the EOD database.
The application incorporates backend services which automatically import the EOD data file and create new EOD databases hosted centrally on Azure, with a backend test automation service and an email notification system. The application integrates with AT’s Azure DevOps Boards, Azure Test Plan and DevOps CI/CD pipelines, fully automating deployments of solution components. The solution automates infrastructure provisioning and provides AT with an EOD test portal.
With this application, testers are relieved of multiple manual and error-prone steps relating to accessing, arranging, and testing the data in the EOD file.
The success of the initial effort established the feasibility of automating what was previously considered an intractable problem. This has resulted in further work, with Assurity identifying the most-encountered change types, delivering proof-of-concept automation, and then delivering automation that manages three of those change types. An additional 30 to 35 change types are being examined with a view to the building of the test database, with a further phase potentially addressing the automation of regression testing.
RESULTS
Automation delivers the immediate and obvious benefit of eliminating tedious, detailed manual work and the inevitable attendant errors. It also accelerates the process, allowing AT to create, apply and introduce changes to a dynamic transport network more efficiently, supporting improved customer service delivery.
While it is still relatively early days, with just three change types automated, there is a lot on the drawing board and a considerable amount of potential for extending the solution. Emani says the idea that AT and Assurity are working towards is empowering the test team to add more change types themselves. “But the reality remains that full automation isn’t possible, and we’ll always need human intuition and flexibility to solve the truly tricky problems.”
“It doesn’t reduce our work because there’s always a constant stream. It reduces our release cycle, making the data and testing teams more responsive to the business. And the business makes those changes because they are responsive to the people who matter – AT’s customers, the citizens of Auckland.”
Related articles
-
Redefining quality for the AI era: Lessons from 20 years at Assurity Consulting
-
Avoid ERP failure: How to build resilience in your Microsoft Dynamics 365 Programme
-
Future-proof Your ERP Transformation Programs