The Quality First Blueprint: Under the microscope

[00:00:01] Reg Prasad

Harvard Business Review recently noted that out of 1.3 trillion dollars spent globally on digital transformations, over 900 billion was essentially wasted. A massive program is announced with huge fanfare, only to go quietly over budget and miss deadlines or fall completely short of expectations. We’ve seen this before. Now why do all these things keep on happening? Because in the rush to deliver faster, we often forget to deliver better.

Thank you for joining us. This is the first edition of The Quality Edge by Assurity Consulting. I’m your host, Reg Prasad. 

In the show, we strip away the corporate jargon and look at what it actually takes to make digital transformation succeed, here in New Zealand and globally. Our thesis is simple. Quality isn’t just a testing phase at the end of the project; it’s a mindset that needs to be engineered into the very beginning.

Now, today, we’re just putting our own ideas under the microscope. We recently published The Quality-First Blueprint, a guide on how to fundamentally shift your delivery culture. Joining me to tear this blueprint apart and look at how it works in the real world are two industry veterans. Now, when I say veterans, they’re not very old, who have spent their careers architecting and executing these exact types of programs.

That’s Shikar Ramchand, our Practice and Solutions Manager at Assurity. With over 19 years of experience in quality assurance, Shikar has been the driving force behind the success of massive ERP replacements in business transformation across logistics, telcos, government, and the financial sector. He specialises in breaking down silos and fostering real collaboration between QA and the wider delivery teams.

Shikar, welcome.

[01:51:19] Shikar Ramchand

Thanks, Reg. Great to be here for episode one.

[01:54:15] Reg Prasad

Awesome. Thank you. Also, Gavin Cochius. Gavin brings a highly pragmatic, results-focused approach to our conversation today, combining deep software test management with a strong background in operations, accredited ISEB Testing and Certified Scrum Master. Gavin has directed complex onshore, offshore and multi-vendor testing teams, particularly in the international finance sector. He’s the leader who ensures these massive strategies actually translate into language and results the stakeholders at all levels can understand.

Gavin, thanks for joining us as well.

[02:30:05] Gavin Cochius

Thanks for that introduction, Reg. It’s great to be here. I’m looking forward to sharing some of my experiences and answering some of the questions with real-life scenarios.

[02:39:02] Reg Prasad

Awesome. Thanks so much, guys. Hey, let’s get straight into it. Now, Shikar, I’ll start at the strategic level. Executives are constantly under pressure for cutting costs at the moment and delivering features which were very important, and they want it yesterday. Now, when you sit down with the CIO and tell them that they need to adopt a quality-first blueprint, their knee-jerk reaction is often, “That sounds expensive and slow.” Now, how do you draw on your experience with those large-scale ERP transformations to prove that building quality in early actually accelerates delivery?

[03:12:20] Shikar Ramchand

I think that’s exactly the reaction we get from executives leading programs of business transformation. They’re always looking to ensure that they’re getting the best value for money, and that’s exactly what they should be doing. And with us talking through the quality-first approach, there’s obviously a cost associated with that. However, the way we articulate the benefits and the value that we will realise for that cost is important.

When we build anything, if we’re building, if you’re constructing a building or if we’re assembling a motor vehicle, we want to ensure that every part of the design is appropriate and applicable for the eventual use case. So it’s important that we’re involved from the outset, from the initial design, the strategy and planning through to build and configure and then to test.

If we leave test for the very end, for quality assurance, for the very end, we risk finding those issues far later. It’s our famous shift-left approach. You know, the further we shift-left, the more cost-effective it is to resolve those issues. So, that’s typically the way we would begin these conversations, so that we make sure that everyone is on board and understands the value that we will bring into the team from an early stage of a transformation.

[04:44:12] Reg Prasad

Now you’ve spent two decades in QA. That’s a long, long time. Yeah. So obviously, QA has changed over the period, especially the last five years, given, you know, AI has made the trend and the buzzwords of QA very different. A big part of your focus is fostering collaboration. In the blueprint, we emphasise moving from quality assurance, which people often view just as a gatekeeper at the end, to quality engineering. What does it actually take to get a solid QA team and a delivery team to collaborate effectively on a complex project?

[05:19:10] Shikar Ramchand

You know, with all of the new technology that has emerged in the last years, it’s a really exciting time for us. It’s somewhat scary because it’s new, but also we see a lot of the work that we’re doing, and the grunt work that we’re doing being taken by AI. Just on that point, I think it’s important that we make sure that we’ve got a consistent way of applying AI approaches and policies in achieving the outcomes that we want, instead of just throwing an LLM at a team and expecting a huge productivity gain. 

In terms of collaboration, I think in moving from just a QA checkbox exercise into having a well-defined collaborative team delivering a solution, I like to compare it to a sports team on a field. We’re all working together. We’re linking up, we’re making passes. We’re assisting one another. We’re not just playing our position and then standing back once we make the pass. In order to achieve the goal, we have to move as a team. We have to collaborate. 

So, the development team, the BAs team, the QAs team we’ve got our three amigos approach in collaboration, in making sure that we can embed quality through a transformation and move from that checkbox exercise into a quality engineering space, and build in all of our processes so that we can achieve those quality outcomes that we want.

[06:51:12] Reg Prasad

I like that analogy of a sports team, you know, like what is the success criteria? We have to win the game, but we have to do it as a team.

Now, Gavin, let’s get you into the mix of things as well. Now, you know, from a Shikar side of things, you can design highly collaborative, quality-first solutions. You’ve got a background of operations and managing multi-vendor teams across multiple sectors. Now, when a project is running late, and you have offshore teams, third-party vendors, internal developers all in the mix. How do you practically enforce a quality-first mindset without operations grinding to a halt?

[07:29:01] Gavin Cochius

What I found has been successful in the times that I’ve worked across things like that is that everybody has to understand right from the beginning and early on in the project what success is going to mean and what the impacts of that success could be. And it just reiterates what Shikar was saying; it’s about a collaborative effort. I think the earlier we involve the end user into the process to understand, from an operational perspective, this is what you’re going to be getting.

This is what it’s going to look like. This is how it’s going to work. This is the impact that it’s going to have on your role and how you’re going to be doing your role. We talk a lot about shift-left from a QA and a quality engineering and a testing perspective, but it’s shift-left from all the way across to the end user perspective.

The quality-first mindset is that the end user understands right from the beginning what they’re getting and how they’re going to be getting it. And what’s their involvement going to be from a UAT perspective, from an integration testing perspective? Is there going to be a reliance on them to actually help with testing, to create data and this type of thing?

So, that’s not getting involved early in the process. And that’s the operational side that we tend to leave right near the end. We go, we’re going to sort out UAT right towards the end of the test cycle. So, I found that bringing them together early on, the whole cycle together early on, from operational and working back all the way through to analysis and design. That the user actually understands, from analysis and design perspective, already what’s coming. How it’s going to impact them. Then you don’t have the things of surprises.

I liken it, a big project like this with multiple vendors and multiple areas of expertise and this type of thing. It’s to me, it’s like the one where the guy sweeping the floor in the Kennedy Space Center, whatever it was with NASA, and he gets asked, what are you doing? And he says, I’m sending a rocket to the moon. 

And I think that’s the whole concept that everybody in the project, from inception, all the way to the end, has got to understand that they’re playing their part in sending their rocket, whatever rocket it is, to wherever it’s got to go to; they’re all playing an integral part in that.

[09:32:01] Reg Prasad

100%. I think communication and clarity is a big part of delivering these large-scale programs. Now it’s part of your expertise in translating QA outcomes into language stakeholders can actually understand now when a transformation is struggling, and you know, we’ve been through quite a few that do start, but I think the big thing is the success criteria is forgotten. You know, how do you actually communicate the value of blueprint’s principles to senior stakeholders who only care about the deployment date? You know, it’s not just about go-live.

[10:05:07] Gavin Cochius

I find, from a program test manager perspective for an operational management perspective, it’s probably more important to communicate upwards than what it is to communicate laterally and downwards. The key people, the key stakeholders, need to be aware of what’s happening from the beginning. And I think that these days we have tools like Jira and that type of thing, which can give you instant data on what’s happening from a testing perspective. But the comms is key, and it’s how you distribute that comms, and who you distribute it to. 

C-level and execs, and senior stakeholders aren’t interested in going to Jira and looking at stats on Jira; they want a one-page or PowerPoint that they can breeze through and have a look at, and Jira and those types of things give you the tools to actually generate that type of information instantly and on a regular basis.

You’ve got information right at hand that you can do it. You can have discussions about risk assessments. You can have discussions on how you’re tracking against the risk. You can have discussions of impacts of where the testing is happening, what issues are being found, what are the impacts of that testing.

It gives analyses and details of defect trends and that type of thing that you can communicate upwards. You can communicate it downwards. You communicate it in the language, and it’s going to be understood by the different parties. If you’re presenting to execs, you’re not presenting the same detail as what you’re presenting to your testing. So, it’s understanding how you communicate that.

[11:35:08] Reg Prasad

So, just from a tool side of things and you know, insights that the tools provide now, where do you think the complexity is in terms of providing the information and versus how stakeholders consume it?

[11:47:06] Gavin Cochius

I think it’s about understanding what information is relevant to which party. And with tools like Jira these days, you’ve got the options of creating a number of dashboards that can display different information. So, you can create a dashboard that’s specific for C-level. You can create a dashboard that’s for the testing. You can create a dashboard for audit stakeholders.

So, wherever it is from a risk and governance perspective, you can create dashboards to suit the need. And that gives you instant access; if you need to report on something, you don’t have to try and correlate from all different sources. You’ve got a snapshot that you can give straight away. 

And that’s the challenge these days, is having information that’s instantly available, because people don’t want to wait for an answer. They want to know straight away: where are we? What’s happening? How are we progressing? But you have to have kept the communication channels open to actually, that when you give them those stats, they understand how you’ve actually got there already.

It’s not just a, wow that’s a surprise. That’s news to me. So, it’s about communicating that all the time.

[12:48:02] Shikar Ramchand

I’m just latching on to some of what Gavin’s saying that. Absolutely agree. And going back to the very first question around shifting-left, I think different organisations and different executives within those organisations measure quality differently. So, their perspective on quality might not necessarily be a tester’s default measure of quality. And we can measure it by default by a number of severity, one and two defects. Or, you know, the pass rate, for example.

So, if when we’re involved early, we were able to then define what success looks like, we’re able to establish that foundation. And then we’re able to tailor our reporting and messaging so that we can deliver the message that the sky is falling when actually the sky is falling. And not to our perspective or some generic perspective of what quality looks like. And then it differs from industry to industry.

If we’re in healthcare or in air travel, then there’s a different level of quality and discipline that we’ll need to do you know, embed. And if we’re in some other industries, then a risk-based approach is more applicable for rapid delivery. So, getting those foundations right at the beginning is very important, so that we can then deliver all of those data points accurately.

[14:13:23] Reg Prasad

You know, you’re one of the people that provide new insights, and you know, wrote The Blueprint. Now it looks airtight on paper, but no rollout is seamless. Between the three of us, we’ve got an immense amount of program rollouts. Now, looking at the government or telco sectors you’ve worked in, what is the single hardest part of this blueprint for an organisation to adopt and where’s that friction point?

[14:40:02] Shikar Ramchand

The blueprint has been written because we’ve had these conversations many times across multiple clients. So, I think it’s valuable that we document it and are able to share it widely with our industry. For me, the most difficult part of getting a solid quality regimen in is getting an automation test regression suite established early on.

The conversations are tough because it’s an investment it’s a notable investment to make from a program’s perspective, and it doesn’t necessarily return value immediately. So, if we do not have many multiples of regression cycles built into the project, it’s not multi-year, with us taking vendor upgrades during the program, then you might not see that return on investment.

However, it becomes very important from go-live and especially with transformation programs and SaaS solutions being implemented, cloud-based solutions, we will be taking in up to eight updates a year. That’s two major, plus six minor, possibly. 

And for each one of those, you would want to do some sort of regression testing. Doing that manually versus having an automation regression suite established makes a world of difference in terms of the cost associated for ownership, and that’s typically left towards the end when the team has mostly left, the project team has mostly left, the IP has left with them. And then starting to build an automation suite from that point on becomes another challenge that we would need to overcome.

So again, in this instance, let’s strike while the iron’s hot and get that done.

[16:32:17] Reg Prasad

Gavin, similar question for you. Now when you’re trying to execute the blueprint, especially within the strict regulatory environment or financial services, what is typically the biggest roadblock you’re seeing?

[16:46:17] Gavin Cochius

That’s interesting because it also goes back to what Shikar has just said about risk-based testing as well. We don’t have the capacity to test everything all the time. So, no matter what part of the financial services sector, you’re going to do risk-based testing, and that’s going to be the focus.

But then there’s, in the financial services sector that I’ve been involved in, there’s two streams of thinking around that. And I’ll use an example. You’ve got somebody who’s aged early 20s that takes out a life insurance policy to mature if they die before age 80. That policy is probably only going to get looked at in maybe 40, 50 years’ time when it matures or when the person passes away and has to pay out. So, all you’ve got to do is make sure that the correct premium is being deducted. It’s been applied to the LIC policy and the policy details, and everything is correct. And that’ll be the next time that that policy is looked at. And that’s happened with some of the legacy policies from the 70s and 80s.

We’ve moved it into a modern scenario. And you’re going, okay, well, what applied 20 years ago doesn’t apply these days, and we need to try and modernise it. So, then you go into things like data migration, and that type of thing becomes key. 

But then you have another policy, same scenario where a person needs an early 20s, and they invest money. And you’ve got to have that on specific right now right there; they’ve got to be able to see what’s the value of it right now. What’s it going to be in 40 years time? If I put a premium on it, is it going to change? Is it market-related that type of thing?

So, your focus from a risk perspective is different. The one you can worry about it’s low risk, only going to get looked at in 40–50 years time. The other one has got to be looked at tomorrow as soon as I apply them. So, it’s about getting that right. 

And things change now these days because of the ways of working that we have with automation and this type of thing, and automation happening early in the process with continuous integration and agile and everything else, and you’ve got your compliance people looking and going, well, from a compliance perspective, I need to know that this output matches that output. And what am I going to be getting, and does it match? And how can you validate that testing actually validated the outputs? 15, 20 years ago, it was it was basically paper outputs that they were looking at. It wasn’t an online digital format. So, it’s basically looking at what risks can we live with, what risks can we accommodate.

A lot of the financial services stuff these days is inherited from a lot of legacy systems from years ago. So, that’s one of the biggest changes also, is about getting the data migration correct in those sectors. 

And another thing, as Shikar mentioned earlier, about if we look at medical and aviation, you’re very risk-averse because you can’t do risk-based testing on some medical stuff, and you can’t do it on aviation, because when the plane’s in the air and something fails, you can’t say, well, we didn’t really test that. We didn’t have the time to. So we removed it out of our testing scope. But you can do that on financial services. You have the luxury. And I’m putting that in quotes. I’ve been able to do that type of thing.

But again, it’s having stakeholders involved early that they understand what the processes are, what the risks are, what the mitigation is, what is going to be displayed, what’s not displayed, and everybody in agreement about what is the risk that we’re going to take in our testing.

[20:08:04] Reg Prasad

Now, before we do wrap up, I just wanted to address our listeners. Now we talk about improving digital transformation in New Zealand and globally. Now, Shikar, do New Zealand organisations face different constraints implementing these solutions compared to global enterprises? Or are these issues exactly the same?

[20:28:01] Shikar Ramchand

I feel due to the size of our population and our capacity to deliver, it’s at a different capacity to some of the larger markets out there, like the USA or Southeast Asia, for example. New Zealand teams are typically very lean, averse to offshoring in some instances. And I feel that we’re also a little bit behind the rest of the world in implementing some of these solutions.

So, we get the feeling that we’re not the first to do it, so it should be simpler, but we also need to do it faster. And we should be doing it with a smaller team. And I feel that those factors contribute to a space that makes it more difficult for us to be able to deliver something successfully. And I think we see that in the rate of successful transformations that occur in New Zealand. 

And I feel again that a lot of what we spoke about today, you know, Gavin spoke about understanding those business processes for transformation. For example, I’ve been in a few spaces where that’s left for the end.

Let’s understand how we want to transform our business right at the very end, after we’ve already built most of the solution. For me, that’s counterproductive and counterintuitive as well; understanding how we want our solution to look when we begin is almost as important as implementing the actual solution. So, I feel that that’s also a very important point that we need to address.

[22:07:08] Reg Prasad

Nice. Thank you. Now, Gavin, from a QA manager or scrum master listening right now, something concrete to kind of put in there is start applying some of these principles tomorrow morning. What is one pragmatic step they can take with their teams?

[22:26:00] Gavin Cochius

I think communication is the key. Whether you’re a CSM, whether you are a QA, QM manager, whatever it is, it’s about communicating upwards and communicating laterally and downwards to the different audiences that you need.

It’s also being pragmatic because you’re probably the biggest change manager in that project, and you have a bigger influence than anybody else because your area drives the outputs, your area influences what’s going to happen from a user perspective, you influence what’s going to happen from an inception perspective, from analysis and design perspective, and quality engineering is probably the only space that’s across the entire life cycle of the project and involved and integrated fully all the way. 

So, you set the tone for what is going to happen, for whether your product or solution is going to be delivered, is actually well received and works, or is it going to be a lemon?

And to me, it’s a case of you can deliver the greatest solution in the world, but if it is not accepted by the end user, it’s a failure, because if they’re not going to use it and they’re going to use workarounds to actually mitigate to them not having to use the system, or they’ve got to go to a whole bunch of manual things which was automated before. It’s not necessarily a success from a project perspective or from a delivery perspective.

Even if there’s no bugs. It’s brilliant. Every feature that was there is there. Every feature works as expected, but if it is not being used, it’s pointless. 

So yeah, it’s just making sure that you communicate from the beginning what’s coming, what’s the measure of success? What’s the definition of done?

[24:15:18] Reg Prasad

Nice. And Shikar for you. From the executives who know that the delivery culture needs an overhaul, but don’t know where to begin. And you know, we’ve seen some firsthand ones of those. What is a first strategic step they should take this week?

[24:31:07] Shikar Ramchand

I think the delivery team should work as what it says on the label, as a team. So, let’s build that culture of assisting one another, of embedding that quality mindset throughout the team, and instead of segregating those teams with the different incentives, let’s incentivise the entire delivery team, the entire project team to work towards that quality outcome.

You know, when we think of something that’s quality, that screams quality. You think of your German sedan, or you’re thinking of your Italian sports car. The quality comes from every single component in every part of utmost quality. 

But then, besides that is also understanding that once all of those parts are fitted together, that we have a fully capable and quality end-to-end solution for that piece of build that we’ve done.

So, changing that culture will definitely contribute to us being able to deliver that quality outcome again.

[25:37:10] Reg Prasad

Thank you.

So, here’s some of the key takeaway points. So, collaboration culture, massive part of things. Communication and developing consistency as well. You know, key things. Thought process, but with quality in mind and having a success criterion that everyone’s working together on. I think these are all fascinating things.

And, you know, for our listeners, if you want to see the strategies we discussed today in detail, you can read more about the full guide by visiting assurity.nz and searching The Quality-First Blueprint. We’ll also drop a link in the show notes.

I’m Reg Prasad on The Quality Edge for Assurity Consulting, and we’ll see you in the next episode.

Thank you.