Date: 06 November 2020
New Zealand companies lose billions of dollars on failed IT projects every year. Failure is scary. As humans, we're wired to run from fear or fight it at all costs. But maybe it’s time to shift the narrative around the fear of failure and start to embrace it.
Failure is scary. As humans, we’re wired to run from fear or fight it at all costs. But maybe it’s time to shift the narrative around the fear of failure and start to embrace it.
Failure is an opportunity to learn and, when harnessed early enough, it can pay dividends in the long run. On the other hand, slow, drawn-out failure, like those in most IT projects, is insidious. By the time it’s recognised and acknowledged, it’s often too late to act. The question becomes ‘how might we harness failure to our benefit?’ How might we purposely engineer for failure, to learn ‘what to do’ through discovering ‘what not to do’?
Our safest way to fail is to test.
Test – test – test
Testing comes in all shapes and forms, so knowing when to apply the different disciplines can be confusing. To help, I’ll guide you through the different disciplines to help you understand which points of the project lifecycle they’re best suited to.
The conventional product lifecycle puts testing at the end of the project once the build is complete. At this point, its purpose is to check that the product functions as intended, which, let’s face it, is a hugely important part of the product development lifecycle. But, where this approach comes unstuck, is if it is the only type of testing carried out on the project. The issue being, we might be testing the functionality of a product that wasn’t even needed to begin with.
To nip this problem in the bud, the solution is as simple as asking the right question. Instead of asking ‘Is the product working properly?’, we should be asking ‘Are we creating the right product?’.
We faced this very scenario as part of our brief to redesign Sky TV’s website. From the business’s perspective, one of the key objectives was to improve the functionality of the cart and increase the sales conversion rate.
To sense check this, we started with a round of exploratory customer testing, observing customers while they interacted with the old website. It became clear that customers were using the cart to learn about the packages and game their way to the cheapest bundle possible. We also realised that the job our customer was trying to do was understand what content Sky had to offer, only to be met with a list of traditional channels, leaving them feeling lost and confused. Ultimately, these insights helped us hone in on the underlying problem – of how to hero the content that Sky has to offer and remove the guesswork and calculations from the buying journey.
The beauty of testing early and often is it creates the opportunity to pivot, evolve or if you’re brave, kill ideas before pouring time and resources into building things users see no value in.
As you start to move from concept to prototype through to build, the testing scope will start to narrow. As confidence and clarity is gained around what our customers need and the details are fleshed out, it will transition from being more exploratory testing through to more functional and technical testing.
A customer-centred approach to multi-stage testing
Discovery testing
As illustrated by the Sky case study earlier, we start by first trying to understand our customer and the jobs they are trying to do. It’s about understanding the challenges they face and opportunities they see, to define the core problem to solve. Traditionally, projects are kicked off by gathering business requirements to define the scope of work. But here we propose flipping it on its head, using customers and their needs to provide an objective set of requirements.
When we are clear on the needs of the customer, we won’t waste time down the line scoping, building or testing functionality that isn’t needed in the first place. Businesses will still have a set of objectives and requirements they are trying to achieve, but by exploring the needs of our customers first, we can ensure we strike a balance in a solution that’s best for both the customer and the business.
Prototype testing
Often customers will say one thing, but do another, so it’s crucial that we get early concepts in front of them as soon as possible to check we’re on track. Something as simple as paper prototype will help you elicit a phenomenal amount of feedback long before a line of code needs to be written.
We will often go through this process a couple of times, iterating, evolving the idea and progressing fidelity until we get to a place where we know it’s working for our customers.
User Experience (UX) testing
Once we’re confident that we’re designing the right solution for customers, we can sharpen our focus on the delivery of our product, focusing on how we can best execute it. Here we might come up with a few alternative options and A/B test them with users to see what they prefer. We can use customers as a sounding board to validate design and development decisions to check that we are still optimising the user experience. From our experience, the more we can involve our development team in this, the better. Because nothing beats hearing feedback first-hand – and it empowers our team to make smarter decisions and reduce technical debt.
Technical testing
Technical testing is an incredibly important part of the process. After all, there’s no point putting something in the market that doesn’t meet the description on the tin. We use functional and non-functional testing to validate that the product is working and performing as it should. By working in parallel with development teams, testers can raise discrepancies and defects that stray from the intended specification. And by testing as early in the build process as practical, test teams can identify issues early and avoid technical debt from building up and stopping insidious failure before it sets in.
As counter intuitive as it may sound, adopting an integrated testing approach will help set projects off on the right foot by preventing complacency and removing bias from decision making. Harnessing prototypes to check in with customers and getting real-time usability feedback will embed customer centricity into your development lifecycle. Together, this approach helps us make informed decisions and deliver a product that hits the mark. It’s not a case of using one type of testing over another, but instead a rock-solid process that delivers products that customers love.
Failure is an inevitable part of innovation and new product development, but we can control the scale of it. After all, failure is less painful when you extract the maximum value from it.
