Agile as a Lean practice in software development
I first learned about Lean Manufacturing when I was studying TQM (Total Quality Management) in college and that’s when I knew I wanted to work in quality control/assurance.
Since I was already interested in quality – and after doing some research – I decided to do my graduation project in Six Sigma as it was already spreading as a powerful technique for process improvement in manufacturing and business processes. I spent some time trying to figure out how I could link Six Sigma to the software industry until I came across Lean Six Sigma. I started reading more about Lean manufacturing and Lean Six Sigma which then inspired me to do my graduation project in Lean software quality management as I thought this would be applicable to software, as well to manufacturing and other businesses.
I started my career in software testing around five years ago and, in that time, I’ve only become more interested in methodologies that improve software quality. I’ve read a lot about Agile methodologies but only started working on an Agile project a year ago. And as a fan of Lean, Quality and Agile, this article aims to link them altogether.
The world around us is changing rapidly and software development isn’t slowing down! We’ve probably all heard the term ‘IT transformation’ within the organisations we’ve worked for. But what is ‘IT transformation’ and what does it have to do with us?
Generally speaking, ‘transformation’ is best defined as something that is changing. ‘IT transformation’ differs from one organisation to another but, whatever the change is, organisations need to have the ability to respond to that change.
I was part of a major change that occurred on the software development level in one of the organisations I worked for – setting up a DevOps environment. There seemed to be a lot of resistance from some teams and management parties against any major change and it took a lot of time and effort to bring different mindsets on board. I quickly realised I wasn’t the first to try and implement such changes. Although we were falling behind without having such processes in place, people didn’t seem to want to get out of their comfort zones as it required learning new skills and techniques.
Take Nokia as an example. “Why did Nokia fall from industry leadership?” asked Julian Birkinshaw, Professor of Strategic and International Management at London Business School in the Fortune article ‘Why corporate giants fail to change’. “The failure of big companies to adapt to changing circumstances is one of the fundamental puzzles in the world of business. Occasionally, a genuinely “disruptive” technology, such as digital imaging, comes along and wipes out an entire industry”.
Organisations are doing their best to adapt to the rapid changes in the market. Having a powerful approach in place will definitely help improve its ability to respond quickly and, even better, continuously learn and improve through those changes.
‘Lean’ and ‘Agile’ are two of the most valuable approaches I’ve come across that work towards continuous improvement. Despite the fact that ‘Agile’ was coined for software development (in the Agile Manifesto), while ‘Lean’ – which usually refers to Lean manufacturing or Lean production and is used within a manufacturing system – a pro-Lean subculture is now emerging from within the Agile community.
In my experience, most IT organisations that go through IT transformation are usually trying to apply ‘Agile’ techniques and some can lose track of its value. In one Agile project I worked on, we had a really good team and were doing our best to follow an Agile methodology. But, at some point, we started to see the Agile ceremonies as a waste of time. Retrospectives got boring and we stopped getting value out of them. It was like we totally forgot the objectives behind the meetings – and even our sprint goals were not being met anymore. In fact, it started to look more like a waterfall project than an Agile one! I’ve also came across team members who weren’t adaptable to Agile; they kept working in their own way and wouldn’t really collaborate with the rest of the team in the Agile activities.
Most teams incorrectly use the Agile principle of ‘be flexible’, which leads them to fail in the implementation of Agile methods as designed. So how about we try to use this rule in a different way? What if we combine Lean with Agile and define how the two methodologies could work hand-in-hand to achieve better outcomes?
To begin with, let’s take a high-level view of ‘Lean’ principles before diving into how we could get the most out of it in an Agile world. Lean Software Development (LSD) is a translation of Lean manufacturing and Lean IT principles. It originated in Mary Poppendieck and Tom Poppendieck’s book Lean Software Development: An Agile Toolkit.
Below is a quick review of the Agile Manifesto:
Agile focuses on satisfying the customer through early and continuous delivery of valuable software by welcoming changing requirements, bringing business people and developers together daily to build trust and support within the team. Agile teams are motivated, self-organising teams that work efficiently and effectively towards continuous delivery and improvement.
So we can see that both methodologies are based upon similar principles that work towards continuous improvement. And, with the right ingredients, we could combine them and make Agile more Lean and get a more beneficial practice.
Let’s dive deeper into Lean principles and define how we can relate it to Agile.
Waste is anything that doesn’t add real value. If we take ‘buggy software’ as an example, we’ll see that it creates a lot of waste – in added testing, additional reporting, re-coding or fixing the code, re-testing and even additional regression testing. In Lean Software Development, Tom and Mary Poppendieck identified some of the wastes relevant to software development:
From the Agile Manifesto: the ‘Individuals and interactions – over processes and tools’ Agile principle encourages the team to be more effective and efficient. The team regularly reflects on how to become more effective and adjusts accordingly.
One of the most common Agile practices that plays an important part in identifying and eliminating waste is the ‘retrospective’. The team meets after every iteration to discuss what went well, what didn’t go so well and what could be done better in the next iteration. They then agree which actions are to be implemented to ensure continuous improvement.
The following image shows the results of the ‘Eliminate Waste’ retrospective that was run, including the different items identified and the actions to be implemented in the next iteration.
‘Value Stream Mapping’ is a Lean technique used to identify waste and end wasteful activities. It is simply a mapping tool that gives you and your team a clear understanding of the current state and allows you to visualise the future state. Software development often suffers from delays, high costs and an inability to meet customers’ needs. VSM can enable organisations to focus on the value-added activities and identify and remove any non-value added ones.
In an interview about VSM in Lean Software Development, Mary Poppendieck said, “We start by asking people to draw a Value Stream Map. You start with a customer problem-need request, and you go to where that request is filled. You draw a map or a timeline of everything that happens from the time the customer request comes in the organization until the customer has their problem solved. You lay out the activities there and how much of the time are you really adding customer value and how much of the time is just sitting there contending with other work that has to happen”.
The next step is to point out sources of waste and to eliminate them. There are many Agile development methods that could help your team eliminate the identified waste:
Let’s have a closer look at ‘Requirements backlogs’ as an example of a software development activity. As a start, you shouldn’t put too much thinking into every requirement in your backlog. You only need to focus on what you will work on in the next two iterative cycles. Get the top priority requirements ready. Your stakeholders will let you know the really important requirements that should be moved to the top of your backlog. Backlogs should be manageable and, if any of the requirements are unnecessary, just get rid of them. Do the same with defects or technical debt backlogs. Keep them limited to what’s really important and be realistic about what you can achieve.
Without having the visibility of the knowledge that’s flowing within the team, it’s easy to fall behind due to a lack of a certain skill or requirement.
On my current Agile project, we have developers who always work on the back end and others who focus on the front-end tasks. It was only when they moved out of their comfort zones and started sharing knowledge, code reviews and started pair programming that we became more productive.
Having live documentation that’s well maintained or training that can grow the team’s skills adds value to the team knowledge. Agile is carried out through short iteration cycles which helps speed up the learning process. It also encourages team members to support each other in getting the work done – with testers doing some business analysis work, BAs picking up some testing work and so on.
Decide as late as possible
In software development, change is the only constant and decisions are usually associated with uncertainty. Things are unlikely to change if decisions are made too early – which is why it’s better to delay decisions until they can be made based on a stronger foundation. The further into the process that critical decisions are made, the safer it is in a Lean world as more information is available. This way, you have a better chance of easily responding to change and building the right product.
Agile processes harness change for the customer's competitive advantage and, to do that, the team must have the ability to adjust and adapt to change, even if it means totally changing how to do things within the team.
Agile development methods break development work into small increments that minimise upfront decision making and planning. Below are some of the approaches that could be used to facilitate regular decision making.
Daily stand-ups – short meetings that involve all the team members including the product owner to track the progress of the sprint – are the right time to agree on decisions or actions.
Deliver as fast as possible
Technology is rapidly changing and the faster the product is delivered, the sooner we can get feedback – and you need to be fast to survive in the current technology market. One of the principles of the Agile Manifesto is that working software is delivered frequently – the fact that Agile is based on regular iterations works greatly towards providing faster delivery, receiving feedback and incorporating it into upcoming iterations.
Agile processes also promote sustainable development which enables the team to achieve and maintain an optimal development pace indefinitely.
Build integrity in
Customers always want a better experience. So, as a development team, we need to work towards achieving that through better understanding of what the customer needs.
Face-to-face conversation is the best form of communication, as mentioned within the Agile principles. Effective communication promotes a better understanding of what the customer needs. Increasing the customer feedback sessions is one form of such communication, as well as performing daily stand-ups.
Continuous attention to technical excellence and good design is also one of the main principles that lead the team to better quality and customer satisfaction – the more features are added to the code base, the less maintainable and flexible it becomes. Code refactoring is one of the activities that helps improve technical requirements and helps keep simplicity and clarity in the code. Automated testing can also be a means to more reliable quality and helps reduce defects.
Empower the team
Lean principles strongly promotes respect through responding to people, listening attentively and hearing their opinion. If you have a happy team, you probably have a successful team. It’s all about respect and trust. Managers need to trust the team’s capabilities in delivering the work. Motivated individuals are always more productive.
Agile projects are built around motivated individuals who thrive on trust – business people and developers having to work together daily throughout the project to be more efficient and effective in conveying information. The best architectures, requirements and designs emerge from self-organising teams – easily achievable when team members take part in the decision making. Put simply, give the team the environment and support they need and then trust them to get the job done. The Agile values below can help create an adequate environment between developers and businesses.
See the whole
A Lean organisation seeks to optimise the whole value stream, not just individual functions or teams. Most of the issues we face in IT organisations are a result of teams being built around skill sets rather than products or projects – meaning individuals don’t have an overview of the high-level value they’re delivering. The team needs to work to the same principles and goals to satisfy the customer through early and continuous delivery of valuable software, giving the team a greater sense of ownership which leads to commitment and better quality. “Think big, act small, fail fast; learn rapidly” – Lean Software Development, Mary and Tom Poppendieck.
To sum up, there’s really no good or bad way to do things in a Lean/Agile environment as long as you make the right decisions that follow in your project’s context. Iterate, learn and improve as you go. By using the above principles and practices, you can help move your Agile engagement to a more Lean one and get the most out of each approach.
There are many approaches, principles and methodologies out there. Explore and use what you think fits within your team. You don’t need to label every technique you want to implement. Feel free to mix it up, add your own thinking and spread your wings. The sky’s your limit!