Scaling Agile is a waste of time.
There, I've given you a bold statement.
Recently, I've been involved in a number of projects and conversations about agile scalability. Everyone wants to scale agile. This implies making agile "bigger". If I have an agile team of 40, why can't I have an agile group of 100? Why can't my multinational corporation be one big agile unit?
Well, scaling agile is like scaling a cherry. If you have a cherry and you like it, maybe you want your whole family to taste that cherry. Cherries are great! Everyone wants cherry.
The obvious solution, then, is to go get a huge cherry big enough to feed your whole family. One massive cherry.
Obviously this isn't how it works. Cherries are of a certain size and consistency. An eight pound cherry would not do.
You appropriately have a lot of cherries and you eat them one at a time.
Your agile teams are working because they are of a size to allow the benefits of agile methods. They don't necessarily need to be bigger.
Agile supercharges team performance by increasing communication through daily stand-up meetings, frequent reviews, rapid release cycles, client representatives and other means. If done correctly, this increased communication gives teams a stronger understanding of the context of their tasks, a focus on completion, and paths of escalation when things go awry.
Agile methods also manage expectations by agreeing ahead of time what work will be completed in an iteration and getting buy-off on the current work plan as well as work just completed.
Internal Prioritization is a strength of Agile (while actual estimation and prioritization is often poorly done). Agile methods are great at coming up with games / methods to estimate and prioritize work. The goal of which is to get the most important things done first. Later, if schedule slips, the items not completed are of the least value.
Where Agile Hurts
Out of the box Agile is great for small teams and, in my opinion, vital to the coherent creation of software.
Historically, Agile methods have trouble integrating into larger environments or into groups that would like a higher level of organizational maturity.
Agile's first hints of trouble tend to center around the estimation and prioritization element. Estimation and prioritization schemes in agile often fall short because - while tasks may meet their rough estimates - the team rarely has a good idea what their actual throughput is. In other words, while they understand velocity (number of user stories done in a given iteration), neither teams nor management have a handle on the variability of velocity from iteration to iteration.
This variability makes it very hard for the programming groups to communicate a predictable level of service to the rest of the organization. The organization wants to be able to say "If I give a request to my programming group, I will have results in about n weeks."
If about means give or take 2 weeks, that's not so bad. If it means give or take 2 months, that is a problem.
In order to have a good internal corporate relationship, communication of throughput throughout the organization is vital.
But it doesn't stop there. The organization itself is a valid object. Agile rhetoric goes Shakespearian and nearly advocates "Kill all the <Managers>." That's not appropriate. Companies have value streams, without which there would not be a company.
Agile deals with this by having client representatives who are ideally in the room. This implies the value stream can be reduced to a person or a single actor. A stream is a lot of drops of water. Often a client representative is merely a drip. A drip was more than we had before Agile, but is still insufficient for organization-wide communication.
Out of the box Agile, doesn't have value stream awareness. It doesn't fully appreciate where requests came from and why. This creates several issues down the line. Not understanding how value flows through the company means that we don't understand the value needs of specific parts of the company.
This impedes interdepartmental communication. Some departments are overlooked, others get reports based on burn-down charts or features completed and not according to their value needs.
An example is necessary:
Ajay, The CEO of Bumbr, an e-commerce site selling umbrellas, sets out a three part initiative to Increase Sales, Decrease On-line Customer Wait Times, and Increase Restock Speed. Everything to Ajay is about stock turnover.
The C Level officers of Bumbr turn this into six Programs. IS1, IS2, WT1, WT2, RS1 and RS2. The CFO and CMO are very interested in IS1 and IS2, the CTO is obsessed with WT1 and WT2, and the COO is all over RS1 and RS2.
Business Analysts, Program Managers, and Project Managers start to work, figuring out exactly what feature sets are needed, what common infrastructure these share, what the user stories are, which teams will handle what, what is of highest priority and so on.
The BA's are at one level of granularity, Program Mangers another and Project Managers yet another.
Then we get to programmers, testers, integrators and so on.
So, by the time programmer Portia Galbraith gets a few tasks for this iteration they look like this:
- Incorporate changes in database schema
- Add ratings to customer satisfaction report
- Create internal data distribution engine
She knows exactly what she needs to do. The user stories these tasks are attached to provide context, as does her daily stand up meetings.
But suppose Ajay is walking down the hall and has this conversation:
Ajay: Hi Portia, what are you working on?
Portia: Well, I'm working on the database schema, some satisfaction report improvements, and the distro engine.
Ajay: How does that help Increase Sales, Decrease On-line Customer Wait Times, or Increase Restock Speed?
Portia: Please don't hurt me.
By the time things got down to implementation, they were so far removed from the original initiative that the context had been lost. Portia could guess, but she really can't say for certain.
But ... the secret is ... the context is there, people just were not tracking it. This is the nature of traceability. Traceability is the Rosetta Stone of business. You need to be able to track corporate activities through the value stream and in the language of value needs.
Let's say that Portia's tasks 2 and 3 were part of User Story 48. User Story 48 was something like:
User Story 48: As an Umbrella Buyer, I want to rate my favorite umbrella purchase so that I can get Bumbr points.
Bumbr points are something that directly relates to sales. More points = discounts = sales. Bumbr points at a certain level also get expedited shipping and processing.
So User Story 48 is part of IS1, WT1 and WT2. These have direct impacts on Increasing Sales and Wait Times. But, due to breakdowns in traceability, there is no easy way to find this out. Ajay shouldn't have even needed to ask.
Value needs are being met and everyone's happy! But, no one knows it. And if you don't know you're happy - then you're probably not happy. Not being able to communicate the value has decreased the value actually created.
Traditionally, to find all this out, we'd usually need to pour over documents to find out which task related to which user story and to what project(s), and then programs and then initiative(s). What the documents didn't tell us, we'd have to directly speak to people throughout the value stream. That's time consuming, boring and valueless to everyone involved. It is waste.
And it's frequently what happens. No traceability means nothing to communicate. The lack of communication upwards is reflected in information requests that take time to fulfill.
Traceability - the tracking of what relates to what - is not, as some Agile proponents might say, overhead. It is risk mitigation and streamlining. Keeping an accurate, digital record of these relationships leads to automated reports distributed to people in their value language.
The COO can watch his automated dashboard daily to see the progress on WT1 and WT2. Task level completion is communicated immediately at the program level (the level of value to the COO).
So the COO sees progress. Progress makes people happy. And, information makes people happy. Even if WT1 is having an issue - the COO is more comfortable knowing about it. Even if no action is required.
Happy people are more likely to be collaborative. Which is good because the hardest thing about making Agile work in a large or highly mature environment is setting limits to work-in-progress (WIP). WIP limits spark intensely collaborative situations. (My counterpart, Corey Ladas, has written extensively about this.)
The upshot of this is that if you overfeed someone work, they become less productive. Teams have an appropriate throughput that is very maintainable and suffers much lower variation when work is introduced into the queue in appropriately sized chunks.
WIP limits are excellent for maintaining team flow, but their side effect is perhaps better than their intended effect. WIP limits force the rest of the value stream to respect the programmers and the value of work.
Wow!
Here's how it works:
Say you have a WIP limit of 10 user stories at a time and you have an initial queue of 7 user stories. As you start to work on a user story, the people upstream (business analysts, managers, the CMO, whatever) can add one story (and only one!) to the queue. They have to choose between all the stories they have. That means they have to really prioritize (Corporate Prioritization).
Through prioritization some wonderful things emerge.
1. Upper management understands how software development works (it's no longer a black box).
2. User Stories get tighter and better defined (helping decrease variability)
3. Waste is filtered out of the system at the front end - where the cost of change is very low - and not after something has been fully implemented
4. Departments in the company negotiate for value (as opposed to demanding it from you at others expense).
and on and on.
Wrap Up
Out of the box Agile misses key elements for being applicable to large or mature environments. Scaling Agile is an invalid concept as it tends to focus on making "everyone agile" as opposed to incorporating agile teams into complex environments. In other words, this isn't a scalability issue, it's an integration issue.
Disclaimer: This blog post is a blog post. A very large book can be written about these ideas (okay, so we're working on that too). What I'm saying here is, don't expect this informal blog post to be a final product for how to create an organization that supports Agile teams. There's just enough here to prove the point.
At Modus Cooperandi, we like to remind people that Knowledge Work is Perishable. If you don't use it, you lose it. You can lose it by neglect. You can lose it by forgetfulness. You can lose it because it's relevance expires.
They focus on actually being an organization. The organization can set the culture. They base activities on their mission. They define the groups infrastructure.