The Death of Agile


, , , , , , , , , ,

A master of martial arts who has spent his whole career studying and practicing is instructing a student. The student says to himself that he is not going to be “religious” about adopting his instructor’s teaching, that he knows best what will work in his particular case. So the master instructs, and rather than learn as much as he can, the student “just tweaks it” so that it “better suits him.” Now they are to have a match. Who would you bet on?

I just attended a webinar – well, I attended it until I couldn’t stand it any more. The title was “Dollars and Dates are Killing Agile!”

Given that the webinar was offered by a company that sells Agile tools and services, one expects that the title was intentionally provocative, and the webinar would really be a discussion of how to deal more effectively with costs, schedules, and other challenges to your Agile projects. Right?

The gist of the talk (as long as I could bear to listen to it) was that lots of companies have adopted Agile now, and it is generally failing. (How their marketing department let this webinar go on is beyond me.) According to the speaker, the main reason Agile is failing is that it has “not met its promise to enable Engineering to make product faster. Companies have been using it for three, six, or nine months, and they still can’t get more product out the door in less time.”

Wow. In my reading and experience of Agile in general and Scrum in particular, it takes three to nine months for a company to get good at it, and product development speed improvement is a later stage fringe benefit, not a major objective, so I hardly know where to begin.

Agile’s first deliverable is improved visibility into what you are doing that is not working well. The desired response is to stop doing what doesn’t work and replace it with things that do work. The actual response in many cases is to proclaim, “We are not going to be ‘religious’ about this Agile thing – even though we are drawn to Agile because what we are doing now is (profoundly) not working, we are going to alter Agile ‘to suit our corporate culture’.”

The company has again and again failed to produce quality product on time. It has sought Agile as a route to understanding and fixing what is not working, but it determines from the get-go that it knows better than Agile, than the people who have turned their whole careers to improving engineering process and have been so effective that they are changing the entre industry. You know so much more about improving engineering process than Ken Schwaber and Jeff Sutherland and the other thought leaders that have gathered around them that you don’t have to try their advice even once before changing it.

Before the student takes his first lesson, he commits to not listening to the master.

Be honest with yourself. If your company was really good at this, then what you have been doing would be working and you wouldn’t be investing in Agile. You have other things to do. Yet, instead of truly moving to Agile, you are chopping it off at the knees and then wondering why “it” doesn’t work.

The speaker’s assertion that lots of companies are doing Agile is without basis in fact. What lots of companies are doing is failing to adopt Agile, and they are paying a lot for it.

In addition to giving visibility to what is not working well, by providing provably working incremental updates with every iteration, Agile provides visibility on how much progress we are making (or not making). We can tell every few weeks. Actual data. If we like the results, we should further institutionalize what caused them. If we don’t like the results, we should again take the opportunity to learn what we are doing that is not working well. Everyone likes this idea in Agile, but what actually happens at your “Agile” company when results are less than desired? Would you call it learning?

In my view Agile’s main promise is greater predictability. Short iterations limit complexity and make estimation much more reliable. The visibility just mentioned gives early and frequent feedback. The ill-controlled mysteries of long term software development are replaced with the paradigm of empirical process control. We can tell what is happening, make corrections, even do experiments and take actions not previously anticipated – all because we are no longer flying blind. You might not like what the data say, but you get the data. This allows you to make business decisions informed by the actual rate of progress, and measures of increasing or decreasing levels of risk in the project.

Let me ask the Product Management folks and CEO frankly – will you run the business based on the capacity that is now plainly documented? It’s not easy because you are always going to naturally and understandably want more than that. As fellow employees and shareholders, your product development team wants more too! That is a natural tension in a healthy business. You do get to push on the product development team, and like all others, that team should always be working to find new efficiencies. But if the gap between goals and capacity is too large, that is a failure of strategy, not of the staff.

Agile does not make things magically faster. It is not a science fiction machine that manufactures time for Engineering while everyone else goes on unaware. Work is still work. Agile creates a pathway to improved predictability and quality in that work. Happily, improved speed also comes into play later, as a secondary consequence. But rather than feeling magically faster, Agile most likely will feel slower, and at first it will be slower.

Yes, it took X months to make a new widget, and we used to do it in 2/3 of X, but when we did it the old way, we had little visibility, low predictability, and we made really crappy widgets. This annoyed our customers and we then had to spend 30 to 60 percent of the time we wanted to put into the next widget on fixing the one we already released. That means we were going slower and increasing risk to the business when we did things the “fast” way!  We are so dissatisfied with the old “fast” way that we are moving to Agile to try to make things better. Let’s not then hold up the old way as the high water mark for success. The truly costly and utterly fake success of the old way is what we are trying to get away from. That is why we are trying something new.

With Agile we take “more time” to make a widget, but when we are done, we have a widget, instead of a pretense of a widget, instead of a shaky 70% of three widgets, we have one solid, actual, usable, valuable widget. We have a better managed business that can make more reliable promises to customers. We create the chance to make a product with fewer defects – fewer defects that alienate customers, fewer security defects that put corporate and customer assets and privacy at risk, fewer defects that create opportunities for competitors, and fewer defects to compete with development and test time for the next release. So, going “slower” at first will inevitably but almost invisibly yield better time to market later. More importantly, done properly, Agile supports a more predictable business that more efficiently makes a higher quality product.

Isn’t that the goal? Or, are we to continue the unspoken conspiracy between software makers and customers that crappy software on questionable schedules is better than good software in predictable increments? By the way, if we are going to continue the conspiracy, let’s stop blaming Product Development for quality and schedule problems, since these problems are occurring by design.

I worked at a company where the program management team led us from Waterfall to an alleged implementation of “Agile”. It was nothing but mini-waterfalls, and the impact was correspondingly mini.

I have introduced Scrum to several companies. In one case, we did it pretty much by the book. Both the staff and the CEO reported vastly improved satisfaction with visibility, predictability, and quality, and we just stopped feeling the need to talk about speed, although it was noted that productivity was higher. The staff felt like they were really accomplishing something tangible, of which they could be genuinely proud, and the CEO became confident of his team’s ability to produce. Is that how people feel at your company?

In other cases well intentioned but “evil” forces prevailed, we did not implement Scrum as it is taught, and we did not get good results. That is not an illustration of Agile failing, that is an illustration of companies failing to adopt Agile. I am no longer patient with discussions of how “we are not going to be rigid and religious about this Agile thing – we are going to adapt it to our company’s culture and specific situation before we even try it”.

This does not mean that Ken Schwaber and Jeff Sutherland are all-knowing and infallible, or that companies should superstitiously seek out and adhere to every subtle innuendo in their published work, or that no one else has any good ideas, or that business conditions won’t change and call on us to innovate again. It just recognizes the painful truth that Ken and Jeff are better at this than you.

Do it as described for a year, then when you have seen the benefit of it, then we can talk about whether or not we need to change it for local conditions. To be clear, this is distinctly my advice. Most of the Scrum coaches and others Agile advocates I have encountered are all too ready to capitulate without a fight if you want to change the lesson before you master it. But this has to do with fearing for their jobs and or avoiding conflict, not good advice.

It matters what we call things. If you want to say your company has failed to achieve its goals for product development speed, be my guest, and let’s do a root cause analysis on the gap between the goals and the results. If you want to say you failed to properly adopt Agile, I will applaud your honesty, and let’s do it right this time. But if you fail to adopt something and then say that it failed, you mislead everyone else. Don’t do that. It impedes everyone’s progress and could even kill off the very thing that can help us the most – if we will actually do it. If you are in enough pain to change from whatever it is your are doing now to Agile, then you darn well ought to make the actual change, not the pretend or partial version of the change.

If you want a more transparent, more predictable, higher quality company and product, then adopt a well-defined Agile approach like Scrum – the actual thing, as those who have mastered it before you teach it, not your own pre-altered version of it. Start with training for everyone and a full time coach. Do it for a year. Throughout that year, when you don’t get the results you want, assume you have more to learn and do it more by the book rather than less. If you just adopt the terminology, and those practices that don’t require much change, then “not much change” is what you are going to get.

If this decade witnesses the death of Agile it will be because careless people have murdered a friend by mistaking it for an evil monster that they witlessly created and dressed in an Agile costume. In the match you are in right now between you and your competitors, you are betting on you, and asking your customers to bet on you, so learn from the masters instead of ignoring their teaching, do a beter job, and win.