Sustainability and Plateaus in Agile Development
I’ve been thinking about how to keep the project pace sustainable a lot lately (my wife says I need a hobby). My experience has been that agile projects generally demand a lot more focused engagement by the team than plan driven projects. This is both good and bad. The short, intense sprints enable some remarkable productivity, but they also come with a cost. Without a doubt, the highly collaborative environments found in most agile projects require a lot of energy on everyone’s part. Eventually it can take a toll on everyone. I am fortunate enough to work in an environment with lots of agile teams, and I think I see a pattern emerging.
Here’s how it goes: The team starts off slowly, builds up some velocity, works out some issues and is soon a fairly highly performing unit. But the challenges don’t stop coming. Some of this is good, challenges are what make the work interesting. However, some of those challenges have an emotional component – and they can be real killers. For example, problems getting the product owner to engage, internal team conflicts, defending the team from micromanagement by upper managers, etc. All of these challenges are quite normal, and frankly they are to be expected in any agile environment. After maybe a year of these challenges you might start to notice the scrum master starting to look a tad bit crispy around the edges (is that a nervous tic or is he winking at me?). The team might start showing up at 10 AM rather than 9. Give them another six months, and while there may have been many successes, they all start to get a bit of that “thousand yard stare”.
One of the benefits of highly transparent environments like those found in agile teams is that problems are often raised more often and more visibly. This is often touted as being good for the overall health of the project, but it can also be hell on the team. Bringing conflicts out into the open is great, but it’s also exhausting. Continuous improvement is great, but it’s also a lot of hard work. I think that it contributes to a very real feeling of burnout.
Recently I realized that I’ve encountered this phenomenon before: back when I used to be involved in amateur weightlifting. Burnout is a very common phenomenon in many different sports. It’s symptoms can range from the very simple (physical exhaustion) to the more subtle (such as a lack of enthusiasm, depression, etc.). And in my opinion, it is probably the single most difficult challenge to face when you are trying to push yourself (or perhaps your team) to the next level of performance – mental or physical.
In sports there is even a terminology used to describe advancement through these different levels of accomplishment: “I’ve hit a plateau” A flat spot, an inability to make further gains. No matter how hard you throw yourself at the problem, nothing you do seems to lead to an improvement. It’s as if the body and the mind have conspired to say, “We aren’t going any further – we’re gonna take a break.” So what do you do?
Maybe you thrash for a while, trying to continue to make those magical gains that you first experienced (you want that endorphine rush!). But at some point it becomes apparent that progress just isn’t going to continue forever. The day to day monotony takes over and you simply deal with the basics: you show up, you do your work, you endure the pain, you go home and you rest. I think there is a very close parallel between these plateaus that we reach in athletics and the plateaus that we reach as software development teams.
With software teams, the challenges are a bit different, but the “plateau” is the same. We’ve got our continuous integration working. We’re using pair programming. We have beautiful burn down charts showing our progress – and yet we still haven’t reached our goal of delivering fully tested software to our customer every 2 weeks. What’s the matter? I think this is the point where we may lose a lot of people who are trying out agile methods for the first time. They think, “We tried it the new way, and it felt good for a while, but in the end it was just more of the same hassles.” Same issues, different methodology. The fundamental problems don’t change.
I’ve seen this happen in weightlifting countless times. Go into any gym the week after New Years day. The place is packed! People are feverishly pounding iron like there is no tomorrow! This is going to be the year that they get in shape! Here comes that six pack! Then the inevitable happens: toward the end of January the gym starts to get less crowded. By mid February the place is almost back to normal again. No surprise I guess, it’s just human nature at work. But what about those guys who are still there in June? How do they manage to keep it up? What is it that makes them different?
Now, in my own experience there are two factors that can make a huge difference for the guys who pound it out day after day: they have almost maniacal discipline, and they have a plan.
After hanging out with the die-hard weightlifters for a while, I discovered that they all had different strategies for pacing themselves in order to hit a long term objective. Say for example, that you wanted to bench press 300 pounds, but right now you were only able to bench press 275. How do you create a long term strategy that will allow you to gradually work your way up to the desired weight?
I think the first strategy was discovered by the ancient Greeks – you simply increase the weight in tiny increments until you reach your goal (actually there was this guy who was fond of lifting a calf every day until it was full grown – don’t ask me for the details – it’s all Greek to me). That strategy of building in tiny increments is a start, but it doesn’t really model the way we typically make gains. Typically athletes make gains after long plateaus or stable periods in their training. Then it sort of happens all at once. One day you are pounding the same weight as yesterday, then BOOM! the energy comes and you are ready to go to the next level.
It suggests that the body needs time to acclimate to each level of performance. That there is no such thing as truly non-stop continuous improvement. You improve for a little while…and then you plateau… You might want to go further, but you are just not ready yet – and there is no forcing it. However, you can plan for it. That made me wonder, are there plateaus for teams? How do we get past those plateaus?
Weightlifters and other athletes use a strategy called “cycling” to try and simulate the highs and lows of normal development (physical development that is…). It’s a plan to enable them to reach their “Peak” without burning out. Cycling involves gradually working your way up to a heavy weight, and then backing off. The next time you start a cycle, you start with a baseline just a few pounds heavier and go a little bit closer to your peak from there. To do all of this, you need to do a fair amount of planning. If I have a competition three months away, then I might put together a workout that allows me to “Peak” or cycle about three times (once a month) before the big event. I calculate the weights that I need to lift on any given day, then it’s just a matter of showing up and doing the work. That’s where the dedication comes in.
The cycles or plan that I have created give me a set of milestones or deliverables to work with. If I discover that one month from now, I’ve added 20 pounds to my bench press – hooray! I’m ahead of schedule. Or I may find out that I’m 5 pounds shy of my goal. Then I adjust my plan for the next cycle and keep moving forward. By the time the day of the competition arrives, I should have had enough experience with my previous “Peaks” or deliverables that I should be fairly confident of what I can accomplish. It shouldn’t be much of a mystery. To me, this technique bears a strong resemblance to the short, delivery oriented iterations in agile – and even to some of the agile planning techniques.
For me, the hardest part is not the planning, but the discipline to consistently execute. It’s that maniacal discipline that I was talking about – and it ain’t easy. There are so many different excuses I can come up with to skip some apparently trivial part of the plan on any given day. The really hard part is to show up and do the work, and report the results the same way – every single day without exception – regardless of whether it is raining, snowing, you have a cold, your mother-in-law is in town – whatever. You get the idea. I think that all too often I have used these excuses on my projects too: Skip the stand-up meeting because we all really “know” what’s going on, forget to report burn down time accurately – I’ll catch up tomorrow, delay creating tests because the code doesn’t *really* need it. You name the lame excuse – I’ve used it.
I see potentially strong parallels in the efforts that development teams and athletes might use to pace themselves whether they be weightlifters, runners, swimmers, or golfers (wait…scratch that, golfing is *not* a sport). So if you find yourself unsatisfied with the pace of your teams progress, if you are wondering if you will ever find a way to meet the next challenge – try looking a the way people in other domains pace themselves. I think there is a lot that we could learn from other disciplines such as athletics. Maybe your development team has reached a plateau – that’s perfectly natural. We talk about continuous improvement, but other disciplines learned a long time ago that it’s much more complicated than just changing a little every day. For example, some teams need to reach a certain level of collective maturity before they are ready to take on something like pair programming. If you try and force them to do it too early, it’s like trying to teach a pig to sing – it frustrates you and irritates the pig.
Speaking of pigs, that brings me back to Scrum. Maybe we need better mechanisms to help assess where teams are in their development, so that we can provide the mentorship and/or the tools they need to make it to the next level. Whatever the answer is, I think that I learned a few valuable lessons from my own athletic experience with regard to keeping the pace of a team sustainable. I would invite you to consider your own experiences outside of the software domain and see if there are similar parallels that arise for you.