Why Scrum does not work (and what can we do about it)
Scrum is the most popular agile framework. Period. If we look at VersionOne 11th State of Agile report, Scrum is practiced by 58% of respondents (68% if we sum up Scrum and Scrum/XP hybrid). Among scaling frameworks Scrum dominates: while Scrum-of-Scrums has 27% (1% less than Scaled Agile Framework), Scrum-of-Scrums, LeSS and Nexus combined have 31%. I believe that there are two main reasons for such a high degree of popularity. First, Scrum is simple to understand and comes with a clear recipe — Scrum Guide. Second — it works by helping a team to succeed in value-driven agile delivery.
So it looks like within the first paragraph I have reached a conclusion that contradicts my initial claim made in the post heading. Well, no. Scrum really works — but only if you succeed in its adoption. I do believe that Scrum is extremely powerful, and its power emerges out of tight interconnection of Scrum roles, artifacts and practices. Its holistic nature emphasized in the Scrum Guide itself: “Each component within the framework serves a specific purpose and is essential to Scrum’s success and usage.” This means that the one and only way of getting the full benefits of Scrum is to literally follow the Scrum Guide. This does not mean that you cannot get at least some benefits from using part of Scrum (often referred as “Scrum, But”), but:
a) one should not call this thing “Scrum
and
b) actual results are highly unpredictable.
Why would you want to implement ScrumBut? Well, assuming that you are not ignorant, the main reason would be “It contradicts our organizations’ culture.” Among the main challenges of Agile adoption the statement “Company philosophy or culture at odds with core agile values” is at the top (and rising) for at least 6 years with 63% last year. So in Agile (and Scrum) adoption the key question is “How can we change organizational culture so that it will embrace (or at least will not contradict) Agile values and principles.” One possible answer to that question is known as Larman’s 5th Law of Organizational Behavior: “Culture follows structure.”
If you want to really change culture, you have to start with changing the structure, because culture does not really change otherwise. Make people act differently, they will adapt their mindset so that this new way of doing things will be aligned with their way of thinking about their actions. This is what Scrum does (when fully implemented) — it creates a structure that forces organization to become Agile.
I believe that Scrum works exactly like that — not only changing the way things are done, but changing the mindset and belief system of people involved. But (and it’s a big butt!) putting people in a new environment with new rules is painful*, and so is changing one’s mindset. And when choosing is between two painful paths, people tend to pick the one that’s associated will less pain. Or at least the one that they think will bring less pain**.
This leads us to understanding a key reason for Scrum failure: it just requires too much change (and too much pain) to do Scrum. If an organization’s culture is close enough to Agile, implementing Scrum works like magic: it will add the necessary tension that will force cultural changes, leading to the success of Scrum adoption and Agile transformation. But as soon as the distance between organizational culture and Agile becomes too big, enforcement of Scrum framework does the opposite. Now it’s not just structure vs culture, it’s structure and culture and pain aversion combined. And we all know how much energy people can direct on pain aversion.
So Scrum is doomed to fail, and there are basically two ways in which events can unfold after that. One is stepping back to what is, claiming that “Scrum (and Agile) does not fit our company”. The other is changing Scrum so that it won’t bring so much tension (and pain). The latter leads to all sorts of “ScrumBut”s and Cargo Cults, and best case will lead to frustration, irritation and unproductive behavior (and worst case it can lead an organization to total disaster).
So Scrum does not work when forced on an organization with a poorly compatible culture. Does this imply that Scrum is bad? No. But what it does imply is that literally following the Scrum Guide is not suitable for every organization, and the Scrum Master has to carefully think about a way of Implementing Scrum. This could possibly mean that we do not do Scrum from the very beginning, and we will try to gradually approximate our structure to Scrum, and avoiding a “get it right from the first time” attitude.
This is the first in a series of posts on my perception regarding Agile coaching, it’s challenges and faults. By no means am I claiming that I have all the answers. Rather, I hope that by writing this (and by reflecting on your feedback) I could think through and maybe find some plausible answer on the main question of agile coaching: How to help an organization make its way through successful Agile transformation. In the next post I will consider some of the questions that a Change Leader should consider designing a plan for Agile transformation.
** Sometimes this comparison can be very tricky, because you should take into account not only the current pain, but although we take a strategic view, and compare expected pains for some wider timeframe. But this does not change the core claim