Scrum of Scrums Is Dead - Emerging Coordination Practices at Scale

Cross-team coordination is decided by the teams.

Prefer decentralized and informal coordination over centralized coordination.

 

Needless to say, that this article is heavily inspired by the principles behind the LeSS framework.

Scrum of Scrums is ... Born Dead

Scrum of Scrums ("SoS") is probably the most well-known coordination practice recommended for scaling Scrum beyond a one-team setting.

 

I've seen many variations of SoS and in fact recommended several of my clients to use it to help coordinate dependencies among their Scrum teams.

 

We all learn and I had to admit - it never worked too well. It was of course better than nothing... but those meetings were never fluent enough and fun to be at.

 

Still the idea somehow it brilliant - if Scrum with its feedback cycles work so great on the team level, then when having teams working together the same framework shall be applied on the higher level.

 

A great idea. And that also implies we need to have Scrum of Scrums, Sprint of Sprints, Backlog of Backlogs, Review of Reviews and Retrospective of Retrospectives... And don't tell me you haven't just thought of ScrumMaster of ScrumMasters.

 

Now you see why this thinking is going all wrong. 

 

What looks great on paper might not eventually turn out to be a good idea in practice. Remember, we are in a domain of complex things where it is not possible to draw conclusions on the future from thought experiments and past observations.

 

And in fact, after years of experiments and observations we (the agile community of practitioners who are still uncovering better ways...) are saying louder than ever before: Scrum of Scrums is not a recommended practice to use at scale. In fact, it should be your last resort, not the first one to start with.

 

Centralization of coordination implies the same "old thinking" that decisions need to be made "on top" based on the information gathered from the "bottom". That's blocking self-management that is the best known way to manage complexity. Such thinking results in over-complicated process based on compliance. Versus lightweight processes emerging from the workers wanting in solving specific issues at hand.

 

But what to do when you're in a multi-team (or scaled) environment with inter-team dependencies?

 

Let's see.

Coordination over Dependencies

Compare these two statements:

  1. We, the team-X, have a dependency on the team-Y.
  2. We, the team-X, need to coordinate with the team-Y.

Same situation. Two different mindsets.

 

In the first example when a team-X has a dependency, they becomes passive hostages of circumstances they are in. In the worst case they finish "their" part of the thing and then wait for the team-Y to deliver.

 

In the very worst case - they just don't start until the other team finishes. And if the team-Y is also "blocked" by being dependent on a team-Z...

 

In the second statement: "we need to coordinate" means an active step into dealing with the situation at hand. That means challenging how things are and making agreements with the other teams. That's an adult behavior. That's what we want to see in mature Scrum teams.

 

So let's stop calling things "dependencies". And let's call them "a need for coordination". That's a game changer.

Coordination and Integration

We need to do more Jenkins than Jira.
Ron Jeffries and Chet Hendrickson

Craig Larman and Bas Vodde in their last book on Large Scale-Scrum define a set of recommended coordination and integration practices (surely, the Scrum of Scrums is not listed as a recommended one).

 

Continuous integration of the overall product increment is in the heart of coordination practices at scale. Why? Because it creates a demand - a pull for coordination.

 

So integration pulls coordination.

 

And why do we coordinate? We coordinate in order to have integrated solutions.

 

So the result of coordination is integration.

 

As you can see these two channels (coordination and integration) strongly overlap, causing, supporting and enforcing each other.

Communicate in code

Continuous integration is integrating much more often than you think.

Craig Larman

And this doesn't mean writing comments to unclear code portions...

 

The modern version control systems (gitGitHub, BitBucket) and continuous integration tooling (like Jenkins, CircleCI) foster communication. If configured and maintained properly your CI server becomes your main enterprise signalling system. These signals visualize coordination that is needed to happen, for instance when a build has just failed because of red tests.

 

Of course, there are ways to abuse the tooling, no matter how great it is. The most common way that I see is following the guidelines of the "git flow" by introducing feature branches where each feature sits in a single branch until "done". 

 

Guess what happens next?

 

 

There are ways (life-hacks) to make your CI run parallel and independent builds for each of the feature branches, but I rarely see how this creates a need for coordination and prevents upcoming integration conflicts.

 

In fact this malpractice creates a collection of really bad engineering habits.

 

Recently I spent a week at a client digging into their build server. The guys did intensive pairing. But each of the pairs used to sit in "their" feature branch for days before merging. Their build server did a magnificent job of building and reporting on each commit in every  branches (I counted 35 branches for 7 pairs of developers). But since the branches were "owned" by a pair, no other pair cared if those branches are red or green. If no one cares, soon you'll stop caring. So 80% of builds in all branches were red.

  

At this firm since the red became the new norm, all pairs continued committing to their branches, producing continuously red builds - and that's a really bad habit. 

 

And that is a total opposite of continuous integration. They say: continuous integration is not a tool; it is a practice, a mindset. I can't agree more.

 

 

Then once a feature was finally "done", a pair spent few hours (and sometimes days) tweaking the tests to be green again. Removing "old broken tests" was not an exception.

 

So communication in code implies having the culture of shared code ownership across all teams (and all components and software layers) involved in development of the product.

 

That's definitely not a new practice at all. Still it is becoming even more important at scale as it creates a need for coordination that glues all teams, pairs and individual developers together better than any other centralized and top-bottom way of coordination (I imply of course the Scrum of Scrums).

More Decentralized Coordination Practices

To be continued with more coordination practices like:

 

  • Communities of Practice
  • Cross-Team Scrum Meetings
  • Component Mentors and Component Communities 
  • Open Spaces, Lean Coffees and Internal Conferences
  • Travellers, Scouts and Leading Teams  

Write a comment

Comments: 1
  • #1

    Mike DePaoli (Friday, 04 November 2016 18:48)

    Great post Alex. You are absolutely correct about this system being in the domain of complex systems. The problem is, IMO, it the system of which you speak is far more complex than most discussing dependency management understand it to be. I say this because folks still focus on the tactics and the tooling in addressing the challenges at hand, but not the human ecosystem in which the challenges are occurring.

    Traditional "dependency theater" has been in most cases, during my last 25+ years of experience with building technology solutions, dependency failure management. Calling the underlying design relationship "a need to coordinate" might suggest a different mindset, but if the human (worker) ecosystem isn't conducive to such a mindset, nothing will change. This is a management / leadership issue, because as Demming and others have observed and have written about, management creates the system in which workers work. The behaviors they model, which workers then interpret and act based on that interpretation, defines the culture of their team / organization.

    A culture not conducive to building awareness and self-responsibility, which does scale from the individual, to the team, to the organization and ultimately to a society, is unlikely to have motivation or understanding of why this "need for coordination" is so important and the self-responsibility at the team level to own it.

    A knowledge worker ecosystem that is can be characterized as being focused, aligned, collaborative, transparent, unimpeded, autonomous and learning is the type of ecosystem where the type of coordination you refer to can flourish. It is conducive to a lean-agile mindset of which awareness and self-responsibility are key. It is management and leadership that needs to cultivate and maintain such a ecosystem.

    I live in a heavily agriculture focused area of California, so to use an agricultural metaphor is very appropriate.
    It doesn't matter if stakeholders make it clear what crops they want to harvest and sell at market, and you as management acquire the best equipment, seed, fertilizer and people that are trained to use it and are experienced at growing crop, if the ecosystem provided to cultivate the crops is a dessert.

    That is the challenge, you can't assume a mindset of cooperation in the experiment you are postulating, that has to be cultivated in a behavioral ecosystem where it is safe to hold. Until the leadership and management of organizations get that and make it a priority of 'transformation efforts', that aspect of the complexity of human systems at scale will run be the limiting factor for scaling the coordination you describe.

    Hint, look to other biological systems for how coordination is achieved. I started to recognize this back in the early days of Object-Oriented theory. Much of the most powerful patterns are derived from the observation of natural system. There are signs of this cognition surfacing now in scaling discussions with topics like using SOA design principles to model and build organizations and with David Anderson's Enterprise Services Planning model which scales the thinking of Kanban and Lean to even larger applications in enterprise.

    These are exciting times to be in the lean-agile community!