Stop fighting reality, take the red pill! A tale about managers

Today I'm on a mood for a rant. Be warned! :)

I constantly get gob-smacked (to borrow a term from a friend) about the lack of simple understanding of reality.

Let me explain. One particularly common symptom in management is that they tend to "believe" (that's the word) that they can affect what happens in a large/complex software development team. They believe that if they ask for more, pressure the teams that the teams will actually do what they ask them. That's almost never the case, and when it is you don't want to live with the result of that (quality problems, missed cases, etc.).

The more they push the teams to work on more things, the less they get. I call this the "reality is a bitch" problem. No matter what you think will happen you will always be surprised and there's nothing you can do about it... until you realize that you are not managing teams, you are managing the system in which they work!

Here's an example. A set of teams go away to plan the next iteration, they come back with their plans. Plans they believe in (i.e. a sprint backlog that is prioritized and estimated). During the Sprint all kinds of surprises happen (reality...) and then they deliver anywhere between 0% and usually 80-90% of what they had planned (it's less often that they deliver everything they planned and that's ok because the Sprint backlog was prioritized anyway).

After this manager scream bloody murder and pushes the teams to be more accurate! To plan more. Guess what happens? Yes, they do indeed plan more, (and in some cases, rare as they may be) plan better, but ultimately deliver between 0% and 80-90% of what they planned.

And here's where the manager's face hits the proverbial wall. The typical manager will try to find someone to blame (likely not himself) and increase the pressure by, perhaps, shouting at some innocent bystander like the project manager or the architect and demand more accuracy in the planning. You know the drill.

The problem with this picture is that the manager does not recognize that a group of teams will always deliver a certain % of their planned work (the larger the group the more reliably so). They may oscillate between, say 0% and 50% most of the time but their accuracy is realiable. What that means is that, any person aware of systems thinking will identify that there is a system at work that prevents the teams from being more accurate. And here is the lead for the informed manager: if you find such a situation don't waste your time shouting at people around you, instead go and study the system. Ask the question: "why can't they deliver on their plans?" and then ask why again (and again, and again). Investigate the deep causes for the planning failure. It is only then that you will be able to do something about it!

Embrace reality, understand reality and then change reality! Fighting reality is a waste of your time and just annoys everybody around you.

Photo credit: clawzctr @ flickr

The curious case of buttoned-up Benjamin

Benjamin works in a consulting company where there was no dress-code unless the client would have a dress-code. Since most consultants were testers and coders (and you know how we dress) the clients understood not to demand a formal dress code.

Then Benjamin's company changed his boss. The new boss required and insisted on everyone wearing a suit and tie for any work for any client.

Benjamin heard of the news while on an assignment that he had started 3 months ago. He had a good relationship with the client, the work was progressing at a fast clip and the client was happy to have Benjamin there despite him not wearing a suit and tie.

Benjamin had a conundrum. He could continue to wear his pull-over sweater and t-shirt at the client as he had done for 3 months and risk being denounced to the new boss or go for the suit and tie which he hated because it made him sweat. After much consideration he decided that he would wear the suit but not the tie.

The boss called a meeting with Benjamin and in between all of the spitting and shouting said "you either do as I say or you're back to low level work here at the home office and you will not be in any client project". The boss felt that Benjamin was personally disrespecting him for not wearing a tie at work.

Benjamin, a veteran of IT projects knew that the best way to handle this situation was to let the boss fall flat on his face and decided to retire from the project and inform the client that, he was no longer allowed to be in the project due to the case of the missing tie.

He left the client's project and the boss put another consultant in his place. The new consultant made a mess of the project to the point that the consultancy company had to put a second consultant on the project for no extra charge to the client in order to have the project finished without a much larger delay and possible penalties.

This same company still has not fired Benjamin's boss, in fact they have probably awarded him a bonus for having the decisiveness of instituting a dress code, and Benjamin is still working on back-office projects at HQ instead of doing work at clients.

This is a true story. It is also one of the (many) reasons why I'm glad that I live in a country where people are valued by the work they do and the results they achieve instead of by spurious shows of dictator-like tendencies and the price of their suit and tie.

Benjamin's boss is a person that does not understand software or software people but he has a position where he can boss them around and that's what he does!

If you find yourself in a position like that don't walk away... RUN! Bosses like that should be given only one reward: the pleasure of working alone!

Update: corrected 2 typos.

Bas, and the world are not ready...

My good friend Bas Vodde just published a review of the panflet Scrumban.

I see that his review is yet another manifestation of what I titled "the world is not ready".

I don't think that his ideas are wrong, on the contrary, just that his ideas show that a lot of ground is still to be covered in software until we understand the nature of software and why certain techniques (like Scrumban) work, despite their counter-intuitive properties.

One example is when Bas states:

Another interesting thought that made me uncomfortable is the idea that there is such a thing as "in-control production" and special/common causes within that. To me, it feels like a manufacturing assumption within product development. Though Corey does cover these assumptions and makes great points about how to control variability, I'm not sure if it is a good thing to do.

Not believing that a team/organization can produce software at a regular pace in terms of number of stories is a common thing. The most common among people that believe strongly that you need to spend a lot of time in estimation of stories.

That is not my experience, I've been working in many projects (in many roles) and one thing that is common to all of the agile projects I've been is that the team's throughput (the number of stories they get DONE in a sprint) is very stable, "in control". This, to me, suggests that the throughput of a team can, indeed be "in control" (actually statistical process control (SPC) is the term).

I recently wrote a post where I detail how you can use the knowledge from SPC in order to help a team improve. The point I make in that post is that you can look at a team's velocity and based on it analyze whether the variation is a special cause (the velocity is all over the place) or if it is a common cause (the variation is predictable i.e. it goes down or up in a predictable fashion).

This distinction is very important. Special causes need to be eliminated one-by-one by instituting error-proofing, for example. Common causes are much harder to eliminate because as Wikipedia states:

Common-cause variation is characterised by:

• Phenomena constantly active within the system;
  
• Variation predictable probabilistically;
  
• Irregular variation within an historical experience base; and
  
• Lack of significance in individual high or low values.
  

In practice this means that a team where special causes are the reason for their velocity variation needs help in doing deep retrospectives with PDCA cycles tackle the root causes of their velocity variation.

However, a team who's velocity varies due to "common causes" will need support from management because they most likely need to change their process or the process of the neighboring teams in order to get any improvement to their throughput.

My experience is that teams of software developers can handle the special causes most of the time on their own, but need support and coaching to be able to overcome the common causes for their lack of constant improvement.

Not recognizing this lesson from manufacturing and applying it to software development is, in my opinion, lose sight of the goal: not falling in love with Agile, but falling in love with the improvement of our industry.

Acceleration, schmeleration. Scott Ambler misses the point...

Through this post on InfoQ I learned that Scott Ambler is writing about productivity in Agile and how to measure it. He comes up with an interesting concept, that of acceleration.

Anyone familiar with statistical process control knows that a process does not have "acceleration" by itself. A process is either under statistic process control (ie churning out about the same amount of output per unit of time, within an acceptable variation) or it is not (ie the output is not reliable).

If a process's output is changing outside the control limits (ie not in statistical process control that actually means that the process is not under statistic process control. When that happens it is due to either one of two:

• Common cause - This is a cause that is known and happens repeatedly (e.g. accumulated technical debt in a way that can be anticipated).
  
• Special cause - Something totally out of left field that cannot be predicted.

If the velocity of the team is decreasing constantly (to the point that you can measure acceleration as Scott proposes) then you are in the presence of a common cause (ie predictable cause that impacts velocity). If that is the case you know that it's not the team that is to blame, it is the system in which the team is working that is causing the decreased velocity.

Scott misses the point completely, when it comes to processes, if you can "predict" the acceleration in velocity then you can do something about it. However, you do have to work at it, you need to investigate the systemic causes for the velocity to decrease and identify the root cause that, being solved, can change the system's behavior. This is not necessarily easy or quick...

In practice this means that every single team out there can improve their productivity (and should) by looking at their velocity, seeing if it is within the statistic control limits and acting on it. If the velocity is in control (as in statistic control) you can improve it by changing the process (the whole process, not just the team's sprint-process). If your velocity is not in control then you need to eliminate the "special causes".

What does this all mean? It means that instead of selling the idea of
"pricing" velocity increases for an Agile team (or any other team) Scott should be trying to tell people about the consequences of actually knowing whether your velocity is increasing/decreasing or it is stable.


Here's what really matters:

• If your velocity is stable (in control) the only way to improve it is to change the process (do a proper retrospective, identify root causes and then act -- the old PDCA). Remember that the "whole" process influences the team's velocity, not just what you do within a sprint (requirements, production, etc. should be looked at also).
  
• If your velocity is not stable (not in control) then you should start by analyzing the reasons for it's oscillation and start solving those. These tend to be random events that happen in an unpredictable fashion (e.g. a developer forgot to submit one change to the VCS before testing started).