Real stories of how estimates destroy value in Software Development

A friend shared with me a few stories about how Estimates are destroying value in his organization. He was kind enough to allow me to share these stories anonymously. Enjoy the reading, I know I did! :)

The story of the customer that wanted software, but got the wrong estimates instead

Once, one of our customers asked for a small feature and, according to this customer it was quite clear from the beginning what he wanted. Alas, in my experience things often don’t go as planned and therefore I have added a good buffer on top of the needed estimates. Just like all developers I know do. Every day.

However, the story was about to get more interesting. A bit later, the sales team said those numbers were too high. "The customer will never accept those numbers!", they said. "And we really want this case."

After much negotiations we reduced our estimates. After all, we had added some buffer just in case. So, we reduced the estimates on the account, and I heard myself say (I should have known better): "Well, I guess that if everything goes well we could do it in that time."

My real surprise was that, a few days after the estimates were given to the sales team and the customer, I heard from the project manager that the company had agreed a Fixed Price Project with the customer. "That is madness", I said to the sales team. I felt betrayed as a developer! I had been asked for an estimate for the project, but I was tricked into accepting a Fixed Price Project through the estimation process! During the estimation I was not told that this would be a Fixed Price project.

The result was that the project was delivered late, we exceeded the estimates we gave. However, I did learn a valuable lesson: If you are forced to create estimates and the customer requires fixed price then never obey wishes from sales team: their target is different. Or, alternatively just do away with the estimate process altogether: just ask the sales team what number they want to hear ;-).

History repeated: never trust the sales team to handle estimates

A few months ago a customer called in and requested a small feature for their existing product. They were asking for an estimate. Part of the work was very clear and would be very easy to implement. But there was a part that wasn’t that clear. After some pre-analysis and discussions with other development team I knew about what would be needed to finish the job. Unlike the story above, this time I knew the customer asked for a Fixed Price Project and therefore added some buffer – just to be on the safe side.

After carefully estimating the work with the team we sent the total estimate to the sales team. This estimate included everything that was needed: from analysis, implementation, test, deployment. I did not split the estimates into its components parts because I didn’t want the customer to think that testing would be optional (Yes! It as it has happened before).

However, the sales teams split my estimate into the different roles. Big mistake! Magically the estimate that the customer received was half-day shorter than the one we provided. Not a big difference, but I learned my lesson!

Surprise number two was about to hit me: The customer said they hadn't used some hours from a previous contract and that they should get a discount. Long story short, when the project was started we had to reduce our "cost" to 50% of the estimation I had originally given to the sales team. I learned that I should never trust the sales team with mathematical calculations!

We did manage to deliver the feature to our customer, thanks to some very aggressive scope management, this meant that we had to deliver functionality that was requested, but did not perform as well as it could have if we had been allowed to work on the feature from the start, without the estimation back-and-forth.

I did learn my lesson: If you sales team doesn’t know how to sell software projects, and before you given them any estimates, add even more buffer on top of things learnt from the previous story! ;-).

Estimation as an organizational smell

Certainly there are many organizations in the world that do not go through similar stories as these. However, many still do! For those organizations that are affected by stories like these I suggest that we look at different ways to manage software projects. #NoEstimates provides a clear alternative that has proved very successful in the past. For example, in the paper below I mention a story where a team would have given an estimate within 4% of the actual time it took the project to complete, if they had used #NoEstimates!

The Silver Lining

The story continues, however. Here's what my friend added at the end of his email:

Well, after all having successfully changed the mindset of many in the company and in our team. We are already doing quite well.

Now I do things differently. The first thing I always discuss with the customer is how much money they would like to spend for the work they want to have done. This already gives me a good picture if we are even close in understanding of the amount of work that is necessary or more in the direction of “insane”.

Today I don't negotiate traditional contracts with my customers. Every job is now build on trust, communication and transparency.

The three main trends in long agile adoption processes

I was having a discussion with fellow agile adopters yesterday and a bunch of interesting patterns emerged from the discussion:

• We are getting to agile/process improvement fatigue
• The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach
• We don't have a process to learn at the organizational level

Here are some ideas I brought with me from yesterday.

We are getting to agile/process improvement fatigue

This is certainly a familiar sentiment among those of us that have been involved in agile adoptions for several years. One trend/common sentiment I felt yesterday is that agile and lean are seen as just another trend that will pass. Many are trying to look for "better" approaches in meta-approaches (like Lean over agile, or Toyota-driven ideas over Lean, etc.)
This meta-approach is seen as solving the problems with the previous approach.

There's certainly some truth to that. One cannot really improve without going deeper in the process of understanding, and that naturally leads to more abstract approaches. Something the Alistair Cockburn readers will recognize as passing from the Shu-level to the Ha-level.

I don't see this as a problem, I see that as a natural learning process. Agile in general and Scrum in particular have been around for long enough for people to recgnize it's shortcomings and to look for "better" ways. It's the learning process. The problem is when people don't really look for the theory behind the patterns they are trying to apply and just change patterns (say from Scrum to Kanban) without understanding why.

As long as we fail to recognize the theory behind what we are applying, we are doomed to make the same mistakes, over and over again.

The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach


This was one trend that I raised in the discussion. Here's my argument summarized.
Agile is the result of a fundamentally different approach to management (analytical management vs. complexity-aware management). However, the management that is driving/leading the adoption of Agile software development is versed and informed by a completely different way of thinking. They live and bread a different paradigm of management. Until that changes it is impossible to make agile adoptions sustainable.

Symptoms of this problem are quite common and likely familiar to you. Managers set targets for agile adoption, but don't change the way they manage the organization (a requirement to meet those targets). Managers say they want agile adoption but at the same time try to micro-manage the teams, don't respect their need for self-organization and set all kinds of policies that go against teams taking responsibility over their work. And the list goes on.

Main point: there's a basic conflict between the analytical management mind-set and agile adoption. Until we solve that the best we can do (and it's a lot already!) is to get our teams to be agile, but not the structure that directly affects and limits their performance.

We don't have a process to learn at the organizational level

The point that follows from the previous is that in order to be able to get out of the analytical management paradigm we need to build a way for the organization to learn, to experiment and then adapt.

The lack of this method of learning, a meta-management method if you will, is what dooms us to be stuck in the previous paradigm. There is plenty of information around that can help managers learn and improve their work the same way we ask teams to do. But that learning is not happening in the vast majority of companies adopting agile. Until that happens agile is firmly stuck in the realm of the team developing software. It's like buying F1 tyres for a family car and expecting it to win a rally race.

Conclusion

What encourages me is to see that fellow colleagues in the agile journey also understand these issues and are (some of them at least) directly addressing these.

I hope they succeed and share their knowledge with the rest of us...

Photo credit: Denis Collette @ flickr

The Proxy Product Owner, a pattern in Scrum adoption: version 1

Below I try to describe a pattern that I've faced and have seen in other teams in the context of an Agile transition.
See the explanation section for more details on each of the sections in the pattern.
If you have faced similar situations let us know of your own story in the comments below.

Proxy Product Owner

Problem:

There are 2 or more Product Owners that give work to the Scrum team and the Scrum team has several systems to maintain. Each Product Owner owns only a subset of the systems the team works on and is only interested in the systems she owns.
The team is faced with unclear priorities, Product Owners who do not want to agree on the priority of each other items, and because each Product Owner has their own Product Backlog, the team has a hard time coordinating the work that they need to manage sometimes leading to the team creating their own Backlog that is hidden from the Product Owners, and where the real priorities are set without their knowledge (this is the Secret Team Backlog anti-pattern).

Context:

This pattern is applicable when the Product Owner community for one development team can not reach agreement without the intervention of a third person that will help them define the right priority for all the backlog items.
This pattern applies equally well if there is a decision within the organization that some products, and therefore the respective Product Owners, should receive a higher level of service.

Forces:

• Typically the team is paralized or hampered in their effort by the lack of clear priorities.
  
• In some cases the team ends up delivering features that are not in line with the company or customer needs
  
• The ScrumMaster will spend most of her day managing the Product Backlog and answering uncoordinated requests from all stakeholders
  
• Stakeholders include the Product Owners, but in some cases the Product Owners don't act as a proxy, but rather delegate their responsibility in the users/customers they represent. In this case the team and the ScrumMaster are faced with many conflicting signals about what the real priorities are.
  

Solution

The ScrumMaster takes charge of the Product Backlog, which in this case will become a Team Backlog, in that it includes items for all of the systems that the team manages, not just one product.
The first goal for the ScrumMaster is to get the Team Backlog in shape so that the team can start working from the Team Backlog without having to spend many hours getting the Backlog In Shape.
Once the team can consistently take items from the backlog without a need for large interactions with the Product Owners, then the ScrumMaster can start using the techniques described below for the proper prioritization of the Backlog by all stakeholders.
When several (2 or more) Product Owners are involved in the prioritization of the backlog it is important that all of them understand the need to serve all Product Owners and not just one, therefore the prioritization technique should reflect that fact.

Equal weight for all Product Owners

In this case all Product Owners have equal importance for the organization and therefore they should be assigned the same amount of decision power. That is done by giving each Product Owner 1000 points that they can distribute among all items in the Product Backlog. Each product owner knows that all others have the same amount of points to distribute, therefore they will be forced to "invest" more points into the stories/use cases/backlog items that they really want to get done in the next few iterations.
Each Product Owner may also invest points into other systems than their own, if that will help them get some feature in their own systems.

Different weight for each Product Owner

In this case one Product Owner is considered more important. He is given a larger percentage of the points available. The points given to each Product Owner should be similar to the company investment level decisions. For example, if a Product Owner's product is to receive 70% of the investment, the she should be assigned 70% of the points available for distribution among product backlog items.

Resulting context

• Once the ScrumMaster takes ownership of the Product Backlog the Product Owners will now focus on influencing her and get her to make decisions that are favorable to their needs. This should be avoided and the ScrumMaster should follow as much as possible the real results from the the prioritization done through the distribution of the points by the Product Owners
  
• In some situations some Product Owner will not do the prioritization before the Planning Meeting. In this case the ScrumMaster has a choice of letting the Product Owner assign a subset of her points during the meeting -- as not to delay the meeting too much; or then not allowing late prioritization, in which case the team will not work on the systems owned by that Product Owner.
  
• For this pattern to be implemented the ScrumMaster will have to spend a large part of each day managing the Team Backlog and communicating with the Product Owners. The team and ScrumMaster should be aware of this and decide to invest the ScrumMaster's time in that task.
  

Example

The ScrumMaster in one of our teams is frustrated with the Product Owners because they are never present, and hardly communicate with the team except to shout at them when something critical needs to be worked on "yesterday".
The ScrumMaster has taken the ownership of the different Product Backlogs and joined them into a Team Backlog, where items for all products are collected.
In this example, the team has not enforced a Force Ranked Priority on the backlog or Sprint Backlog, which then leads to a more difficult decision on the real priorities for each item that ends up in the Sprint Backlog. This leads to the stakeholders, who are not engaged, classifying every item as Critical or High priority. This situation then leads to the team in practice deciding which items are done first and which Product Owners are served first.

See Also (patterns and anti-patterns not defined yet):

• Team Backlog
• Backlog In Shape
• Planning Meeting
• Secret Team Backlog anti-pattern

Pattern format explanation

In "Name" I give a name to the pattern.
In "Problem" I describe the problem the team is facing in the form of a root cause and a set of symptoms that may be detected and lead to this problem.
In "Context" I explain where the pattern is applicable.
In "Forces" I try to describe the constraints or forces that the solution has to take into account.
In "Solution" I try to describe the instructions and practices to be used to solve the problem as described.
In "Example" I briefly explain the case of one team at work that has faced this problem and how they solved it.