Real stories of how estimates destroy value in Software Development
The story of the customer that wanted software, but got the wrong estimates instead
Once, one of our customers asked for a small feature and, according to this customer it was quite clear from the beginning what he wanted. Alas, in my experience things often don’t go as planned and therefore I have added a good buffer on top of the needed estimates. Just like all developers I know do. Every day.
However, the story was about to get more interesting. A bit later, the sales team said those numbers were too high. "The customer will never accept those numbers!", they said. "And we really want this case."
After much negotiations we reduced our estimates. After all, we had added some buffer just in case. So, we reduced the estimates on the account, and I heard myself say (I should have known better): "Well, I guess that if everything goes well we could do it in that time."
My real surprise was that, a few days after the estimates were given to the sales team and the customer, I heard from the project manager that the company had agreed a Fixed Price Project with the customer. "That is madness", I said to the sales team. I felt betrayed as a developer! I had been asked for an estimate for the project, but I was tricked into accepting a Fixed Price Project through the estimation process! During the estimation I was not told that this would be a Fixed Price project.
The result was that the project was delivered late, we exceeded the estimates we gave. However, I did learn a valuable lesson: If you are forced to create estimates and the customer requires fixed price then never obey wishes from sales team: their target is different. Or, alternatively just do away with the estimate process altogether: just ask the sales team what number they want to hear ;-).
History repeated: never trust the sales team to handle estimates
A few months ago a customer called in and requested a small feature for their existing product. They were asking for an estimate. Part of the work was very clear and would be very easy to implement. But there was a part that wasn’t that clear. After some pre-analysis and discussions with other development team I knew about what would be needed to finish the job. Unlike the story above, this time I knew the customer asked for a Fixed Price Project and therefore added some buffer – just to be on the safe side.
After carefully estimating the work with the team we sent the total estimate to the sales team. This estimate included everything that was needed: from analysis, implementation, test, deployment. I did not split the estimates into its components parts because I didn’t want the customer to think that testing would be optional (Yes! It as it has happened before).
However, the sales teams split my estimate into the different roles. Big mistake! Magically the estimate that the customer received was half-day shorter than the one we provided. Not a big difference, but I learned my lesson!
Surprise number two was about to hit me: The customer said they hadn't used some hours from a previous contract and that they should get a discount. Long story short, when the project was started we had to reduce our "cost" to 50% of the estimation I had originally given to the sales team. I learned that I should never trust the sales team with mathematical calculations!
We did manage to deliver the feature to our customer, thanks to some very aggressive scope management, this meant that we had to deliver functionality that was requested, but did not perform as well as it could have if we had been allowed to work on the feature from the start, without the estimation back-and-forth.
I did learn my lesson: If you sales team doesn’t know how to sell software projects, and before you given them any estimates, add even more buffer on top of things learnt from the previous story! ;-).
Estimation as an organizational smell
Certainly there are many organizations in the world that do not go through similar stories as these. However, many still do! For those organizations that are affected by stories like these I suggest that we look at different ways to manage software projects. #NoEstimates provides a clear alternative that has proved very successful in the past. For example, in the paper below I mention a story where a team would have given an estimate within 4% of the actual time it took the project to complete, if they had used #NoEstimates!
The Silver Lining
The story continues, however. Here's what my friend added at the end of his email:
Well, after all having successfully changed the mindset of many in the company and in our team. We are already doing quite well.
Now I do things differently. The first thing I always discuss with the customer is how much money they would like to spend for the work they want to have done. This already gives me a good picture if we are even close in understanding of the amount of work that is necessary or more in the direction of “insane”.
Today I don't negotiate traditional contracts with my customers. Every job is now build on trust, communication and transparency.
Labels: #NoEstimates, adoption, agile, agile estimation, business, estimation, project management, story, story points
RSS link
8 Comments:
Well, basicly you say that fixed price contracts for software development are EVIL and should be avoided.
Is that correct?
By Алексей, at May 02, 2014 11:47 AM
Estimates destroying value? I read your friend's experiences very differently - estimates being abused as cost commitments (without contingency), leading to all manner of problem behaviours.
As a result, his move towards greater transparency means that clients have no basis to plan what they might have, when, for how much. No basis to decide whether to invest in the software. No accountability if he under-delivers. Transparency for him maybe - but not for the poor client footing the bill.
There are lessons here. #NoEstimates isn't one of them.
Guy
By Guy Strelitz, at May 02, 2014 4:38 PM
@Alexey
I would not say that based on these stories. And whether fixed price projects are good or bad very much depends on what your business is.
However, in this case the Fixed Price projects were part of the problem as they were a possible cause for the Sales team to behave like they did and negotiate down the estimate to achieve an "acceptable" price.
By Unknown, at May 02, 2014 5:45 PM
This comment has been removed by the author.
By Unknown, at May 02, 2014 5:45 PM
@Guy
In these stories estimates were used as a negotiation technique by both the sales team as well as the customer to "get more for less".
In practice destroying value (in the first case for the vendor, and for the customer in the second case).
I don't understand what you mean by transparency in your comment. In these stories there is no "move towards greater transparency". Perhaps even the opposite happened, depending on your point of view.
One way that would have increased the transparency in my view would be if the customer would define the investment ceiling and the scope of the project would be managed to meet that cost.
In fact, many projects that require a specific release time-frame (any consumer software) would benefit from that discussion. In those cases the time acts as the cost multiplier and the scope is managed to meet the release date. Which is what the author of the story describes at the end of the post.
By Unknown, at May 02, 2014 5:49 PM
Vasco, The number of stories of bad estimates, bad use of estimates, and project failure based on bad estimates and bad use abounds in all domains. In our defense and space SW business this is epidemic.
And like the large post hanging in the lobby of building 771 says "Don't do stupid things in purpose." http://en.wikipedia.org/wiki/File:Rocky_Flats_Plant_Historic_District.jpg
But the need to know the cost of any value still remains. I look forward to receiving your white paper on how you provide the business with rhe numbers needed to calculate ROI in your domain, since this is a "never ending" issue in all domains.
Subscribed an hour ago , nothing arrived yet
By Glen Alleman, at May 02, 2014 6:33 PM
Thanks for your reply Vasco.
I entirely agree that use of estimates as a negotiating tool was destructive in these stories. That's not a case against estimates, it's a case against abusive sales and procurement techniques, and against separating the practitioners - the people who have to do the work! - from sales.
It's interesting that you propose investment-limited projects. That's the approach of DSDM, at the 'heavier' end of the agile spectrum. However the customer still needs an idea of what they'll get for their money - in DSDM they'll definitely get all their Musts and probably a good range of their Shoulds and Coulds.
"trust, communication and transparency" was the author's final point. He saw the change positively. I see it much more ambivalently.
Otherwise I should probably hold my tongue. I'm only chanelling Glen Alleman, who's joined the conversation himself!
By Guy Strelitz, at May 06, 2014 12:44 PM
Thanks for your reply Vasco.
I entirely agree that use of estimates as a negotiating tool was destructive in these stories. That's not a case against estimates, it's a case against abusive sales and procurement techniques, and against separating the practitioners - the people who have to do the work! - from sales.
It's interesting that you propose investment-limited projects. That's the approach of DSDM, at the 'heavier' end of the agile spectrum. However the customer still needs an idea of what they'll get for their money - in DSDM they'll definitely get all their Musts and probably a good range of their Shoulds and Coulds.
"trust, communication and transparency" was the author's final point. He saw the change positively. I see it much more ambivalently.
Otherwise I should probably hold my tongue. I'm only chanelling Glen Alleman, who's joined the conversation himself!
By Guy Strelitz, at May 06, 2014 12:46 PM
Post a Comment
<< Home