What is an Estimate?

If you don’t know what an estimate is, you can’t avoid using them. So here’s my attempt to define what is an estimate.
The "estimates" that I'm concerned about are those that can easily (by omission, incompetence or malice) be turned into commitments. I believe Software Development is better seen as a discovery process (even simple software projects). In this context, commitments remove options and force the teams/companies to follow a plan instead of responding to change.

Here's my definition: "Estimates, in the context of #NoEstimates, are all estimates that can be (on purpose, or by accident) turned into a commitment regarding project work that is not being worked on at the moment when the estimate is made."

The principles behind this definition of an estimate

In this definition I have the following principles in mind:

• Delay commitment, create options: When we commit to a particular solution up front we forego possible other solutions and, as a consequence we will make the plan harder to change. Each solution comes with explicit and implicit assumptions about what we will tackle in the future, therefore I prefer to commit only to what is needed in order to validate or deliver value to the customer now. This way, I keep my options open regarding the future.
• Responding to change over following a plan: Following a plan is easy and comforting, especially when plans are detailed and very clear: good plans. That’s why we create plans in the first place! But being easy does not make it right. Sooner or later we are surprised by events we could not predict and are no longer compatible with the plan we created upfront. Estimation up front makes it harder for us to change the plan because as we define the plan in detail, we commit ourselves to following it, mentally and emotionally.
• Collaboration over contract negotiation: Perhaps one of the most important Agile principles. Even when you spend time and invest time in creating a “perfect” contract there will be situations you cannot foresee. What do you do then? Hopefully by then you’ve established a relationship of trust with the other party. In that case, a simple conversation to review options and chose the best path will be enough. Estimation locks us down and tends to put people on the defensive when things don’t happen as planned. Leaving the estimation open and relying on incremental development with constant focus on validating the value delivered will help both parties come to an agreement when things don’t go as planned. Thereby focusing on collaboration instead of justifying why an estimated release date was missed.

  Here are some examples that fit the definition of Estimates that I outlined above:

• An estimate of time/duration for work that is several days, weeks or months in the future.
• An estimate of value that is not part of an experiment (the longer the time-frame the more of a problem it is).
• A long term estimate of time and/or value that we can only validate after that long term is over.

How do you define Estimates in your work? Are you still doing estimates that fit the definition above? What is making it hard to stop doing such estimates? Share below in the comments what you think of this definition and how you would improve it.

This definition of an estimate was sent to the #NoEstimates mailing list a few weeks ago. If you want to receive exclusive content about #NoEstimates just sign up below. You will receive a regular email on the topic of #NoEstimates as Agile Software Development.

Subscribe to our mailing list

* indicates required

Email Address *

First Name

Last Name

Picture credit: Pascal @ Flickr

Coming out of the closet - the life and adventure of a traditional project manager turned Agilist

I’m coming out of the closet today. No, not that closet. Another closet, the tabu closet in the Agile community. Yes, I was (and to a point still am) a control freak, traditional, command and control project manager. Yes, that’s right you read it correctly. Here’s why this is important: in 2003 when I first started to consider Agile in any shape or form I was a strong believer of the Church of Order. I did all the rites of passage, I did my Gantt charts, my PERT charts, my EVM-charts and, of course, my certification.

I was certified Project Manager by IPMA, the European cousin of PMI.

I too was a control freak, order junkie, command and control project manager. And I've been clean for 9 years and 154 days.

Why did I turn to Agile? No, it wasn’t because I was a failed project manager, just ask anyone who worked with me then. It was the opposite reason. I was a very successful project manager, and that success made me believe I was right. That I had the recipe. After all, I had been successful for many years already at that point.

I was so convinced I was right, that I decided to run our first Agile project. A pilot project that was designed to test Agile - to show how Agile fails miserably (I thought, at that time). So I decided to do the project by the book. I read the book and went to work.

I was so convinced I was right that I wanted to prove Agile was wrong. Turned out, I was wrong.

The project was a success... I swear, I did not see that coming! After that project I could never look back. I found - NO! - I experienced a better way to develop software that spoiled me forever. I could no longer look back to my past as a traditional project manager and continue to believe the things I believed then. I saw a new land, and I knew I was meant to continue my journey in that land. Agile was my new land.

Many of you have probably experienced a similar journey. Maybe it was with Test-Driven Development, or maybe it was with Acceptance Testing, or even Lean Startup. All these methods have one thing in common: they represent a change in context for software development. This means: they fundamentally change the assumptions on which the previous methods were based. They were, in our little software development world a paradigm shift.

Test-driven development, acceptance testing, lean startup are methods that fundamentally change the assumptions on which the previous software development methods were based.

NoEstimates is just another approach that challenges basic assumptions of how we work in software development. It wasn’t the first, it will not be the last, but it is a paradigm shift. I know this because I’ve used traditional, Agile with estimation, and Agile with #NoEstimates approaches to project management and software delivery.

A world premier?

That’s why me and Woody Zuill will be hosting the first ever (unless someone jumps the gun ;) #NoEstimates public workshop in the world. It will happen in Finland, of course, because that’s the country most likely to change the world of software development. A country of only five million people yet with a huge track record of innovation: The first ever mobile phone throwing world championship was created in Finland. The first ever wife-carrying world championship was created in Finland. The first ever swamp football championship was created in Finland. And my favourite: the Air Guitar World Championship is hosted in Finland.

#NoEstimates being such an exotic approach to software development it must, of course, have its first world-premier workshop in Finland as well! Me and Woody Zuill (his blog) will host a workshop on #NoEstimates on the week of October 20th in Helsinki. So whether you love it, or hate it you can meet us both in Helsinki!

In this workshop will cover topics such as:

• Decision making frameworks for projects that do not require estimates.
• Investment models for software projects that do not require estimates.
• Project management (risk management, scope management, progress reporting, etc.) approaches that do not require estimates.
• We will give you the tools and arguments you need to prove the value of #NoEstimates to your boss, and how to get started applying it right away.
• We will discuss where we see #NoEstimates going and what are the likely changes to software development that will come next. This is the future delivered to you!

Which of these topics interest you the most? What topics would you like us to cover in the workshop. Tell us now and you have a chance to affect the topics we will cover.

Contact us at vasco.duarte@oikosofy.com and tell us. We will reply to all emails, even flame bombs! :)

You can receive exclusive content (not available on the blog) on the topic of #NoEstimates, just subscribe to the #NoEstimates mailing list below. As a bonus you will get my #NoEstimates whitepaper, where I review the background and reasons for using #NoEstimates

Subscribe to our mailing list

* indicates required

Email Address *

First Name

Last Name

Picture credit: John Hammink, follow him on twitter

How injecting randomness into your project can help it succeed

Success and failure differ, very often, by very little. Take nature as an example. A small change in our DNA (a few mutated genes) can have catastrophic consequences. On the other hand, without these mutations humans would never have come into existence.

Humans and other species in the planet evolved because of chance (although not totally chaotic) changes in their DNA. As small change after small change happened, the surviving individuals were able to spread the viable, and ultimately the most adequate changes to the next generation - and the process repeats still in our own bodies today. If Nature has taught us something about improvement and evolution is that you must have a little bit of luck involved!

The DNA of a project

In software projects - my domain - I see some similarities to this natural model that are worth exploring. Projects also have a DNA which includes the structure and the communication connections between individuals. Communication being one of the most quoted causes for failure in software projects, and therefore one of its critical success factors.

The More I Practice the Luckier I Get

Once a project gets started, a particular structure is put in place (governance, management, teams, etc.). Very often that structure remains in place, and static until the end of the project. But is this a smart choice?

Another aspect of the project that rarely changes is the network of connections between the individuals. This network is what carries information from individual to individual and eventually "selects" the information that is acted upon by the project team. If a piece of information reaches an individual in the project, and that individual does not consider it relevant, that information will not spread further. In contrast, when a piece of information is perceived as important by an individual, that information will be actively spread by that individual.

The Illusion of Control

The structure that is set for a project strongly influences the communication links that can emerge between individuals. Who talks to whom? What meetings exist where people interact regularly? etc. But the reverse is also true. The existing communication network within an organization is a strong influence on what governance structure is put in place, how teams are formed, etc. These co-dependent processes create a positive (or negative) spiral of events for the project, but because neither (structure, or communication network) is often changed, that spiral is unstoppable. If it is positive, it will help the project succeed. If this co-dependent processes create, instead a negative spiral, that will relentlessly remove the chances of success for that project.

This co-dependent relation between two key processes (structure and communication network) is why we can increase the chances of success in our projects by simply causing random perturbations in the project. Randomness helps us explore different patterns for structure and communication networks.

As we carefully design and inject small - safe-to-fail - changes into the project, we can observe how it reacts and adjusts. Later, we can retrospectively amplify or remove/dampen the changes based on the outcome we see. If something works, keep it and ask how can you make it grow. If something creates a net negative outcome, dampen or remove it completely.

The cycle goes like this:

• Define and implement small, safe-to-fail changes in the structure or communication network for the project
• These changes lead to emergent behavior as the individuals and teams adapt to the changes
• A new project configuration emerges which drives new project results
• Finally, we evaluate the results of these changes and decide which components we will try to amplify or dampen

What Do You Mean Random?

There are many ways in which we can, randomly, explore better configurations for our projects. Below I list only a few to illustrate the concept:

• At the start of a project, let the teams chose their own composition. This establishes new connections between individuals in the project as some will choose to work with new team members. However, these new connections will not eliminate the previous connections between individuals. The net effect should be that your project now has a more connected communication network (more individuals with strong connections to each other). In practice this works just like when we form new neural connections in our brain: more connections leads to different thinking and acting patterns
• Organize common project events where people interact with each other outside the day-to-day routine. Planning events can be organized following a structured approach, but including also some unstructured time (à lá unconference, using e.g. Open Space) where people can interact based on their interests. This will also help form new connections between individuals in the project and spread information that would otherwise be locked down and not accessible
• Have regular project "coffee breaks" that happen at the same time and same place. In these events people can connect with other individuals and ask questions from the project management team or the most connected individuals in the project. This unstructured communication increases the chance that some piece of information will "jump the hierarchical borders" and reach people in decision-making positions, or people with influence that can later act on that information.

Conclusion

These practices are designed to inject randomness (unplanned situations or connections) into the project. The goal is not to create Chaos (a scary word for many project teams), but to generate new pathways for the information to flow within the project team, as well as novel project structures that are more adapted to the current challenges the teams face.

Using these (or other) practices will increase your project's chances for success. They don't eliminate the need for the project to be managed, or to have structure. They do increase the chance of success for the project by exploring new organizations and structures through a process of small (safe-to-fail) changes that can lead to unplanned, but ultimately superior performance.

These were just a few examples. How would you inject randomness into your project? Have you done that in the past? Please share your experiences in the comments for the benefit of others.

Photo credit: John Hammink, follow him on twitter

Use #NoEstimates to create options and deliver value reliably

Why do I use #NoEstimates? Certainly the first reason is obvious in the name of this movement. But there are may other reasons. One of the reasons that does not get enough airtime is that it creates more options than using pre-estimated methods to manage software projects.

Value is unpredictable, generate options to explore the value space

When we estimate a software project we make certain assumptions about what work and how that work will be completed. We need to do that in order to sequence the work, handle the dependencies, assign the work to the right people, etc. Estimation is a process of planning, and as such it requires us to make commitments. Those commitments reduce our options, because we immediately eliminate possible implementation strategies.

Let's look at an example. I want to implement a IT-request queue in my organization. Something that allows us to manage the requests from employees in a way that none is lost and the employee can easily get information on how the item is proceeding through it's life-cycle. A project is started. How to plan this project?

• The Estimated approach: In this approach I get the team together we list all the work that needs to be completed (database changes, UI for form input, queries and IT ticket management, server deployment, testing, etc.). We schedule this work according to skills and availability of people and end up with a schedule. This schedule will be implemented by different people. Because of this, I will have to follow-up with each person, to make sure we have each necessary task completed before the work on the next tasks begins. This is how we would typically handle the dependencies between these tasks.
• The #NoEstimates approach: In this approach I ask the team: "What is the most important work that we need to complete NOW?". Tracy says: "The IT ticket handling is the most important part. We can add the customer-side UI later and handle incoming requests by email until then". Bingo! I get the list of tasks for that specific functionality from the team, and it looks much smaller: Database changes + IT UI development. The immediate benefit is that the dependencies are fewer (and we will make them even fewer with User Story splitting), which means less coordination, and valuable functionality delivered earlier. But the key point is this: We have more options! We have made the necessary commitment to deliver something of value, but not made a commitment to the planning for the whole project.

One of the key benefits of using #NoEstimates as a planning approach is that we create more Options, making our projects more valuable and more flexible.

In the #NoEstimates approach we release a partial set of functionality (value) earlier. We may find that for the time being that functionality is enough and may decide not to release any further functionality (an option). Or we may find something we did not know before, and change the project (another option). While in the Estimated approach we will make commitments in terms of time, people and resource allocation that are going to be harder to change later (reducing options), and may commit more money to the problem than we actually need (less options still).

With the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on

Note also that with the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on. The reason is simple: requirements have a "best before" date and expire. We should not commit to requirements too far into the future - but that we will tackle in another post.

Have you applied this planning heuristic in your projects? What were the benefits, and obstacles you faced when using it?

Photo credit: psiaki @ flickr

Using #NoEstimates when deciding which projects to take on

One of the questions (Part I, Part II) that emerged from the #NoEstimatesQuestions discussion on twitter was: “How do I choose between options or opportunities (without having estimates regarding the cost)?”

The thinking goes: I can spend some time doing A or B, but which should I do first? To answer this question we can look at it from two angles:

• 1) How much does A or B cost and how long to they take?
• 2) How much will I learn from doing A or doing B? Which knowledge is more important for me now?

The first point of view assumes that both A and B have similar value (or Return) and that the investment part (the I in ROI) is the most important factor. I happen to think that in software this is not the case. In my experience the R (Return in Return On Investment - ROI) is often more important than the Investment in defining the overall value of a certain decision. In my experience, the second approach will help you make a better decision when selecting between different options. Asking yourself "what is the value I expect to get from option A or option B?" is often a much more insightful discussion regarding the Business Model for your product or service. Here is how I would answer the question: “How do I choose between options of opportunities (without having estimates regarding cost)?”

• Step 1- Define what you expect from doing A and B. Example: A) If I implement “login in with facebook” I will expect to have more users to access the site due to the simpler registration. B) If I implement “invite a friend” I expect to have more visits to the site and ultimately more people registering than if I choose A.
• Step 2- Now, break the work down into risk-neutral chunks (i.e. chunks that don’t kill your project if you fail to reach the expected goal). This leads me to goal decomposition: given that I can’t test any of the options by tomorrow, is there something that I can test by tomorrow which would help me choose futher? Can I implement the “invite a friend” feature by tomorrow? Yes: great! lets do that. The reason is simple: by tomorrow I can start to test option B, if I had gone with Option A I would have to wait longer to be able to test my expectation.
• Step 3- Now it is time to get to work, deliver the smallest/easiest possible implementation of “invite a friend” by tomorrow, and review the impact of that feature. Once this functionality is live you can can either continue with B (if more changes are needed), or ask yourself what are the next options you need to decide on.

How to tackle Large Problems(™)

The example above will, rightfully, called a “simple” problem. So, the next question is: what if I have to decide between 2 Large Problems(tm), which can take several years of development and months of deployment? For example:

I work in the Ministry of Health and I need to decide between developing a large system to manage the health-care records of 10’s of millions of people, or maintain an existing system with potentially huge costs.

In this environment the Return part of the ROI equation is not easily accessible. In fact, one could argue that in this type of projects the I (or Investment/Cost) is the most important factor in making a decision. Most governments do that and spend considerable amounts of time estimating the costs of both options (with terrible consequences). #NoEstimates asks for a completely different approach to these problems (where cost is the key factor in deciding). In fact I would work to completely eliminate the “develop new system from scratch” option from the table. The reason is simple: we are terribly bad at estimating large software projects, so we should avoid them completely. Instead, we must use the flexibility of software to our advantage: start small and slowly grow the team as well as the final technical solution. How I would do it:

• Step 1- Determine what is the most operationally critical part of the system (e.g. by number of people assigned to work on that part of the system, or by contingency plan size -- don’t spend time estimating criticality, there is plenty information already available). Don’t start by changing this part of the system - it is far too critical (as you just determined). Select any other part of the system that you can change without any long-winded approval processes (random selection is ok, but you probably have a hunch).
• Step 2- With a small team (say 5-7 people) start a 1 week project to change this part of the system (a new version of the same software, including tests to ensure that the new implementation works as expected). Can’t do it in a week? Select a smaller part of that system component until you can. Don’t bother estimating, just ask the team until most agree it can be done. In parallel start listing, every day, other system components you will need to change next. Use the ideas in Step 1 to prioritize that list. By the end of week #1 you will have the first “changeable system component backlog” (don’t worry about the comprehensiveness of this list, you will update it every constantly).
• Step 3- By the end of week #1 you should have a small part of the system component re-implemented to the new standards (maybe a new technology? maybe a new privacy law?). Now ask, how many such components do you have to change to complete the project. 10? good, then it will take 10 weeks or so (don’t worry about being right, you will update your projections as you go)
• Step 4- Take the next part of the system and start another 1 week project. Later you may want to start more than one part of the project by adding one more team. But don’t add all teams at the same time

In this approach we try to achieve the following:

• Evolving understanding of the system in terms that can be used for data-driven progress projection (a- changeable system components backlog, b- how many system components can we change per week?)
• An evolutionary change to the system in a way that the system is always functional and the project - even if stopped prematurely - constantly delivers value to the owners of the system (in this case the taxpayer).

Conclusion

I’m sure that there are still quite many questions about the steps and approach I described here. Leave those questions below so that we can continue the dialog.

The #NoEstimates questions, Part II

Last week we did not cover all the questions on #NoEstimates. Twitter gave us a lot more questions than could be listed last week, and even more interesting ones. Read on for the questions and my take on their meaning.

How do I choose between options or opportunities (without having estimates regarding the cost)?
• Choices are hard to make. When you are deep into the development process you inevitably come up with many ideas. But how should we choose between all those ideas without a detailed estimate? I believe this question implies that we have no way to assess the opportunity cost of selecting one of the options. I propose we challenge this assumption in a later post.
How to win a government contract without estimates?
• A very common question, also present implicitly in last week’s post when a questions asked: How to practice #NoEstimates in companies where estimates are "mandatory"? When the legal framework (or contract) requires you to provide an estimate of the length of the work you just have to do it, of course. But a more interesting question to tackle will be: how do we use #NoEstimates to our own (and the customer’s) benefit even when we have to provide estimates before we can win the contract?
What visual indicators do you have in #NoEstimates to show if you deviate from the "optimum"?
• Are we going in the right direction? Are we progressing towards the goal? Do we have the right features? Will we be on time? There are many implicit questions behind this one. While exploring the answers to these questions, I hope we can generate “heuristics” that are relevant and actionable for people interested in the “end game” for a project or a delivery. Including heuristics based on visualization.
Will #NoEstimates give me better predictability even when not using estimates?
• The holy Grail: predictability. In my view, the temptation is to say: if you have time-boxes you don’t need more predictability, you already have it. However, I think this question goes beyond “delivering on time”. In fact, I believe that this question is about predicting “everything” (scope, quality, cost, length, …) We have to explore what #NoEstimates can help with, and what it can’t help with when it comes to looking into the future.
If I just "play the game" with project managers and give estimates that I know aren't right, am I already doing #NoEstimates?
• This question is perhaps related to the one above where we tackle how to use #NoEstimates in an environment where we are asked to provide estimates. It must be said that lying (providing information you know not to be true) is not right, deceiving is not right - no matter what method you use to manage your projects. I hope to clarify how my practice of #NoEstimates can help you tackle the estimation requests you get, but still benefit from the added information #NoEstimates can provide.
Is there some analogy between SQL/NoSQL and Estimates/#NoEstimates?
• Many people comment on the choice of tag (#NoEstimates) for this wider discussion on estimating and managing projects. It is true that if you look at the hash tag used and the previous SQL/NoSQL discussions there is a similarity in the surface. However, I think there are further similarities between these two movements. I’ll explore those in a later blog post.
Does #NoEstimates aim to change existing social constructs (e.g. capitalism, budgets) to remedy the need to estimate?
• As far as I understand this question explores the possible impact and consequences of #NoEstimates, including on the budgeting cycle and the business models. I don’t think #NoEstimates tries to tackle business models at all, in fact I think it aims to enable more business models. On the other hand, #NoEstimates does try to change the way we look at budgets - project budgets at least - with the aim to make those more transparent and reliable. This topic promises to be very interesting as it has generated a lot of discussion on twitter.
Is it accurate to say #NoEstimates is for projects in Cynefin's complex domain?
• For those of you that haven’t followed the Cynefin model or Complex Systems discussion, let me just clarify that Complex Systems are such that causality can only be attributed in retrospect, i.e. we cannot anticipate how the system will react to a certain stimulus, therefore we must experiment (Probe-Sense) and Respond to the results of those experiments. While I believe that #NoEstimates is a perfect fit for Complex environments, I don’t think it applies only in that domain. There are benefits to be gained in other domains: Cynefin/Complicated, Cynefin/Chaotic and even Cynefin/Simple.
If we elect not to take on the risk of predicting an uncertain future, who do we think should, and why?
• I believe this question is trying to answer: Who should take the risk for the (future) results of our actions? Obviously I think we - the people in the project - should take the responsibility for tackling the project in all its characteristics and risk. I don’t think #NoEstimates leads to a shift of the risk carried in any project, and in fact I believe #NoEstimates approaches can be much more effective at uncovering and tackling emergent project threats or risks. More on this later.
Why would we choose not to judge likely outcomes, when worthwhile human endeavors necessarily carry risk?
• My interpretation of this question is that it asks: Should we even look into the future at all? Equating #NoEstimates with #NotLookingIntoTheFuture is a red-herring in my view. I’ll try to explain in a later post how, in my view, #NoEstimates approaches can deliver a much better model of “the future” for any project.
What is and isn't 'estimating' when people form and act on expectations of an uncertain future every day?
• If I remember correctly somebody tweeted: “how can you say “don’t estimate!” when even the act of getting up in the morning requires estimating?”. Perhaps this comment comes from a misunderstanding of the domain we are talking about when we use the term #NoEstimates. It is important to state that we are using #NoEstimates in the context of estimating large software endeavors (i.e. projects), not simple tasks or even fixed/variable-costs calculations (e.g. servers, bandwidth, person-hours, etc.).

This is the list of questions that I’ve collected in the last few weeks on twitter. I hope to answer them here on the blog with your help. Please contribute to the prioritization of these (and last week’s) questions by leaving a comment below with the questions you’d like to tackle first.

Once prioritized, I’ll start answering those questions and we can have a longer discussion about each of them here. After all, twitter, is a great conversation starter, but we need blogs to develop them further ;) Photo Credit: Brad Hoc @ flickr

Story Points Considered Harmful - Or why the future of estimation is really in our past...

This article is the companion to a talk that myself and @josephpelrine gave at OOP 2012.


We have a lot to learn from our ancestors. One that I want to focus on for this post is Galileo.
Galileo was what we would call today a techie. He loved all things tech and was presented an interesting technology that he could not put down. Through that work he developed optic technology to build first a telescope and later a microscope.
Through the use of the telescope and other approaches he came to realize and defend the Heliocentric view of the universe: the Earth was not the center of the Universe, but rather moved around the Sun.
This discovery caused no controversy until Galileo wrote it down and apparently discredited the view held by the Church at that time. The Church believed and defended that the Universe was neatly organized around the Earth and everything moved around our lanet.
We now know that Galileo was right and that the Church was - as it often tends to be with uncritical beliefs - wrong. We now say obviously the Earth is round and moves around the Sun. Or do we...

The Flat Earth Society

Actually, there are still many people around (curious word, isn't it?) the planet that do not even believe that the Earth is round! Don't believe me? Then check The Flat Earth Society.

The fact that is that even today many people hold uncritical beliefs about how our world really works. Or our projects in the case of this post...

Estimation soup

We've all been exposed to various estimation techniques, in an Agile or traditional project. Here are some that quickly come to mind: Expert Estimation, Consensus Estimation, Function Point Analysis, etc. Then we have cost (as opposed to only time) estimation techniques: COCOMO, SDM, etc. And of course, the topic of this post: Story Point Estimation.
What do all of these techniques have in common? They all look towards the future!
Why is this characteristic important?

The Human condition

This characteristic is nt because looking at the future is always difficult! We humans are very good at anticipating immediate events in the physical world, but in the software world what we estimate is neither immediate, nor does it follow any physical laws that we intuitively understand!
Take the example of a goal-keeper in a football (aka soccer) match. She can easily predict how a simple kick will propel the ball towards the goal, and she can do that with quite a high accuracy (as proven by the typically low scores in today's football games). But even in soccer, if you face a player like Maradona, or Beckham, or Crisitiano Ronaldo it is very difficult to predict the trajectory of the ball. Some physicists have spent considerable amount of time analyzing the trajectory of Beckham's amazing free kicks to try to understand how the ball moves and why. Obviously a goal-keeper does not have the computers or the time to analyze the trajectory of Beckham's free kicks therefore Beckham ends up scoring quite a few goals that way. Even in football, where well-known physics laws always apply it is some times hard to predict the immediate future!
The undisputed fact is that we, humans are very bad at predicting the future.
But that is not all!

This is when things get Complex

Lately, and especially in the agile field we have been finding a new field of study: Complexity Sciences.
A field of study that tries to identify rules that help us navigate a world where even causality (cause and effect) are challenged.
An example may be what you may have heard of, the Butterfly effect: "where a small change at one place in a nonlinear system can result in large differences to a later state".
Complexity Sciences are helping us develop our own understanding of software development based on the theories developed in the last few years.
Scrum being a perfect example of a method that has used Complexity to inspire and justify its approach to many of the common problems we face in Software development.
Scrum has used "self-organization", and "emergence" as concepts in explaining why the Scrum approach works. Here's the problem: there's a catch.

Why did this just happen?

In a complex environment we don’t have discernible causality!
Sometimes this is due to delayed effects from our actions, most often it is so that we attribute causality to events in the past when in fact no cause-effect relationship exists (Retrospective Coherence). But, in the field of estimation this manifests itself in a different way.
In order for us to be able to estimate we need to assume that causality exists (if I ask Tom for the code review, then Helen will be happy with my pro-activeness and give me a bonus. Or will she?) The fact is: in a Complex environment, this basic assumption of the existence of discernible Causality is not valid! Without causality, the very basic assumption that justifies estimation falls flat!

Solving the the lack of internal coherence in Scrum

So, which is it? Do we have a complex environment in software development or not? If we do then we cannot - at the same time - argue for estimation (and build a whole religion on it)! In contrast, if we are not in a complex environment we cannot then claim that Scrum - with it’s focus on solving a problem in the complex domain - can work!
So then, the question for us is: Can this Story Point based estimation be so important to the point of being promoted and publicized in all Scrum literature?
Luckily we have a simple alternative that allows for the existence of a complex environment and solves the same problems that Story Points were designed (but failed to) solve.

The alternative prediction device

The alternative to Story Point estimation is simple: just count the number of Stories you have completed (as in "Done") in the previous iterations. They are the best indicator of future performance! Then use that information to project future progress. Basically, the best predictor of the future is your past performance!
Can it really be that simple? To test this approach I looked at data from different projects and tried to answer a few simple questions

The Experiment

• Q1: Is there sufficient difference between what Story Points and ’number of items’ measure to say that they don’t measure the same thing?
• Q2: Which one of the two metrics is more stable? And what does that mean?
• Q3: Are both metrics close enough so that measuring one (number of items) is equivalent to measuring the other (Story Points)?

I took data from 10 different teams in 10 different projects. I was not involved in any of the projects (I collected data from the teams directly or through requests for data in Agile-related mailing lists). Another point to highlight is that this data came from different size companies as well as different size teams and projects.
And here's what I found:

• Regarding Question 1: I noticed that there was a stable medium-to-high correlation between the Story Point estimation and the simple count of Stories completed (0,755; 0,83; 0,92; 0,51(!); 0,88; 0,86; 0,70; 0,75; 0,88). With such a high correlation it is likely that both metrics represent a signal of the same underlying information.
• Regarding Question 2: The normalized data (normalized for Sprint/Iteration length) has similar value of Standard Deviation(equally stable). Leading me to conclude that there is no significant difference in stability of either of the metrics. Although in absolute terms the Story Point estimations vary much more between iterations than the number of completed/Done Stories
• Regarding Question 3: Both metrics (Story Points completed vs Number of Stories completed) seem to measure the same thing. So...

At this point I was interested in analyzing the claims that justify the use of Story Points, as the data above does not seem to suggest any significant advantage of using Story Points as a metric. So I searched for the published justification for the use of Story Points and found a set of claims in Mike Cohn's book "User Stories Applied" (page 87, first edition):

• Claim 1: The use of Story points allows us to change our mind whenever we have new information about a story
• Claim 2: The use of Story points works for both epics and smaller stories
• Claim 3: The use of Story points doesn’t take a lot of time
• Claim 4: The use of Story points provides useful information about our progress and the work remaining
• Claim 5: The use of Story points is tolerant of imprecision in the estimates
• Claim 6: The use of Story points can be used to plan releases

This these claims hold?

Claim 1: The use of Story points allows us to change our mind whenever we have new information about a story

Although there's no explanation about what "change our mind" means in the book, one can infer that the goal is not to have to spend too much time trying to be right. The reason for this is, of course, that if a story changes the size slightly there's no impact on the Story Point estimate, but what if the story changes size drastically?
Well, at this time you would probably have another estimation session, or you would break down that story into some smaller granularity stories to have a better picture of it's actual size and impact on the project.
On the other hand, if we were to use a simple metric like the number of stories completed we would be able to immediately assess the impact of the new items in the progress for the project.
As illustrated in the graph, if we have a certain number of stories to complete (80 in our example) and suddenly some 40 are added to our backlog (breaking down an Epic for example) we can easily see the impact of that in our project progress.
In this case, as we can see from the graph, the impact of a story changing it's meaning or a large story being broken down into smaller stories has an impact on the project and we can see that immediate impact directly in the progress graph.
This leads me to conclude that regarding Claim 1, Story Points offer no advantage over just simply counting the number of items left to be Done.

Claim 2: The use of Story points works for both epics and smaller stories

Allowing for large estimates for items in the backlog (say a 100SP Epic) does help to account in some way for the uncertainty that large pieces of work represent.
However, the same uncertainty exists in any way we may use to measure progress. The fact is that we don’t really know if an Epic (say 100 SPs) is really equivalent to a similar size aggregate of User Stories (say 100 times 1 SP story). Conclusion: there is no significant added information by classifying a story in a 100 SP category which in turn means that calling something an "Epic" is about the same information as classifying it as a 100 Story Points Epic.

Claim 3: The use of Story points doesn’t take a lot of time

Having worked with Story Points for several years this is not my experience. Although some progress has been done by people like Ken Power (at Cisco) with the Silent Grouping technique, the fact that we need such technique should dispute any idea that estimating in SP’s "doesn’t take a lot of time". In fact, as anybody that has tried a non-trivial project knows it can take days of work to estimate the initial backlog for a reasonable size project.

Claim 5: The use of Story points is tolerant of imprecision in the estimates

Although you can argue that this claim holds - even if the book does not explain how - there's no data to justify the belief that Story Points do this better than merely counting the number of Stories Done. In fact, we can argue that counting the number of stories is even more tolerant of imprecisions (see below for more details on this)

Claim 6: Story points can be used to plan releases

Fair enough. On the other hand we can use any estimation technique to do this, so how would Story Points be better in this particular claim than any other estimation technique? Also, as we will see when analysis Claim 4, counting the number of Stories Done (and left to be Done) is a very effective way to plan a release (be patient, the example is coming up).

Claim 4: The use of Story points provides useful information about our progress and the work remaining

This claim holds true if, and only if you have estimated all of your stories in the Backlog and go through the same process for each new story added to the Backlog. Even the stories that will only be developed a few months or even a year later (for long projects) must be estimated! This approach is not very efficient (which in fact contradicts Claim 3).
Basing your progress assessment on the Number of Items completed in each Sprint is faster to calculate (number of items in the PBL / velocity in number of items Done per Sprint = number of Sprints left) and can be used to provide critical information about project progress. Here's a real-life example:

The real-life use of a simpler metric for project progress measurement

In a company I used to work at we had a new product coming to market. It was not a "first-mover" which meant that the barrier to entry was quite high (at least that was the belief from Product Management and Sales).
This meant that significant effort was made to come up with a coherent Product Backlog. The Backlog was reviewed by Sales and Pre-Sales (technical sales) people. All agreed, we really needed to deliver around 140 Stories (not points, Stories) to be able to compete.
As we were not the first in the market we had a tight market window. Failing to meet that window would invalidate the need to enter that market at all.
So, we started the project and in the first Sprint we complete 1 single Story (maybe it was a big story -- truth is I don't remember). Worst, in the same period another 20 stories were added to the Product Backlog. As expected, the Product Management and Sales discovered a few more stories that were really a "must" and could not be left out of the product.
The team was gaining speed and in the second Sprint they got 8 stories to "Done". They were happy. At the same time the Product Manager and the Sales agreed to a cut-down version of the Product Backlog and removed some 20 stories from the Backlog.
After the third sprint the team had achieved velocities of 1 (first Sprint), 8 (second) and 8 (third). The fourth sprint was about to start and the pressure was high on the team and on the Product Manager. During the Sprint planning meeting the team committed to 15 new stories. This was a good number, as a velocity of 15 would make the stakeholders believe that the project could actually deliver the needed product. They would need to keep a velocity of 15 stories per sprint for 11 months. Could they make it?

The climax

As the fourth sprint started I made a bet with the Product Manager. I asked him how many items he believed that the team could complete and he said 15 (just as the team had committed to). I disagreed and said 10. How many items would you have said the team could complete?
I ask this question from the audience every time I tell this story. I get many different answers. Every audience comes up with 42 as a possible answer (to be expected given the crowds I talk to), but most say 8, 10, some may say 15 (very few), some say 2 (very few). The consensus seems to be around 8-10.
At this point I ask the audience why they would say 8-10 instead of 15 as the Product Manager for that team said. Obviously the Product Manager knew the team and the context better, right?
At the end of the fourth sprint the team completed 10 items, which even if it was 20% more than what they had done in previous sprints was still very far from the velocity they needed to make the project a success. The management reflected on the situation and clearly decided that the best decision for the company was to cancel that product.

Story Points Myth: Busted!

That company did that extremely hard decision based on data, not speculation from Project Managers, not based on some bogus estimation using whatever technique. Real data. They looked at the data available to them and decided to cancel the project 10 months before its originally planned release. This project had a team of about 20 people. Canceling the project saved the company 200 man-month of investment in a product they had no hope of getting out of the door!
We avoided a death-march project and were able to focus on other more important products for the company's future. Products that now bring in significant amount of money!

OK, I get your point, but how does that technique work?

Most people will be skeptical at this point (if you've read this far you probably are too). So let me explain how this works out.
Don't estimate the size of a story further than this: when doing Backlog Grooming or Sprint Planning just ask: can this Story be completed in a Sprint by one person? If not, break the story down!
For large projects use a further level of abstraction: Stories fit into Sprints, therefore Epics fit into meta-Sprints (for example: meta-Sprint = 4 Sprints). Ask the same question of Epics that you do of Sprints (can one team implement this Epic in half a meta-Sprint, i.e. 2 Sprints?) and break them down if needed.

By continuously harmonizing the size of the Stories/Epics you are creating a distribution of the sizes around the median:


Assuming a normal distribution of the size of the stories means that you can assume that for the purposes of looking at the long term (remember: this only applies on the long term, i.e. more than 3 sprints into the future) estimation/progress of the project, you can assume that all stories are the same size, and can therefore measure progress by measuring the number of items completed per Sprint.

Final words

As with all techniques this one comes with a disclaimer: you may not see the same effects that I report in this post. That's fine. If that is the case please share the data you have with me and I'm happy to look at it.
My aim with this post is to demystify the estimation in Agile projects. The fact is: the data we have available (see above) does not allow us to accept any of the claims by Mike Cohn regarding the use of Story Points as a valid/useful estimation technique, therefore you are better off using a much simpler technique! Let me know if you find an even simpler one!

Oh, and by the way: stop wasting time trying to estimate a never ending Backlog. There's no evidence that that will help you predict the future any better than just counting the number of stories "Done"!

How do I apply #NoEstimates to improve estimation? Here's how...

Photo credit: write_adam @ flickr

PMI and the meta-planning process, or why software development planning is different

In the previous post I said I'd talk about Project Integration Management. Of that knowledge area from the PMBOK, I'll focus on one specific Project Management Process within the Project Integration Management (this is how PMBOK calls the different components for each Knowledge Area).The process I'll focus on is "Develop Project Management Plan".

There's an important distinction between what PMBOK suggests and what I've seen in the field (in software development organizations). The PMBOK talks about the project manager having to build a project "management" plan. Note the difference: not a project plan, but a project "management" plan.

The reason for that is, according to PMBOK: This process is needed

for defining, preparing, integrating and coordinating all subsidiary plans into a project management plan. The project management plan becomes the primary source of information for how the project will be planned, executed, monitored and controlled, and closed.[1]

In plain English this means that before you start the project you need a "meta-plan", i.e. a plan on how you plan to plan & manage the project (read that sentence again, it makes sense...)

In many (but not all) software organizations this particular process makes no sense. Why? Because it is very likely that you are involved in developing the next dot-version or major version of an existing software product/service. This is a major difference to what PMBOK and the PMI certification/approach is about.

In software, a lot of what we do is very much akin to what PMBOK calls "operations", i.e., a time-limited development effort within a context/organization that is pre-existing and has processes in place to deal with this (ongoing work vs. temporary and unique work). For the majority of the software organizations and projects out there this process is simply not useful.

It is important, of course, to be aware of this "meta-plan" or processes. For that reason alone it would be beneficial for many to read books like Alistair Cockburn's Agile Software Development or Jim Highsmith's Agile Project Management or even PMBOK (less entertaining as it may be).

Being aware of this "meta-plan" or process is very useful, but not enough. In other words software project managers can still benefit from what PMBOK describes: PMBOK is useful, but not enough for software development endeavors. This will be, as we shall see, a running theme in this series of posts.

Tomorrow we will talk about the Project Scope Management Knowledge Area and why the Scope Planning process in PMBOK is out of synch with today's (agile) methods for developing products in a software organization.

[1] PMBOK Guide, 3rd edition, 2004