Using #NoEstimates when deciding which projects to take on

One of the questions (Part I, Part II) that emerged from the #NoEstimatesQuestions discussion on twitter was: “How do I choose between options or opportunities (without having estimates regarding the cost)?”

The thinking goes: I can spend some time doing A or B, but which should I do first? To answer this question we can look at it from two angles:

• 1) How much does A or B cost and how long to they take?
• 2) How much will I learn from doing A or doing B? Which knowledge is more important for me now?

The first point of view assumes that both A and B have similar value (or Return) and that the investment part (the I in ROI) is the most important factor. I happen to think that in software this is not the case. In my experience the R (Return in Return On Investment - ROI) is often more important than the Investment in defining the overall value of a certain decision. In my experience, the second approach will help you make a better decision when selecting between different options. Asking yourself "what is the value I expect to get from option A or option B?" is often a much more insightful discussion regarding the Business Model for your product or service. Here is how I would answer the question: “How do I choose between options of opportunities (without having estimates regarding cost)?”

• Step 1- Define what you expect from doing A and B. Example: A) If I implement “login in with facebook” I will expect to have more users to access the site due to the simpler registration. B) If I implement “invite a friend” I expect to have more visits to the site and ultimately more people registering than if I choose A.
• Step 2- Now, break the work down into risk-neutral chunks (i.e. chunks that don’t kill your project if you fail to reach the expected goal). This leads me to goal decomposition: given that I can’t test any of the options by tomorrow, is there something that I can test by tomorrow which would help me choose futher? Can I implement the “invite a friend” feature by tomorrow? Yes: great! lets do that. The reason is simple: by tomorrow I can start to test option B, if I had gone with Option A I would have to wait longer to be able to test my expectation.
• Step 3- Now it is time to get to work, deliver the smallest/easiest possible implementation of “invite a friend” by tomorrow, and review the impact of that feature. Once this functionality is live you can can either continue with B (if more changes are needed), or ask yourself what are the next options you need to decide on.

How to tackle Large Problems(™)

The example above will, rightfully, called a “simple” problem. So, the next question is: what if I have to decide between 2 Large Problems(tm), which can take several years of development and months of deployment? For example:

I work in the Ministry of Health and I need to decide between developing a large system to manage the health-care records of 10’s of millions of people, or maintain an existing system with potentially huge costs.

In this environment the Return part of the ROI equation is not easily accessible. In fact, one could argue that in this type of projects the I (or Investment/Cost) is the most important factor in making a decision. Most governments do that and spend considerable amounts of time estimating the costs of both options (with terrible consequences). #NoEstimates asks for a completely different approach to these problems (where cost is the key factor in deciding). In fact I would work to completely eliminate the “develop new system from scratch” option from the table. The reason is simple: we are terribly bad at estimating large software projects, so we should avoid them completely. Instead, we must use the flexibility of software to our advantage: start small and slowly grow the team as well as the final technical solution. How I would do it:

• Step 1- Determine what is the most operationally critical part of the system (e.g. by number of people assigned to work on that part of the system, or by contingency plan size -- don’t spend time estimating criticality, there is plenty information already available). Don’t start by changing this part of the system - it is far too critical (as you just determined). Select any other part of the system that you can change without any long-winded approval processes (random selection is ok, but you probably have a hunch).
• Step 2- With a small team (say 5-7 people) start a 1 week project to change this part of the system (a new version of the same software, including tests to ensure that the new implementation works as expected). Can’t do it in a week? Select a smaller part of that system component until you can. Don’t bother estimating, just ask the team until most agree it can be done. In parallel start listing, every day, other system components you will need to change next. Use the ideas in Step 1 to prioritize that list. By the end of week #1 you will have the first “changeable system component backlog” (don’t worry about the comprehensiveness of this list, you will update it every constantly).
• Step 3- By the end of week #1 you should have a small part of the system component re-implemented to the new standards (maybe a new technology? maybe a new privacy law?). Now ask, how many such components do you have to change to complete the project. 10? good, then it will take 10 weeks or so (don’t worry about being right, you will update your projections as you go)
• Step 4- Take the next part of the system and start another 1 week project. Later you may want to start more than one part of the project by adding one more team. But don’t add all teams at the same time

In this approach we try to achieve the following:

• Evolving understanding of the system in terms that can be used for data-driven progress projection (a- changeable system components backlog, b- how many system components can we change per week?)
• An evolutionary change to the system in a way that the system is always functional and the project - even if stopped prematurely - constantly delivers value to the owners of the system (in this case the taxpayer).

Conclusion

I’m sure that there are still quite many questions about the steps and approach I described here. Leave those questions below so that we can continue the dialog.