Coming out of the closet - the life and adventure of a traditional project manager turned Agilist

I’m coming out of the closet today. No, not that closet. Another closet, the tabu closet in the Agile community. Yes, I was (and to a point still am) a control freak, traditional, command and control project manager. Yes, that’s right you read it correctly. Here’s why this is important: in 2003 when I first started to consider Agile in any shape or form I was a strong believer of the Church of Order. I did all the rites of passage, I did my Gantt charts, my PERT charts, my EVM-charts and, of course, my certification.

I was certified Project Manager by IPMA, the European cousin of PMI.

I too was a control freak, order junkie, command and control project manager. And I've been clean for 9 years and 154 days.

Why did I turn to Agile? No, it wasn’t because I was a failed project manager, just ask anyone who worked with me then. It was the opposite reason. I was a very successful project manager, and that success made me believe I was right. That I had the recipe. After all, I had been successful for many years already at that point.

I was so convinced I was right, that I decided to run our first Agile project. A pilot project that was designed to test Agile - to show how Agile fails miserably (I thought, at that time). So I decided to do the project by the book. I read the book and went to work.

I was so convinced I was right that I wanted to prove Agile was wrong. Turned out, I was wrong.

The project was a success... I swear, I did not see that coming! After that project I could never look back. I found - NO! - I experienced a better way to develop software that spoiled me forever. I could no longer look back to my past as a traditional project manager and continue to believe the things I believed then. I saw a new land, and I knew I was meant to continue my journey in that land. Agile was my new land.

Many of you have probably experienced a similar journey. Maybe it was with Test-Driven Development, or maybe it was with Acceptance Testing, or even Lean Startup. All these methods have one thing in common: they represent a change in context for software development. This means: they fundamentally change the assumptions on which the previous methods were based. They were, in our little software development world a paradigm shift.

Test-driven development, acceptance testing, lean startup are methods that fundamentally change the assumptions on which the previous software development methods were based.

NoEstimates is just another approach that challenges basic assumptions of how we work in software development. It wasn’t the first, it will not be the last, but it is a paradigm shift. I know this because I’ve used traditional, Agile with estimation, and Agile with #NoEstimates approaches to project management and software delivery.

A world premier?

That’s why me and Woody Zuill will be hosting the first ever (unless someone jumps the gun ;) #NoEstimates public workshop in the world. It will happen in Finland, of course, because that’s the country most likely to change the world of software development. A country of only five million people yet with a huge track record of innovation: The first ever mobile phone throwing world championship was created in Finland. The first ever wife-carrying world championship was created in Finland. The first ever swamp football championship was created in Finland. And my favourite: the Air Guitar World Championship is hosted in Finland.

#NoEstimates being such an exotic approach to software development it must, of course, have its first world-premier workshop in Finland as well! Me and Woody Zuill (his blog) will host a workshop on #NoEstimates on the week of October 20th in Helsinki. So whether you love it, or hate it you can meet us both in Helsinki!

In this workshop will cover topics such as:

• Decision making frameworks for projects that do not require estimates.
• Investment models for software projects that do not require estimates.
• Project management (risk management, scope management, progress reporting, etc.) approaches that do not require estimates.
• We will give you the tools and arguments you need to prove the value of #NoEstimates to your boss, and how to get started applying it right away.
• We will discuss where we see #NoEstimates going and what are the likely changes to software development that will come next. This is the future delivered to you!

Which of these topics interest you the most? What topics would you like us to cover in the workshop. Tell us now and you have a chance to affect the topics we will cover.

Contact us at vasco.duarte@oikosofy.com and tell us. We will reply to all emails, even flame bombs! :)

You can receive exclusive content (not available on the blog) on the topic of #NoEstimates, just subscribe to the #NoEstimates mailing list below. As a bonus you will get my #NoEstimates whitepaper, where I review the background and reasons for using #NoEstimates

Subscribe to our mailing list

* indicates required

Email Address *

First Name

Last Name

Picture credit: John Hammink, follow him on twitter

The most valuable question for your software project

Every time a software project is started, a dance starts: the dance of project approval. Decision makers and project delivery team take different positions at a table, some ask questions, others do their best to respond, given that these questions are typically about the future. Some of the questions are so much into the future that it would be foolish to pretend we know the answer. Yet we do.

This was the main lesson I learned in my life as a project manager: you have to learn how to dance.

How to dance...

In the dance-like meetings there are 3 types of people:

• A) Those that pretend to know the future
• B) Those that know they can't know the future
• C) And those that don't want to know the future

The question below is only valid for the people in category B). The other two categories are beyond help from me or anyone else.

What is the likelihood this project will fail?

When a project starts, the best question anyone can ask about the project is this:
The most important question: What is the likelihood this project will fail? Why is this question important?

• First, if you don't have an answer to this question you are either in the first or third category of those listed above, and there's nothing I (or anyone else) can do to help you.
• Second, by discussing the possibility of failure we are actively looking for possible causes of problems for the project (rule 1 of risk management).
• Third, having an actual conversation about the failure probability of a project will give you an indication of the "value at risk" for the project.

For example: let's say the project budget is about 100K dollar, and to make it a worthwhile investment we would need to make it between 120K and 90K in total cost (i.e. if it is likely it will cost more than 120K you should not do it). If the answer to the most important question is: 50/50, then you know that the project is not worth doing. It has very little chance of delivering the necessary value to justify it's existence, in fact: it is likely it will be above 100K and up to as much as 200K in cost.
However, if the answer is that we are 80% confident we will deliver the project on timecost, then the project is worth pursuing (although with some strict scope control in place) because the expected worst case scenario would be a total cost of 120K dollars (the 20% margin implicit in the confidence above). In practice an 80% confidence means you expect the project to cost up to 120K (100K + 20%) in the *worst* case.

In a later post I'll explain how we can answer this most important question using the #NoEstimates approach. For now, let me know what you think of the question and the concept of "value at risk" above.

Definition of Value-at-risk [PDF]: "value-at-risk summarizes the expected maximum loss (or worst loss) over a target horizon within a given confidence interval"

Note: A special thanks to @galleman for the idea of the 3 categories of people from which I derived the 3 categories I present in this post.

Photo credit: Steven Depolo @ flick

The three main trends in long agile adoption processes

I was having a discussion with fellow agile adopters yesterday and a bunch of interesting patterns emerged from the discussion:

• We are getting to agile/process improvement fatigue
• The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach
• We don't have a process to learn at the organizational level

Here are some ideas I brought with me from yesterday.

We are getting to agile/process improvement fatigue

This is certainly a familiar sentiment among those of us that have been involved in agile adoptions for several years. One trend/common sentiment I felt yesterday is that agile and lean are seen as just another trend that will pass. Many are trying to look for "better" approaches in meta-approaches (like Lean over agile, or Toyota-driven ideas over Lean, etc.)
This meta-approach is seen as solving the problems with the previous approach.

There's certainly some truth to that. One cannot really improve without going deeper in the process of understanding, and that naturally leads to more abstract approaches. Something the Alistair Cockburn readers will recognize as passing from the Shu-level to the Ha-level.

I don't see this as a problem, I see that as a natural learning process. Agile in general and Scrum in particular have been around for long enough for people to recgnize it's shortcomings and to look for "better" ways. It's the learning process. The problem is when people don't really look for the theory behind the patterns they are trying to apply and just change patterns (say from Scrum to Kanban) without understanding why.

As long as we fail to recognize the theory behind what we are applying, we are doomed to make the same mistakes, over and over again.

The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach


This was one trend that I raised in the discussion. Here's my argument summarized.
Agile is the result of a fundamentally different approach to management (analytical management vs. complexity-aware management). However, the management that is driving/leading the adoption of Agile software development is versed and informed by a completely different way of thinking. They live and bread a different paradigm of management. Until that changes it is impossible to make agile adoptions sustainable.

Symptoms of this problem are quite common and likely familiar to you. Managers set targets for agile adoption, but don't change the way they manage the organization (a requirement to meet those targets). Managers say they want agile adoption but at the same time try to micro-manage the teams, don't respect their need for self-organization and set all kinds of policies that go against teams taking responsibility over their work. And the list goes on.

Main point: there's a basic conflict between the analytical management mind-set and agile adoption. Until we solve that the best we can do (and it's a lot already!) is to get our teams to be agile, but not the structure that directly affects and limits their performance.

We don't have a process to learn at the organizational level

The point that follows from the previous is that in order to be able to get out of the analytical management paradigm we need to build a way for the organization to learn, to experiment and then adapt.

The lack of this method of learning, a meta-management method if you will, is what dooms us to be stuck in the previous paradigm. There is plenty of information around that can help managers learn and improve their work the same way we ask teams to do. But that learning is not happening in the vast majority of companies adopting agile. Until that happens agile is firmly stuck in the realm of the team developing software. It's like buying F1 tyres for a family car and expecting it to win a rally race.

Conclusion

What encourages me is to see that fellow colleagues in the agile journey also understand these issues and are (some of them at least) directly addressing these.

I hope they succeed and share their knowledge with the rest of us...

Photo credit: Denis Collette @ flickr

Nokia goes agile. Will the rest follow?

Ari Jaaksi from maemo.org is blogging about Scrum in Nokia's Linux/Open source projects.
Also, Juha-Markus Aalto (of Object development fame) presented what I would call the "Nokia Agile Requirements Model" (and what Dean Leffingwell calls "Lean, Scalable Requirements Model") at the Object days in Tampere. See the slides here(PDF alert).

With all of this I can only say that Nokia is an exciting place to work at if you are interested in the improvement of our industry!

Now, if only the other Finnish software organization would start following the lead... Yes, you know I mean also the Banks, the Post office, the Tax office and the agonizing slow adopters that are the big consulting firms.

If others start to follow, Finland can become the most exciting place for a software person to work, even better than Silicon Valley!

You are not ready for agile if...

This is a post I've been meaning to write for a long time. Who am I kidding, I don't have the time right now, so I'll let Andy do the talking:

You’re not ready for agile if you:

* behave territorially. You mark your territory like an aggressive dog, refusing to share power or information, and let everyone know if you weren’t included in the decision. When something new comes up, avoid it or kill it.
* are inflexible. You’ve got a policy for everything. If it’s not in the book, it doesn’t exist. This is the way we’ve always done it.
* grow uncomfortable with uncertainty. In an agile project, you will not know the project end date or even what features will be delivered in the next iteration. You cannot stand this, and will insist on fixed dates and costs right up front.
* treat developers as a commodity; a uniform, fungible resource. They are all alike. You can’t trust them to think for themselves, you’ve got to make the important decisions for them.
* believe development is a linear process. You ignore unpleasant feedback. Rather than acting on it, you always stick to the plan, just like a politician in an ill-advised military quagmire.

Read the whole post here.

As Andy says: Agility is a mindset. Call me back when you are willing to change your mind.

End the never ending design discussions: Timeboxed estimation for User Stories, a pattern in Agile adoption

In this post, Jason explains one common problem with the estimation and planning sessions in Agile teams. The problem he describes however, is quite common and has lead to implementation of the Separate Design and Requirements anti-pattern in the traditional software development world.
So I thought I'd write a pattern about a simple and common solution that I've seen and applied to solve that problem.

Timeboxed User Story estimation

Problem:

In the first part of the planning meeting the Scrum team and the Product Owner will review the top User Stories in the backlog to make sure the team and the Product Owner share the same understanding of each User Story.
In the second part of the planning meeting the team will do a short design discussion for each of the items. The discussion should be around what needs to get done for each User Story in order to collect Meaningful Tasks and make the work clear to the whole team.
When the team gets to the definition of meaningful tasks they will get stuck in design discussions often spending too much time in the first one or two stories and then not having a complete decomposition of the other User Stories that would organize the sprint for the team.
Some consequences of this problem are:

• missed tasks in the sprint backlog that end up not being done;
• mis-understanding of the User Story design that leads to inconsistent design decisions;
• unclear definition of the design for a User Story that leads to inconsistent and sometime incompatible architectural patterns in components that need to interact (especially when in the presence of Component Teams);
• Cowboy Coding.
  

Context:

This pattern is applicable in the context of the planning meeting, but can be used also when reviewing a Product Backlog before the planning meeting (see Prepared Product Backlog).
The same pattern can be used to estimate any attribute of a task, such as effort, risk or business value.
In order for this pattern to be applied the team needs to agree to use the same technique. Sometimes some of the team members will refuse to use this practice by stating that it is not "serious" enough or challenging it's credibility. Alternatively people can say that with this method we cannot get "good" design, however it is important to point out that the aim of the planning meeting is not to set the "final" design, but rather the first iteration of the design and especially to achieve consensus with the team on the design patterns and architectural approaches selected. The design, as well as the product, should evolve during the sprint.
In my experience this technique can be successfully used in teams of up to 10 people during the planning meeting.

Solution

The solution is to time-box the estimation for every single User Story, not just for the meeting. Note that the team needs to agree how much time they will spend for each item they accepted for the sprint that is starting. We have used 5 minute time boxes for development stories and 2 minutes for management-related stories.
At the start of the meeting the team agrees on the size of the time box in such a way that it allows to cover all the stories needed within the length of the meeting. For example, if you have 10 stories to cover and 60 minutes meeting that means that after 5 minutes you need to move to the next story (that will give you a 10 minute buffer).
The process for each story is:

1. The facilitator reads the story from the product backlog
   
2. The teams ask questions or define tasks by talking about the design of the implementation
• Example questions are: what modules will this story touch?; do we need help from any other team?; do we already have module X in our version control?; do we have the design for the UI?; will we need to manually test this feature or is it enough to have the automated user acceptance tests?; etc.
  
3. While the team discusses the needed design changes/additions the facilitator writes down the clear tasks (example: refactor class X, change the algorithm for Y, design a view for Z, get support from the DB admin for the DB upgrade in the test environment, etc.)
• For some tasks to be clear, the facilitator may need to ask clarifying questions. See step 2 for examples.
  
4. At 4 minutes (if the timebox is 5 minutes), the facilitator will request everybody to estimate the story (using the Planning Poker method for example).
5. If there is no consensus on the estimation, the discussion continues
• At this time the facilitator can focus the discussion by asking the highest and lowest estimate person: "Why do you think that is the case?"
6. At 5 minutes the facilitator will ask everybody to estimate the story and note down the most popular estimate value
7. The facilitator closes the estimation for this story. Return to step 1.

After the meeting the team has a preliminary list of tasks (that will be improved during the sprint) and a consensus on some basic, yet critical design/architectural decisions.
Even if this method seems "minimalist", our experience is that it works well-enough even for 4 week sprints. Although, due to the size of the tasks it is easier to apply to 2 week sprints.

Resulting context

• The first few times the team will try this technique their discussion may be very superficial, especially if the team has never had design discussions in short spurts before. This is to be expected and the experienced ScrumMaster will keep her ears open for symptoms of this during the daily meetings and call for additional design discussions during the sprint, when needed.
  
• The first few times the team may feel that the process is "artificially" strict. This is a common critique. However the team should stick to the practice for at least 4 sprints before changing it or "customizing" it. Our experience is that people will adapt to the method and, after some sprints, start producing high-quality high-level design already during the short discussions in the planning meeting.
  
• If the first part of the planning meeting was not enough to clarify the aim of each of the stories to be estimated the team may have to discuss the stories from the requirements perspective before they can have a design conversation. In this case the facilitator should stop the discussion and defer the estimation of that task to when the Product Owner has explained well enough the story (this is the Unclear Stories anti-pattern).
  

See Also (patterns and anti-patterns not defined yet):

• User Stories for communication across functions
  
• Meaningful tasks
  
• Component teams (anti-pattern)
• Cowboy coding (anti-pattern)
• Unclear stories (anti-pattern)
• Prepared product backlog
  

Resources

A good practice for the actual estimation is the "planning poker game". Here are some resources:

• A timer for planning poker
• Planning poker explained
• A web-based planning poker for distributed teams
  

The Proxy Product Owner, a pattern in Scrum adoption: version 1

Below I try to describe a pattern that I've faced and have seen in other teams in the context of an Agile transition.
See the explanation section for more details on each of the sections in the pattern.
If you have faced similar situations let us know of your own story in the comments below.

Proxy Product Owner

Problem:

There are 2 or more Product Owners that give work to the Scrum team and the Scrum team has several systems to maintain. Each Product Owner owns only a subset of the systems the team works on and is only interested in the systems she owns.
The team is faced with unclear priorities, Product Owners who do not want to agree on the priority of each other items, and because each Product Owner has their own Product Backlog, the team has a hard time coordinating the work that they need to manage sometimes leading to the team creating their own Backlog that is hidden from the Product Owners, and where the real priorities are set without their knowledge (this is the Secret Team Backlog anti-pattern).

Context:

This pattern is applicable when the Product Owner community for one development team can not reach agreement without the intervention of a third person that will help them define the right priority for all the backlog items.
This pattern applies equally well if there is a decision within the organization that some products, and therefore the respective Product Owners, should receive a higher level of service.

Forces:

• Typically the team is paralized or hampered in their effort by the lack of clear priorities.
  
• In some cases the team ends up delivering features that are not in line with the company or customer needs
  
• The ScrumMaster will spend most of her day managing the Product Backlog and answering uncoordinated requests from all stakeholders
  
• Stakeholders include the Product Owners, but in some cases the Product Owners don't act as a proxy, but rather delegate their responsibility in the users/customers they represent. In this case the team and the ScrumMaster are faced with many conflicting signals about what the real priorities are.
  

Solution

The ScrumMaster takes charge of the Product Backlog, which in this case will become a Team Backlog, in that it includes items for all of the systems that the team manages, not just one product.
The first goal for the ScrumMaster is to get the Team Backlog in shape so that the team can start working from the Team Backlog without having to spend many hours getting the Backlog In Shape.
Once the team can consistently take items from the backlog without a need for large interactions with the Product Owners, then the ScrumMaster can start using the techniques described below for the proper prioritization of the Backlog by all stakeholders.
When several (2 or more) Product Owners are involved in the prioritization of the backlog it is important that all of them understand the need to serve all Product Owners and not just one, therefore the prioritization technique should reflect that fact.

Equal weight for all Product Owners

In this case all Product Owners have equal importance for the organization and therefore they should be assigned the same amount of decision power. That is done by giving each Product Owner 1000 points that they can distribute among all items in the Product Backlog. Each product owner knows that all others have the same amount of points to distribute, therefore they will be forced to "invest" more points into the stories/use cases/backlog items that they really want to get done in the next few iterations.
Each Product Owner may also invest points into other systems than their own, if that will help them get some feature in their own systems.

Different weight for each Product Owner

In this case one Product Owner is considered more important. He is given a larger percentage of the points available. The points given to each Product Owner should be similar to the company investment level decisions. For example, if a Product Owner's product is to receive 70% of the investment, the she should be assigned 70% of the points available for distribution among product backlog items.

Resulting context

• Once the ScrumMaster takes ownership of the Product Backlog the Product Owners will now focus on influencing her and get her to make decisions that are favorable to their needs. This should be avoided and the ScrumMaster should follow as much as possible the real results from the the prioritization done through the distribution of the points by the Product Owners
  
• In some situations some Product Owner will not do the prioritization before the Planning Meeting. In this case the ScrumMaster has a choice of letting the Product Owner assign a subset of her points during the meeting -- as not to delay the meeting too much; or then not allowing late prioritization, in which case the team will not work on the systems owned by that Product Owner.
  
• For this pattern to be implemented the ScrumMaster will have to spend a large part of each day managing the Team Backlog and communicating with the Product Owners. The team and ScrumMaster should be aware of this and decide to invest the ScrumMaster's time in that task.
  

Example

The ScrumMaster in one of our teams is frustrated with the Product Owners because they are never present, and hardly communicate with the team except to shout at them when something critical needs to be worked on "yesterday".
The ScrumMaster has taken the ownership of the different Product Backlogs and joined them into a Team Backlog, where items for all products are collected.
In this example, the team has not enforced a Force Ranked Priority on the backlog or Sprint Backlog, which then leads to a more difficult decision on the real priorities for each item that ends up in the Sprint Backlog. This leads to the stakeholders, who are not engaged, classifying every item as Critical or High priority. This situation then leads to the team in practice deciding which items are done first and which Product Owners are served first.

See Also (patterns and anti-patterns not defined yet):

• Team Backlog
• Backlog In Shape
• Planning Meeting
• Secret Team Backlog anti-pattern

Pattern format explanation

In "Name" I give a name to the pattern.
In "Problem" I describe the problem the team is facing in the form of a root cause and a set of symptoms that may be detected and lead to this problem.
In "Context" I explain where the pattern is applicable.
In "Forces" I try to describe the constraints or forces that the solution has to take into account.
In "Solution" I try to describe the instructions and practices to be used to solve the problem as described.
In "Example" I briefly explain the case of one team at work that has faced this problem and how they solved it.