PMI vs Agile, the debate continues

Prompted by a comment by Murray on my first blog in this debate I wanted to expand some of the ideas that have not yet been explored.

Here are the key excerpts from Murray's comment:

PMBOK V4 is compatible with and supportive of Agile processes. Here are some relevant quotes from CHAPTER 2 − PROJECT LIFE CYCLE AND ORGANIZATION

"There is no single way to define the ideal structure for a project"

"it is up to the project manager and the project management team to determine the most appropriate method of carrying out the project"

"There are three basic types of phase-to-phase relationships:

• A sequential relationship (...)

• An overlapping relationship, where the phase starts prior to completion of the previous one (...)

• An iterative relationship, where only one phase is planned at any given time (...)

The PMBOK v4 is quite recent and I have to admit I'm not familiar with it.
However, PMBOK v3 completely omitted the discussion about phase-to-phase relationship in the way described above.

Another aspect that is seldom discussed is that in IT projects (specifically Software Products) we are very quickly distancing our methods from the project-concept. In companies where I've worked, projects no longer represent in any meaningful way how the company works.

Another caveat is that Linear-iterative projects (like RUP proposed) are significantly different from the Incremental-iterative project structure that Agile methods in general and Scrum specifically propose.

All in all, there are significant differences that warrant a very skeptic look at what PMBOK has to offer.

If you want to check some other views into why the "project" pattern is not necessarily useful for software product development take a look at this blog post by Ari, who has quite a long experience in product companies: "Focusing on projects ruins your business"

Photo credit: Dunechaser @ Flickr

The challenge to the Agile community: can't we do better than PMBOK?

Another Knowledge Area in the PMBOK is Human Resource Management. The very name of the knowledge area already gives away the values behind it. It should not say “resource management”, it should say People Management!

The old-skool project management is very much based on the values of Scientific Management and Hierarchical organizations that have permeated our society for hundreds of years. Surely there is a lot of useful things to learn from this, useful things for sure, but not enough.
In order to be able to better handle the unpredictable situations we will regularly face, we need to be able to self-organize to better respond to the challenges presented.

What is self-organization? A technique to enable a faster way for a team to answer any problems that cross their path. Before, when following the command-and-control values a team would have to stop and wait for the boss to leave a meeting and finally ask the boss what to do next. Today, teams are required to self-organize, find a solution or several solutions, experiment and come-up with the final result. When the development iteration ends, the customer will tell the team whether the solution is good enough or not.

Not allowing a team to self-organize turns the old-skool decision makers into bottlenecks that delay the team and potentially the whole project.
Self-organization, however is not easy, you cannot order or wish it into reality. You have to work hard to enable that to happen. Scrum is a process framework that already has some of the needed ingredients for self-organization to happen.

Through the setting of clear goals and it's governance framework, Scrum allows the team to be left alone during an iteration (after having agreed to a set of clear goals). In the Sprint Review, the customer/Product Owner will come together with the team and evaluate what was delivered. This evaluation, in turn, is input to the planning of the next Sprint

So, the message of this series of posts is embrace change.
However, even changing some aspects of the project management body of knowledge (PMBOK) is useful, it is not enough!

A mind shift is needed;. So many new concepts are in play that we must start thinking about Software project management in a completely different way.

We need to change the culture in our companies/organizations. Companies as a whole need to change.

The challenge that is presented to the Agile community (and to the PMI Agile community by extension) is: "how do we benefit from the breakthroughs that have enabled Agile SW development to emerge"?

As a person that has tried the "change PMBOK" approach, my answer is that we need to forget about PMBOK and start afresh from a different set of principles to those that were at the origin of PMBOK.

The recent field of Product Development with authors such as James Morgan, Donald G. Reinertsen and others, together with the people writing about software development present a sufficiently convincing and engaging view to the software development system. Those views demand a serious inspection of PMI/PMBOK approaches. The Agile community must take up that task and come up with something credible. Just embracing PMI's Agile community is not enough. The world has changed enough to warrant a different approach.

"Déjà vu" all over again, or why PMI/PMBOK are dangerous for the agile movement

There are significantly different sets of beliefs between what I call Old-Skool project management (embodied by PMBOK) and what is emerging as Agile project management for Software Development.

Old-skool project management is based on values and scientific breakthroughs that are over 100 years old.

• Scientific management (Frederick Taylor, circa 1911)
  
• Cost based management (optimizing costs vs optimizing the whole system)
  
• Deterministic process control
  
• Don’t trust the people, give them a process that cannot fail!
  

Looking at those values to build today’s development processes is: useful, but not enough.

Just like in architecture we can learn a lot from the old pyramids, but that is not enough to build today’s skyscrapers.

Agile project management tries to incorporate breakthroughs that have happened over the last 100 years and give a fundamentally new perspective to work and work management. Some of the breakthroughs are:

• The effects on productivity of empowered and self-managing teams
  
• Empirical process control (inspect and adapt). Software development project environments are complex. The customer may not know enough about the problem domain, the developers may not understand the problem correctly, the problem domain may be very difficult and multi-dimensional, etc. Things are changing all the time. You need a process that adapts to that change.
  
• Complexity Science: how large complex systems behave and the effect of trying to model their behavior on predictability (or lack thereof).
  
• Shewart/Deming’s Cycle of Plan-Do-Study-Act
  
• Lean Manufacturing
  
• Toyota production system / Toyota product development system
  
• Knowledge work how it is different from the work we previously understood
  
• Deming’s work on the importance of quality in costs, productivity and profitability
  

The important thing to remember is that the new way of looking at project management incorporates many years of evolution. The new project manager is faced with a lot more science and background today than 100 years ago. Our work must take that into account.

Using a paradigm that is rooted in the old principles of work management is not just bad, it's wrong. It can still be useful, but it is not adjusted to our current reality.

To quote Don Reinertsen in his latest book[1]:

Today's orthodoxy has institutionalized a set of internally consistent but dysfunctional beliefs. This has created a tightly interlocking and self-reinforcing system, a system from which it is very difficult to break free. Even when we change one piece, the other pieces hold us back by blocking the benefits of our change. When our change fails to produce benefits, we revert to our old approaches.

Without challenging and removing these self-reinforcing beliefs we are doomed to stay in the same place. And this is why our recent look at PMBOK and PMI in the Agile community is worrying. Not that PMBOK is wrong in itself (read that quote again), but that it represents a set of beliefs that is fundamentally contrary to what is behind the Agile movement.

I will continue this series of posts to look at specific parts of the PMBOK, but keep in mind that what I'm trying to do here is to expose things that are fundamentally wrong with PMI/PMBOK, not just suggesting minor adjustments.

[1] The Principles of Product Development Flow, Donald G. Reinertsen

Photo credit: Shyald @ Flickr

On how PMBOK Change management creates variability and reduces predicatbility

In the last post I tried to point out how the analytical approach of any standard (and specifically PMBOK's) will create problems for those actually having to implement those standards.
Change management in PMBOK is a particularly large problem in this respect. Scope Control (PMBOK's process for change management in scope) comes at 19 different activities. This will leave the most experienced project manager grasping for air when it comes to implementing these activities (they are not implemented in a vaccum obviously, but that does not make it easier...)

Contrast that with the approach that Scrum uses: Re-plan every sprint based on the improved knowledge you have of the product and the market.

Now, that's simple! Simple, but not easy.

For Scrum's change management to work properly the people in charge need to understand the stakeholders needs, the market needs and the current state of development. None of these is easy to achieve, but -- and this is the point -- they are easy to explain.

In Scrum, every Sprint will have a small number of ceremonies:

• The Sprint planning: where the previous sprint's result as well as the changes in environment (stakeholders, market, etc.) are input for the planning process
  
• The Daily meeting: where the progress is reviewed and plans quickly adjusted to meet the Sprint goals
  
• The Sprint review + demonstration: where the status of development is analyzed as well as the reasons for possible problems
  
• The Retrospective: where we analyze what went well/wrong and take actions based on that to improve for the next Sprint, i.e. change our process.
  

That's it. That's how Scrum addresses change in scope: by being prepared for it at the very core of the process. Every sprint change is reviewed and handled. Plans are adjusted.

Interestingly this has an important (yet often forgotten side-effect). Because the stakeholders know that their needs will be taken into account in the next Sprint at the latest, they don't feel the need to disturb the teams during the Sprint. This allows the team to focus on the ongoing work and meet their Sprint goals while at the same time not avoiding change, rather embracing it!

Implementing a process based on PMBOK will not take this aspect into account. It is my experience that in practice, a PMBOK based approach will lead to a separate change management process (often through Change Management Boards), which will regularly but at unpredictable intervals submit changes to the teams. Teams, then need to react "immediately" to those changes: reviewing, commenting and sometimes even accepting them. Anyone familiar with Queuing Theory recognizes this problem immediately: adding more work to a team will make them late and reduce predictability because of the added variability inherent in the "change" related tasks.

This is why PMBOK fails when it separates processes like change management into their own "process" within a larger process which is a software development process.

I should however emphasize: there is value in PMBOK. Read it if you can, but you should not follow PMBOK when defining your processes. Rather you should look at Scrum, Kanban or other processes for inspiration on how to run your software development processes. This is because PMBOK is useful, but not enough!

Photo credit: Will Lion @ Flickr

Why PMBOK fails to embrace change

Yesterday we talked about the Project Management Plan process and how it is not adjusted to many software organizations. Today we will talk about the Project Scope Management knowledge area in the PMBOK.

PMBOK defines that within this Knowledge Area there are five processes[1]

1. Scope Planning: plan on how the scope will be defined [plus other activities]
   
2. Scope Definition: a detailed project scope statement as a basis for future [scope related] project decisions.
   
3. Create Work-Breakdown-Structure: dividing [large] project deliverables and project work into smaller, more manageable components [but this particular component is also used for "cost accounting" of the project]
   
4. Scope Verification: formalizing acceptance of the completed project deliverables
   
5. Scope Control: controlling [unwanted] changes to the project scope
   

NOTE: the []'s above are my own additions.

According to PMBOK each of these 5 processes has at least 8 activities and some up to 19 separate activities (Scope Control specifically)[1].

Having so many separately defined, yet inter-dependent activities in a book like the PMBOK points to a key failure in much of the PMI focus. In the drive to standardize the project management process (PMBOK is ANSI standard 99-001-2004) the description of the activities is exhaustive and detailed. It is analytical. In Agile, many of the major breakthroughs have been in not being analytical, but rather synthesizing knowledge into practices. Take the example of Scope Control, one of the processes in the Project Scope Management knowledge area.

According to PMBOK:

Project scope control is concerned with influencing the factors that create project scope changes and controlling the impact of those changes. (...) Project scope control is also used to manage the actual changes when they occur and is integrated with the other control processes. Uncontrolled changes are often referred to as project scope creep. Change is inevitable, thereby mandating some type of change control process.

As you can understand from the quote above there's an emphasis on activities related to change control (you have to love their choice of words), because "change is inevitable".

In Agile project management approaches we try to avoid having explicitly separating change control by incorporating it into the core process being used to develop software. This is a HUGE (I'd like to use even bigger letters) difference between what PMBOK and Agile are about.

In Agile we really take seriously the fact that "change is inevitable", and that's why we don't build an extra process for change management. Why? Because often these change control processes are the very reason why projects fail. See for example government fixed-price & fixed-scope projects.

The counter-intuitive insight here is that by isolating change control as a separate process we make change management stiff and slow, rendering it an obstacle rather than an enabler to success.

Now, it still is useful to talk explicitly about how we intend to manage change. That is useful, but ultimately not enough.

If I was advising a new project manager, I certainly would ask them to read about change management in projects. Not just from PMBOK, but also from other books like Mary Poppendieck's Implementing Lean Software Development or any of the many Agile Project management books for example.

On the other hand, I would never, ever suggest that we should have a separate change management process in a software project, and that is, for all intents and purposes what the PMBOK suggests by taking it's heavily analytical approach.

Again we see that PMBOK is indeed useful, but certainly not enough!

In the next post I'll look at some specific ways in which Agile processes explicitly take into account change management without creating an extra process but rather building change management into the everyday activities: synthesizing one of the most important processes into "how we run the day-to-day activities in projects".

[1], PMBOK 3rd edition, 2004

Photo credit: David Reece @ Flickr

PMI and the meta-planning process, or why software development planning is different

In the previous post I said I'd talk about Project Integration Management. Of that knowledge area from the PMBOK, I'll focus on one specific Project Management Process within the Project Integration Management (this is how PMBOK calls the different components for each Knowledge Area).The process I'll focus on is "Develop Project Management Plan".

There's an important distinction between what PMBOK suggests and what I've seen in the field (in software development organizations). The PMBOK talks about the project manager having to build a project "management" plan. Note the difference: not a project plan, but a project "management" plan.

The reason for that is, according to PMBOK: This process is needed

for defining, preparing, integrating and coordinating all subsidiary plans into a project management plan. The project management plan becomes the primary source of information for how the project will be planned, executed, monitored and controlled, and closed.[1]

In plain English this means that before you start the project you need a "meta-plan", i.e. a plan on how you plan to plan & manage the project (read that sentence again, it makes sense...)

In many (but not all) software organizations this particular process makes no sense. Why? Because it is very likely that you are involved in developing the next dot-version or major version of an existing software product/service. This is a major difference to what PMBOK and the PMI certification/approach is about.

In software, a lot of what we do is very much akin to what PMBOK calls "operations", i.e., a time-limited development effort within a context/organization that is pre-existing and has processes in place to deal with this (ongoing work vs. temporary and unique work). For the majority of the software organizations and projects out there this process is simply not useful.

It is important, of course, to be aware of this "meta-plan" or processes. For that reason alone it would be beneficial for many to read books like Alistair Cockburn's Agile Software Development or Jim Highsmith's Agile Project Management or even PMBOK (less entertaining as it may be).

Being aware of this "meta-plan" or process is very useful, but not enough. In other words software project managers can still benefit from what PMBOK describes: PMBOK is useful, but not enough for software development endeavors. This will be, as we shall see, a running theme in this series of posts.

Tomorrow we will talk about the Project Scope Management Knowledge Area and why the Scope Planning process in PMBOK is out of synch with today's (agile) methods for developing products in a software organization.

[1] PMBOK Guide, 3rd edition, 2004