About when I stopped worrying and embraced Engineering!

Product Managers often forget that they can make or break a project. If your team is using Scrum that is even more true, because Product Managers are (or should be) involved regularly in the planning of the work with the team. Either as Product Manager or in the role of Product Owner.

It is therefore very important to carefully craft the messages that the Product Manager gives the team. It took me a while to understand this simple message, and this is my story.

As a Product Manager I was eager to give the team direction and clarity on the goals for the product. I communicated regularly with the team with more information on the market, the product, the customers and generic feedback on the direction they were taking.

Eager to achieve the goals we had set for ourselves the team was churning new features at a very good pace. Then the bomb hit. Every time we were supposed to go to production we faced major hurdles. The Retrospectives did not point to anything that could cause the quality problems we had, so I went on an investigation. Why was deployment to production so hard to this team that was delivering features at such a good pace (uncommon for other teams at that time)?

As a Product Manager I had to forget about what I wanted and had to concentrate on finding the root-cause for the problem. I interviewed the developers but nothing I found would explain the situation. The team was testing regularly in their environments and they were practicing Scrum "by the book".

It was then that it hit me. Having a conversation about the development process with one of the developers I understood that they were neglecting their unit and integration tests, which in turn led them to have a long feedback cycle for integration (some days only, but still too long).

After I heard that, it was fairly easy to trace the deployment problems to the lack of automated, fast-cycle integration testing. In their eagerness to deliver more features the developers would be developing up to the last day and would not have time to do the integration testing for those last changes. In turn, that led to many problems when it came to deploy.

Through that conversation I realized that the problem the team had was that they were being implicitly rewarded for delivering more features every time I praised them about the new feature they delivered. However, they were not being rewarded for building the unit and integration tests that would prevent the quality problems.

The end result was that quality-sustaining practices were being neglected. Having understood that, I changed my communication with the team. From that time on I started asking them if they had the integration and unit tests for every feature they delivered and started giving them praise for delivering tested features, not just any feature.

Before this happened to me I was not aware of how strong an influence the Product Manager's message can have on team behavior. "They are the engineers" - I thought - "they should know what they need to do". It's not that simple!

Photo credit: pcalcado @ Flickr

The challenge to the Agile community: can't we do better than PMBOK?

Another Knowledge Area in the PMBOK is Human Resource Management. The very name of the knowledge area already gives away the values behind it. It should not say “resource management”, it should say People Management!

The old-skool project management is very much based on the values of Scientific Management and Hierarchical organizations that have permeated our society for hundreds of years. Surely there is a lot of useful things to learn from this, useful things for sure, but not enough.
In order to be able to better handle the unpredictable situations we will regularly face, we need to be able to self-organize to better respond to the challenges presented.

What is self-organization? A technique to enable a faster way for a team to answer any problems that cross their path. Before, when following the command-and-control values a team would have to stop and wait for the boss to leave a meeting and finally ask the boss what to do next. Today, teams are required to self-organize, find a solution or several solutions, experiment and come-up with the final result. When the development iteration ends, the customer will tell the team whether the solution is good enough or not.

Not allowing a team to self-organize turns the old-skool decision makers into bottlenecks that delay the team and potentially the whole project.
Self-organization, however is not easy, you cannot order or wish it into reality. You have to work hard to enable that to happen. Scrum is a process framework that already has some of the needed ingredients for self-organization to happen.

Through the setting of clear goals and it's governance framework, Scrum allows the team to be left alone during an iteration (after having agreed to a set of clear goals). In the Sprint Review, the customer/Product Owner will come together with the team and evaluate what was delivered. This evaluation, in turn, is input to the planning of the next Sprint

So, the message of this series of posts is embrace change.
However, even changing some aspects of the project management body of knowledge (PMBOK) is useful, it is not enough!

A mind shift is needed;. So many new concepts are in play that we must start thinking about Software project management in a completely different way.

We need to change the culture in our companies/organizations. Companies as a whole need to change.

The challenge that is presented to the Agile community (and to the PMI Agile community by extension) is: "how do we benefit from the breakthroughs that have enabled Agile SW development to emerge"?

As a person that has tried the "change PMBOK" approach, my answer is that we need to forget about PMBOK and start afresh from a different set of principles to those that were at the origin of PMBOK.

The recent field of Product Development with authors such as James Morgan, Donald G. Reinertsen and others, together with the people writing about software development present a sufficiently convincing and engaging view to the software development system. Those views demand a serious inspection of PMI/PMBOK approaches. The Agile community must take up that task and come up with something credible. Just embracing PMI's Agile community is not enough. The world has changed enough to warrant a different approach.

"Déjà vu" all over again, or why PMI/PMBOK are dangerous for the agile movement

There are significantly different sets of beliefs between what I call Old-Skool project management (embodied by PMBOK) and what is emerging as Agile project management for Software Development.

Old-skool project management is based on values and scientific breakthroughs that are over 100 years old.

• Scientific management (Frederick Taylor, circa 1911)
  
• Cost based management (optimizing costs vs optimizing the whole system)
  
• Deterministic process control
  
• Don’t trust the people, give them a process that cannot fail!
  

Looking at those values to build today’s development processes is: useful, but not enough.

Just like in architecture we can learn a lot from the old pyramids, but that is not enough to build today’s skyscrapers.

Agile project management tries to incorporate breakthroughs that have happened over the last 100 years and give a fundamentally new perspective to work and work management. Some of the breakthroughs are:

• The effects on productivity of empowered and self-managing teams
  
• Empirical process control (inspect and adapt). Software development project environments are complex. The customer may not know enough about the problem domain, the developers may not understand the problem correctly, the problem domain may be very difficult and multi-dimensional, etc. Things are changing all the time. You need a process that adapts to that change.
  
• Complexity Science: how large complex systems behave and the effect of trying to model their behavior on predictability (or lack thereof).
  
• Shewart/Deming’s Cycle of Plan-Do-Study-Act
  
• Lean Manufacturing
  
• Toyota production system / Toyota product development system
  
• Knowledge work how it is different from the work we previously understood
  
• Deming’s work on the importance of quality in costs, productivity and profitability
  

The important thing to remember is that the new way of looking at project management incorporates many years of evolution. The new project manager is faced with a lot more science and background today than 100 years ago. Our work must take that into account.

Using a paradigm that is rooted in the old principles of work management is not just bad, it's wrong. It can still be useful, but it is not adjusted to our current reality.

To quote Don Reinertsen in his latest book[1]:

Today's orthodoxy has institutionalized a set of internally consistent but dysfunctional beliefs. This has created a tightly interlocking and self-reinforcing system, a system from which it is very difficult to break free. Even when we change one piece, the other pieces hold us back by blocking the benefits of our change. When our change fails to produce benefits, we revert to our old approaches.

Without challenging and removing these self-reinforcing beliefs we are doomed to stay in the same place. And this is why our recent look at PMBOK and PMI in the Agile community is worrying. Not that PMBOK is wrong in itself (read that quote again), but that it represents a set of beliefs that is fundamentally contrary to what is behind the Agile movement.

I will continue this series of posts to look at specific parts of the PMBOK, but keep in mind that what I'm trying to do here is to expose things that are fundamentally wrong with PMI/PMBOK, not just suggesting minor adjustments.

[1] The Principles of Product Development Flow, Donald G. Reinertsen

Photo credit: Shyald @ Flickr

Response to Lynda Bourne on Agile and the Business message

The following is a comment to Lynda that I was not able to submit to her blog. You can find Lynda's original post here.

@Lynda

Nice post. I'm glad that the conversation is continuing. Both PMI and Agile people need this dialogue.

A couple of comments on your post.

1. I agree that Agile Advocates need to learn to speak to the business people. However,

2. Agile adoption has been fast in many places, not slow.

It is true that many Agile Advocates (AAs as you call them -- like the analogy BTW, and think it fits! :) don't talk "business-ese". That's been a failure of the community, however that is far, very far from being a problem in adoption. Having been involved in two major organization change projects I can vouch for the lack of business-oriented communication, but I can also assure you that speed of adoption has been anything but slow. I know of a company that has several 10's of thousands of employees where Agile is a topic of discussion at all levels, from top management to the team level. This is very encouraging, especially because this company has not been shy in removing some of the building blocks that are in PMBOK (and inadequate to software development) and build new processes at all levels based on the principles of Agile Software Development.

3. Agile software development needs discipline

I agree with this, but the lack of discipline existed in software organizations when PMBOK was the prevailing framework for organizing projects. Lack of discipline is a problem that Agile encoutered when it started, not a problem created by adopting Agile software development methods. Indeed, it is my experience that agile software development builds a much higher level of discipline into the development process than was previously the norm.

4. Agile alignment with business objectives.

This is a topic for a post in itself, but I'll just refer one example of how agile brings focus and alignment with business goals. Check this post by Dean Leffingwell where he explains how requirements management in Agile can be done in a way that directly reflects the strategy of the company.

Let's continue the dialogue. I've learned a lot and hope to continue learning from it! :)

On how PMBOK Change management creates variability and reduces predicatbility

In the last post I tried to point out how the analytical approach of any standard (and specifically PMBOK's) will create problems for those actually having to implement those standards.
Change management in PMBOK is a particularly large problem in this respect. Scope Control (PMBOK's process for change management in scope) comes at 19 different activities. This will leave the most experienced project manager grasping for air when it comes to implementing these activities (they are not implemented in a vaccum obviously, but that does not make it easier...)

Contrast that with the approach that Scrum uses: Re-plan every sprint based on the improved knowledge you have of the product and the market.

Now, that's simple! Simple, but not easy.

For Scrum's change management to work properly the people in charge need to understand the stakeholders needs, the market needs and the current state of development. None of these is easy to achieve, but -- and this is the point -- they are easy to explain.

In Scrum, every Sprint will have a small number of ceremonies:

• The Sprint planning: where the previous sprint's result as well as the changes in environment (stakeholders, market, etc.) are input for the planning process
  
• The Daily meeting: where the progress is reviewed and plans quickly adjusted to meet the Sprint goals
  
• The Sprint review + demonstration: where the status of development is analyzed as well as the reasons for possible problems
  
• The Retrospective: where we analyze what went well/wrong and take actions based on that to improve for the next Sprint, i.e. change our process.
  

That's it. That's how Scrum addresses change in scope: by being prepared for it at the very core of the process. Every sprint change is reviewed and handled. Plans are adjusted.

Interestingly this has an important (yet often forgotten side-effect). Because the stakeholders know that their needs will be taken into account in the next Sprint at the latest, they don't feel the need to disturb the teams during the Sprint. This allows the team to focus on the ongoing work and meet their Sprint goals while at the same time not avoiding change, rather embracing it!

Implementing a process based on PMBOK will not take this aspect into account. It is my experience that in practice, a PMBOK based approach will lead to a separate change management process (often through Change Management Boards), which will regularly but at unpredictable intervals submit changes to the teams. Teams, then need to react "immediately" to those changes: reviewing, commenting and sometimes even accepting them. Anyone familiar with Queuing Theory recognizes this problem immediately: adding more work to a team will make them late and reduce predictability because of the added variability inherent in the "change" related tasks.

This is why PMBOK fails when it separates processes like change management into their own "process" within a larger process which is a software development process.

I should however emphasize: there is value in PMBOK. Read it if you can, but you should not follow PMBOK when defining your processes. Rather you should look at Scrum, Kanban or other processes for inspiration on how to run your software development processes. This is because PMBOK is useful, but not enough!

Photo credit: Will Lion @ Flickr

Why PMBOK fails to embrace change

Yesterday we talked about the Project Management Plan process and how it is not adjusted to many software organizations. Today we will talk about the Project Scope Management knowledge area in the PMBOK.

PMBOK defines that within this Knowledge Area there are five processes[1]

1. Scope Planning: plan on how the scope will be defined [plus other activities]
   
2. Scope Definition: a detailed project scope statement as a basis for future [scope related] project decisions.
   
3. Create Work-Breakdown-Structure: dividing [large] project deliverables and project work into smaller, more manageable components [but this particular component is also used for "cost accounting" of the project]
   
4. Scope Verification: formalizing acceptance of the completed project deliverables
   
5. Scope Control: controlling [unwanted] changes to the project scope
   

NOTE: the []'s above are my own additions.

According to PMBOK each of these 5 processes has at least 8 activities and some up to 19 separate activities (Scope Control specifically)[1].

Having so many separately defined, yet inter-dependent activities in a book like the PMBOK points to a key failure in much of the PMI focus. In the drive to standardize the project management process (PMBOK is ANSI standard 99-001-2004) the description of the activities is exhaustive and detailed. It is analytical. In Agile, many of the major breakthroughs have been in not being analytical, but rather synthesizing knowledge into practices. Take the example of Scope Control, one of the processes in the Project Scope Management knowledge area.

According to PMBOK:

Project scope control is concerned with influencing the factors that create project scope changes and controlling the impact of those changes. (...) Project scope control is also used to manage the actual changes when they occur and is integrated with the other control processes. Uncontrolled changes are often referred to as project scope creep. Change is inevitable, thereby mandating some type of change control process.

As you can understand from the quote above there's an emphasis on activities related to change control (you have to love their choice of words), because "change is inevitable".

In Agile project management approaches we try to avoid having explicitly separating change control by incorporating it into the core process being used to develop software. This is a HUGE (I'd like to use even bigger letters) difference between what PMBOK and Agile are about.

In Agile we really take seriously the fact that "change is inevitable", and that's why we don't build an extra process for change management. Why? Because often these change control processes are the very reason why projects fail. See for example government fixed-price & fixed-scope projects.

The counter-intuitive insight here is that by isolating change control as a separate process we make change management stiff and slow, rendering it an obstacle rather than an enabler to success.

Now, it still is useful to talk explicitly about how we intend to manage change. That is useful, but ultimately not enough.

If I was advising a new project manager, I certainly would ask them to read about change management in projects. Not just from PMBOK, but also from other books like Mary Poppendieck's Implementing Lean Software Development or any of the many Agile Project management books for example.

On the other hand, I would never, ever suggest that we should have a separate change management process in a software project, and that is, for all intents and purposes what the PMBOK suggests by taking it's heavily analytical approach.

Again we see that PMBOK is indeed useful, but certainly not enough!

In the next post I'll look at some specific ways in which Agile processes explicitly take into account change management without creating an extra process but rather building change management into the everyday activities: synthesizing one of the most important processes into "how we run the day-to-day activities in projects".

[1], PMBOK 3rd edition, 2004

Photo credit: David Reece @ Flickr

PMI and the meta-planning process, or why software development planning is different

In the previous post I said I'd talk about Project Integration Management. Of that knowledge area from the PMBOK, I'll focus on one specific Project Management Process within the Project Integration Management (this is how PMBOK calls the different components for each Knowledge Area).The process I'll focus on is "Develop Project Management Plan".

There's an important distinction between what PMBOK suggests and what I've seen in the field (in software development organizations). The PMBOK talks about the project manager having to build a project "management" plan. Note the difference: not a project plan, but a project "management" plan.

The reason for that is, according to PMBOK: This process is needed

for defining, preparing, integrating and coordinating all subsidiary plans into a project management plan. The project management plan becomes the primary source of information for how the project will be planned, executed, monitored and controlled, and closed.[1]

In plain English this means that before you start the project you need a "meta-plan", i.e. a plan on how you plan to plan & manage the project (read that sentence again, it makes sense...)

In many (but not all) software organizations this particular process makes no sense. Why? Because it is very likely that you are involved in developing the next dot-version or major version of an existing software product/service. This is a major difference to what PMBOK and the PMI certification/approach is about.

In software, a lot of what we do is very much akin to what PMBOK calls "operations", i.e., a time-limited development effort within a context/organization that is pre-existing and has processes in place to deal with this (ongoing work vs. temporary and unique work). For the majority of the software organizations and projects out there this process is simply not useful.

It is important, of course, to be aware of this "meta-plan" or processes. For that reason alone it would be beneficial for many to read books like Alistair Cockburn's Agile Software Development or Jim Highsmith's Agile Project Management or even PMBOK (less entertaining as it may be).

Being aware of this "meta-plan" or process is very useful, but not enough. In other words software project managers can still benefit from what PMBOK describes: PMBOK is useful, but not enough for software development endeavors. This will be, as we shall see, a running theme in this series of posts.

Tomorrow we will talk about the Project Scope Management Knowledge Area and why the Scope Planning process in PMBOK is out of synch with today's (agile) methods for developing products in a software organization.

[1] PMBOK Guide, 3rd edition, 2004

A message to PMI: let's start a much needed dialogue. First post

Why PMI style project management is different and why should we care? Continues...

Given the brouha about my last post, I thought I'd give my self some posts to clarify where I think the big differences are between Agile project management and the PMI-style project management (this last one I will affectionately call old-skool, because I too was once an old-skool project manager).

I will try to be fair and balanced, but my bias (borne out of experience) is towards Agile project management, so be warned. This series will not be an account of how PMI-style (or old-skool) project management is good, which it no doubt has been, but is more likely that this series will be seen as an account of the old-skool project managment's Swan Song.

The odissey starts

When thinking about this series of posts (of which this is the first), and how to convey the message I have in the best possible way I could not come with a better way then to just start by stating it up-front:

Embrace change!

If there’s one thing that I would like you to take with you after this series of posts is this: The world has changed, and project management has changed, and will continue to change with it. You should embrace that change.

There are many reasons to embrace this change in project management.

I give you only 3:

1. Your are more likely to succeed if you change
   
2. And it is more Fun to succeed.
   And if the previous 2 reasons did not convince you here’s one more:
   
3. If you don’t change others will, and soon you will be out of business
   

So, in this series of posts I’ll try to cover some things of what I think any project manager (PMI or not) needs to know in this new world of project management for software projects. I’ll look at the Project Management Body of Knowledge (PMBOK) from Project Management Institute – which is an ANSI standard in the US and I’ll try to tell you what new you need to know regarding the 9 areas of knowledge in the PMBOK.

The first area we will focus on one is Project integration management (or project planning for those that are not familiar with PMBOK): This area includes Project plan development, project plan execution and integrated change control (and some other items).

Check back tomorrow for my comment on how an Agile project manager should look at Project Integration Management.

Image credit: Louisville Joe @ Flickr

PMI vs. Agile, what is different, and why should we care

The PMI people don't seem to stop trying to "guess" what Agile is. Guessing is the right term, because anyone with more than a few hours experience in Agile software development can see their cluelessness from afar!

Take this article by Lynda Bourne DMP, PMP (no, I'm not making those TLA's up, she uses them). She says that Agile is different from waterfall because:

• The need for robust change management and configuration management to track the evolution of the Agile project
  
• The critical importance of developing the correct strategy and architecture at the beginning of the Agile project

Someone that says that in Agile project management you need "robust change management and configuration management" probably does not even understand what those are, let alone Agile. Hear me out: Change Management is not needed in Agile, Agile *IS* change management. Take Scrum for example, the whole idea with Scrum is to provide a framework for accepting and managing changes. Scrum *IS* change management. To say that Agile project management "needs" strong change management is to miss one of the most elementary points of Agile Software Development.

Then comes the killer: we Agile project managers (supposedly) need to focus on "developing the correct strategy and architecture at the beginning of the Agile project", missing this - Lynda writes - will lead to failure. Only one word: WTF? C'mon Lynda, that is probably the largest mis-conception of Agile Project Management that I've seen in my (admittedly) short life!

There are thousands of posts about why Agile people focus on "growing" architectures rather than "getting them right up-front" (aka BDUF). Please read up, just google for Agile Architecture and you will find many articles that explain how in Agile we look at Architecture development (here's an example link).

There seems to be a lot of discussion happening in the PMI circles about Agile, but PMI people need to understand that the practices they've developed for building, acquiring companies, etc. don't all apply to software. PMI people should first learn about software and then Agile. Trying to bypass software and going straight for an Agile take-over will only get us (the software industry) another 10-years back in time and lose so much of the evolution we gained with the Agile movement.

To all PMBOK, PMP and PMI people: you are missing 1 million points! Stop trying to explain something you don't understand!

The traditionalists are starting to be quite dangerous in the hostile "take over" of Agile and Scrum (Kanban can't be far behind).

In this post Glen tries to tell us that it is ok to call "project management" what we do in a Scrum (f. ex.) development effort.

Now, that would not be so serious if he did not go and quote the list of activities in the PMBOK and explain that because you are doing those, then you are (by syllogism, one supposes) doing Project Management. Well, it's not that simple as I try to explain in my comment to this article:

It's not that simple. One of the biggest changes that Agile brings to the development of software is that, for most (usually) so-called projects, Scrum (or other methods) change the approach to a more continuous work approach, i.e. the work is always ongoing, just the input (requirements) are being managed in a time frame (release).

So, to answer your question (are you doing project management in Scrum?), in many software projects you DON'T have project management, you have WORK management. Nevertheless you have all of the activities you list (except project initiation). Which also shows how useless the list is, because you always have those activities in any endeavour, project or not.

To sum it up. I think that PMBOK is a good read for newbies and people starting in WORK or PROJECT management, just like many other books are, but PMBOK is dangerous in one aspect: it singles out different aspects that should/must be part of every day work. Take the example of Risk management: in Scrum all activities in the process are risk management activities (planning, daily meeting, demo, retrospective...). If you follow PMBOK you get the idea that Risk Management is actually a different activity that may even include different actors/stakeholders. This is BS, pure and simple.

In my opinion PMBOK and the prevailing approaches to Project Management today are simply dangerous and destroy customer/shareholder value.

We need a new paradigm in project/work management and Scrum is a good start!

The CMMI folks are out to get you! Seriously, they are starting with Scrum, what's next?

The folks talking about CMMI and Scrum are dangerous! Not the least of which because they miss some of the most basic points about software development! You would think they would know about software, but you would be wrong!

Check this out:

In an article about CMMI and Scrum the site ExecutiveBrief (go there at your own risk) states something like:

There is a tradeoff between cost and quality

The person who wrote that line cannot have written one single line of production code in his/her life! What a f*?€%# idiot (excuse my french, I'm sounding like @bellware).

The point is that quality software is cheaper, not more expensive than crappy software. For everybody: not just the customer, but also the vendor/developer.

When people try to sell you the idea that CMMI and Scrum are complementary they are only disguising Scrum in clothes that are easier to swallow for the gray-suited execs that don't understand software at all and should have been fired a long time ago!

Agile scales, Waterfall doesn't! The myth is busted!

I have, for many years, been working in software development and for about 6 years with Agile Software Development adoption. First as a project manager and later driving the adoption of Agile in the organizations where I've worked.

One clear characteristic has become clear to me over time. You may not hear about it much today, but some years ago the biggest fault people would point to Agile methods was that they did not scale! (despite some work by Alistair Cockburn to prove them wrong).

Today, it is quite clear that Agile does scale, and scales to very large organization like the one where I'm working right now. This organization is adopting Agile at a scale that is probably unprecedented, a colossal scale.

So, the cat is out of the bag. Agile does scale! But during my years in this industry I've started to note another trend: Waterfall-like processes do not scale. Many of the problems that we are now solving with Agile are problems that were originally caused by the ideas developed for larger projects using the waterfall model of software development (I include the V-model in this group).

Take an example: Requirements Engineering. In the waterfall-like models the requirements were collected upfront with considerable efforts to try and be as exhaustive as possible (everybody remembers Boehm's cost model for late-found requirements). This approach basically created a huge inventory of requirements (overproduction anyone?) that would then require large groups of people to manage (in large organization, remember this post is about scaling).

These large groups of requirements would not deliver any value to the end-customers, and in fact many of them would never be implemented because others would be found later. This inevitable late discovery of requirements in turn created even more requirements that needed more people to manage them.

The end result was that large software organizations ended up with large requirements-related departments (analysts, user-specialists, user-research, product management, product concepts, etc.) and relatively small engineering departments. It is no wonder then that companies that followed that model would end up slow and unadapted to the fast changing world of software development (remember IBM?).

At the end, the conclusion can only be that Waterfall does not scale! But Agile does!

I'll be exploring this and other topics regarding Agile at scale in my talk at Agile Eastern Europe that will take place on September 18-19 in Kiev.

If you are there or near by don't miss the conference that promises to have extremely interesting talks!



Photo credit: "It's this big" by bobu @ flickr