Stop committing to iteration scope!

This may seem like heresy in the world of "Scrum requires commitments from the team", but think about it.

If your iteration is about two weeks (give or take) and your team is pretty much fixed for that period of time (you can't really "change" a team that fast), then what else is left? Yes, precisely: only scope if left.

In a typical iteration your time is fixed (it better be if you are doing Scrum), your capacity/team is fixed so you are left with only one decision to make: choose to respect either scope or Definition of Done.

Definition of Done is probably the single most important practice as it is the practice that makes sure you have a future, so your choice should be pretty obvious! Don't commit to scope!

It's not that simple, mister!

Of course, the devil is in the details. How can you commit to "anything" for an iteration? Here in lies the crucial point of this post: User Stories are not fixed scope! If you have read anything about User Stories you know of the INVEST properties that each story should respect. The second property is N-Negotiable. This means that even if you commit to deliver the value explicit in the User Story the actual way you deliver that value is to be negotiated with the Product Owner/Customer. This negotiation is the key dynamic for teams that want to succeed at meeting their iteration commitments.

In an iteration you will discuss which are the key User Stories to be delivered. However, later on, when you find out that you are over-budget you should take clear actions, together with the Product Owner, to find out what is the key functionality for each of the remaining stories that should be delivered within the ongoing iteration. And drop the rest!

There's no point in doing over-time and delivering something you will have to throw away later on (because of quality problems or other). Sit down with the product owner and decide what to drop, what to keep in order to deliver the maximum amount of customer value without breaking the most important commitments Schedule and Quality.

Keep your relationship with the product owner as that is the most important relationship for an Agile team.

Photo credit: d_oracle @ Flickr

Can Scrum and Kanban be used for non-software development work?

Can Scrum or Kanban be used to manage any other work than software development? Do the same concepts apply?

The past week-end there's been some chatter on twitter about using Kanban or Scrum for managing work outside the software development realm.

People seem (pleasently) surprised that it can be done. But can it really? The answer is YES! But why do methods that were clearly developed with software development in mind expand to manage other type of work?

Well, we have to look at the roots of these methods to understand that. Both Kanban and Scrum based their core practices around a list of "stuff" to do. Features/stories in software development fills these lists or backlogs, but in an advertising agency, for example, the "stuff" will be something like drawing proposals for clients, brainstorming meetings, research work, etc.

Through these lists, Kanban and Scrum establish an order of work (what to do now?) and allow what is normally a disconnected mass of work to be organized in a way that a team can use, not only to execute, but also to follow-up their work (e.g. through a burndown/burnup chart)

Indeed, at the root both Kanban and Scrum are just formalized work management methods that were created to overcome the chaos that is still so normal in software development organizations. The hidden truth however, is that the same chaos is present elsewhere and those companies can benefit as much from Scrum/Kanban as the software industry is benefiting now.

The corollary of this is that Scrum/Kanban can be used to do anything, from building a space ship to planting potatoes, and I have even used it for personal non-software development work as well as helped others establish some of the ideas behind Scrum as their own "working method".

There's a catch, though!

Getting back to the backlogs, both these methodologies need a steady stream of work to be analyzed, prioritized and ultimately picked up to be worked on. Because the constraint (i.e. the part in the system that is always busy) in the process is normally the project team, it is imperative that special attention be given to the backlog, where work is listed and described. It is often a good practice to regularly spend time analyzing the work in the backlog to make sure that it fits the Vision for the product/project and that it is sufficiently well defined so that the team will not lose their time trying to figure out what the work-item really means.

For this we normally have a manager (called product owner in Scrum) whose time is spent analyzing the project/product and deciphering the customer needs into a format that the team can understand.

This work is crucial, and the real reason why teams need "managers" (product owner in Scrum).

Manage your work for better performance

The key message here is this: Your work performance will only be as good as you are at managing the backlogs.

Because Scrum/Kanban are work management methods, it follows that the team's performance will depend heavily on how the backlogs are managed by the team and their manager. Therefore, the hidden pearl in Scrum and Kanban is that you should focus a lot (but not all) of energy in defining and improving how your manage your work. And this is valid for any type of work. Be it software or creative projects in an advertising agency.

Waterfall in a Scrum world, yes it is possible. See how below

Can you do waterfall at the same time you are doing Scrum? Let's explore the subject a bit...

We all know what problems there are with waterfall and long term plans that are not changed. We start executing the plans and see that things don't really work out as we thought but we stick to the original plan anyway just delaying the project but trying to deliver the same scope we agreed to.

This has a knock-on effect on the subsequent projects (there's never only one project), and other projects get delayed, but not changed because that would require changing the plans (portfolio, roadmap, strategy you name it!) and the chain goes on, ad infinitum...

The fact is that any plan that is not changed based on what we learn during the project will have the same effect that a waterfall plan has: delay later projects and provide no visibility when the projects will be completed.

So, here's the problem. Many teams and companies are doing Scrum. They have their Sprints, their Sprint plans and they change the overall scope to meet the deadlines. They may even be releasing those projects on time. But there's something more macabre at play here. Despite being able to release projects on time these teams and companies are still riddled with problems: overtime, being late in some (but not all) projects, not being able to react fast to customer requests such as support requests, fix requests, etc.

How come? What's the problem? Well, the fact is that for Scrum (or any other Agile method) to work you need to really have plans that change. It is not enough to have team-level or single-project plans that change, you need to have the whole R&D, the whole company understand that portfolio plans, roadmaps also need to change!

Imagine this. Your team is working on one project and delivering on time, but, without your knowledge, someone in the company has agreed a roadmap that actually requires that your team start working on another project before you finish the current one. Obviously they never thought to ask you your opinion and your project manager even agreed to it, albeit many months ago.
The consequence: your resource plans are now fixed! Your company booked their R&D 100% without any room for changes (after all, we don't want any people twiddling their thumbs do we?).

Since the project that you are working on changes (as it should), you actually get to work on some features until the last sprint, which in practice means that the project that was supposed to start, cannot. Your roadmap just changed, but the VP of R&D does not accept it. "Just push harder" he says. Your team continues to work overtime and you can actually deliver that "extra" project almost on time. But as it happens, there were other projects you were supposed to be working on at that time, and they could not be started. So the roadmap was again affected. And this negative spiral has no end unless the whole organization acknowledges that company-level plans need to change as much as team-level plans do!

See, you can do waterfall at the portfolio level while you are doing scrum at the team level.

Be at least as careful with your commitments to the long term as you are your Sprint commitments!

Response to critical article on Kanban

I was reading Chris McMahon's post "Against Kanban" and could not help but think that a response was in order. The post below is a response, in detail to the (flawed) arguments that Chris makes against Kanban.

"were revolutionary, because they were software development processes actually created from scratch by people who were very very good at creating software."

In here, however you forget that Mike Beedle and Ken Schwaber explicitly mention the chemical industry as a source of learning when putting together the explanation for the success of Scrum.
You further forget that Jeff Sutherland (and others) have often referred to Takeuchi's and Nonaka's paper (The New New product development game) that is straight from, you guessed it: manufacturing!

"if your team measures velocity and uses velocity to plan iterations, and if your team always does pair programming, test-driven development, and continuous integration, then you will either release excellent software every iteration; or you will quickly expose and face the reasons why not."

In this phrase you seem to imply that one cannot use the same XP practices with Kanban. Here again you are wrong. All of the practices that make Scrum "work" can be used to make Kanban "work".

"Before we started releasing every iteration, my team had been using a kanban-like process."

This argument is disingenuous. You should have said that you did not have timeboxed iterations. Kanban is much more than not having timeboxed iterations and you fail your readers when you associate a previously used process (not working by your own admission) with something you quite clearly do not fully grasp. A bit like what you blame on your so called "kanban expert".

"Furthermore, it was an enormous headache for the marketing people. Without timeboxes or estimates, it was difficult to know when a particular feature would be available, which made it almost impossible to do effective marketing for the product."

If you had any experience with Kanban or indeed if you knew more about Lean you would know that this phrase does not make any sense in the Lean context. In a (properly working) Lean process you always know when you are going to ship something, that's why Lean is being adopted in the first place. The fact taht you state lack of predictability as a problem inherent to Kanban just means that you don't understand it -- which is fair, but in that case you should not write condemning articles, you should first prove your Hypothesis (BTW: the Hypothsis-Test-Theory cycle, aka PDCA, is at the heart of Lean. Just google for "a3 report problem solving")

"The fundamental difference between software development and manufacturing or engineering is that in software development we do not manipulate physical objects."

This is an obvious truth, but you don't explain the consequences of this phrase. The (other) fact is that manufacturing, like other industries, are very process oriented and processes from all industries have similar theoretical frameworks! Therefore what you learn from processes in the chemical industry (to use Beedle and Schwaber's example) can be used in the software industry!

"I'm sympathetic to those who see estimation and iteration planning as waste, but I think the risk inherent in not having a mechanism to expose our inadequacies is too much risk to take, even for the best of teams."

In this phrase you make an implicit sylogism by which not having estimation and iteration planning leads to not being able to see what is wrong. This is clearly due to your lack of understanding of how a "flow" process can work. In a flow-like process (e.g. Kanba) you don't just retrospect at the end of the iteration (there's none), you retrospect much more frequently, even sometimes a day. This is indeed a much more powerful way to expose "inadequacies" as any tester knows.

"can you deliver running tested features on a consistent schedule? Iteration timeboxing exposes that metric in the most obvious way possible."

I agree that timeboxing exposes the features delivered/iteration metric. Heck, I've been using that metric myself to explain why you don't actually need to estimate (one of the basic problems in Kanban according to you). The point is that the metric you say is impossible to gather with Kanban is the very metric that makes Kanban transparent to the business owners (the cycle time, or how long does it take to deliver 1 feature). You seem to have missed this point also.

"What we find in both literature and anecdote about the application of processes taken from manufacturing and engineering, puzzlingly, is that for every team for which the process is successful and helpful, there are other teams who implement the same process only to meet with failure.

What I suspect is that ANY process or methodology (that is not actively destructive), applied to a skilled, disciplined, high-functioning, motivated team, will succeed, regardless of the process. Likewise, any process applied to a low-functioning team will likely fail."

I agree with these two paragraphs, but the same is true (like you say) for ANY process, even those that did not originate from the manufacturing industry. At this point your post is contradicting itself...

"With an XP/Scrum sort of iterative process, we will know very soon exactly how and why we are failing. This seems to be not true of kanban or of any other software development process taken from the manufacturing arena."

Having read the other comments above you know how false this statement is. The point is that in Kanban restrospectives are part of everyday work (when it is working), therefore you actually find out "how and why we are failing" much faster than in a timeboxed iteration-based process!

Maybe you should also take a look at this post: http://bit.ly/7NhAg in it the author claims (and I can't verify it) that they release software 50 times a day. If you get to that point, what's the reason to have iterations. By your own metric (features released/iteration) this team is already over achieving to the point of making the metric useless...

A feature team needs more than a cross-functional group!

In a conversation with a friend we identified an anti-pattern that can often happen to "feature teams".

Feature teams are supposed to be cross-functional groups that focus the software development process in valuable-chunks of work, which normally cross the whole system (vertical slices of the system).[1]

There's a way in which you can meet that definition and still not be a feature team. Here's the example.

In a certain company, that shall go unnamed, one feature team was developing their features all through the project. At the same time a set of UI designers were developing their version of what would be a new user interface for the product. Come the last Sprint of the project and the UI people gave the complete UI specification to the feature team. Lots of changes were needed.

At that point the feature team was struggling to develop all of the new screens defined by the UI people as well as changing the ones that were already implemented.

On the face of it, the feature team was working like a feature team. It had UI people, architects, testers and engineers that worked together daily to develop and deliver complete features at the end of every sprint. It was the proverbial cross-functional team. However there was a problem. That's when it hit me the feature team definition has been wrong all along! A feature team is not defined by it's structure it is defined by it's output!

This particular team had no way to create a complete output of the product, they were powerless to influence what the UI team would work on and could only wait and cross their fingers that the UI team would not ask them to completely re-implement the product in the last sprint.

Note that the feature team had UI people present in the planning and development of each feature they developed, but there was another UI team working in a different organization that was shielded from the development work.

Also, the feature team could deliver so called features in every sprint, so by the "old" definition they were indeed a feature team; except their were not.

Note that this same problem could happen if, instead of UI team, you talk about Marketing, IT, Sales, Localization, Documentation, Certification testing, etc.

A feature team is not defined by it's structure (being cross-functional), it's defined by it's output! (delivering shippable pieces of the system)

Proposal for a new definition of Feature Team

So here's a proposal for an improved definition of a feature team:

A feature team is a long-lived group of people that typically involves engineering and non-engineering functions and delivers a complete and potentially shippable feature at the end of every Sprint.
A Feature Team may, as needed, include roles such as: UI designers, UX testers, IT personnel, Marketing, Sales. A feature team will in any case involve all roles that influence the development of a feature to the point of being shippable.

[1] Scaling Lean and Agile Development, Larman, Vodde: Feature team definition on page 153