Humans suck at statistics - how agile velocity leads managers astray

Humans are highly optimized for quick decision making. The so-called System 1 that Kahneman refers to in his book "Thinking fast, thinking slow". One specific area of weakness for the average human is understanding statistics. A very simple exercise to review this is the coin-toss simulation.

Humans are highly optimized for quick decision making.

Get two people to run this experiment (or one computer and one person if you are low on humans :). One person throws a coin in the air and notes down the results. For each "heads" the person adds one to the total; for each "tails" the person subtracts one from the total. Then she graphs the total as it evolves with each throw.

The second person simulates the coin-toss by writing down "heads" or "tails" and adding/subtracting to the totals. Leave the room while the two players run their exercise and then come back after they have completed 100 throws.

Look at the graph that each person produced, can you detect which one was created by the real coin, which was "imagined"? Test your knowledge by looking at the graph below (don't peak at the solution at the end of the post). Which of these lines was generated by a human, and which by a pseudo-random process (computer simulation)?

One common characteristic in this exercise is that the real random walk, which was produced by actually throwing a coin in the air, is often more repetitive than the one simulated by the player. For example, the coin may generate a sequence of several consecutive heads or tails throws. No human (except you, after reading this) would do that because it would not "feel" random. We, humans, are bad at creating randomness and understanding the consequences of randomness. This is because we are trained to see meaning and a theory behind everything.

Take the velocity of the team. Did it go up in the latest sprint? Surely they are getting better! Or, it's the new person that joined the team, they are already having an effect! In the worst case, if the velocity goes down in one sprint, we are running around like crazy trying to solve a "problem" that prevented the team from delivering more.

The fact is that a team's velocity is affected by many variables, and its variation is not predictable. However, and this is the most important, velocity will reliably vary over time. Or, in other words, it is predictable that the velocity will vary up and down with time.

The velocity of a team will vary over time, but around a set of values that are the actual "throughput capability" of that team or project. For us as managers it is more important to understand what that throughput capability is, rather than to guess frantically at what might have caused a "dip" or a "peak" in the project's delivery rate.

The velocity of a team will vary over time, but around a set of values that are the actual "throughput capability" of that team or project.

When you look at a graph of a team's velocity don't ask "what made the velocity dip/peak?", ask rather: "based on this data, what is the capability of the team?". This second question will help you understand what your team is capable of delivering over a long period of time and will help you manage the scope and release date for your project.

The important question for your project is not, "how can we improve velocity?" The important question is: "is the velocity of the team reliable?"

Picture credit: John Hammink, follow him on twitter

Solution to the question above: The black line is the one generated by a pseudo-random simulation in a computer. The human generated line is more "regular", because humans expect that random processes "average out". Indeed that's the theory. But not the the reality. Humans are notoriously bad at distinguishing real randomness from what we believe is random, but isn't.

As you know I've been writing about #NoEstimates regularly on this blog. But I also send more information about #NoEstimates and how I use it in practice to my list. If you want to know more about how I use #NoEstimates, sign up to my #NoEstimates list. As a bonus you will get my #NoEstimates whitepaper, where I review the background and reasons for using #NoEstimates

Subscribe to our mailing list

* indicates required

Email Address *

First Name

Last Name

Dealing with Complexity in Software Projects - The theory that explains why Agile Project Management works

Why do projects fail?

This is a question that haunts all project managers. Good and bad, experienced and beginners, Agile or non-Agile. The reason for that is simple: we believe that if we crack the code of why projects fail, we will be able to avoid failure in our own projects. After all who does not want to be in a successful project?

We believe that if we crack the code of why projects fail, we will be able to avoid failure in our own projects.

But before we can can answer such a difficult question, we need to understand the factors that influence project failure. Many will immediately say that lack of planning or bad planning are major causes for project failure. We've all been told that more planning is the solution. And we have been told that the "right" planning is the solution. Sure, we all know that, but does that "right kind of planning" look in practice?

Enter Agile...

We know that "the right planning" is the solution, but we need to have a functional definition of what that "right planning" looks like. Agile has contributed greatly to further our understanding of software projects. For example: thanks to Agile we now can confidently state that individuals and their interactions are a key enabler for successful projects, not just "the right kind of planning". But there is more...

We are also starting to understand that there are some fundamental failures in the existing Theory of Project Management. Thanks for researchers like Koskela and Howell[1] we even have a framework to analyze what is wrong - very specifically - with traditional Project Management. Most importantly, that framework also helps us understand what needs to be different for projects to succeed in the new world of Knowledge Work.

In the article below (sign-up required) I explore the differences between the existing theory of project management and what Agile methods (such as Scrum) define as the new way to look at project management.

This paper explains some of the ideas that are part of  the "Chaos Theory in Software Projects" workshop. In that workshop we review the lessons learned from Complexity and Chaos theory and how they apply to Project Management.

The goal of the workshop is to give project managers a new idea of what is wrong in the current view of project management and how that can be changed to adapt project management to the world of Knowledge work. To know more about the workshop don't hesitate to get in touch: vasco.duarte@oikosofy.com.

[1] Koskela and Howell, The Theory of Project Management: Explanation to Novel Methods, retrieved on April 2014
Picture credit: John Hammink, follow him on twitter

How Chaos Theory will influence management and management styles in the future

Managers all over the world are faced with a critical challenge to their role. They ideas about management and their management style is being challenged. And this is even more important because many managers have reached a position of in their career where they thought they could "take it easy". Nothing could be further from the truth.

Today the role of managers in all industries is shifting. And in no industry more than the knowledge industry.

In this video I explore why this is happening and where we may be able to look for solutions. I also present a concrete set of consequences that will affect you as a manager from the trends we are witnessing in the knowledge industry.

Do you want to know more?

Ready to explore what you as a manager can learn from The Science of Chaos?

You came to the right place! :) Mystes in Finland organizes a workshop about Chaos Science applied to the challenges of managing small and large knowledge work organizations. You can visit their site to know more about the workshop and to sign up. Places are limited. In that workshop I will touch on the following topics:

• Current theoretical base for managing projects
• What is wrong with managing software projects today and why?
• What can we learn from Chaos Theory and how to apply it in real life projects?
• A model for a successful project using what we have learned from Chaos Theory

Do you have specific questions that intrigue you? Send them to us and we promise to address them during the workshop!

Fractals, the solution to all of your scaling problems. Including Scaling Agile

It is no secret that I love planning. I'm not coming out of the closet now, that's been true forever! And at some point in my life I was even "cool" with that.

Additionally, I want you to know (although you will not yet understand why) that I still love planning. That's me :) Now pay attention, I'm about to shoot myself in the foot.

Loading the gun

When I was in college there was a topic that I loved, the topic was Information Theory. There's so much stuff in that area of research that I can't even begin to touch the tip of the proverbial iceberg, so I'll just say - for now - that information theory is an area of research that investigates information and how it is codified. For example: how do I compress a text file? Turns out, text can be very efficiently compressed. Perhaps the blog post you are reading can be codified in a few hundred bytes. The cool thing is why that happens: redundancy. Redundancy is an unappreciated quality of all systems.

So unappreciated in fact, that we all praise it's mirror twin: efficiency. A compressed text file is very efficient (i.e. it codifies a lot of information in a very small amount of disk space - I could probably compress this post into 140 bytes and tweet it out - that would be efficient).

If we can create so efficient representation of information why would we stick the old-skool inefficient representation (for example: language)? I'm glad you ask! The reason is that language with all its redundancy is easy to understand even if you cut parts out or mangle the letters. Try reading the following paragraph (click for a larger version):

Did you understand what that said? Of course you did! Redundancy saves the day! Yay!

Now about software and organizations

In organizations and companies, we also have redundancy - plenty of it! And just as well, because without it most companies and organizations would stop working altogether. Redundancy gives us resilience! Just like in the language example above: even with parts of the words cut out of the phrase you were able to understand it! And this is just how organizations work: through, and thanks to, redundancy. This is the reason why some "downsizing" efforts end up killing whole companies, and the often touted "efficiencies" or "synergies" leaders try to gain from mergers and acquisitions end up destroying economic value more often than they create.

The trick with redundancy is to repeat

By now you probably agree that redundancy is good - and it is. But how do you apply it to your organization and processes?

Before we go there, we have to tackle a very neat concept of mathematics. Fractals. Fractals have a property that is mind-boggling. Fractals are concepts that once explored end up generating infinite (yes, infinite!) information. In fact, a fractal line has infinite length even while fitting in a finite space! I won't bore you with the math details, but check this page on Wikipedia about the length of the British coast line - it has a neat demonstration of how a finite space can hold a line of infinite length.

This means that fractals are generative when it comes to information: they generate infinite amounts of information. And this happens to be a very useful property to have in mind when we explore how organizations work.

Making the case for infinity (and beyond)

In this post I argued that removing rules from your company's process book is actually better for your business and for your teams. The next step is to remove as many rules as you can, so that you end up with a small and simple set of generative rules - just like a fractal. Fractals are very simple equations that have in themselves an infinite number of solutions. And that's exactly what our processes should be: a small set of rules that, once in use, accommodate an infinite amount of possible behaviors - this is what I mean by "complex behavior" in the post on disciplined organizations.

Conclusion

Turns out fractals are perfect (yes, perfect - as in perfectly efficient) compression algorithms: a simple equation can be solved in an infinite number of ways, which when plotted in 2D or even 3D generate an infinite line in a finite space.

This property is extremely useful when applied to processes in your company because you cannot predict how people should behave in the future, but you can create an environment that - just like a fractal - allows every actor / person in the company to act in an infinite (and therefore practically unpredictable) number of ways and adapt to whatever the ever changing reality throws at them.

If you believe that your business environment is constantly changing, and that your organization is akin to a living organism you have to embrace the concept of fractal organizations.

Fractals work for you when they allow your blood vessels to reach every cell of your body (within a few cells distance), and when they allow your brain to store vast quantities of information even if it is small enough to fit in your head. Fractal organizations are organizations that can adapt in an infinite number of ways in response to an unpredictable environment.

If change is the only constant, how do I adapt to that?

Epilogue

Before we can understand how to apply the concept of fractal organizations and benefit from that, a very serious question must be answered: If people can behave in infinitely different ways, how do we prevent organizations from turning into chaos? That's a question we will explore soon - stay tuned! :)

Do you want to know more?

Title image credit: John Hammink, follow him on twitter.

Why projects fail, is why (we think) they succeed!

When I started my career as a Project Manager, I too was convinced that following a plan was a mandatory requirement for project success. As I tried to manage my first projects, my emphasis was on making sure that the plan was known, understood and then followed by everyone involved.

When I started my career as a Project Manager, I too was convinced that following a plan was a mandatory requirement for project success

I wrote down all the work packages needed, and discussed with the teams involved when those work packages could be worked on, and completed. I checked that all the dependencies were clear, and that we did not have delays in the critical path (the linear path through a project plan that has no buffer).

I made sure that everyone knew what to do, to the point that I even started using daily meetings before I heard of Scrum which would, later on, institutionalize that practice in many software organizations.

As my projects succeeded, I was more and more convinced that the Great Plan was the cause for their success.

As my projects succeeded, I was more and more convinced that the Great Plan was the cause for their success. The better the plan the more likely the project would succeed - I thought. And I was good at planning!

Boy, was I wrong!

It was only later - after several successful, and some failed projects - that I realized that The Plan had little effect on the success of the projects. I could only reach this conclusion through experience. Some of the projects I ran were "rushed", which made it impossible to create a Great Plan, but had to be managed "by the seat of the pants". Many of them were successful nonetheless.

In other cases, I did create a plan that I was happy with. Then I had to change it. And then change it again, and again, and again - to the point that I did little else but change The Plan.

Confusing the chicken with the egg, which came first?

The example above is one where I had confused the final cause (chicken) with the original cause (egg).

When something works well, we will often retrospectively analyze the events that led to success, and create a story/narrative about why that particular approach succeeded. We will assign a "final cause" to the success: in my example I assigned the "final cause" of project success to having a Great Plan, and the events that created the Great Plan.

This is normal, and it is so prevalent in humans that there is a name for it: retrospective coherence. Retrospective coherence is what we create when we evaluate events after-the-fact and create a logical path that leads from the initial state to the final state via logical causality, that can easily be explained to others. These causality relationships are what lead us to create lists of "Best Practices".

Best Practice lists are the result of Retrospective Coherence, and because of that many are useless

When the solution becomes the problem

However, this phenomena of Retrospective Coherence is not necessarily a good thing. In my initial example about Project Management I was convinced that the Great Plan and the related activities were the reason for success because that is what I could "make sense" of when I looked back in time. But as I gained experience I was forced to recognize that my "Best Practice" did not, in fact, help me in other projects. This realization, in turn led me to question the real reasons for success in my previous projects.

After many years of research and reflection I came to realize that many projects are successful purely by random reasons. For example: someone did an heroic effort to come to work during the week-end and recover the Visual SourceSafe database that had been corrupted once again and for the 1000th time!

But there are many other reasons why projects succeed by pure random chance. Here are some:

• In one project we had a few great testers that were not willing to wait to the end of the project to test the product. What they found changed the requirements and made the project a success
• Some projects were started so that we could deliver the "perfect feature set" to our customers. But as time went by and the deadlines were closing in, some managers - sometimes even me - understood that delivering on time was more important, and therefore changed the project significantly by reducing scope.
• Some developers were single handedly able to both, increase product functionality, while reducing the code base by 30%. This feat increased quality massively and made a delivery on time even possible.
• In one project we tried to use Agile. As a result, we started practicing timeboxed iterations and eventually ended up releasing so often that we could never be late

These are only a few of the reasons why projects succeed despite having a Great Plan, rather than because of it.

The Original Cause

The reasons for project success that I listed above are only a few that can be called "original cause" for project success. Original causes are those that actually start a chain of events that lead to success (or failure), but are too detailed or far into the past to be remembered while doing a retrospectively coherent analysis of project success (after-the-fact).

The kicker

But the kicker is this: when we get caught in "Final Cause" assignment through the retrospective coherence lenses or our logical mind, we lose a massive opportunity to actually learn something. By removing the role of "luck" or "randomness" from our success scorecard we miss the opportunity to study the system that we are part of (the teams, the organization, the market). We miss the opportunity to understand how we can influence this system and therefore increase our chances of success in the future.

Many people in the project management community still think - today - that having a Great Plan is a "Best Practice", and that you cannot succeed without one. I would be the first to agree that having a plan will increase your chances of success, but I will also claim that the Great Plan alone (including following that Great Plan) can never deliver success without those random events that you will never recognize because you are blind to the effects of chance in your own success.

In our lives we must, always, strive to separate Original Cause (what actually caused success) from Final Cause (why we think success happened by analysing it after-the-fact).

In a later post I will discuss how to increase the chances of project success by - on purpose - inserting randomness and chance into the project. Stay tuned...

Image credit: John Hammink, follow him on twitter

What is Chaos? And how we found out about it...

As he looked at numbers e noticed that the oscillations of his model did not repeat.

In fact he entered the numbers again, and again, and again but his model would refuse to behave the same way twice.

During our long education we were taught that given the initial state of a system (the present parameters that define the state of the system) plus the equations that fully describe the system, you will be able to plot the future behavior of the system forever.

And we have plenty of evidence that this approach works. For one, we are able to predict when Comet Halley will next visit our corner of the galaxy. And many astronomers were able to predict that last visit thousands of years ago!

So, why was his model stubbornly behaving differently every time he punched the numbers in? After all he knew the system perfectly - he had designed it!

The best way he had to describe this "unpredictable" behavior was with the word "chaotic", a never ending sequence of never repeating patters. Nothing was the same even if the initial state was the same and he was the one defining the equations for this toy-weather model. I mean he had defined ALL the equations...

It took a few days, but he figured it out. He had entered the parameters with a precision of 1/1000, but during processing the computer executing the model would use numbers with a precision of 1/1000000. On initial consideration this did not seem a relevant difference, after all a difference of 1/1000 was equivalent to having a butterfly flap its wings in China and having that create a storm in North America.

Systems that would never repeat in behavior even if they ran for ever

Later, this and other experiments would be repeated all over the world, in many different domains but the results would be similar. All over the world scientists were discovering other systems that were "sensitive dependence to initial condition" (aka suffered from the Butterfly effect), the scientific definition for "chaos" which later became the popular term to describe systems that would never repeat in behavior even if they ran for ever. These systems exhibited infinite variety of behavior when certain conditions were met. What were those conditions? That we will explore in a later post

Photo credit: John Hammink, follow him on twitter

The #NoEstimates questions, Part II

Last week we did not cover all the questions on #NoEstimates. Twitter gave us a lot more questions than could be listed last week, and even more interesting ones. Read on for the questions and my take on their meaning.

How do I choose between options or opportunities (without having estimates regarding the cost)?
• Choices are hard to make. When you are deep into the development process you inevitably come up with many ideas. But how should we choose between all those ideas without a detailed estimate? I believe this question implies that we have no way to assess the opportunity cost of selecting one of the options. I propose we challenge this assumption in a later post.
How to win a government contract without estimates?
• A very common question, also present implicitly in last week’s post when a questions asked: How to practice #NoEstimates in companies where estimates are "mandatory"? When the legal framework (or contract) requires you to provide an estimate of the length of the work you just have to do it, of course. But a more interesting question to tackle will be: how do we use #NoEstimates to our own (and the customer’s) benefit even when we have to provide estimates before we can win the contract?
What visual indicators do you have in #NoEstimates to show if you deviate from the "optimum"?
• Are we going in the right direction? Are we progressing towards the goal? Do we have the right features? Will we be on time? There are many implicit questions behind this one. While exploring the answers to these questions, I hope we can generate “heuristics” that are relevant and actionable for people interested in the “end game” for a project or a delivery. Including heuristics based on visualization.
Will #NoEstimates give me better predictability even when not using estimates?
• The holy Grail: predictability. In my view, the temptation is to say: if you have time-boxes you don’t need more predictability, you already have it. However, I think this question goes beyond “delivering on time”. In fact, I believe that this question is about predicting “everything” (scope, quality, cost, length, …) We have to explore what #NoEstimates can help with, and what it can’t help with when it comes to looking into the future.
If I just "play the game" with project managers and give estimates that I know aren't right, am I already doing #NoEstimates?
• This question is perhaps related to the one above where we tackle how to use #NoEstimates in an environment where we are asked to provide estimates. It must be said that lying (providing information you know not to be true) is not right, deceiving is not right - no matter what method you use to manage your projects. I hope to clarify how my practice of #NoEstimates can help you tackle the estimation requests you get, but still benefit from the added information #NoEstimates can provide.
Is there some analogy between SQL/NoSQL and Estimates/#NoEstimates?
• Many people comment on the choice of tag (#NoEstimates) for this wider discussion on estimating and managing projects. It is true that if you look at the hash tag used and the previous SQL/NoSQL discussions there is a similarity in the surface. However, I think there are further similarities between these two movements. I’ll explore those in a later blog post.
Does #NoEstimates aim to change existing social constructs (e.g. capitalism, budgets) to remedy the need to estimate?
• As far as I understand this question explores the possible impact and consequences of #NoEstimates, including on the budgeting cycle and the business models. I don’t think #NoEstimates tries to tackle business models at all, in fact I think it aims to enable more business models. On the other hand, #NoEstimates does try to change the way we look at budgets - project budgets at least - with the aim to make those more transparent and reliable. This topic promises to be very interesting as it has generated a lot of discussion on twitter.
Is it accurate to say #NoEstimates is for projects in Cynefin's complex domain?
• For those of you that haven’t followed the Cynefin model or Complex Systems discussion, let me just clarify that Complex Systems are such that causality can only be attributed in retrospect, i.e. we cannot anticipate how the system will react to a certain stimulus, therefore we must experiment (Probe-Sense) and Respond to the results of those experiments. While I believe that #NoEstimates is a perfect fit for Complex environments, I don’t think it applies only in that domain. There are benefits to be gained in other domains: Cynefin/Complicated, Cynefin/Chaotic and even Cynefin/Simple.
If we elect not to take on the risk of predicting an uncertain future, who do we think should, and why?
• I believe this question is trying to answer: Who should take the risk for the (future) results of our actions? Obviously I think we - the people in the project - should take the responsibility for tackling the project in all its characteristics and risk. I don’t think #NoEstimates leads to a shift of the risk carried in any project, and in fact I believe #NoEstimates approaches can be much more effective at uncovering and tackling emergent project threats or risks. More on this later.
Why would we choose not to judge likely outcomes, when worthwhile human endeavors necessarily carry risk?
• My interpretation of this question is that it asks: Should we even look into the future at all? Equating #NoEstimates with #NotLookingIntoTheFuture is a red-herring in my view. I’ll try to explain in a later post how, in my view, #NoEstimates approaches can deliver a much better model of “the future” for any project.
What is and isn't 'estimating' when people form and act on expectations of an uncertain future every day?
• If I remember correctly somebody tweeted: “how can you say “don’t estimate!” when even the act of getting up in the morning requires estimating?”. Perhaps this comment comes from a misunderstanding of the domain we are talking about when we use the term #NoEstimates. It is important to state that we are using #NoEstimates in the context of estimating large software endeavors (i.e. projects), not simple tasks or even fixed/variable-costs calculations (e.g. servers, bandwidth, person-hours, etc.).

This is the list of questions that I’ve collected in the last few weeks on twitter. I hope to answer them here on the blog with your help. Please contribute to the prioritization of these (and last week’s) questions by leaving a comment below with the questions you’d like to tackle first.

Once prioritized, I’ll start answering those questions and we can have a longer discussion about each of them here. After all, twitter, is a great conversation starter, but we need blogs to develop them further ;) Photo Credit: Brad Hoc @ flickr

Kanban vs Scrum, the ultimate fight? Don't think so, here's why:...

Wow, what a week! A BIG post on Kanban by Scrum evangelist @jcoplien litterally put the blogosphere (and twittersphere on fire!).

It is good to have these family fights in the Agile family once in a while. As a life-philosopher once said: "These things gotta happen every five years or so, ten years. Helps to get rid of the bad blood".

But what are the differences between Kanban and Scrum, really? What are they?

Here's my take on the differences:

Kanban innovation

Kanban is, from it's origin a more systematic approach to measuring, visualizing and following up work in a "system" (one team, many teams, a company you name it). Thanks to the work by the Poppendiecks, David Andersson and Don Reinertsen (and others I'm sure) a pretty interesting and innovative economic framework as been put into the software process development lingo. Cost of Delay, Queues, optimize the whole, etc.
This economic framework is, in my view, the major innovation that Kanban proponents bring to the table. This was to be expected as many of the early adopters of Kanban were using Scrum before and felt the need to quantify and analyze some aspects of software development that Scrum did not tackle.

Scrum innovation

Scrum brought to the fore-front of process discussions issues that had never made it to our attention before: Self-organization, people/team dynamics, problem solving within a short cycle with feedback loops in place (Sprint), etc.
The major innovation of Scrum in my view was the introduction of the Socio-technical system of software development as Liker describes it in the Toyota way. Other similar software development processes took some of the ideas that Scrum also took, but shied away from the self-organization at the team level and blocker-removal focus of roles such as the scrum master. Those methods did not last long because the teams felt like puppets instead of adults with the possibility (and responsibility) to produce the best product they could.

Wrap-up

Both Kanban and Scrum brought significant innovations to the software development world. Those are just two types of innovations. There is more coming from the community and instead of focusing only on these two methods (which all of you should experiment with!) we should also look at what else is missing in the software development ecosystem.
My next interest area is "Complex Systems" (Complexity + Systems Thinking) and Management as a profession. I'll be exploring those subjects more and more in the future as a way to complement my use of Kanban *AND* Scrum. I suggest you do the same and find what else is missing in your environment and look for what else is around that could help you complement what you are already using. Here's a suggestion: start with @jurgenappelo's book on Management!

Happy reading, happy learning!

Kanban in a networked process -- Visualise the network!

It seems that the idea of a work-network instead of work-flow is catching on and I'd like to throw a bit more fuel in that fire!

Knowledge work is not linear! This we have learned from the experiences with failed Waterfall projects. There are things we do over and over again, and there are things we do only once in a project. None of those can easily be predicted. Scrum and Kanban are examples of our quest to distill what we do to the bare essential. Only then can we understand and manage our work.

Scrum does this by strictly limiting the work in progress through the concept of time boxes. The idea is: do something that fits a (reasonably) short timebox, and nothing else. Get that done and released before you go on to the next thing.

Kanban does this by strictly obeying Work In Progress (WIP) limits and establishing a Pull system (instead of push as much content as you can into an iteration). Both have pros and cons which I will not discuss here. http://www.blogger.com/img/blank.gif

But most interestingly, both tackle the flow of our work in a linear fashion. Both Scrum and Kanban assume that some work can "flow" orderly to completion. However that is not the case! In knowledge work we have many loops and re-loops in the process that are hard to visualize and follow. Without visualizing those loops we cannot effectively manage our work!

Jurgen Appelo and Allan Shalloway had (what looks like was) an interesting discussion on the non-linearity of work and consequently on the non-linearity of Kanban boards. Check out Jurgen's and Allan's posts on the subject.

Since I've been experimenting with Personal Kanban for 3 months now I'd like to share some of my own experiences.

The picture below depicts my process boards as they evolved (click for larger images). The initial one was simple and linear, and it did not disclose enough information for me.


The second one was much better in highlighting some of the bottlenecks in my process (waiting for my action, waiting for others action, waiting for meeting), but was still too linear to reflect the work in a way that was useful to visualize.


The final Kanban board is actually what I use now and is a slight modification of the second one. As you can see it depicts the networked nature of my work and therefore also helps me set my WIP limits appropriately (for example why not have many things waiting for meetings?).


A conclusion of mine from the above boards is that it is not possible to set WIP limits without understanding the work (for example what "wait" states do you have in your process). On the other hand, trying to follow the WIP limits you have set will help you visualize those "wait" states as a consequence of the search for sustainable WIP limits

How about you? What are your experiences in using Kanban boards or other visualization techniques to uncover the networked nature of our work? Leave a comment below with a link to your post!

Photo credit: sjcockell @ flickr

Dude! Where's my manager? or Why you should attend LESS2011

Many teams start their agile transition from the "developer-side". This is quite normal, developers (coders and testers) feel the pain more than others because they need to actually get the product/system finished. But by focusing on developers, aren't we missing something? Aren't we forgetting that many of the consequences we so detest come from un-informed decisions by people higher in the chain?

In my experience many agile transitions fail by not involving managers in the process. Sure, it is possible to change how your team works with minimal involvement from your manager, but at some point the team is constrained more by management decisions than actual technical practices or even understanding of the product.

I remember a story of a team that was on their path to agile adoption, but could not progress further because management had decided that certain tools should be used. The usual explanations were given: "IT can only support one tool", "we need to harmonize our tool landscape", etc. Whatever the reason for the decision what we can say is that in this case management had a real - and negative - impact on the team's capability to deliver working software.

Sure we can go rogue and use our own tool chain "hidden" from IT and our manager (and many teams have done that), but that is not a long term strategy. Sooner or later we will bump again against a set of decisions that will hinder us from progressing.

I believe we need a new model for Agile adoption. One that includes the managers, leaders, VP- C-level people in our organizations.

It is this belief that has led me to participate in a project to organize a set of conferences focused on the role of leaders and managers. Last year we organized the first of that set of conferences in Helsinki and named it LESS2010. This year we are continuing that project with LESS2011 in Stockholm.

There are many reasons for you or your manager to attend the LESS2011 conference in Stockholm. From the individual speakers (check-out our keynote line up) to the people you will meet. But If I had to name one reason it is this: Agile and Lean adoptions require our managers to understand the new mindset, and without that we are bound to fail. So, get yourself and your manager to LESS2011 and talk to other managers! Share your experiences and questions and come learn from people that have been facing long lasting agile transitions (the kind that requires management involvement)!

LESS goes Stockholm for LESS2011, join us there in October

Last year I was involved in the organization of a unique conference. LESS2010 was a unique conference because it brought together different communities. We wanted to make sure that people with different points of view would discuss those points of view. The aim was to create a space for sharing ideas from different communities in the hope that new ideas would emerge. And emerge they did!

When the conference was done we had brought together the academic community which is involved in researching Agile adoption in the real world; the Agile community that is day-in-day-out applying Agile ideas in their own places of work; the Lean community and finally the Beyond Budgeting community that is sharing novel ways to manage and lead companies. This year we are doing the same, only better!

LESS goes Stockholm for LESS2011

You can check the details of the conference in the web-site, but here's a hint. We are working again on bringing different communities together!

The tracks that we have chosen are around the topics of Agile, Lean, Beyond Budgeting, Complexity Sciences, Systems Thinking and one topic that we deal with every day: Organization Transformation.

This year the trend is even more clear that LESS2011 is a conference for leaders in R&D organizations. Perhaps the only leadership focused conference in Europe at this point.

I do hope that more of these start happening because that's the next frontier for Agile and Lean adoption in our places of work.

So, I encourage you to submit a paper/session to the conference. Being there and talking to the the people with the experience and ideas is very important for us to continue our day-to-day work with new ideas and fresh energy!

See you in Stockholm!

You cannot transition to Agile. Stop and just embrace it!

I am writing this blog post to explore a concept. So bear with me, I'll probably ask more questions than I'll answer.

Why do most Agile fail in our companies (or government organizations for that matter)? My view is that we cannot actually transition from a command and control management paradigm to Agile / Complex management paradigm. The reasons are not fully clear to me, but I believe that it has something to do with the fact that we actually (typically) try to use a pre-determined way to make those transitions happen.

Case in point: When we try to move from Waterfall software development to Agile software development, we will typically draw a plan up for the transition with "steps" or "phases". Those "phases" or "steps" will typically be "stable points" in the evolution of our system (the company or organization). However, the Agile / Complex management paradigm assumes, at its core that software work is complex, therefore there is no predictable causality. The consequence of this is that the "steps" or "phases" in between the command and control paradigm and the Agile paradigm cannot themselves be "stable" in the sense that predictability can be recognized.

By following the argument above I'd state that: transitions fail because we try to move from a command and control paradigm to an Agile / Complex paradigm by applying command and control models. It is impossible to 'move orderly to a complex environment'.

What does it mean in practice for us? Well, for starters we cannot "plan" the transition in the same way we tried to plan our waterfall projects in the past. We can, and should have a goal or an idea of where we want to be. But after that we must embrace the new paradigm, or "Adopt the new Philosophy" as Deming put it. There are no intermediate steps between the "old command and control mindset" and the new "complex / agile mindset".

As this is an idea I'm still developing, I'll probably return to this subject and write some more, but in the meanwhile: what do you think? Does this make sense? What did you get from the above?

Photo credit: Marc Soller @ flickr

What can we learn from illusions that helps us manage projects better?

Just had the opportunity to watch a BBC documentary about illusions.

The show went on to detail how we as humans get deceived by our own senses. There were 3 things that opened my eyes in that documentary:

The things that caused me to think were:

• when we mix senses one of them tends to take over: this is the case when we are watching a magic act. The example on the documentary was a magician throwing a ball in the air for a few times, and then making the same gesture but keeping the ball in his had. Although the subjects eyes stayed on the face of the magician, the brain perceives the movement of the ball and later realizes that the ball has vanished.
  
• some of our senses are completely/100% contextual (see image above): the example they had here is a cube with many small squares on it's faces. Some faces are lighter, others are darker. When asked about the difference between 2 specific squares in the faces of the cube people would clearly state that the colors were different. However, when you moved one of the squares to the other face of the cube, you could see that the colors were exactly the same! Our perception of color was completely dependent on the colors surrounding the colors we were comparing.
  
• we can learn new senses if we are exposed to the right stimuli: The amazing examples they had for this was a blind-person that had learned to use sound and echo to see and used that learned sense to ride a bike (while blind!) The second example was quite amazing. A person would use a belt what was wired with buzzers/vibrators (like mobile phones) so that they could constantly feel where the magnetic north was. Later they would use that learned sense to navigate their way in a place while blind folded!
  

All of these observations from the documentary point to one thing. We are not wired to perceive reality. We jump to conclusions far too quickly, but we can also take advantage of that if we train our brain with the right stimuli! That was the most interesting find from this documentary.

The implications are huge in a software project setting, where it is quite common that people use only a very small number of stimuli (some reports, a couple of graphs, etc.) to jump into conclusions. We need to "learn" how to sense the status of a software project, and that can be done by following the example of the person who was taught how to "feel" the magnetic north...

Can you think of applications of this knowledge in your project?

Short multimedia review of LESS 2010 conference

The LESS 2010 conference just ended on a positive note of shared knowledge, bridge building between different communities and celebration of great work in the area of Software Development.

LESS was started with a great ambition of combining, from the starting point three different communities globally. Those communities were the Agile community, which is focused on the improvement of the software industry. The Lean community with both people from manufacturing as well as from software industry. And the Beyond Budgeting community, which comes from large financial and industrial companies and represents a turn in the way those companies tackle the problem of managing large organizations.

I hope that you have the chance to dive into each of these communities' body of knowledge as there are great contributions from each of those complementing what we have been doing in the Agile community for nearly a decade.

There was one piece of data that particularly impressed me. That data does emphasize something that many of us have felt, but putting a number on it does make it even more impressive. The first keynote presented a figure from an evaluation of work done in a company. The amount of tasks that were blocked (could not progress) was 62%. This is amazing, most of the work in that company was blocked, and so people would start new tasks and, guess what: get blocked on those. The queuing theory's prediction of "the more tasks you start the less you finish" was quite clear here.

There was one talk and one paper that were highlighted by the organizers as the best in the conference. These selections are always subjective, of course. But it's worth highlighting them as they were very good sources of information about Agile adoption (the paper) and new ways of looking at the organization and inform the way we adopt Agile and Lean software development (the talk).

Maarit Laanti received the award for the best paper of the conference. Here's Maarit receiving the award.

Paper: Agile Transformation Study at Nokia - One Year After

Here's Maarit receiving the award:
UPDATE: This video has been removed


Jurgen Appelo received the award for best talk in the conference. His talk: "Complexity vs. Lean, the Big Showdown". You can find Jurgen's slides here.

Here's Jurgen receiving his award:

Jurgen Appelo receiving award for best presentation at LESS2010 from Vasco Duarte on Vimeo.

Finally the conference ended with the gala dinner, which I thought was a wonderful way to end the event where we meet so many new people. We celebrate the fact that we spent time together trying to understand the issues that we face every day, but with the help of different points of view and mental frameworks.

Very good 3 days. I'm already looking forward for next year's conference!

PS: Watch this space as I'll publish an amazing surprise that the LESS2010 organizers prepared for the participants! :)

The three main trends in long agile adoption processes

I was having a discussion with fellow agile adopters yesterday and a bunch of interesting patterns emerged from the discussion:

• We are getting to agile/process improvement fatigue
• The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach
• We don't have a process to learn at the organizational level

Here are some ideas I brought with me from yesterday.

We are getting to agile/process improvement fatigue

This is certainly a familiar sentiment among those of us that have been involved in agile adoptions for several years. One trend/common sentiment I felt yesterday is that agile and lean are seen as just another trend that will pass. Many are trying to look for "better" approaches in meta-approaches (like Lean over agile, or Toyota-driven ideas over Lean, etc.)
This meta-approach is seen as solving the problems with the previous approach.

There's certainly some truth to that. One cannot really improve without going deeper in the process of understanding, and that naturally leads to more abstract approaches. Something the Alistair Cockburn readers will recognize as passing from the Shu-level to the Ha-level.

I don't see this as a problem, I see that as a natural learning process. Agile in general and Scrum in particular have been around for long enough for people to recgnize it's shortcomings and to look for "better" ways. It's the learning process. The problem is when people don't really look for the theory behind the patterns they are trying to apply and just change patterns (say from Scrum to Kanban) without understanding why.

As long as we fail to recognize the theory behind what we are applying, we are doomed to make the same mistakes, over and over again.

The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach


This was one trend that I raised in the discussion. Here's my argument summarized.
Agile is the result of a fundamentally different approach to management (analytical management vs. complexity-aware management). However, the management that is driving/leading the adoption of Agile software development is versed and informed by a completely different way of thinking. They live and bread a different paradigm of management. Until that changes it is impossible to make agile adoptions sustainable.

Symptoms of this problem are quite common and likely familiar to you. Managers set targets for agile adoption, but don't change the way they manage the organization (a requirement to meet those targets). Managers say they want agile adoption but at the same time try to micro-manage the teams, don't respect their need for self-organization and set all kinds of policies that go against teams taking responsibility over their work. And the list goes on.

Main point: there's a basic conflict between the analytical management mind-set and agile adoption. Until we solve that the best we can do (and it's a lot already!) is to get our teams to be agile, but not the structure that directly affects and limits their performance.

We don't have a process to learn at the organizational level

The point that follows from the previous is that in order to be able to get out of the analytical management paradigm we need to build a way for the organization to learn, to experiment and then adapt.

The lack of this method of learning, a meta-management method if you will, is what dooms us to be stuck in the previous paradigm. There is plenty of information around that can help managers learn and improve their work the same way we ask teams to do. But that learning is not happening in the vast majority of companies adopting agile. Until that happens agile is firmly stuck in the realm of the team developing software. It's like buying F1 tyres for a family car and expecting it to win a rally race.

Conclusion

What encourages me is to see that fellow colleagues in the agile journey also understand these issues and are (some of them at least) directly addressing these.

I hope they succeed and share their knowledge with the rest of us...

Photo credit: Denis Collette @ flickr

Don't just plan, plan to be surprised!

Walking a tightrope without a net maybe good entertainment, but it's hardly what project teams enjoy the most. 

When the deadline comes and your team is late, you know why that is. No matter how well you plan (and you must) there will always be a volcano or a flood or simpler things like changing requirements or power-outages that will delay the project.

Delays are inevitable, we plan to get rid of the obvious ones, we do creative risk management to get rid of the less obvious ones, but ultimately it is impossible to avoid problems! It is smarter to be ready to deal with problems when they do happen.

This is why I liked this post by Dave Snowden about two seemingly contradictory strategies to deal with environmental disasters (or project problems, which ever applies to you). In the post he states:

They did recognise that some failures were inevitable, so they focused on speedy detection and fast recovery. In other words they adopted a strategy of resilience rather than one of robustness.  

There's no reason why you should use only one of these strategies in your projects, in fact they are complementary. Investing in planning is about implementing a "robust" strategy. Investing in adaptation mechanisms (like iterations, re-planning, demo with retrospectives, etc.) is about implementing a "resilient" strategy.

It is insane to think that a single-minded focus on "resilience" is a good idea, but so is a single-minded focus on "robustness"! Why do so many projects prefer either over the other? 

Photo credit: wollbinho @ flickr