5 No Estimates Decision-Making Strategies

One of the questions that I and other #NoEstimates proponents hear quite often is: How can we make decisions on what projects we should do next, without considering the estimated time it takes to deliver a set of functionality?

Although this is a valid question, I know there are many alternatives to the assumptions implicit in this question. These alternatives - which I cover in this post - have the side benefit of helping us focus on the most important work to achieve our business goals.

Below I list 5 different decision-making strategies (aka decision making models) that can be applied to our software projects without requiring a long winded, and error prone, estimation process up front.

What do you mean by decision-making strategy?

A decision-making strategy is a model, or an approach that helps you make allocation decisions (where to put more effort, or spend more time and/or money). However I would add one more characteristic: a decision-making strategy that helps you chose which software project to start must help you achieve business goals that you define for your business. More specifically, a decision-making strategy is an approach to making decisions that follows your existing business strategy.

Some possible goals for business strategies might be:

• Growth: growing the number of customer or users, growing revenues, growing the number of markets served, etc.
• Market segment focus/entry: entering a new market or increasing your market share in an existing market segment.
• Profitability: improving or maintaining profitability.
• Diversification: creating new revenue streams, entering new markets, adding products to the portfolio, etc.

Other types of business goals are possible, and it is also possible to mix several goals in one business strategy.

Different decision-making strategies should be considered for different business goals. The 5 different decision-making strategies listed below include examples of business goals they could help you achieve. But before going further, we must consider one key aspect of decision making: Risk Management.

The two questions that I will consider when defining a decision-making strategy are:

• 1. How well does this decision proposal help us reach our business goals?
• 2. Does the risk profile resulting from this decision fit our acceptable risk profile?

Are you taking into account the risks inherent in the decisions made with those frameworks?

All decisions have inherent risks, and we must consider risks before elaborating on the different possible decision-making strategies. If you decide to invest in a new and shiny technology for your product, how will that affect your risk profile?

A different risk profile requires different decisions

Each decision we make has an impact on the following risk dimensions:

• Failing to meet the market needs (the risk of what).
• Increasing your technical risks (the risk of how).
• Contracting or committing to work which you are not able to staff or assign the necessary skills (the risk of who).
• Deviating from the business goals and strategy of your organization (the risk of why).

The categorization above is not the only possible. However it is very practical, and maps well to decisions regarding which projects to invest in.

There may good reasons to accept increasing your risk exposure in one or more of these categories. This is true if increasing that exposure does not go beyond your acceptable risk profile. For example, you may accept a larger exposure to technical risks (the risk of how), if you believe that the project has a very low risk of missing market needs (the risk of what).

An example would be migrating an existing product to a new technology: you understand the market (the product has been meeting market needs), but you will take a risk with the technology with the aim to meet some other business need.

Aligning decisions with business goals: decision-making strategies

When making decisions regarding what project or work to undertake, we must consider the implications of that work in our business or strategic goals, therefore we must decide on the right decision-making strategy for our company at any time.

Decision-making Strategy 1: Do the most important strategic work first

If you are starting to implement a new strategy, you should allocate enough teams, and resources to the work that helps you validate and fine tune the selected strategy. This might take the form of prioritizing work that helps you enter a new segment, or find a more valuable niche in your current segment, etc. The focus in this decision-making approach is: validating the new strategy. Note that the goal is not "implement new strategy", but rather "validate new strategy". The difference is fundamental: when trying to validate a strategy you will want to create short-term experiments that are designed to validate your decision, instead of planning and executing a large project from start to end. The best way to run your strategy validation work is to the short-term experiments and re-prioritize your backlog of experiments based on the results of each experiment.

Decision-making Strategy 2: Do the highest technical risk work first

When you want to transition to a new architecture or adopt a new technology, you may want to start by doing the work that validates that technical decision. For example, if you are adopting a new technology to help you increase scalability of your platform, you can start by implementing the bottleneck functionality of your platform with the new technology. Then test if the gains in scalability are in line with your needs and/or expectations. Once you prove that the new technology fulfills your scalability needs, you should start to migrate all functionality to the new technology step by step in order of importance. This should be done using short-term implementation cycles that you can easily validate by releasing or testing the new implementation.

Decision-making Strategy 3: Do the easiest work first

Suppose you just expanded your team and want to make sure they get to know each other and learn to work together. This may be due to a strategic decision to start a new site in a new location. Selecting the easiest work first will give the new teams an opportunity to get to know each other, establish the processes they need to be effective, but still deliver concrete, valuable working software in a safe way.

Decision-making Strategy 4: Do the legal requirements first

In medical software there are regulations that must be met. Those regulations affect certain parts of the work/architecture. By delivering those parts first you can start the legal certification for your product before the product is fully implemented, and later - if needed - certify the changes you may still need to make to the original implementation. This allows you to improve significantly the time-to-market for your product. A medical organization that successfully adopted agile, used this project decision-making strategy with a considerable business advantage as they were able to start selling their product many months ahead of the scheduled release. They were able to go to market earlier because they successfully isolated and completed the work necessary to certify the key functionality of their product. Rather then trying to predict how long the whole project would take, they implemented the key legal requirements first, then started to collect feedback about the product from the market - gaining a significant advantage over their direct competitors.

Decision-making Strategy 5: Liability driven investment model

This approach is borrowed from a stock exchange investment strategy that aims to tackle a problem similar to what every bootstrapped business faces: what work should we do now, so that we can fund the business in the near future? In this approach we make decisions with the aim of generating the cash flows needed to fund future liabilities.

These are just 5 possible investment or decision-making strategies that can help you make project decisions, or even business decisions, without having to invest in estimation upfront.

None of these decision-making strategies guarantees success, but then again nothing does except hard work, perseverance and safe experiments!

In the upcoming workshops (Helsinki on Oct 23rd, Stockholm on Oct 30th) that me and Woody Zuill are hosting, we will discuss these and other decision-making strategies that you can take and start applying immediately. We will also discuss how these decision making models are applicable in day to day decisions as much as strategic decisions.

If you want to know more about what we will cover in our world-premiere #NoEstimates workshops don't hesitate to get in touch!

Your ideas about decision-making strategies that do not require estimation

You may have used other decision-making strategies that are not covered here. Please share your stories and experiences below so that we can start collecting ideas on how to make good decisions without the need to invest time and money into a wasteful process like estimation.

The Release Paradox: releasing less often makes your teams slower and decreases quality

Herman is a typical agile coach. He works with teams to help them learn how to deliver high-quality software quickly.

Many teams want to focus on design, architecture, or (sometimes) even on business value. But they are usually not in a hurry to release quickly.

Recently Herman conveyed a story to me that illustrates how releasing quickly can help teams deliver high-quality software much faster than if they would focus on quality in the first place. This is the case of a team that was working on a long overdue project. They had used a traditional and linear process in the past and had been able to release software only very recently, after more than 12 months of work on the latest release.

Not surprisingly, they were having trouble releasing regularly. The software was not stable; once it was live it had many problems that needed to be fixed quickly, and worst of all: all of this was having a direct impact on the company’s business.

The teams were extremely busy fixing the problems they had added to the product in the last year and could not focus on solving the root causes of those problems.

They were in full-fledged firefighting mode. They worked hard every day to fix yet another problem and release yet another hot fix.

This lasted for a few weeks, but once the fire-fighting mode was over, Herman worked with the teams to improve their release frequency. During their work with Herman, those teams went from one year without any release to a regular release every two weeks.

At first the releases were not always possible, but with time they improve their processes, removed the obstacles preventing them from releasing every two weeks and started releasing regularly.

What happened next was surprising for the teams. The list of problems after each release did not grow - as they expected - but instead shrank.

When some problems came in from the customers after a 2-week release, they were also much faster to fix the problem and quicker to release a fix if that was required. When the fix was not critical, they waited for the following release which was, after all, only 2 weeks away.

By focusing on releasing every two weeks, Herman’s teams were able to focus on small, incremental changes to their product. That, in turn, enabled them to fine-tune their development and release processes.

Here are some of the key changes the teams implemented

1. They started with a 4 week release cycle, and fine-tuned their daily builds and release testing process to enable a release every 2 weeks.
2. They invested time and energy to improve their test automation strategy and automated the critical tests to enable them to run “enough” tests to be confident that the quality was at release level.
3. They had some teams on maintenance duty in the first few iterations to make sure that any problem found after release could quickly be fixed, and released to customers if necessary.
4. They changed their source code management strategy to enable some teams to work on longer term changes while others worked on the next release.
5. They involved all teams necessary to complete a release in their iterations. This affected especially: production/operations team, localization team, documentation team, marketing team, and other teams when needed.

This list of changes was the result of the drive to complete each release and learning from the failures in the previous release. Some changes were harder to implement, and especially the testing strategy to allow for 2-week release cycles had to be changed and adjusted several times.

One of the key problems the teams had to solve, was the lack of coordination with departments that directly contributed to the release but were not previously involved in their day-to-day work.

This process lasted several months, and would not have been possible without a clear Vision set forth by the teams in cooperation with Herman, who helped them discover the right way to reach that Vision within their context.

Herman’s work as a coach was that of a catalyst for management and the teams in that organization. He was able to create in their minds a clear picture of what was possible. Once that was clear, the teams and the management took ownership of the process and achieved a step-change in their ability to fulfill market demands and customer needs.

Customers have no reason to change provider as they have an ever-improving experience when using this company’s services.

Today, this organization releases a new version of their product every two weeks. Unaware of it, their customers receive regular improvements to the product they use, and have no reason to change provider as they have an ever-improving experience when using this company’s services.

Picture credit: John Hammink, follow him on twitter

How to create a knowledge worker Gemba

I am a big fan of the work by Jim Benson and Tonianne Barry ever since I read their book: Personal Kanban.

In this article Jim describes an idea that I would like to highlight and expand. He says: we need a knowledge worker Gemba. He goes on to describe how to create that Gemba:

• Create a workcell for knowledge work: Where you can actually observe the team work and interact
• Make work explicit: Without being able to visualize the work in progress, you will not be able to understand the impact of certain dynamics between the team members. Also, you will miss the necessary information that will allow you to understand the obstacles to flow in the team - what prevents value from being delivered.

These are just some steps you can take right now to understand deeply how work gets done in your team, your organization or by yourself if you are an independent knowledge worker. This understanding, in turn will help you define concrete changes to the way work gets done in a way that can be measured and understood.

I've tried the same idea for my own work and described it here. How about you? What have you tried to implement to create visibility and understanding in your work?

Management science's impossible quest: in search of predictability

Many years ago some people came up with a brilliant idea: "what if, we could turn abundant lead (Pb on the table of elements) into scarce gold?". A brilliant idea to be sure, but as we know now, impossible. What made Alchemy grow and thrive as a science, was that people at that time did not know that it was impossible. They did not have enough knowledge of chemistry to know that you cannot turn an element's atom into another element's atom. That did not stop them however! And thanks to those crazy idealists today we have Chemistry. How about that for a twist of fate?

Alchemy's history can be traced back for millennia, and that name stands as the prototypical failed science (except for poets and … idealists). Failed because it based its existence on a belief that was impossible to make reality in our world.

I find many parallels between Alchemy and Management science today. Alchemy wanted to find the magic Philosopher's Stone that could turn led into gold, Management scientists are in a similar quest: in search of the method that can make management succeed as a science. Management science's goal today is to find the magic ingredient that can give us predictability.

Basic Science

Many management-fads are based on the hypothesis that you can control the world around you, mold it in a way that can give you predictability. This (mythical) predictability is a core assumption that enables the management's idea of success: I will do X so that I can achieve Y and therefore succeed.

This predictability is, however, a form of causality. Many, perhaps including you, will not think of it that way, but predictability is only present in systems where you can determine causality. A predictable world is one where we can reliably predict what happens when we do X, i.e. a world where we can establish causality.

Newton's laws gave us predictability in the physical macroscopic world of objects, because they told us what would happen if, for example, you rolled a sphere in an inclined plane, or shot a bullet from a canon, or shot an arrow with a bow. Relativity theory gave us predictability (to a certain extent) in the microscopic world of atoms, or sub-atomic particles also because it established useful and applicable causality rules

Medicine gives us (mostly) predictability in the world of diseases and biologic systems like the human body because it has a system of studying and documenting causality: if you take Aspirin after a great Saturday night party, you will feel milder effects of the oncoming hangover.

You can see why management theorists (and practitioners) would want to achieve the same goal: predictability.

Predictability is management science's Philosopher's stone.

The Philosopher's stone

Planning is one of those attempts at achieving predictability. We plan, because we believe that planning will give us the predictability we seek. Without the goal of predictability we would not plan, there would be no point.

But planning itself rests on the assumption that we can predict (here it comes again) causality in the world where the plan is to be executed. If we did not believe that we could predict B to happen after doing A, then planning would not be possible. It would be illogical.

We plan to be able to predict, but plans only work if we can predict in the first place!

The problem with what is stated above is that it is a circular dependency: we plan to be able to predict, but the act of planing rests on the assumption that the world is predictable. You see where this is going, right?

The sad fact is that there are many other management techniques that both rest on, and try to achieve predictability. For example: pay for performance, management by objectives, or the yearly strategic plan.

Going around in circles

All these management techniques have one thing in common. They are static. They exist at a precise point in time: Strategy is defined in the Spring or Autumn and is supposed to be executed for the next 12, 18, 24 or more months. Objectives or goals upon which our evaluation is based are set several months in advance of the point of evaluation.

All these techniques require perfect, 20/20 foresight. How likely is that?

The basic problem with most perspectives on management today is that they are static analyses of a future environment. And all decisions are made because we believe we can predict the future.

We make these predictions to help our businesses be more predictable, but our predictions depend on our business being predictable. This circular dependency is why most management approaches are doomed to failure. Like Alchemy.

Looking forward

We need a new Management Science, one that does not require the existence of a predictable world. When it defines goals, it does so in a way that is dynamic, i.e. the goals change with the observation of reality. This dynamic adaptation process is what we, in the Agile community, call a feedback loop.

A future-proof management approach must start by basing it's core assumptions on the existence of feedback loops that must be studied and tested as we make decisions. And these feedback loops must be the driving factors for action in management. Not the plans!

How would such a management approach look like?

The first assumption of such a management approach must be that the world is not predictable beyond a very short time. This may be a simple statement, but the consequences of this simple statement are fundamental.

• First, management techniques must not be based on the existence of a perfect, predictable future. Management techniques must be based on the acknowledgement that we cannot predict what is the best (or worst) possible outcome of our actions. What good is it to plan to sell 100 widgets when the market is - unexpectedly - demanding us to produce 1000 widgets? or 10 widgets?
• Second, management techniques must be designed to include feedback loops that inform action. Not just as information collection loops, but as fundamental action-defining loops. For example: every single decision is temporary. Every decision must state it's assumptions and when one assumption is proven wrong, the decision must be reverted or at a minimum re-evaluated. When we execute our plans, we must be alert to surprises; without detailing our assumptions we can never be surprised. Surprises are triggers for action - no surprises, no learning, no adaptation.
• Third, an individual's performance cannot be predicted up front. This tells us that we can only evaluate an individual's performance after we have the facts of what happened. Maybe a sales person failed to meet their sales target, but that could be because they helped R&D get the first pilot customers for an entirely new business!

What would your world look like if you believed and followed the three principles above? How would it differ from your current perspective on management?

Image credit: John Hammink, follow him on twitter.

The simple recipe for disciplined organizations

One question puzzles non-Agilists more than any other question. It is the question that uncovers why Agile does not fit their view of the world. A question that makes non-Agilists feel insecure and reject Agile completely or mostly. This question is: how can less structure, and less planning deliver software more reliably, and with higher quality, and faster, and with better usability, and with better customer satisfaction?

Do you know the answer to this question? Many Agilists don't. Here's one attempt at answering that question.

Agile is different

Those of us that have been practicing Agile for a while have learned to "break the rules" - or reach the Ha or Ri levels in the Shu-Ha-Ri model. We've practiced, and practiced, and practiced. We've learned what works (some times) and what does not work (most of the time), and we have been able to hone our practice to a point where Agile just feels "natural". But what is "normal"? How would you, a Ha or Ri level practitioner of Agile define Agile?

This reminds me of a story I heard from Angel Medinilla, a dear friend and fellow Agile journeyman:

In a dojo somewhere in a southern country, the Aikido apprentice reaches out to the sensei to understand the secret of Aikido.

Aikido apprentice: Sensei, what is the secret of Aikido?
Sensei: The secret is to breath, to balance, to connect with your opponent and whole body movement.

The apprentice was frustrated, he was expecting something enlightened and concrete for him to use.
Then, after 40 years of practice the apprentice finally says to his sensei:

Apprentice: Sensei, finally I understand the secret of Aikido
Sensei: Good. So, what is it?
Apprentice: It was simple and in front of my eyes all along. The secret is to breath, to balance, to connect with your opponent and whole body movement.

The sensei in that story knew that the apprentice would eventually come to realize that Aikido was indeed "just" to breath, to balance, connecting with the opponent and whole body movement. So, how about Agile, what is Agile all about? And how does it deliver more discipline, with less rules?

Chaos is very disciplined

While I tried to make sense of Agile, I explored many possible models. One of them was Chaos Theory. As I explored the Theory of Chaos I found a very interesting video about one of the favorite topics in the Agile community: emergence. Emergence is the process by which a system (any system) organizes itself based on real rules and exhibits what we would call complex behavior. For example, in this post I explained how a colony of Ants can exhibit complex behavior by following just a few rules. That complex, adaptive behavior emerges from the rules, and the action of the individuals. The rules are real (there is a benefit in following them), and the resulting behavior is complex and adaptable but also, very very disciplined (because there is a benefit to that discipline, you follow it or you die of hunger).

Going back in time

The counter-intuitive aspect of emergence is that it goes against all accepted "truths" about organizational design. You see, emergence can not be explained by reducing our model of the system to simply modeling of the agents' independent behaviors. Emergence is a result of all the parts interacting in the system. If you remove one of the rules from the ant colony you would get something different (possibly even the break down of the whole colony).

Simple rules lead to complex behavior. Complex rules lead to stupid behavior.
--adapted from original quote by Dee Hock

Reductionism is the most common process for organization design used by humans today. We tend to start by modeling small independent parts, and when we are satisfied with each, we put them together. But there's a problem with that approach. When you try to design an organization, or a process, by first identifying all the necessary steps individually, and then you sequence them orderly you make the system stupid. Or, in other words, you create rules that remove the possibility for complex, adaptive behavior to emerge.

This process or analyzing, ordering, prescribing organizational and process design is labeled analytic, and it is based on the theory that if you break something apart (like the organization), and analyze those parts independently you will understand it sufficiently enough to know how the whole system works. The problem is that an organization, like any system, has dynamics that can only be seen when all the parts are assembled and never when they are analyzed separately.

By contrast, if we define clear and simple rules (think the rules of chess) and nothing else, we are allowing the system to - through emergence - exhibit complex, adaptive behavior. The big question comes next: where do you draw the line between simple rules that allow for complex behavior to emerge, and the analytic process? When do we cross from one to the other?

Drawing a line in the sand

This is not an easy question to answer, and it is highly context dependent. However, I have found that the moment you think of rules that apply to single agents in a system you are approaching that line (and possibly crossing it). For example: when you have specific rules for every type of agent in the system you are designing through a process of Analytic Reduction. Like when you have specific rules for Project Managers to follow; you have specific rules for Developers follow; you have specific rules for Testers to follow; etc. ad infinitum.

A heuristic that I have found useful is to remove rules whenever possible. Scrum, for example, does this by assigning no special role to any team member, but instead giving the team a set of simple rules and boundaries (timeboxed iterations, interaction with PO, Definition of Done, etc.)

This realization led me to understand the critical role that generalists play in facilitating complex adaptive behavior (but that will have to wait for a later post).

Conclusion

Complex adaptive behavior emerges in systems where the rules are simple, and the agents have a high degree of freedom to make decisions. By trying to analyze systems with an reductionist process we are removing the possibility for agents (us, the people) to make intelligent decisions based on their context, and therefore we are reducing the possibility for complex adaptive behavior to emerge. The analytical mindset kills emergence. But more importantly, when it comes to performance Emergence kicks Reductionism in the groin.

If you tell people where to go, but not how to get there, you'll be amazed at the results. - George S. Patton

Next time you are designing a process or an organization take this into account and remove as many rules as you can. That will help you achieve more! Define the goals and let the people surprise you with their results!

Photo credit: John Hammink, follow him on twitter

How injecting randomness into your project can help it succeed

Success and failure differ, very often, by very little. Take nature as an example. A small change in our DNA (a few mutated genes) can have catastrophic consequences. On the other hand, without these mutations humans would never have come into existence.

Humans and other species in the planet evolved because of chance (although not totally chaotic) changes in their DNA. As small change after small change happened, the surviving individuals were able to spread the viable, and ultimately the most adequate changes to the next generation - and the process repeats still in our own bodies today. If Nature has taught us something about improvement and evolution is that you must have a little bit of luck involved!

The DNA of a project

In software projects - my domain - I see some similarities to this natural model that are worth exploring. Projects also have a DNA which includes the structure and the communication connections between individuals. Communication being one of the most quoted causes for failure in software projects, and therefore one of its critical success factors.

The More I Practice the Luckier I Get

Once a project gets started, a particular structure is put in place (governance, management, teams, etc.). Very often that structure remains in place, and static until the end of the project. But is this a smart choice?

Another aspect of the project that rarely changes is the network of connections between the individuals. This network is what carries information from individual to individual and eventually "selects" the information that is acted upon by the project team. If a piece of information reaches an individual in the project, and that individual does not consider it relevant, that information will not spread further. In contrast, when a piece of information is perceived as important by an individual, that information will be actively spread by that individual.

The Illusion of Control

The structure that is set for a project strongly influences the communication links that can emerge between individuals. Who talks to whom? What meetings exist where people interact regularly? etc. But the reverse is also true. The existing communication network within an organization is a strong influence on what governance structure is put in place, how teams are formed, etc. These co-dependent processes create a positive (or negative) spiral of events for the project, but because neither (structure, or communication network) is often changed, that spiral is unstoppable. If it is positive, it will help the project succeed. If this co-dependent processes create, instead a negative spiral, that will relentlessly remove the chances of success for that project.

This co-dependent relation between two key processes (structure and communication network) is why we can increase the chances of success in our projects by simply causing random perturbations in the project. Randomness helps us explore different patterns for structure and communication networks.

As we carefully design and inject small - safe-to-fail - changes into the project, we can observe how it reacts and adjusts. Later, we can retrospectively amplify or remove/dampen the changes based on the outcome we see. If something works, keep it and ask how can you make it grow. If something creates a net negative outcome, dampen or remove it completely.

The cycle goes like this:

• Define and implement small, safe-to-fail changes in the structure or communication network for the project
• These changes lead to emergent behavior as the individuals and teams adapt to the changes
• A new project configuration emerges which drives new project results
• Finally, we evaluate the results of these changes and decide which components we will try to amplify or dampen

What Do You Mean Random?

There are many ways in which we can, randomly, explore better configurations for our projects. Below I list only a few to illustrate the concept:

• At the start of a project, let the teams chose their own composition. This establishes new connections between individuals in the project as some will choose to work with new team members. However, these new connections will not eliminate the previous connections between individuals. The net effect should be that your project now has a more connected communication network (more individuals with strong connections to each other). In practice this works just like when we form new neural connections in our brain: more connections leads to different thinking and acting patterns
• Organize common project events where people interact with each other outside the day-to-day routine. Planning events can be organized following a structured approach, but including also some unstructured time (à lá unconference, using e.g. Open Space) where people can interact based on their interests. This will also help form new connections between individuals in the project and spread information that would otherwise be locked down and not accessible
• Have regular project "coffee breaks" that happen at the same time and same place. In these events people can connect with other individuals and ask questions from the project management team or the most connected individuals in the project. This unstructured communication increases the chance that some piece of information will "jump the hierarchical borders" and reach people in decision-making positions, or people with influence that can later act on that information.

Conclusion

These practices are designed to inject randomness (unplanned situations or connections) into the project. The goal is not to create Chaos (a scary word for many project teams), but to generate new pathways for the information to flow within the project team, as well as novel project structures that are more adapted to the current challenges the teams face.

Using these (or other) practices will increase your project's chances for success. They don't eliminate the need for the project to be managed, or to have structure. They do increase the chance of success for the project by exploring new organizations and structures through a process of small (safe-to-fail) changes that can lead to unplanned, but ultimately superior performance.

These were just a few examples. How would you inject randomness into your project? Have you done that in the past? Please share your experiences in the comments for the benefit of others.

Photo credit: John Hammink, follow him on twitter

Why projects fail, is why (we think) they succeed!

When I started my career as a Project Manager, I too was convinced that following a plan was a mandatory requirement for project success. As I tried to manage my first projects, my emphasis was on making sure that the plan was known, understood and then followed by everyone involved.

When I started my career as a Project Manager, I too was convinced that following a plan was a mandatory requirement for project success

I wrote down all the work packages needed, and discussed with the teams involved when those work packages could be worked on, and completed. I checked that all the dependencies were clear, and that we did not have delays in the critical path (the linear path through a project plan that has no buffer).

I made sure that everyone knew what to do, to the point that I even started using daily meetings before I heard of Scrum which would, later on, institutionalize that practice in many software organizations.

As my projects succeeded, I was more and more convinced that the Great Plan was the cause for their success.

As my projects succeeded, I was more and more convinced that the Great Plan was the cause for their success. The better the plan the more likely the project would succeed - I thought. And I was good at planning!

Boy, was I wrong!

It was only later - after several successful, and some failed projects - that I realized that The Plan had little effect on the success of the projects. I could only reach this conclusion through experience. Some of the projects I ran were "rushed", which made it impossible to create a Great Plan, but had to be managed "by the seat of the pants". Many of them were successful nonetheless.

In other cases, I did create a plan that I was happy with. Then I had to change it. And then change it again, and again, and again - to the point that I did little else but change The Plan.

Confusing the chicken with the egg, which came first?

The example above is one where I had confused the final cause (chicken) with the original cause (egg).

When something works well, we will often retrospectively analyze the events that led to success, and create a story/narrative about why that particular approach succeeded. We will assign a "final cause" to the success: in my example I assigned the "final cause" of project success to having a Great Plan, and the events that created the Great Plan.

This is normal, and it is so prevalent in humans that there is a name for it: retrospective coherence. Retrospective coherence is what we create when we evaluate events after-the-fact and create a logical path that leads from the initial state to the final state via logical causality, that can easily be explained to others. These causality relationships are what lead us to create lists of "Best Practices".

Best Practice lists are the result of Retrospective Coherence, and because of that many are useless

When the solution becomes the problem

However, this phenomena of Retrospective Coherence is not necessarily a good thing. In my initial example about Project Management I was convinced that the Great Plan and the related activities were the reason for success because that is what I could "make sense" of when I looked back in time. But as I gained experience I was forced to recognize that my "Best Practice" did not, in fact, help me in other projects. This realization, in turn led me to question the real reasons for success in my previous projects.

After many years of research and reflection I came to realize that many projects are successful purely by random reasons. For example: someone did an heroic effort to come to work during the week-end and recover the Visual SourceSafe database that had been corrupted once again and for the 1000th time!

But there are many other reasons why projects succeed by pure random chance. Here are some:

• In one project we had a few great testers that were not willing to wait to the end of the project to test the product. What they found changed the requirements and made the project a success
• Some projects were started so that we could deliver the "perfect feature set" to our customers. But as time went by and the deadlines were closing in, some managers - sometimes even me - understood that delivering on time was more important, and therefore changed the project significantly by reducing scope.
• Some developers were single handedly able to both, increase product functionality, while reducing the code base by 30%. This feat increased quality massively and made a delivery on time even possible.
• In one project we tried to use Agile. As a result, we started practicing timeboxed iterations and eventually ended up releasing so often that we could never be late

These are only a few of the reasons why projects succeed despite having a Great Plan, rather than because of it.

The Original Cause

The reasons for project success that I listed above are only a few that can be called "original cause" for project success. Original causes are those that actually start a chain of events that lead to success (or failure), but are too detailed or far into the past to be remembered while doing a retrospectively coherent analysis of project success (after-the-fact).

The kicker

But the kicker is this: when we get caught in "Final Cause" assignment through the retrospective coherence lenses or our logical mind, we lose a massive opportunity to actually learn something. By removing the role of "luck" or "randomness" from our success scorecard we miss the opportunity to study the system that we are part of (the teams, the organization, the market). We miss the opportunity to understand how we can influence this system and therefore increase our chances of success in the future.

Many people in the project management community still think - today - that having a Great Plan is a "Best Practice", and that you cannot succeed without one. I would be the first to agree that having a plan will increase your chances of success, but I will also claim that the Great Plan alone (including following that Great Plan) can never deliver success without those random events that you will never recognize because you are blind to the effects of chance in your own success.

In our lives we must, always, strive to separate Original Cause (what actually caused success) from Final Cause (why we think success happened by analysing it after-the-fact).

In a later post I will discuss how to increase the chances of project success by - on purpose - inserting randomness and chance into the project. Stay tuned...

Image credit: John Hammink, follow him on twitter

Why processes fail us, and what to do about it

Organizations go through different levels of maturity as they grow. Some will say that they "learn to perform", others will say that they "become ossified and slow", yet others would say that they "mature". In my view neither of these classifications is accurate, they all touch on possible outcomes of a process of "aging". Organizations grow older, but what dynamics do we see in these organizations?

Organizations grow older, they age just like us!

A recurring pattern in organizations is that they create, develop and install processes. Processes are, for practical purposes, sets of rules that define how work should happen in those organizations. They are the rules we follow daily when we work.

These rules are necessary for a common understanding of expectations and roles for each of us. We need those rules or processes so that we know what to expect, and what is expected of us. Or do we?

What is the role for rules in an organization?

In the study of Chaos and Complex systems scientists have found that Complex or Chaotic systems exhibit infinitely complex behavior starting from very simple - perhaps even simplistic - rules. The most common example of these simple rules in documentaries about Chaos and Complexity is the way ants find their way to a new food source. The rules they follow are simple:

• Walk around randomly and lay a pheromone path
• When you find food, turn around and follow your pheromone path to the colony
• While you walk around, if you find (bump into) an existing pheromone path, follow it

PS: you can find a much more detailed explanation here.

Following these simple rules Ants can not only find food, but feed an entire colony. In fact, when observed from an external view point we see complex System behavior even if one Ant alone follows a very simple set of rules.

The complex behavior we witness is Complex, and Adaptive. Hence the term Complex Adaptive System (or CAS).

What does this have to do with us - humans - and companies?

In investigating CAS we have found that the more complex the rules that we define, the less likely that the system (or company/organization) will be Adaptive. In fact, the opposite is true. Companies often put rules in place to "clarify, and specify" the expected behavior, thereby making it simple - or even simplistic. One glaring example of this phenomena is the way companies develop highly sophisticated goal-setting processes that eventually end up setting goals that distort the behavior of the organization in a way that makes it lose sight of what matters: their adapation to the environment they exist in (customers, suppliers, society, etc.).

The more complex the process and rules, the less Adaptable the organization will be!

But there are more examples of this phenomena whereby defining complex rule systems leads, invariably, to simple - even simplistic - behavior.

What's the role for rules?

What is now clear from research, is that simple rules can lead to Complex and Adaptive behavior in the "system" or organization. For us managers, this means that we must avoid the temptation to develop complex set of rules and must be on the lookout for rules that add burden to the organization and possibly constraining behavior to the point that the organization is unable to adapt to the changes it faces in the market.

The recipe to foster adaptability in the organization is simple: when possible remove rules, when in doubt remove rules. Add rules only when the cost of not doing so is prohibitive (legal boundaries for example) or when you've learned something about your environment that should be codified for everybody to follow (you found out that a certain technology is too expensive or unstable).

But here is the most important rule for you: All rules should be created as a result of a root-cause analysis, never as a result of a knee-jerk reaction to some unplanned or unpredictable outcome.

The most important rule: No rules should be established without a thorough Root-cause analysis!

The quote "Keep it Simple" really means: use less rules and more feedback! Like Agile...

Image Credit: John Hammik, follow him on twitter

My secret to flow and a productive day

As I start my day, I have this routine I do. I review all the loose threads that I have in my mind, and write them down (a mindmap, usually). After I write them down I quickly evaluate what are the most important goals for the day. Then I write those goals on a post-it that I carry with me all day.

This piece of paper is my ticket to "flow".

This piece of paper is my ticket to "flow". Flow, is a semi-mythical state of mind that allows us to do our best work and achieve our goals in a state that some would call "happiness". I call this a "semi-mythical state of mind", because in today's fast-paced, interruption rich world that state is very elusive. Actually, according to some it is impossible to reach unless we have large chunks of uninterrupted time. I have a different experience.

I don't worry my brain with decisions on what to do during the day, I do that in the morning when I plan my day.

My secret to "flow" is not that I get a lot of uninterrupted time. That is impossible for me. The secret to "flow" in my line of work is that I have externalized the order and type of work that I need to do for that day. I don't worry my brain with decisions on what to do during the day, I do that in the morning when I plan my day. Once I have externalized the list of objectives to achieve, my brain can focus on how to achieve those goals and complete those tasks. Simple and effective - for me.

After that, my whole day is in the "flow". No matter how many interruptions I get (meetings, managers, colleagues, etc.), I always know what to do next. That allows me to be in the flow for the whole day, even when I'm interrupted.

How about you? How do you ensure that you achieve "flow" every day at work?

Photo credit: Dustin Diaz @ Flickr

Use #NoEstimates to create options and deliver value reliably

Why do I use #NoEstimates? Certainly the first reason is obvious in the name of this movement. But there are may other reasons. One of the reasons that does not get enough airtime is that it creates more options than using pre-estimated methods to manage software projects.

Value is unpredictable, generate options to explore the value space

When we estimate a software project we make certain assumptions about what work and how that work will be completed. We need to do that in order to sequence the work, handle the dependencies, assign the work to the right people, etc. Estimation is a process of planning, and as such it requires us to make commitments. Those commitments reduce our options, because we immediately eliminate possible implementation strategies.

Let's look at an example. I want to implement a IT-request queue in my organization. Something that allows us to manage the requests from employees in a way that none is lost and the employee can easily get information on how the item is proceeding through it's life-cycle. A project is started. How to plan this project?

• The Estimated approach: In this approach I get the team together we list all the work that needs to be completed (database changes, UI for form input, queries and IT ticket management, server deployment, testing, etc.). We schedule this work according to skills and availability of people and end up with a schedule. This schedule will be implemented by different people. Because of this, I will have to follow-up with each person, to make sure we have each necessary task completed before the work on the next tasks begins. This is how we would typically handle the dependencies between these tasks.
• The #NoEstimates approach: In this approach I ask the team: "What is the most important work that we need to complete NOW?". Tracy says: "The IT ticket handling is the most important part. We can add the customer-side UI later and handle incoming requests by email until then". Bingo! I get the list of tasks for that specific functionality from the team, and it looks much smaller: Database changes + IT UI development. The immediate benefit is that the dependencies are fewer (and we will make them even fewer with User Story splitting), which means less coordination, and valuable functionality delivered earlier. But the key point is this: We have more options! We have made the necessary commitment to deliver something of value, but not made a commitment to the planning for the whole project.

One of the key benefits of using #NoEstimates as a planning approach is that we create more Options, making our projects more valuable and more flexible.

In the #NoEstimates approach we release a partial set of functionality (value) earlier. We may find that for the time being that functionality is enough and may decide not to release any further functionality (an option). Or we may find something we did not know before, and change the project (another option). While in the Estimated approach we will make commitments in terms of time, people and resource allocation that are going to be harder to change later (reducing options), and may commit more money to the problem than we actually need (less options still).

With the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on

Note also that with the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on. The reason is simple: requirements have a "best before" date and expire. We should not commit to requirements too far into the future - but that we will tackle in another post.

Have you applied this planning heuristic in your projects? What were the benefits, and obstacles you faced when using it?

Photo credit: psiaki @ flickr

Which do we need the most, Project Managers or Line Managers?

Should we organize around the delivery of products or along the structure of our organization? Which organizational paradigm is the most effective or efficient?

This is a question that most managers struggle with regularly. At least when they are asked to "re-organize" their respective organizations.

In Europe (and I suspect elsewhere) this has led to a huge growth in what I would call Hybrid organizations. Also known as "Matrix organizations". These are organizations where managers try to blend delivery-focused (project) organizations with hierarchical-focused organizations (the line organization).

Which organization is better?

Just this week I had an argument with a colleague about this. Which of this organizational paradigms is better? It is my view that both of these options, actually all three (if you count the matrix organization) represent a fundamental lack of understanding of how knowledge work happens and how it gets done.

The Hierarchy-focused organization is based on the principles that we need to have someone to "manage" the worker. As if the worker herself would not be able to make decisions about allocation of resources or conflict resolution without the contribution of a higher manager.

The Project-focused organization is based on a similar principle, the principle that the worker cannot make execution-relevant decisions herself. For example, which piece of work to do first, or where to get some necessary information.

The Matrix organization may be based on both of these ideas, i.e., that the worker cannot do any type of decisions herself...

I believe these principles (and the associated practices) are wrong. I believe that we can have organizations that are focused on delivering concrete products / services to the customers without having to have any of these layers to "manage the work and the worker".

Why should we have Project Managers and other interesting arguments...

My colleague was arguing that we don't need Project Managers in an organization because they don't deliver anything. "They are overhead that produces no value outside coordination" he said. Sure, I can buy that, but what do line managers do that cannot be classified in the same way? The answer is nothing. Line manager's work is mostly overhead as well. They don't - on their own - deliver anything to the end customer, and do mostly coordination work (for example: coordination of performance reviews).

So the question is: If we believe that Project Managers "add no value" to a knowledge organization, why do we have line managers?

What do you think would be my colleagues answer to that question?

Photo credit: miuenski @ flickr

Kanban vs Scrum, the ultimate fight? Don't think so, here's why:...

Wow, what a week! A BIG post on Kanban by Scrum evangelist @jcoplien litterally put the blogosphere (and twittersphere on fire!).

It is good to have these family fights in the Agile family once in a while. As a life-philosopher once said: "These things gotta happen every five years or so, ten years. Helps to get rid of the bad blood".

But what are the differences between Kanban and Scrum, really? What are they?

Here's my take on the differences:

Kanban innovation

Kanban is, from it's origin a more systematic approach to measuring, visualizing and following up work in a "system" (one team, many teams, a company you name it). Thanks to the work by the Poppendiecks, David Andersson and Don Reinertsen (and others I'm sure) a pretty interesting and innovative economic framework as been put into the software process development lingo. Cost of Delay, Queues, optimize the whole, etc.
This economic framework is, in my view, the major innovation that Kanban proponents bring to the table. This was to be expected as many of the early adopters of Kanban were using Scrum before and felt the need to quantify and analyze some aspects of software development that Scrum did not tackle.

Scrum innovation

Scrum brought to the fore-front of process discussions issues that had never made it to our attention before: Self-organization, people/team dynamics, problem solving within a short cycle with feedback loops in place (Sprint), etc.
The major innovation of Scrum in my view was the introduction of the Socio-technical system of software development as Liker describes it in the Toyota way. Other similar software development processes took some of the ideas that Scrum also took, but shied away from the self-organization at the team level and blocker-removal focus of roles such as the scrum master. Those methods did not last long because the teams felt like puppets instead of adults with the possibility (and responsibility) to produce the best product they could.

Wrap-up

Both Kanban and Scrum brought significant innovations to the software development world. Those are just two types of innovations. There is more coming from the community and instead of focusing only on these two methods (which all of you should experiment with!) we should also look at what else is missing in the software development ecosystem.
My next interest area is "Complex Systems" (Complexity + Systems Thinking) and Management as a profession. I'll be exploring those subjects more and more in the future as a way to complement my use of Kanban *AND* Scrum. I suggest you do the same and find what else is missing in your environment and look for what else is around that could help you complement what you are already using. Here's a suggestion: start with @jurgenappelo's book on Management!

Happy reading, happy learning!

Adapting to a different country and culture - a story

This is the story of Jerome (names changed to protect the innocent), and his adventures while changing countries for work.

Jerome always wanted to work in another country. He had bought into the whole globalization/Cultural experimentation ideas of the 90's. The economy was growing quite strongly, there was a strong need for IT professionals and he wanted to explore the world.

Armed with a degree and the will to explore he left his country, which was warm and sunny, for Finland. He arrived in the early afternoon of one gray and rainy day. "The prototypical autumn day" - he was told.

"Never mind" - he thought. I can certainly survive a bit of rain and snow. He was met at the airport by a colleague who drove him to this first apartment. Jerome had never seen that apartment but that did not worry him. "How bad could it be?" he thought.
As he pushed his large suit case up the ramp to the apartment, he thanked his colleague for the ride and entered the room. This was the first shock. The apartment was a room, a single room. "Talk about nordic architecture!" he said out loud.

He sat down and appreciated the last few moments of calm before the real battle started. Next morning he would have to visit many offices before his arrival and relocation to Finland could be completed.

Relocation to a new country can be a daunting process for a person. Many new people to interact with, many laws and rules to learn, many things to buy. A whole new life to build! In fact, adapting to a new country can be down right scary. Really scary. However Jerome's employer seemed to be aware of this and had organized for a person to be with him through the "legal" process of relocation. This, Jerome learned, was called a "Relocation Service". "What a great idea!", Jerome thought to himself, "this makes my life so much easier, and already gives me a link to someone I can call in case of an emergency".

So, he went about visiting the "Alien office" (Yes, Jerome was now officially an 'Alien'). "What a way to welcome people, by calling them aliens!". Later he visited the Maistraati, then the bank, then the local phone company and finally the most used relocation service around the world: IKEA!

That night, while reviewing in his mind what had happened during the day he realized that even if much of the communication with the many clerks he met happened in Finnish, all the paperwork was readily available in English. Never for a moment did he feel lost in the paperwork process. Even the web bank (and there was a web bank! -- in 1998!) was in English. Reflecting back Jerome realized that having all the bureaucratic documents in English was one of the most important reasons he felt welcome in his new country. He felt that in this country they were serious about accepting and welcoming people into the society. The barrier was low enough that in retrospect he felt that he would do the same again, if given a chance.

The first barrier was overcome. He was now an official resident of his new country. Next came the integration to the work life, after all work had been the reason for him to move to this new country.

Next was his first day at work. Jerome was apprehensive. he had visited the company offices before during his job interview, but now was his first work day. He entered the office and asked for his manager. After a few minutes waiting, a smiling person came to greet him. Jerome remembered him from the Interview a few months prior to this encounter. He felt immediately at ease.

In the company he was working for the hierarchy was not emphasized, people worked together at all levels and he never felt like an "inferior". He was hired for his knowledge and experience and he felt that his presence was welcome. The fact that he had the support of the relocation service also helped as he did not have to bother which colleagues with questions about the bureaucratic issues and could fully focus on the work at hand. He reckoned that, all in all the relocation service support actually saved him and the company many hours and stress by handling the legal issues and allowing him to focus on work from day 1. This would become even more clear years later when he moved to Germany and did not have that extra support from a relocation services company. But I'm getting ahead of the story. Let's get back to the process of integration to the new culture.

It was now early 1999 and winter had settled in. When he arrived he was warned about winter, and he fully expected cold and snow, but that year was a record year. The boy from down south - where snow meant "no school!" was fully surprised by the -25C winter that year. The snow was another big impact surprise: he had bought some boots that were obviously not fit for Icy roads and spent most of the November/December measuring the temperature of the snow with his lower back. "Auch!"

Getting used to the cold and dark was something that many of his Alien friends struggled with. You can actually see it happening as the level of complaints about culture, bureaucracy and other topics goes up, way up in the winter. Some of the those complaints are justified like the issue with the language learning. No matter how you try if you don't pronounce "kertalippu kiitos" perfectly the tram driver will always reply "and where are you going sir?" - in perfect Oxford English, of course! Language is one of the biggest obstacles to completing the process of adaptation. After many years of attending Finnish courses, Jerome finally learned enough words to confidently order his food in Finnish and not be completely surprised by what was brought to his plate later on.

The first year was gone and one surprise awaited him. One day he got home and saw a letter from the police. "oho" he thought "what now?" After consulting with his colleagues he found out that he had to go to the police to renew his residence permit. This was his first encounter with the local bureaucracy on his own. Would he able to understand what he was told?

Jerome timidly entered the room and saw tens of other "aliens" waiting to be received. Certainly many in the same situation as him. This made him think "why do the police isolate us 'aliens' into a different office? It probably has to do with the efficiency of having all 'alien' issues handled by the same people" he thought. But is this how it really should be? What is the message that his sends to those so called aliens? How does an alien feel to be put into the alien-ghetto like that?

Gladly the Tax Office (verotoimisto) has totally different approach and all tax payers are welcome to any office, despite being an alien. "I much prefer the Tax Office" Jerome thought while at the same time realizing he would never have imagined himself thinking that way.

His preference for the Tax Office was about to grow as later that year he received his first tax return. "It is already pre-filled for me! What a great idea!" Jerome thought while realizing that he would not have to spend endless hours trying to decipher local tax code. "I can get used to this!" he said. This was the first painless tax season of his life! Definitely a plus when considering where to live in the future!

Back to work Jerome could not help but feel that his personality was very much in synch with the way his company was organized. There was a clear purpose and people were encouraged to be autonomous and collaborate across department borders to achieve their common purpose - what a difference to other countries where nothing happens without the boss giving the green light to everything. This focus on autonomy and purpose eased Jerome's integration to the work-place.

Some years later Jerome would receive an offer to move to Germany. He was tempted. Germany's political tradition and strong economic growth were strong attractors, but perhaps the final reason that pushed Jerome to leave Finland was the fact that many companies that he contacted just flat-out refused to hire people who were not native or near native speakers of Finnish. This was in conflict with everything Finnish companies and entrepreneurs were saying publicly! Publicly many companies talked about wanting to go international and about the fact that their companies depended heavily on trade, but when it came to their hiring policies that was not the case! This was 2010! and in Finland we are still hiring people depending on whether or note they speak a language that is probably harder to learn than most of the jobs we hire for! Here's a suggestion: why not have language learning as part of the job? That way we would get highly qualified 'aliens' and would be able to keep them in Finland! But I am digressing now.

Later on Jerome thought about what made his integration at work and in the society harder or easier and he came up this list:

• Easier
  
  
  
  • Relocation Service support to a new comer
    
  • Low hierarchy and clear-purpose at work
    
  • All bureaucratic processes had been documented in English
    
  • Web / Internet banking
    
  • Pre-filled tax returns
    
  • And... lots of friends, many in the same situation
    
  
  
• Harder
  
  
  
  • New culture: when to be friendly? When is that just akward? How to make friends? What do people value?
    
  • Dealing with many different authorities / entities (although relocation services helped!)
    
  • Learning a new language (although knowing the language made the integration process easier later on)
    
  
  

Recently Jerome moved to Germany, and he had a chance to go through the same process again, in a new country. Now more experienced with the process of moving he felt confident that he could tackle the challenges ahead. Little did he know!

Upon arrival to Germany he was informed that he had to decide on a health insurance, and fast because his salary could only be paid after that. "Hmmm, they don't make this very basic thing very easy here, do they?" he thought to himself, and went on: "Strike one for Finland with it's universal health care system. In retrospect KELA makes integration much, much easier by having a universal health coverage from the start!"

Jerome decided to register with a national health care provider. He could have registered with a smaller / local health care provider, but he had heard enough horror stories about smaller / local providers that could go bankrupt at any time. Scary stuff for an Alien!. He then had to register as a "legal alien" - to quote Sting's. He went to the local Amt (that's office in German) and registered, not an "alien" but as an Ausländer. Don't know about you, but Jerome felt much better with the term Ausländer, maybe because he did not speak German.

Jerome arrived at the Ausländeramt desk and politely asked "Sprechen Sie Englisch?" - that being all the German that he could muster. The clerk, however, replied "Oh, but we are in Germany. We must speak German." "Strike 2 for Finland" Jerome thought to himself. Struggling with understanding what the clerk was trying to say he was able to successfully register as a resident of the city. "We have to applaud the idea of an open Europe, this is the stuff that makes us happy we pay our taxes. Long gone are the days when we had to sneak through borders illegally to find a work place to our liking. Now we can freely travel and settle with minimal bureaucracy. That is a real tangible benefit to the people! Borders only benefit the powerful who can find legal, and not so legal ways around them."

Political rant aside it was now time to start enjoying life in a new country. Until...

Later that week, Jerome received a letter at home. He had to present himself in the local city office. His registration had a problem... "Oho! I spoke too soon about freedom from borders..."

Jerome reached the office and asked what the issue was about. After he was explained what the issue was he could not believe it. "Really? in the XXI century? We can send a man to the moon but we can't solve a problem like this?" -- he thought to himself.

The issue was that when he completed his original registration, he had used the wrong postal code and therefore he now had to correct it. But the process for that was worthy of a Kafka plot. He would have to drive to neighboring town, register there (with the original date of his initial registration); then he had to come back to the city office and register again with his current address so that his registration could be considered valid! Basically he had to register twice in two different places to finalize his original registration process. "Talk about bureaucracy! Strike 3 for Finland!"

But the situation got worse later on! Jerome received a letter from the local court where he was instructed to pay a fine for not having registered on time in his current municipality! "The nerve" he said out loud as he read the translation from Google translate. "I've now registered 4 times" - he had registered once more when he changed apartments in between these events - "and they have the nerve to tell me that I have not registered on time?" "Strike 4 for Finland!".

After this ordeal Jerome revised his table of things that make your integration easier or harder:

• Easier
  
  
  
  • Relocation Service support to a new comer
    
  • Low hierarchy and clear-purpose at work
    
  • All bureaucratic processes had been documented in English
    
  • Web / Internet banking
    
  • Pre-filled tax returns
    
  • And... lots of friends, many in the same situation
    
  
  
• Harder
  
  
  
  • New culture: when to be friendly? When is that just akward? How to make friends? What do people value?
    
  • Dealing with many different authorities / entities (although relocation services helped!)
    
  • Learning a new language (although knowing the language made the integration process easier later on)
  
  
  • Bureaucracy!
  
  
  
  

The end!

Photo credit:

Rob Young @ Flickr
mac_filko @ Flickr
Giorgos @ Flickr

You cannot transition to Agile. Stop and just embrace it!

I am writing this blog post to explore a concept. So bear with me, I'll probably ask more questions than I'll answer.

Why do most Agile fail in our companies (or government organizations for that matter)? My view is that we cannot actually transition from a command and control management paradigm to Agile / Complex management paradigm. The reasons are not fully clear to me, but I believe that it has something to do with the fact that we actually (typically) try to use a pre-determined way to make those transitions happen.

Case in point: When we try to move from Waterfall software development to Agile software development, we will typically draw a plan up for the transition with "steps" or "phases". Those "phases" or "steps" will typically be "stable points" in the evolution of our system (the company or organization). However, the Agile / Complex management paradigm assumes, at its core that software work is complex, therefore there is no predictable causality. The consequence of this is that the "steps" or "phases" in between the command and control paradigm and the Agile paradigm cannot themselves be "stable" in the sense that predictability can be recognized.

By following the argument above I'd state that: transitions fail because we try to move from a command and control paradigm to an Agile / Complex paradigm by applying command and control models. It is impossible to 'move orderly to a complex environment'.

What does it mean in practice for us? Well, for starters we cannot "plan" the transition in the same way we tried to plan our waterfall projects in the past. We can, and should have a goal or an idea of where we want to be. But after that we must embrace the new paradigm, or "Adopt the new Philosophy" as Deming put it. There are no intermediate steps between the "old command and control mindset" and the new "complex / agile mindset".

As this is an idea I'm still developing, I'll probably return to this subject and write some more, but in the meanwhile: what do you think? Does this make sense? What did you get from the above?

Photo credit: Marc Soller @ flickr

The three main trends in long agile adoption processes

I was having a discussion with fellow agile adopters yesterday and a bunch of interesting patterns emerged from the discussion:

• We are getting to agile/process improvement fatigue
• The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach
• We don't have a process to learn at the organizational level

Here are some ideas I brought with me from yesterday.

We are getting to agile/process improvement fatigue

This is certainly a familiar sentiment among those of us that have been involved in agile adoptions for several years. One trend/common sentiment I felt yesterday is that agile and lean are seen as just another trend that will pass. Many are trying to look for "better" approaches in meta-approaches (like Lean over agile, or Toyota-driven ideas over Lean, etc.)
This meta-approach is seen as solving the problems with the previous approach.

There's certainly some truth to that. One cannot really improve without going deeper in the process of understanding, and that naturally leads to more abstract approaches. Something the Alistair Cockburn readers will recognize as passing from the Shu-level to the Ha-level.

I don't see this as a problem, I see that as a natural learning process. Agile in general and Scrum in particular have been around for long enough for people to recgnize it's shortcomings and to look for "better" ways. It's the learning process. The problem is when people don't really look for the theory behind the patterns they are trying to apply and just change patterns (say from Scrum to Kanban) without understanding why.

As long as we fail to recognize the theory behind what we are applying, we are doomed to make the same mistakes, over and over again.

The management of software development is changing, but agile adoption processes do not have a mandate to change the management approach


This was one trend that I raised in the discussion. Here's my argument summarized.
Agile is the result of a fundamentally different approach to management (analytical management vs. complexity-aware management). However, the management that is driving/leading the adoption of Agile software development is versed and informed by a completely different way of thinking. They live and bread a different paradigm of management. Until that changes it is impossible to make agile adoptions sustainable.

Symptoms of this problem are quite common and likely familiar to you. Managers set targets for agile adoption, but don't change the way they manage the organization (a requirement to meet those targets). Managers say they want agile adoption but at the same time try to micro-manage the teams, don't respect their need for self-organization and set all kinds of policies that go against teams taking responsibility over their work. And the list goes on.

Main point: there's a basic conflict between the analytical management mind-set and agile adoption. Until we solve that the best we can do (and it's a lot already!) is to get our teams to be agile, but not the structure that directly affects and limits their performance.

We don't have a process to learn at the organizational level

The point that follows from the previous is that in order to be able to get out of the analytical management paradigm we need to build a way for the organization to learn, to experiment and then adapt.

The lack of this method of learning, a meta-management method if you will, is what dooms us to be stuck in the previous paradigm. There is plenty of information around that can help managers learn and improve their work the same way we ask teams to do. But that learning is not happening in the vast majority of companies adopting agile. Until that happens agile is firmly stuck in the realm of the team developing software. It's like buying F1 tyres for a family car and expecting it to win a rally race.

Conclusion

What encourages me is to see that fellow colleagues in the agile journey also understand these issues and are (some of them at least) directly addressing these.

I hope they succeed and share their knowledge with the rest of us...

Photo credit: Denis Collette @ flickr

Setting targets actually decreases your performance. Don't believe me? See this video...

Many of the readers of this blog have probably faced this situation already in the past. When I was adopting Scrum in a local company, I was faced with the tyranny of setting targets. Targets and bonuses were part of that company's HR policies. "This is how we motivate people" as the saying goes.

Now picture this. When we started adopting Scrum we were, as expected, adopting also timeboxed software development (a key part of scrum). But the targets were that we should always be on time (+-10%).

So, get this: you get more money if you are on time (to motivate you to be on time, I guess) but your process is timeboxed! Easy money as they say...

The morale in this story is that targets are useless! If you want to improve the system (R&D for example) you need to manage the system, not set targets!

By adopting Scrum (effectively changing the system) we were able to be 100% on time! And that had nothing to do with the target setting, the system design (using Scrum) was the reason!

This is an insight that many managers don't have today: as manager you must design and manage the system. Setting arbitrary targets and not providing a method (system) is useless!

The video below of a conversation with John Seddon, is a very good explanation of how targets are useless and even counter productive. I especially loved his points:

1. there's no reliable method for setting a target
2. when you use targets or any other arbitrary measure you drive waste into your system
3. once you learn how to use measures derived from the work, and you involve the people who do the work you achieve a level of performance you would never have set as a target.

Response to Lynda Bourne on Agile and the Business message

The following is a comment to Lynda that I was not able to submit to her blog. You can find Lynda's original post here.

@Lynda

Nice post. I'm glad that the conversation is continuing. Both PMI and Agile people need this dialogue.

A couple of comments on your post.

1. I agree that Agile Advocates need to learn to speak to the business people. However,

2. Agile adoption has been fast in many places, not slow.

It is true that many Agile Advocates (AAs as you call them -- like the analogy BTW, and think it fits! :) don't talk "business-ese". That's been a failure of the community, however that is far, very far from being a problem in adoption. Having been involved in two major organization change projects I can vouch for the lack of business-oriented communication, but I can also assure you that speed of adoption has been anything but slow. I know of a company that has several 10's of thousands of employees where Agile is a topic of discussion at all levels, from top management to the team level. This is very encouraging, especially because this company has not been shy in removing some of the building blocks that are in PMBOK (and inadequate to software development) and build new processes at all levels based on the principles of Agile Software Development.

3. Agile software development needs discipline

I agree with this, but the lack of discipline existed in software organizations when PMBOK was the prevailing framework for organizing projects. Lack of discipline is a problem that Agile encoutered when it started, not a problem created by adopting Agile software development methods. Indeed, it is my experience that agile software development builds a much higher level of discipline into the development process than was previously the norm.

4. Agile alignment with business objectives.

This is a topic for a post in itself, but I'll just refer one example of how agile brings focus and alignment with business goals. Check this post by Dean Leffingwell where he explains how requirements management in Agile can be done in a way that directly reflects the strategy of the company.

Let's continue the dialogue. I've learned a lot and hope to continue learning from it! :)