This blog has moved. Go to SoftwareDevelopmentToday.com for the latest posts.

Monday, December 30, 2013

Why processes fail us, and what to do about it


Organizations go through different levels of maturity as they grow. Some will say that they "learn to perform", others will say that they "become ossified and slow", yet others would say that they "mature". In my view neither of these classifications is accurate, they all touch on possible outcomes of a process of "aging". Organizations grow older, but what dynamics do we see in these organizations?

Organizations grow older, they age just like us!

A recurring pattern in organizations is that they create, develop and install processes. Processes are, for practical purposes, sets of rules that define how work should happen in those organizations. They are the rules we follow daily when we work.

These rules are necessary for a common understanding of expectations and roles for each of us. We need those rules or processes so that we know what to expect, and what is expected of us. Or do we?

What is the role for rules in an organization?

In the study of Chaos and Complex systems scientists have found that Complex or Chaotic systems exhibit infinitely complex behavior starting from very simple - perhaps even simplistic - rules. The most common example of these simple rules in documentaries about Chaos and Complexity is the way ants find their way to a new food source. The rules they follow are simple:

  • Walk around randomly and lay a pheromone path
  • When you find food, turn around and follow your pheromone path to the colony
  • While you walk around, if you find (bump into) an existing pheromone path, follow it

PS: you can find a much more detailed explanation here.

Following these simple rules Ants can not only find food, but feed an entire colony. In fact, when observed from an external view point we see complex System behavior even if one Ant alone follows a very simple set of rules.

The complex behavior we witness is Complex, and Adaptive. Hence the term Complex Adaptive System (or CAS).

What does this have to do with us - humans - and companies?

In investigating CAS we have found that the more complex the rules that we define, the less likely that the system (or company/organization) will be Adaptive. In fact, the opposite is true. Companies often put rules in place to "clarify, and specify" the expected behavior, thereby making it simple - or even simplistic. One glaring example of this phenomena is the way companies develop highly sophisticated goal-setting processes that eventually end up setting goals that distort the behavior of the organization in a way that makes it lose sight of what matters: their adapation to the environment they exist in (customers, suppliers, society, etc.).

The more complex the process and rules, the less Adaptable the organization will be!

But there are more examples of this phenomena whereby defining complex rule systems leads, invariably, to simple - even simplistic - behavior.

What's the role for rules?

What is now clear from research, is that simple rules can lead to Complex and Adaptive behavior in the "system" or organization. For us managers, this means that we must avoid the temptation to develop complex set of rules and must be on the lookout for rules that add burden to the organization and possibly constraining behavior to the point that the organization is unable to adapt to the changes it faces in the market.

The recipe to foster adaptability in the organization is simple: when possible remove rules, when in doubt remove rules. Add rules only when the cost of not doing so is prohibitive (legal boundaries for example) or when you've learned something about your environment that should be codified for everybody to follow (you found out that a certain technology is too expensive or unstable).

But here is the most important rule for you: All rules should be created as a result of a root-cause analysis, never as a result of a knee-jerk reaction to some unplanned or unpredictable outcome.

The most important rule: No rules should be established without a thorough Root-cause analysis!

The quote "Keep it Simple" really means: use less rules and more feedback! Like Agile...

Image Credit: John Hammik, follow him on twitter

Labels: , , , ,

at 08:00 | 2 comments links to this post
RSS link

Bookmark and Share

Friday, December 27, 2013

What is Chaos? And how we found out about it...


As he looked at numbers e noticed that the oscillations of his model did not repeat.

In fact he entered the numbers again, and again, and again but his model would refuse to behave the same way twice.

During our long education we were taught that given the initial state of a system (the present parameters that define the state of the system) plus the equations that fully describe the system, you will be able to plot the future behavior of the system forever.

And we have plenty of evidence that this approach works. For one, we are able to predict when Comet Halley will next visit our corner of the galaxy. And many astronomers were able to predict that last visit thousands of years ago!

So, why was his model stubbornly behaving differently every time he punched the numbers in? After all he knew the system perfectly - he had designed it!

The best way he had to describe this "unpredictable" behavior was with the word "chaotic", a never ending sequence of never repeating patters. Nothing was the same even if the initial state was the same and he was the one defining the equations for this toy-weather model. I mean he had defined ALL the equations...

It took a few days, but he figured it out. He had entered the parameters with a precision of 1/1000, but during processing the computer executing the model would use numbers with a precision of 1/1000000. On initial consideration this did not seem a relevant difference, after all a difference of 1/1000 was equivalent to having a butterfly flap its wings in China and having that create a storm in North America.

Systems that would never repeat in behavior even if they ran for ever

Later, this and other experiments would be repeated all over the world, in many different domains but the results would be similar. All over the world scientists were discovering other systems that were "sensitive dependence to initial condition" (aka suffered from the Butterfly effect), the scientific definition for "chaos" which later became the popular term to describe systems that would never repeat in behavior even if they ran for ever. These systems exhibited infinite variety of behavior when certain conditions were met. What were those conditions? That we will explore in a later post

Photo credit: John Hammink, follow him on twitter

Labels: , , , , ,

at 08:00 | 2 comments links to this post
RSS link

Bookmark and Share

Wednesday, December 18, 2013

The most valuable question for your software project


Every time a software project is started, a dance starts: the dance of project approval. Decision makers and project delivery team take different positions at a table, some ask questions, others do their best to respond, given that these questions are typically about the future. Some of the questions are so much into the future that it would be foolish to pretend we know the answer. Yet we do.

This was the main lesson I learned in my life as a project manager: you have to learn how to dance.

How to dance...

In the dance-like meetings there are 3 types of people:

  • A) Those that pretend to know the future
  • B) Those that know they can't know the future
  • C) And those that don't want to know the future
The question below is only valid for the people in category B). The other two categories are beyond help from me or anyone else.

What is the likelihood this project will fail?

When a project starts, the best question anyone can ask about the project is this:
The most important question: What is the likelihood this project will fail? Why is this question important?

  • First, if you don't have an answer to this question you are either in the first or third category of those listed above, and there's nothing I (or anyone else) can do to help you.
  • Second, by discussing the possibility of failure we are actively looking for possible causes of problems for the project (rule 1 of risk management).
  • Third, having an actual conversation about the failure probability of a project will give you an indication of the "value at risk" for the project.

For example: let's say the project budget is about 100K dollar, and to make it a worthwhile investment we would need to make it between 120K and 90K in total cost (i.e. if it is likely it will cost more than 120K you should not do it). If the answer to the most important question is: 50/50, then you know that the project is not worth doing. It has very little chance of delivering the necessary value to justify it's existence, in fact: it is likely it will be above 100K and up to as much as 200K in cost.
However, if the answer is that we are 80% confident we will deliver the project on timecost, then the project is worth pursuing (although with some strict scope control in place) because the expected worst case scenario would be a total cost of 120K dollars (the 20% margin implicit in the confidence above). In practice an 80% confidence means you expect the project to cost up to 120K (100K + 20%) in the *worst* case.

In a later post I'll explain how we can answer this most important question using the #NoEstimates approach. For now, let me know what you think of the question and the concept of "value at risk" above.

Definition of Value-at-risk [PDF]: "value-at-risk summarizes the expected maximum loss (or worst loss) over a target horizon within a given confidence interval"

Note: A special thanks to @galleman for the idea of the 3 categories of people from which I derived the 3 categories I present in this post.

Photo credit: Steven Depolo @ flick

Labels: , , , , , , , ,

at 07:30 | 2 comments links to this post
RSS link

Bookmark and Share

Tuesday, December 10, 2013

My secret to flow and a productive day


As I start my day, I have this routine I do. I review all the loose threads that I have in my mind, and write them down (a mindmap, usually). After I write them down I quickly evaluate what are the most important goals for the day. Then I write those goals on a post-it that I carry with me all day.

This piece of paper is my ticket to "flow".

This piece of paper is my ticket to "flow". Flow, is a semi-mythical state of mind that allows us to do our best work and achieve our goals in a state that some would call "happiness". I call this a "semi-mythical state of mind", because in today's fast-paced, interruption rich world that state is very elusive. Actually, according to some it is impossible to reach unless we have large chunks of uninterrupted time. I have a different experience.

I don't worry my brain with decisions on what to do during the day, I do that in the morning when I plan my day.

My secret to "flow" is not that I get a lot of uninterrupted time. That is impossible for me. The secret to "flow" in my line of work is that I have externalized the order and type of work that I need to do for that day. I don't worry my brain with decisions on what to do during the day, I do that in the morning when I plan my day. Once I have externalized the list of objectives to achieve, my brain can focus on how to achieve those goals and complete those tasks. Simple and effective - for me.

After that, my whole day is in the "flow". No matter how many interruptions I get (meetings, managers, colleagues, etc.), I always know what to do next. That allows me to be in the flow for the whole day, even when I'm interrupted.

How about you? How do you ensure that you achieve "flow" every day at work?

Photo credit: Dustin Diaz @ Flickr

Labels: , , , ,

at 06:30 | 2 comments links to this post
RSS link

Bookmark and Share

Monday, December 02, 2013

Use #NoEstimates to create options and deliver value reliably

Why do I use #NoEstimates? Certainly the first reason is obvious in the name of this movement. But there are may other reasons. One of the reasons that does not get enough airtime is that it creates more options than using pre-estimated methods to manage software projects.
Value is unpredictable, generate options to explore the value space

When we estimate a software project we make certain assumptions about what work and how that work will be completed. We need to do that in order to sequence the work, handle the dependencies, assign the work to the right people, etc. Estimation is a process of planning, and as such it requires us to make commitments. Those commitments reduce our options, because we immediately eliminate possible implementation strategies.

Let's look at an example. I want to implement a IT-request queue in my organization. Something that allows us to manage the requests from employees in a way that none is lost and the employee can easily get information on how the item is proceeding through it's life-cycle. A project is started. How to plan this project?

  • The Estimated approach: In this approach I get the team together we list all the work that needs to be completed (database changes, UI for form input, queries and IT ticket management, server deployment, testing, etc.). We schedule this work according to skills and availability of people and end up with a schedule. This schedule will be implemented by different people. Because of this, I will have to follow-up with each person, to make sure we have each necessary task completed before the work on the next tasks begins. This is how we would typically handle the dependencies between these tasks.
  • The #NoEstimates approach: In this approach I ask the team: "What is the most important work that we need to complete NOW?". Tracy says: "The IT ticket handling is the most important part. We can add the customer-side UI later and handle incoming requests by email until then". Bingo! I get the list of tasks for that specific functionality from the team, and it looks much smaller: Database changes + IT UI development. The immediate benefit is that the dependencies are fewer (and we will make them even fewer with User Story splitting), which means less coordination, and valuable functionality delivered earlier. But the key point is this: We have more options! We have made the necessary commitment to deliver something of value, but not made a commitment to the planning for the whole project.

One of the key benefits of using #NoEstimates as a planning approach is that we create more Options, making our projects more valuable and more flexible.

In the #NoEstimates approach we release a partial set of functionality (value) earlier. We may find that for the time being that functionality is enough and may decide not to release any further functionality (an option). Or we may find something we did not know before, and change the project (another option). While in the Estimated approach we will make commitments in terms of time, people and resource allocation that are going to be harder to change later (reducing options), and may commit more money to the problem than we actually need (less options still).

With the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on

Note also that with the #NoEstimates approach we don't commit to requirements that we are not going to immediately work on. The reason is simple: requirements have a "best before" date and expire. We should not commit to requirements too far into the future - but that we will tackle in another post.

Have you applied this planning heuristic in your projects? What were the benefits, and obstacles you faced when using it?

Photo credit: psiaki @ flickr

Labels: , , , , ,

at 07:00 | 3 comments links to this post
RSS link

Bookmark and Share

 
(c) All rights reserved