Fractals, the solution to all of your scaling problems. Including Scaling Agile

It is no secret that I love planning. I'm not coming out of the closet now, that's been true forever! And at some point in my life I was even "cool" with that.

Additionally, I want you to know (although you will not yet understand why) that I still love planning. That's me :) Now pay attention, I'm about to shoot myself in the foot.

Loading the gun

When I was in college there was a topic that I loved, the topic was Information Theory. There's so much stuff in that area of research that I can't even begin to touch the tip of the proverbial iceberg, so I'll just say - for now - that information theory is an area of research that investigates information and how it is codified. For example: how do I compress a text file? Turns out, text can be very efficiently compressed. Perhaps the blog post you are reading can be codified in a few hundred bytes. The cool thing is why that happens: redundancy. Redundancy is an unappreciated quality of all systems.

So unappreciated in fact, that we all praise it's mirror twin: efficiency. A compressed text file is very efficient (i.e. it codifies a lot of information in a very small amount of disk space - I could probably compress this post into 140 bytes and tweet it out - that would be efficient).

If we can create so efficient representation of information why would we stick the old-skool inefficient representation (for example: language)? I'm glad you ask! The reason is that language with all its redundancy is easy to understand even if you cut parts out or mangle the letters. Try reading the following paragraph (click for a larger version):

Did you understand what that said? Of course you did! Redundancy saves the day! Yay!

Now about software and organizations

In organizations and companies, we also have redundancy - plenty of it! And just as well, because without it most companies and organizations would stop working altogether. Redundancy gives us resilience! Just like in the language example above: even with parts of the words cut out of the phrase you were able to understand it! And this is just how organizations work: through, and thanks to, redundancy. This is the reason why some "downsizing" efforts end up killing whole companies, and the often touted "efficiencies" or "synergies" leaders try to gain from mergers and acquisitions end up destroying economic value more often than they create.

The trick with redundancy is to repeat

By now you probably agree that redundancy is good - and it is. But how do you apply it to your organization and processes?

Before we go there, we have to tackle a very neat concept of mathematics. Fractals. Fractals have a property that is mind-boggling. Fractals are concepts that once explored end up generating infinite (yes, infinite!) information. In fact, a fractal line has infinite length even while fitting in a finite space! I won't bore you with the math details, but check this page on Wikipedia about the length of the British coast line - it has a neat demonstration of how a finite space can hold a line of infinite length.

This means that fractals are generative when it comes to information: they generate infinite amounts of information. And this happens to be a very useful property to have in mind when we explore how organizations work.

Making the case for infinity (and beyond)

In this post I argued that removing rules from your company's process book is actually better for your business and for your teams. The next step is to remove as many rules as you can, so that you end up with a small and simple set of generative rules - just like a fractal. Fractals are very simple equations that have in themselves an infinite number of solutions. And that's exactly what our processes should be: a small set of rules that, once in use, accommodate an infinite amount of possible behaviors - this is what I mean by "complex behavior" in the post on disciplined organizations.

Conclusion

Turns out fractals are perfect (yes, perfect - as in perfectly efficient) compression algorithms: a simple equation can be solved in an infinite number of ways, which when plotted in 2D or even 3D generate an infinite line in a finite space.

This property is extremely useful when applied to processes in your company because you cannot predict how people should behave in the future, but you can create an environment that - just like a fractal - allows every actor / person in the company to act in an infinite (and therefore practically unpredictable) number of ways and adapt to whatever the ever changing reality throws at them.

If you believe that your business environment is constantly changing, and that your organization is akin to a living organism you have to embrace the concept of fractal organizations.

Fractals work for you when they allow your blood vessels to reach every cell of your body (within a few cells distance), and when they allow your brain to store vast quantities of information even if it is small enough to fit in your head. Fractal organizations are organizations that can adapt in an infinite number of ways in response to an unpredictable environment.

If change is the only constant, how do I adapt to that?

Epilogue

Before we can understand how to apply the concept of fractal organizations and benefit from that, a very serious question must be answered: If people can behave in infinitely different ways, how do we prevent organizations from turning into chaos? That's a question we will explore soon - stay tuned! :)

Do you want to know more?

Title image credit: John Hammink, follow him on twitter.

How injecting randomness into your project can help it succeed

Success and failure differ, very often, by very little. Take nature as an example. A small change in our DNA (a few mutated genes) can have catastrophic consequences. On the other hand, without these mutations humans would never have come into existence.

Humans and other species in the planet evolved because of chance (although not totally chaotic) changes in their DNA. As small change after small change happened, the surviving individuals were able to spread the viable, and ultimately the most adequate changes to the next generation - and the process repeats still in our own bodies today. If Nature has taught us something about improvement and evolution is that you must have a little bit of luck involved!

The DNA of a project

In software projects - my domain - I see some similarities to this natural model that are worth exploring. Projects also have a DNA which includes the structure and the communication connections between individuals. Communication being one of the most quoted causes for failure in software projects, and therefore one of its critical success factors.

The More I Practice the Luckier I Get

Once a project gets started, a particular structure is put in place (governance, management, teams, etc.). Very often that structure remains in place, and static until the end of the project. But is this a smart choice?

Another aspect of the project that rarely changes is the network of connections between the individuals. This network is what carries information from individual to individual and eventually "selects" the information that is acted upon by the project team. If a piece of information reaches an individual in the project, and that individual does not consider it relevant, that information will not spread further. In contrast, when a piece of information is perceived as important by an individual, that information will be actively spread by that individual.

The Illusion of Control

The structure that is set for a project strongly influences the communication links that can emerge between individuals. Who talks to whom? What meetings exist where people interact regularly? etc. But the reverse is also true. The existing communication network within an organization is a strong influence on what governance structure is put in place, how teams are formed, etc. These co-dependent processes create a positive (or negative) spiral of events for the project, but because neither (structure, or communication network) is often changed, that spiral is unstoppable. If it is positive, it will help the project succeed. If this co-dependent processes create, instead a negative spiral, that will relentlessly remove the chances of success for that project.

This co-dependent relation between two key processes (structure and communication network) is why we can increase the chances of success in our projects by simply causing random perturbations in the project. Randomness helps us explore different patterns for structure and communication networks.

As we carefully design and inject small - safe-to-fail - changes into the project, we can observe how it reacts and adjusts. Later, we can retrospectively amplify or remove/dampen the changes based on the outcome we see. If something works, keep it and ask how can you make it grow. If something creates a net negative outcome, dampen or remove it completely.

The cycle goes like this:

• Define and implement small, safe-to-fail changes in the structure or communication network for the project
• These changes lead to emergent behavior as the individuals and teams adapt to the changes
• A new project configuration emerges which drives new project results
• Finally, we evaluate the results of these changes and decide which components we will try to amplify or dampen

What Do You Mean Random?

There are many ways in which we can, randomly, explore better configurations for our projects. Below I list only a few to illustrate the concept:

• At the start of a project, let the teams chose their own composition. This establishes new connections between individuals in the project as some will choose to work with new team members. However, these new connections will not eliminate the previous connections between individuals. The net effect should be that your project now has a more connected communication network (more individuals with strong connections to each other). In practice this works just like when we form new neural connections in our brain: more connections leads to different thinking and acting patterns
• Organize common project events where people interact with each other outside the day-to-day routine. Planning events can be organized following a structured approach, but including also some unstructured time (à lá unconference, using e.g. Open Space) where people can interact based on their interests. This will also help form new connections between individuals in the project and spread information that would otherwise be locked down and not accessible
• Have regular project "coffee breaks" that happen at the same time and same place. In these events people can connect with other individuals and ask questions from the project management team or the most connected individuals in the project. This unstructured communication increases the chance that some piece of information will "jump the hierarchical borders" and reach people in decision-making positions, or people with influence that can later act on that information.

Conclusion

These practices are designed to inject randomness (unplanned situations or connections) into the project. The goal is not to create Chaos (a scary word for many project teams), but to generate new pathways for the information to flow within the project team, as well as novel project structures that are more adapted to the current challenges the teams face.

Using these (or other) practices will increase your project's chances for success. They don't eliminate the need for the project to be managed, or to have structure. They do increase the chance of success for the project by exploring new organizations and structures through a process of small (safe-to-fail) changes that can lead to unplanned, but ultimately superior performance.

These were just a few examples. How would you inject randomness into your project? Have you done that in the past? Please share your experiences in the comments for the benefit of others.

Photo credit: John Hammink, follow him on twitter

Why processes fail us, and what to do about it

Organizations go through different levels of maturity as they grow. Some will say that they "learn to perform", others will say that they "become ossified and slow", yet others would say that they "mature". In my view neither of these classifications is accurate, they all touch on possible outcomes of a process of "aging". Organizations grow older, but what dynamics do we see in these organizations?

Organizations grow older, they age just like us!

A recurring pattern in organizations is that they create, develop and install processes. Processes are, for practical purposes, sets of rules that define how work should happen in those organizations. They are the rules we follow daily when we work.

These rules are necessary for a common understanding of expectations and roles for each of us. We need those rules or processes so that we know what to expect, and what is expected of us. Or do we?

What is the role for rules in an organization?

In the study of Chaos and Complex systems scientists have found that Complex or Chaotic systems exhibit infinitely complex behavior starting from very simple - perhaps even simplistic - rules. The most common example of these simple rules in documentaries about Chaos and Complexity is the way ants find their way to a new food source. The rules they follow are simple:

• Walk around randomly and lay a pheromone path
• When you find food, turn around and follow your pheromone path to the colony
• While you walk around, if you find (bump into) an existing pheromone path, follow it

PS: you can find a much more detailed explanation here.

Following these simple rules Ants can not only find food, but feed an entire colony. In fact, when observed from an external view point we see complex System behavior even if one Ant alone follows a very simple set of rules.

The complex behavior we witness is Complex, and Adaptive. Hence the term Complex Adaptive System (or CAS).

What does this have to do with us - humans - and companies?

In investigating CAS we have found that the more complex the rules that we define, the less likely that the system (or company/organization) will be Adaptive. In fact, the opposite is true. Companies often put rules in place to "clarify, and specify" the expected behavior, thereby making it simple - or even simplistic. One glaring example of this phenomena is the way companies develop highly sophisticated goal-setting processes that eventually end up setting goals that distort the behavior of the organization in a way that makes it lose sight of what matters: their adapation to the environment they exist in (customers, suppliers, society, etc.).

The more complex the process and rules, the less Adaptable the organization will be!

But there are more examples of this phenomena whereby defining complex rule systems leads, invariably, to simple - even simplistic - behavior.

What's the role for rules?

What is now clear from research, is that simple rules can lead to Complex and Adaptive behavior in the "system" or organization. For us managers, this means that we must avoid the temptation to develop complex set of rules and must be on the lookout for rules that add burden to the organization and possibly constraining behavior to the point that the organization is unable to adapt to the changes it faces in the market.

The recipe to foster adaptability in the organization is simple: when possible remove rules, when in doubt remove rules. Add rules only when the cost of not doing so is prohibitive (legal boundaries for example) or when you've learned something about your environment that should be codified for everybody to follow (you found out that a certain technology is too expensive or unstable).

But here is the most important rule for you: All rules should be created as a result of a root-cause analysis, never as a result of a knee-jerk reaction to some unplanned or unpredictable outcome.

The most important rule: No rules should be established without a thorough Root-cause analysis!

The quote "Keep it Simple" really means: use less rules and more feedback! Like Agile...

Image Credit: John Hammik, follow him on twitter

What can we learn from illusions that helps us manage projects better?

Just had the opportunity to watch a BBC documentary about illusions.

The show went on to detail how we as humans get deceived by our own senses. There were 3 things that opened my eyes in that documentary:

The things that caused me to think were:

• when we mix senses one of them tends to take over: this is the case when we are watching a magic act. The example on the documentary was a magician throwing a ball in the air for a few times, and then making the same gesture but keeping the ball in his had. Although the subjects eyes stayed on the face of the magician, the brain perceives the movement of the ball and later realizes that the ball has vanished.
  
• some of our senses are completely/100% contextual (see image above): the example they had here is a cube with many small squares on it's faces. Some faces are lighter, others are darker. When asked about the difference between 2 specific squares in the faces of the cube people would clearly state that the colors were different. However, when you moved one of the squares to the other face of the cube, you could see that the colors were exactly the same! Our perception of color was completely dependent on the colors surrounding the colors we were comparing.
  
• we can learn new senses if we are exposed to the right stimuli: The amazing examples they had for this was a blind-person that had learned to use sound and echo to see and used that learned sense to ride a bike (while blind!) The second example was quite amazing. A person would use a belt what was wired with buzzers/vibrators (like mobile phones) so that they could constantly feel where the magnetic north was. Later they would use that learned sense to navigate their way in a place while blind folded!
  

All of these observations from the documentary point to one thing. We are not wired to perceive reality. We jump to conclusions far too quickly, but we can also take advantage of that if we train our brain with the right stimuli! That was the most interesting find from this documentary.

The implications are huge in a software project setting, where it is quite common that people use only a very small number of stimuli (some reports, a couple of graphs, etc.) to jump into conclusions. We need to "learn" how to sense the status of a software project, and that can be done by following the example of the person who was taught how to "feel" the magnetic north...

Can you think of applications of this knowledge in your project?