Story Points Considered Harmful - Or why the future of estimation is really in our past...

This article is the companion to a talk that myself and @josephpelrine gave at OOP 2012.


We have a lot to learn from our ancestors. One that I want to focus on for this post is Galileo.
Galileo was what we would call today a techie. He loved all things tech and was presented an interesting technology that he could not put down. Through that work he developed optic technology to build first a telescope and later a microscope.
Through the use of the telescope and other approaches he came to realize and defend the Heliocentric view of the universe: the Earth was not the center of the Universe, but rather moved around the Sun.
This discovery caused no controversy until Galileo wrote it down and apparently discredited the view held by the Church at that time. The Church believed and defended that the Universe was neatly organized around the Earth and everything moved around our lanet.
We now know that Galileo was right and that the Church was - as it often tends to be with uncritical beliefs - wrong. We now say obviously the Earth is round and moves around the Sun. Or do we...

The Flat Earth Society

Actually, there are still many people around (curious word, isn't it?) the planet that do not even believe that the Earth is round! Don't believe me? Then check The Flat Earth Society.

The fact that is that even today many people hold uncritical beliefs about how our world really works. Or our projects in the case of this post...

Estimation soup

We've all been exposed to various estimation techniques, in an Agile or traditional project. Here are some that quickly come to mind: Expert Estimation, Consensus Estimation, Function Point Analysis, etc. Then we have cost (as opposed to only time) estimation techniques: COCOMO, SDM, etc. And of course, the topic of this post: Story Point Estimation.
What do all of these techniques have in common? They all look towards the future!
Why is this characteristic important?

The Human condition

This characteristic is nt because looking at the future is always difficult! We humans are very good at anticipating immediate events in the physical world, but in the software world what we estimate is neither immediate, nor does it follow any physical laws that we intuitively understand!
Take the example of a goal-keeper in a football (aka soccer) match. She can easily predict how a simple kick will propel the ball towards the goal, and she can do that with quite a high accuracy (as proven by the typically low scores in today's football games). But even in soccer, if you face a player like Maradona, or Beckham, or Crisitiano Ronaldo it is very difficult to predict the trajectory of the ball. Some physicists have spent considerable amount of time analyzing the trajectory of Beckham's amazing free kicks to try to understand how the ball moves and why. Obviously a goal-keeper does not have the computers or the time to analyze the trajectory of Beckham's free kicks therefore Beckham ends up scoring quite a few goals that way. Even in football, where well-known physics laws always apply it is some times hard to predict the immediate future!
The undisputed fact is that we, humans are very bad at predicting the future.
But that is not all!

This is when things get Complex

Lately, and especially in the agile field we have been finding a new field of study: Complexity Sciences.
A field of study that tries to identify rules that help us navigate a world where even causality (cause and effect) are challenged.
An example may be what you may have heard of, the Butterfly effect: "where a small change at one place in a nonlinear system can result in large differences to a later state".
Complexity Sciences are helping us develop our own understanding of software development based on the theories developed in the last few years.
Scrum being a perfect example of a method that has used Complexity to inspire and justify its approach to many of the common problems we face in Software development.
Scrum has used "self-organization", and "emergence" as concepts in explaining why the Scrum approach works. Here's the problem: there's a catch.

Why did this just happen?

In a complex environment we don’t have discernible causality!
Sometimes this is due to delayed effects from our actions, most often it is so that we attribute causality to events in the past when in fact no cause-effect relationship exists (Retrospective Coherence). But, in the field of estimation this manifests itself in a different way.
In order for us to be able to estimate we need to assume that causality exists (if I ask Tom for the code review, then Helen will be happy with my pro-activeness and give me a bonus. Or will she?) The fact is: in a Complex environment, this basic assumption of the existence of discernible Causality is not valid! Without causality, the very basic assumption that justifies estimation falls flat!

Solving the the lack of internal coherence in Scrum

So, which is it? Do we have a complex environment in software development or not? If we do then we cannot - at the same time - argue for estimation (and build a whole religion on it)! In contrast, if we are not in a complex environment we cannot then claim that Scrum - with it’s focus on solving a problem in the complex domain - can work!
So then, the question for us is: Can this Story Point based estimation be so important to the point of being promoted and publicized in all Scrum literature?
Luckily we have a simple alternative that allows for the existence of a complex environment and solves the same problems that Story Points were designed (but failed to) solve.

The alternative prediction device

The alternative to Story Point estimation is simple: just count the number of Stories you have completed (as in "Done") in the previous iterations. They are the best indicator of future performance! Then use that information to project future progress. Basically, the best predictor of the future is your past performance!
Can it really be that simple? To test this approach I looked at data from different projects and tried to answer a few simple questions

The Experiment

• Q1: Is there sufficient difference between what Story Points and ’number of items’ measure to say that they don’t measure the same thing?
  
• Q2: Which one of the two metrics is more stable? And what does that mean?
  
• Q3: Are both metrics close enough so that measuring one (number of items) is equivalent to measuring the other (Story Points)?
  

I took data from 10 different teams in 10 different projects. I was not involved in any of the projects (I collected data from the teams directly or through requests for data in Agile-related mailing lists). Another point to highlight is that this data came from different size companies as well as different size teams and projects.
And here's what I found:

• Regarding Question 1: I noticed that there was a stable medium-to-high correlation between the Story Point estimation and the simple count of Stories completed (0,755; 0,83; 0,92; 0,51(!); 0,88; 0,86; 0,70; 0,75; 0,88). With such a high correlation it is likely that both metrics represent a signal of the same underlying information.
  
• Regarding Question 2: The normalized data (normalized for Sprint/Iteration length) has similar value of Standard Deviation(equally stable). Leading me to conclude that there is no significant difference in stability of either of the metrics. Although in absolute terms the Story Point estimations vary much more between iterations than the number of completed/Done Stories
  
• Regarding Question 3: Both metrics (Story Points completed vs Number of Stories completed) seem to measure the same thing. So...
  

At this point I was interested in analyzing the claims that justify the use of Story Points, as the data above does not seem to suggest any significant advantage of using Story Points as a metric. So I searched for the published justification for the use of Story Points and found a set of claims in Mike Cohn's book "User Stories Applied" (page 87, first edition):

• Claim 1: The use of Story points allows us to change our mind whenever we have new information about a story
  
• Claim 2: The use of Story points works for both epics and smaller stories
  
• Claim 3: The use of Story points doesn’t take a lot of time
  
• Claim 4: The use of Story points provides useful information about our progress and the work remaining
  
• Claim 5: The use of Story points is tolerant of imprecision in the estimates
  
• Claim 6: The use of Story points can be used to plan releases

This these claims hold?

Claim 1: The use of Story points allows us to change our mind whenever we have new information about a story

Although there's no explanation about what "change our mind" means in the book, one can infer that the goal is not to have to spend too much time trying to be right. The reason for this is, of course, that if a story changes the size slightly there's no impact on the Story Point estimate, but what if the story changes size drastically?
Well, at this time you would probably have another estimation session, or you would break down that story into some smaller granularity stories to have a better picture of it's actual size and impact on the project.
On the other hand, if we were to use a simple metric like the number of stories completed we would be able to immediately assess the impact of the new items in the progress for the project.
As illustrated in the graph, if we have a certain number of stories to complete (80 in our example) and suddenly some 40 are added to our backlog (breaking down an Epic for example) we can easily see the impact of that in our project progress.
In this case, as we can see from the graph, the impact of a story changing it's meaning or a large story being broken down into smaller stories has an impact on the project and we can see that immediate impact directly in the progress graph.
This leads me to conclude that regarding Claim 1, Story Points offer no advantage over just simply counting the number of items left to be Done.

Claim 2: The use of Story points works for both epics and smaller stories

Allowing for large estimates for items in the backlog (say a 100SP Epic) does help to account in some way for the uncertainty that large pieces of work represent.
However, the same uncertainty exists in any way we may use to measure progress. The fact is that we don’t really know if an Epic (say 100 SPs) is really equivalent to a similar size aggregate of User Stories (say 100 times 1 SP story). Conclusion: there is no significant added information by classifying a story in a 100 SP category which in turn means that calling something an "Epic" is about the same information as classifying it as a 100 Story Points Epic.

Claim 3: The use of Story points doesn’t take a lot of time

Having worked with Story Points for several years this is not my experience. Although some progress has been done by people like Ken Power (at Cisco) with the Silent Grouping technique, the fact that we need such technique should dispute any idea that estimating in SP’s "doesn’t take a lot of time". In fact, as anybody that has tried a non-trivial project knows it can take days of work to estimate the initial backlog for a reasonable size project.

Claim 5: The use of Story points is tolerant of imprecision in the estimates

Although you can argue that this claim holds - even if the book does not explain how - there's no data to justify the belief that Story Points do this better than merely counting the number of Stories Done. In fact, we can argue that counting the number of stories is even more tolerant of imprecisions (see below for more details on this)

Claim 6: Story points can be used to plan releases

Fair enough. On the other hand we can use any estimation technique to do this, so how would Story Points be better in this particular claim than any other estimation technique? Also, as we will see when analysis Claim 4, counting the number of Stories Done (and left to be Done) is a very effective way to plan a release (be patient, the example is coming up).

Claim 4: The use of Story points provides useful information about our progress and the work remaining

This claim holds true if, and only if you have estimated all of your stories in the Backlog and go through the same process for each new story added to the Backlog. Even the stories that will only be developed a few months or even a year later (for long projects) must be estimated! This approach is not very efficient (which in fact contradicts Claim 3).
Basing your progress assessment on the Number of Items completed in each Sprint is faster to calculate (number of items in the PBL / velocity in number of items Done per Sprint = number of Sprints left) and can be used to provide critical information about project progress. Here's a real-life example:

The real-life use of a simpler metric for project progress measurement

In a company I used to work at we had a new product coming to market. It was not a "first-mover" which meant that the barrier to entry was quite high (at least that was the belief from Product Management and Sales).
This meant that significant effort was made to come up with a coherent Product Backlog. The Backlog was reviewed by Sales and Pre-Sales (technical sales) people. All agreed, we really needed to deliver around 140 Stories (not points, Stories) to be able to compete.
As we were not the first in the market we had a tight market window. Failing to meet that window would invalidate the need to enter that market at all.
So, we started the project and in the first Sprint we complete 1 single Story (maybe it was a big story -- truth is I don't remember). Worst, in the same period another 20 stories were added to the Product Backlog. As expected, the Product Management and Sales discovered a few more stories that were really a "must" and could not be left out of the product.
The team was gaining speed and in the second Sprint they got 8 stories to "Done". They were happy. At the same time the Product Manager and the Sales agreed to a cut-down version of the Product Backlog and removed some 20 stories from the Backlog.
After the third sprint the team had achieved velocities of 1 (first Sprint), 8 (second) and 8 (third). The fourth sprint was about to start and the pressure was high on the team and on the Product Manager. During the Sprint planning meeting the team committed to 15 new stories. This was a good number, as a velocity of 15 would make the stakeholders believe that the project could actually deliver the needed product. They would need to keep a velocity of 15 stories per sprint for 11 months. Could they make it?

The climax

As the fourth sprint started I made a bet with the Product Manager. I asked him how many items he believed that the team could complete and he said 15 (just as the team had committed to). I disagreed and said 10. How many items would you have said the team could complete?
I ask this question from the audience every time I tell this story. I get many different answers. Every audience comes up with 42 as a possible answer (to be expected given the crowds I talk to), but most say 8, 10, some may say 15 (very few), some say 2 (very few). The consensus seems to be around 8-10.
At this point I ask the audience why they would say 8-10 instead of 15 as the Product Manager for that team said. Obviously the Product Manager knew the team and the context better, right?
At the end of the fourth sprint the team completed 10 items, which even if it was 20% more than what they had done in previous sprints was still very far from the velocity they needed to make the project a success. The management reflected on the situation and clearly decided that the best decision for the company was to cancel that product.

Story Points Myth: Busted!

That company did that extremely hard decision based on data, not speculation from Project Managers, not based on some bogus estimation using whatever technique. Real data. They looked at the data available to them and decided to cancel the project 10 months before its originally planned release. This project had a team of about 20 people. Canceling the project saved the company 200 man-month of investment in a product they had no hope of getting out of the door!
We avoided a death-march project and were able to focus on other more important products for the company's future. Products that now bring in significant amount of money!

OK, I get your point, but how does that technique work?

Most people will be skeptical at this point (if you've read this far you probably are too). So let me explain how this works out.
Don't estimate the size of a story further than this: when doing Backlog Grooming or Sprint Planning just ask: can this Story be completed in a Sprint by one person? If not, break the story down!
For large projects use a further level of abstraction: Stories fit into Sprints, therefore Epics fit into meta-Sprints (for example: meta-Sprint = 4 Sprints). Ask the same question of Epics that you do of Sprints (can one team implement this Epic in half a meta-Sprint, i.e. 2 Sprints?) and break them down if needed.

By continuously harmonizing the size of the Stories/Epics you are creating a distribution of the sizes around the median:


Assuming a normal distribution of the size of the stories means that you can assume that for the purposes of looking at the long term (remember: this only applies on the long term, i.e. more than 3 sprints into the future) estimation/progress of the project, you can assume that all stories are the same size, and can therefore measure progress by measuring the number of items completed per Sprint.

Final words

As with all techniques this one comes with a disclaimer: you may not see the same effects that I report in this post. That's fine. If that is the case please share the data you have with me and I'm happy to look at it.
My aim with this post is to demystify the estimation in Agile projects. The fact is: the data we have available (see above) does not allow us to accept any of the claims by Mike Cohn regarding the use of Story Points as a valid/useful estimation technique, therefore you are better off using a much simpler technique! Let me know if you find an even simpler one!

Oh, and by the way: stop wasting time trying to estimate a never ending Backlog. There's no evidence that that will help you predict the future any better than just counting the number of stories "Done"!

Photo credit: write_adam @ flickr

Kanban vs Scrum, the ultimate fight? Don't think so, here's why:...

Wow, what a week! A BIG post on Kanban by Scrum evangelist @jcoplien litterally put the blogosphere (and twittersphere on fire!).

It is good to have these family fights in the Agile family once in a while. As a life-philosopher once said: "These things gotta happen every five years or so, ten years. Helps to get rid of the bad blood".

But what are the differences between Kanban and Scrum, really? What are they?

Here's my take on the differences:

Kanban innovation

Kanban is, from it's origin a more systematic approach to measuring, visualizing and following up work in a "system" (one team, many teams, a company you name it). Thanks to the work by the Poppendiecks, David Andersson and Don Reinertsen (and others I'm sure) a pretty interesting and innovative economic framework as been put into the software process development lingo. Cost of Delay, Queues, optimize the whole, etc.
This economic framework is, in my view, the major innovation that Kanban proponents bring to the table. This was to be expected as many of the early adopters of Kanban were using Scrum before and felt the need to quantify and analyze some aspects of software development that Scrum did not tackle.

Scrum innovation

Scrum brought to the fore-front of process discussions issues that had never made it to our attention before: Self-organization, people/team dynamics, problem solving within a short cycle with feedback loops in place (Sprint), etc.
The major innovation of Scrum in my view was the introduction of the Socio-technical system of software development as Liker describes it in the Toyota way. Other similar software development processes took some of the ideas that Scrum also took, but shied away from the self-organization at the team level and blocker-removal focus of roles such as the scrum master. Those methods did not last long because the teams felt like puppets instead of adults with the possibility (and responsibility) to produce the best product they could.

Wrap-up

Both Kanban and Scrum brought significant innovations to the software development world. Those are just two types of innovations. There is more coming from the community and instead of focusing only on these two methods (which all of you should experiment with!) we should also look at what else is missing in the software development ecosystem.
My next interest area is "Complex Systems" (Complexity + Systems Thinking) and Management as a profession. I'll be exploring those subjects more and more in the future as a way to complement my use of Kanban *AND* Scrum. I suggest you do the same and find what else is missing in your environment and look for what else is around that could help you complement what you are already using. Here's a suggestion: start with @jurgenappelo's book on Management!

Happy reading, happy learning!

XP2012: The Extreme Challenge

"Agile has crossed the chasm" we hear often. But has it really? In the last few years Agile has gone from being "the cousin in the corner that shouts and screams but no one wants to talk to" to being "the famous cousin who everyones wants to talk to". It was not an easy transition, and to be sure there are still many in the Software industry that call that cousin the "bastard child" of software. Need an example? just take a look at the title of this book.

Agile has turned a corner, there's no doubt about that. However, we have also grown as community, and what was the focus of the last 5 years is also being challenged from within. J.B. Rainsberger described the evolution in a very critical as well as prescient way: the future of Agile is XP (the irony is not lost on many, I'm sure).

JB Rainsberger's keynote "The Extreme Decade: Progress, Pain, Paradox" from Agile Eastern Europe on Vimeo.

XP2012 happens in this context and in itself has a challenge of re-inventing the tired old format of Gurus coming to install the faith on the rest of us. Erik Lundh started a project for XP2012 that deserves some attention as well as exposure. XP2012 will be the first conference in a few years to live up to its name *because* of this challenge!

The XP2012 challenge!

The challenge is to do full cycle from Agilehttp://www.blogger.com/img/blank.gif Planning Game or similar to DoneDone deployed into production at least once in a week.

A set of teams of developers will be invited to attend the conference and effectively develop and release software at the conference in one week or less. They will start and end an iteration during the conference. Random people from the audience will use the software in the technical demos (not the team!) to check that everything is working.

So, if you work in a really good Agile team why not take the challenge? Come to XP2012 to *prove* that you can do Agile Software Development just as it should be done!

Photo credit: sanfora @ flickr

Kanban in a networked process -- Visualise the network!

It seems that the idea of a work-network instead of work-flow is catching on and I'd like to throw a bit more fuel in that fire!

Knowledge work is not linear! This we have learned from the experiences with failed Waterfall projects. There are things we do over and over again, and there are things we do only once in a project. None of those can easily be predicted. Scrum and Kanban are examples of our quest to distill what we do to the bare essential. Only then can we understand and manage our work.

Scrum does this by strictly limiting the work in progress through the concept of time boxes. The idea is: do something that fits a (reasonably) short timebox, and nothing else. Get that done and released before you go on to the next thing.

Kanban does this by strictly obeying Work In Progress (WIP) limits and establishing a Pull system (instead of push as much content as you can into an iteration). Both have pros and cons which I will not discuss here. http://www.blogger.com/img/blank.gif

But most interestingly, both tackle the flow of our work in a linear fashion. Both Scrum and Kanban assume that some work can "flow" orderly to completion. However that is not the case! In knowledge work we have many loops and re-loops in the process that are hard to visualize and follow. Without visualizing those loops we cannot effectively manage our work!

Jurgen Appelo and Allan Shalloway had (what looks like was) an interesting discussion on the non-linearity of work and consequently on the non-linearity of Kanban boards. Check out Jurgen's and Allan's posts on the subject.

Since I've been experimenting with Personal Kanban for 3 months now I'd like to share some of my own experiences.

The picture below depicts my process boards as they evolved (click for larger images). The initial one was simple and linear, and it did not disclose enough information for me.


The second one was much better in highlighting some of the bottlenecks in my process (waiting for my action, waiting for others action, waiting for meeting), but was still too linear to reflect the work in a way that was useful to visualize.


The final Kanban board is actually what I use now and is a slight modification of the second one. As you can see it depicts the networked nature of my work and therefore also helps me set my WIP limits appropriately (for example why not have many things waiting for meetings?).


A conclusion of mine from the above boards is that it is not possible to set WIP limits without understanding the work (for example what "wait" states do you have in your process). On the other hand, trying to follow the WIP limits you have set will help you visualize those "wait" states as a consequence of the search for sustainable WIP limits

How about you? What are your experiences in using Kanban boards or other visualization techniques to uncover the networked nature of our work? Leave a comment below with a link to your post!

Photo credit: sjcockell @ flickr

Adapting to a different country and culture - a story

This is the story of Jerome (names changed to protect the innocent), and his adventures while changing countries for work.

Jerome always wanted to work in another country. He had bought into the whole globalization/Cultural experimentation ideas of the 90's. The economy was growing quite strongly, there was a strong need for IT professionals and he wanted to explore the world.

Armed with a degree and the will to explore he left his country, which was warm and sunny, for Finland. He arrived in the early afternoon of one gray and rainy day. "The prototypical autumn day" - he was told.

"Never mind" - he thought. I can certainly survive a bit of rain and snow. He was met at the airport by a colleague who drove him to this first apartment. Jerome had never seen that apartment but that did not worry him. "How bad could it be?" he thought.
As he pushed his large suit case up the ramp to the apartment, he thanked his colleague for the ride and entered the room. This was the first shock. The apartment was a room, a single room. "Talk about nordic architecture!" he said out loud.

He sat down and appreciated the last few moments of calm before the real battle started. Next morning he would have to visit many offices before his arrival and relocation to Finland could be completed.

Relocation to a new country can be a daunting process for a person. Many new people to interact with, many laws and rules to learn, many things to buy. A whole new life to build! In fact, adapting to a new country can be down right scary. Really scary. However Jerome's employer seemed to be aware of this and had organized for a person to be with him through the "legal" process of relocation. This, Jerome learned, was called a "Relocation Service". "What a great idea!", Jerome thought to himself, "this makes my life so much easier, and already gives me a link to someone I can call in case of an emergency".

So, he went about visiting the "Alien office" (Yes, Jerome was now officially an 'Alien'). "What a way to welcome people, by calling them aliens!". Later he visited the Maistraati, then the bank, then the local phone company and finally the most used relocation service around the world: IKEA!

That night, while reviewing in his mind what had happened during the day he realized that even if much of the communication with the many clerks he met happened in Finnish, all the paperwork was readily available in English. Never for a moment did he feel lost in the paperwork process. Even the web bank (and there was a web bank! -- in 1998!) was in English. Reflecting back Jerome realized that having all the bureaucratic documents in English was one of the most important reasons he felt welcome in his new country. He felt that in this country they were serious about accepting and welcoming people into the society. The barrier was low enough that in retrospect he felt that he would do the same again, if given a chance.

The first barrier was overcome. He was now an official resident of his new country. Next came the integration to the work life, after all work had been the reason for him to move to this new country.

Next was his first day at work. Jerome was apprehensive. he had visited the company offices before during his job interview, but now was his first work day. He entered the office and asked for his manager. After a few minutes waiting, a smiling person came to greet him. Jerome remembered him from the Interview a few months prior to this encounter. He felt immediately at ease.

In the company he was working for the hierarchy was not emphasized, people worked together at all levels and he never felt like an "inferior". He was hired for his knowledge and experience and he felt that his presence was welcome. The fact that he had the support of the relocation service also helped as he did not have to bother which colleagues with questions about the bureaucratic issues and could fully focus on the work at hand. He reckoned that, all in all the relocation service support actually saved him and the company many hours and stress by handling the legal issues and allowing him to focus on work from day 1. This would become even more clear years later when he moved to Germany and did not have that extra support from a relocation services company. But I'm getting ahead of the story. Let's get back to the process of integration to the new culture.

It was now early 1999 and winter had settled in. When he arrived he was warned about winter, and he fully expected cold and snow, but that year was a record year. The boy from down south - where snow meant "no school!" was fully surprised by the -25C winter that year. The snow was another big impact surprise: he had bought some boots that were obviously not fit for Icy roads and spent most of the November/December measuring the temperature of the snow with his lower back. "Auch!"

Getting used to the cold and dark was something that many of his Alien friends struggled with. You can actually see it happening as the level of complaints about culture, bureaucracy and other topics goes up, way up in the winter. Some of the those complaints are justified like the issue with the language learning. No matter how you try if you don't pronounce "kertalippu kiitos" perfectly the tram driver will always reply "and where are you going sir?" - in perfect Oxford English, of course! Language is one of the biggest obstacles to completing the process of adaptation. After many years of attending Finnish courses, Jerome finally learned enough words to confidently order his food in Finnish and not be completely surprised by what was brought to his plate later on.

The first year was gone and one surprise awaited him. One day he got home and saw a letter from the police. "oho" he thought "what now?" After consulting with his colleagues he found out that he had to go to the police to renew his residence permit. This was his first encounter with the local bureaucracy on his own. Would he able to understand what he was told?

Jerome timidly entered the room and saw tens of other "aliens" waiting to be received. Certainly many in the same situation as him. This made him think "why do the police isolate us 'aliens' into a different office? It probably has to do with the efficiency of having all 'alien' issues handled by the same people" he thought. But is this how it really should be? What is the message that his sends to those so called aliens? How does an alien feel to be put into the alien-ghetto like that?

Gladly the Tax Office (verotoimisto) has totally different approach and all tax payers are welcome to any office, despite being an alien. "I much prefer the Tax Office" Jerome thought while at the same time realizing he would never have imagined himself thinking that way.

His preference for the Tax Office was about to grow as later that year he received his first tax return. "It is already pre-filled for me! What a great idea!" Jerome thought while realizing that he would not have to spend endless hours trying to decipher local tax code. "I can get used to this!" he said. This was the first painless tax season of his life! Definitely a plus when considering where to live in the future!

Back to work Jerome could not help but feel that his personality was very much in synch with the way his company was organized. There was a clear purpose and people were encouraged to be autonomous and collaborate across department borders to achieve their common purpose - what a difference to other countries where nothing happens without the boss giving the green light to everything. This focus on autonomy and purpose eased Jerome's integration to the work-place.

Some years later Jerome would receive an offer to move to Germany. He was tempted. Germany's political tradition and strong economic growth were strong attractors, but perhaps the final reason that pushed Jerome to leave Finland was the fact that many companies that he contacted just flat-out refused to hire people who were not native or near native speakers of Finnish. This was in conflict with everything Finnish companies and entrepreneurs were saying publicly! Publicly many companies talked about wanting to go international and about the fact that their companies depended heavily on trade, but when it came to their hiring policies that was not the case! This was 2010! and in Finland we are still hiring people depending on whether or note they speak a language that is probably harder to learn than most of the jobs we hire for! Here's a suggestion: why not have language learning as part of the job? That way we would get highly qualified 'aliens' and would be able to keep them in Finland! But I am digressing now.

Later on Jerome thought about what made his integration at work and in the society harder or easier and he came up this list:

• Easier
  
  
  
  • Relocation Service support to a new comer
    
  • Low hierarchy and clear-purpose at work
    
  • All bureaucratic processes had been documented in English
    
  • Web / Internet banking
    
  • Pre-filled tax returns
    
  • And... lots of friends, many in the same situation
    
  
  
• Harder
  
  
  
  • New culture: when to be friendly? When is that just akward? How to make friends? What do people value?
    
  • Dealing with many different authorities / entities (although relocation services helped!)
    
  • Learning a new language (although knowing the language made the integration process easier later on)
    
  
  

Recently Jerome moved to Germany, and he had a chance to go through the same process again, in a new country. Now more experienced with the process of moving he felt confident that he could tackle the challenges ahead. Little did he know!

Upon arrival to Germany he was informed that he had to decide on a health insurance, and fast because his salary could only be paid after that. "Hmmm, they don't make this very basic thing very easy here, do they?" he thought to himself, and went on: "Strike one for Finland with it's universal health care system. In retrospect KELA makes integration much, much easier by having a universal health coverage from the start!"

Jerome decided to register with a national health care provider. He could have registered with a smaller / local health care provider, but he had heard enough horror stories about smaller / local providers that could go bankrupt at any time. Scary stuff for an Alien!. He then had to register as a "legal alien" - to quote Sting's. He went to the local Amt (that's office in German) and registered, not an "alien" but as an Ausländer. Don't know about you, but Jerome felt much better with the term Ausländer, maybe because he did not speak German.

Jerome arrived at the Ausländeramt desk and politely asked "Sprechen Sie Englisch?" - that being all the German that he could muster. The clerk, however, replied "Oh, but we are in Germany. We must speak German." "Strike 2 for Finland" Jerome thought to himself. Struggling with understanding what the clerk was trying to say he was able to successfully register as a resident of the city. "We have to applaud the idea of an open Europe, this is the stuff that makes us happy we pay our taxes. Long gone are the days when we had to sneak through borders illegally to find a work place to our liking. Now we can freely travel and settle with minimal bureaucracy. That is a real tangible benefit to the people! Borders only benefit the powerful who can find legal, and not so legal ways around them."

Political rant aside it was now time to start enjoying life in a new country. Until...

Later that week, Jerome received a letter at home. He had to present himself in the local city office. His registration had a problem... "Oho! I spoke too soon about freedom from borders..."

Jerome reached the office and asked what the issue was about. After he was explained what the issue was he could not believe it. "Really? in the XXI century? We can send a man to the moon but we can't solve a problem like this?" -- he thought to himself.

The issue was that when he completed his original registration, he had used the wrong postal code and therefore he now had to correct it. But the process for that was worthy of a Kafka plot. He would have to drive to neighboring town, register there (with the original date of his initial registration); then he had to come back to the city office and register again with his current address so that his registration could be considered valid! Basically he had to register twice in two different places to finalize his original registration process. "Talk about bureaucracy! Strike 3 for Finland!"

But the situation got worse later on! Jerome received a letter from the local court where he was instructed to pay a fine for not having registered on time in his current municipality! "The nerve" he said out loud as he read the translation from Google translate. "I've now registered 4 times" - he had registered once more when he changed apartments in between these events - "and they have the nerve to tell me that I have not registered on time?" "Strike 4 for Finland!".

After this ordeal Jerome revised his table of things that make your integration easier or harder:

• Easier
  
  
  
  • Relocation Service support to a new comer
    
  • Low hierarchy and clear-purpose at work
    
  • All bureaucratic processes had been documented in English
    
  • Web / Internet banking
    
  • Pre-filled tax returns
    
  • And... lots of friends, many in the same situation
    
  
  
• Harder
  
  
  
  • New culture: when to be friendly? When is that just akward? How to make friends? What do people value?
    
  • Dealing with many different authorities / entities (although relocation services helped!)
    
  • Learning a new language (although knowing the language made the integration process easier later on)
  
  
  • Bureaucracy!
  
  
  
  

The end!

Photo credit:

Rob Young @ Flickr
mac_filko @ Flickr
Giorgos @ Flickr

Dude! Where's my manager? or Why you should attend LESS2011

Many teams start their agile transition from the "developer-side". This is quite normal, developers (coders and testers) feel the pain more than others because they need to actually get the product/system finished. But by focusing on developers, aren't we missing something? Aren't we forgetting that many of the consequences we so detest come from un-informed decisions by people higher in the chain?

In my experience many agile transitions fail by not involving managers in the process. Sure, it is possible to change how your team works with minimal involvement from your manager, but at some point the team is constrained more by management decisions than actual technical practices or even understanding of the product.

I remember a story of a team that was on their path to agile adoption, but could not progress further because management had decided that certain tools should be used. The usual explanations were given: "IT can only support one tool", "we need to harmonize our tool landscape", etc. Whatever the reason for the decision what we can say is that in this case management had a real - and negative - impact on the team's capability to deliver working software.

Sure we can go rogue and use our own tool chain "hidden" from IT and our manager (and many teams have done that), but that is not a long term strategy. Sooner or later we will bump again against a set of decisions that will hinder us from progressing.

I believe we need a new model for Agile adoption. One that includes the managers, leaders, VP- C-level people in our organizations.

It is this belief that has led me to participate in a project to organize a set of conferences focused on the role of leaders and managers. Last year we organized the first of that set of conferences in Helsinki and named it LESS2010. This year we are continuing that project with LESS2011 in Stockholm.

There are many reasons for you or your manager to attend the LESS2011 conference in Stockholm. From the individual speakers (check-out our keynote line up) to the people you will meet. But If I had to name one reason it is this: Agile and Lean adoptions require our managers to understand the new mindset, and without that we are bound to fail. So, get yourself and your manager to LESS2011 and talk to other managers! Share your experiences and questions and come learn from people that have been facing long lasting agile transitions (the kind that requires management involvement)!

Guest post: Building and scaling a startup - Feature or component teams?

There are some really cool startups in Berlin. Last week I met Alexander Grosse (@klangberater) from SoundCloud at ALE2011. Alexander is their VP of engineering and he had been thinking and writing about Feature teams. After reading his article I thought it would be a good and balanced view at that topic that could generate a good (and balanced) discussion on the blog. Alexander kindly agreed to let me publish it, so here it is!

Building and Scaling a startup - the Feature vs Component teams debate

Guest post by: Alexander Grosse from SoundCloud

The first team in a startup is a feature team by default. But as the company grows and you hire more developers, a common question pops up: How do you structure your engineering department when you have more than one team? Two common approaches are teams per component/service/layer or feature teams. This article describes the pros and cons of each approach based on experience.

So you have a startup. Let’s assume for the sake of simplicity it is a consumer facing website done in Ruby on Rails with an underlying database (but this article applies to other kind of startups too). You have around 10-15 employees, of which 6-10 are developers. These developers are one team, some of them concentrate on front-end work, some on the back-end, the mysql database is maintained by a handful of the developers. The team feels like it has reached the natural size limit and is considering splitting into smaller teams. Before discussing team structure let’s first talk about some important goals while scaling up the company.These goals should determine what you do:

• Developers should have as much knowledge of the system as possible, so that there is no single point of failure and technical decisions are made with an understanding of the overall system.
  
• There should be common approaches to most technologies like CSS and for some technologies, like databases, specialists are needed.
  
• Communication overhead should be avoided where possible as well as handovers between teams.
  

Overall there should as much common knowledge as possible but specialists where needed. Let’s first discuss the terms “Feature Team” and “Component Team”, especially in a startup context. The usual definition of a Feature Team is: Feature teams are supposed to be cross-functional groups that focus the software development process in valuable-chunks of work, which normally cross the whole system (vertical slices of the system).[1]

While this sounds good you usually reach pretty fast the team size limit when trying to compose teams which can touch every part of the system. At SoundCloud we currently have 35 engineers working on topics like Android, iOS, Search, Anti Spam, MySQL, Operations, Web Front-end, Back-end and so on. So you can see very easily that if you define seven as an ideal team size, we have difficulties forming teams which can touch every part of the system.

A lot of consultants say that engineers should be able to work on any part of the system, but in reality and especially if you operate under scale, where for example every single SQL query can impact the site, this just does not work. Component Teams are usually defined as horizontal slices of a system, so for example front-end, presentation layer, business logic, database and operations. But I haven’t seen a startup which organizes it’s teams exactly that way. This is usually a team structure big companies choose. But what I’ve seen at a lot of startups is the separation between front-end, backend and operations. There are lot of obvious disadvantages associated with component teams:

• Features often touch several teams, this leads to costly handovers
  
• Teams often act as silos and lose sight of the coherency and goals of the product But especially in large organisations component teams also have some advantages, like having dedicated teams for components - like databases - under scale. (please look at [3] for details)
  

So, we now have two approaches which on the surface both are not applicable for startups either at all or from a certain size on. Reading the post from Vasco Duarte ([4]) clarifies the situation in a very good way. Feature teams are not necessarily defined by their structure, but the most important thing is that a feature team can deliver value to the customer without dependencies on other teams. The reason for that is to reduce handover between teams which usually produces what “waste” is in terms of lean. Citing Vasco: A feature team is not defined by it's structure (being cross-functional), it's defined by it's output! (delivering shippable pieces of the system)[4]

A common anti pattern for the implementation of Feature Teams is (similar to the one mentioned in the article): design is not integrated into front-end teams. This often causes blocked user stories (waste) when design specs are either unclear or during implementation a necessary change in design has been discovered. In reality there is no such thing like having absolutely no dependencies on other teams. So we defined teams, which can work on the vast majority (at least 90%) of the stories in their backlog without needing other teams. How those teams are formed changes over time.

We at SoundCloud did not always have a stable API. So in the beginning front-end and back-end developers were one team until we were able to extract a well defined API, which offered the necessary functionality for all front-end clients (mobile and web). At that point we separated the joint web team as the front-end teams were now enabled to work on the majority of features without involvement of the back-end. Looking back at the start of the article where we defined that we want to have developers as much overall knowledge of the system as possible it seems like the chosen setup at SoundCloud violates that. To achieve a compromise, we encourage developers to rotate through teams. Some developers want to stay in their comfort zone (these are often the specialists mentioned above), some are interested in all parts of the system. As long as half of the developers in a team are aware of what other teams are doing, that team will be fine.

Combine both approaches The question is not so much which formal definition of team to follow. The importance is for teams to deliver the majority of stories in their backlog without depending on other teams. And team composition ought to change over time as the overall software stack matures. Analyze your teams if they can deliver value without depending on other teams (for at least 90% of the features). Do this regularly and team composition changes over time just as architecture does!

Inspired by:
[1] Scaling Lean and Agile Development, Larman, Vodde: Feature team definition on page 153
[2] http://blog.mountaingoatsoftware.com/the-benefits-of-feature-teams
[3] http://scalingsoftwareagility.wordpress.com/2009/07/15/organizing-agile-at-scale-feature-teams-versus-component-teams-part-1/
[4] http://softwaredevelopmenttoday.blogspot.com/2009/07/feature-team-needs-more-than-cross.html

You can find out more about SoundCloud at the Goto conference in Århus. Eric Wahlforss from SoundCloud will be speaking there.