The Release Paradox: releasing less often makes your teams slower and decreases quality

Herman is a typical agile coach. He works with teams to help them learn how to deliver high-quality software quickly.

Many teams want to focus on design, architecture, or (sometimes) even on business value. But they are usually not in a hurry to release quickly.

Recently Herman conveyed a story to me that illustrates how releasing quickly can help teams deliver high-quality software much faster than if they would focus on quality in the first place. This is the case of a team that was working on a long overdue project. They had used a traditional and linear process in the past and had been able to release software only very recently, after more than 12 months of work on the latest release.

Not surprisingly, they were having trouble releasing regularly. The software was not stable; once it was live it had many problems that needed to be fixed quickly, and worst of all: all of this was having a direct impact on the company’s business.

The teams were extremely busy fixing the problems they had added to the product in the last year and could not focus on solving the root causes of those problems.

They were in full-fledged firefighting mode. They worked hard every day to fix yet another problem and release yet another hot fix.

This lasted for a few weeks, but once the fire-fighting mode was over, Herman worked with the teams to improve their release frequency. During their work with Herman, those teams went from one year without any release to a regular release every two weeks.

At first the releases were not always possible, but with time they improve their processes, removed the obstacles preventing them from releasing every two weeks and started releasing regularly.

What happened next was surprising for the teams. The list of problems after each release did not grow - as they expected - but instead shrank.

When some problems came in from the customers after a 2-week release, they were also much faster to fix the problem and quicker to release a fix if that was required. When the fix was not critical, they waited for the following release which was, after all, only 2 weeks away.

By focusing on releasing every two weeks, Herman’s teams were able to focus on small, incremental changes to their product. That, in turn, enabled them to fine-tune their development and release processes.

Here are some of the key changes the teams implemented

1. They started with a 4 week release cycle, and fine-tuned their daily builds and release testing process to enable a release every 2 weeks.
2. They invested time and energy to improve their test automation strategy and automated the critical tests to enable them to run “enough” tests to be confident that the quality was at release level.
3. They had some teams on maintenance duty in the first few iterations to make sure that any problem found after release could quickly be fixed, and released to customers if necessary.
4. They changed their source code management strategy to enable some teams to work on longer term changes while others worked on the next release.
5. They involved all teams necessary to complete a release in their iterations. This affected especially: production/operations team, localization team, documentation team, marketing team, and other teams when needed.

This list of changes was the result of the drive to complete each release and learning from the failures in the previous release. Some changes were harder to implement, and especially the testing strategy to allow for 2-week release cycles had to be changed and adjusted several times.

One of the key problems the teams had to solve, was the lack of coordination with departments that directly contributed to the release but were not previously involved in their day-to-day work.

This process lasted several months, and would not have been possible without a clear Vision set forth by the teams in cooperation with Herman, who helped them discover the right way to reach that Vision within their context.

Herman’s work as a coach was that of a catalyst for management and the teams in that organization. He was able to create in their minds a clear picture of what was possible. Once that was clear, the teams and the management took ownership of the process and achieved a step-change in their ability to fulfill market demands and customer needs.

Customers have no reason to change provider as they have an ever-improving experience when using this company’s services.

Today, this organization releases a new version of their product every two weeks. Unaware of it, their customers receive regular improvements to the product they use, and have no reason to change provider as they have an ever-improving experience when using this company’s services.

Picture credit: John Hammink, follow him on twitter

Real stories of how estimates destroy value in Software Development

A friend shared with me a few stories about how Estimates are destroying value in his organization. He was kind enough to allow me to share these stories anonymously. Enjoy the reading, I know I did! :)

The story of the customer that wanted software, but got the wrong estimates instead

Once, one of our customers asked for a small feature and, according to this customer it was quite clear from the beginning what he wanted. Alas, in my experience things often don’t go as planned and therefore I have added a good buffer on top of the needed estimates. Just like all developers I know do. Every day.

However, the story was about to get more interesting. A bit later, the sales team said those numbers were too high. "The customer will never accept those numbers!", they said. "And we really want this case."

After much negotiations we reduced our estimates. After all, we had added some buffer just in case. So, we reduced the estimates on the account, and I heard myself say (I should have known better): "Well, I guess that if everything goes well we could do it in that time."

My real surprise was that, a few days after the estimates were given to the sales team and the customer, I heard from the project manager that the company had agreed a Fixed Price Project with the customer. "That is madness", I said to the sales team. I felt betrayed as a developer! I had been asked for an estimate for the project, but I was tricked into accepting a Fixed Price Project through the estimation process! During the estimation I was not told that this would be a Fixed Price project.

The result was that the project was delivered late, we exceeded the estimates we gave. However, I did learn a valuable lesson: If you are forced to create estimates and the customer requires fixed price then never obey wishes from sales team: their target is different. Or, alternatively just do away with the estimate process altogether: just ask the sales team what number they want to hear ;-).

History repeated: never trust the sales team to handle estimates

A few months ago a customer called in and requested a small feature for their existing product. They were asking for an estimate. Part of the work was very clear and would be very easy to implement. But there was a part that wasn’t that clear. After some pre-analysis and discussions with other development team I knew about what would be needed to finish the job. Unlike the story above, this time I knew the customer asked for a Fixed Price Project and therefore added some buffer – just to be on the safe side.

After carefully estimating the work with the team we sent the total estimate to the sales team. This estimate included everything that was needed: from analysis, implementation, test, deployment. I did not split the estimates into its components parts because I didn’t want the customer to think that testing would be optional (Yes! It as it has happened before).

However, the sales teams split my estimate into the different roles. Big mistake! Magically the estimate that the customer received was half-day shorter than the one we provided. Not a big difference, but I learned my lesson!

Surprise number two was about to hit me: The customer said they hadn't used some hours from a previous contract and that they should get a discount. Long story short, when the project was started we had to reduce our "cost" to 50% of the estimation I had originally given to the sales team. I learned that I should never trust the sales team with mathematical calculations!

We did manage to deliver the feature to our customer, thanks to some very aggressive scope management, this meant that we had to deliver functionality that was requested, but did not perform as well as it could have if we had been allowed to work on the feature from the start, without the estimation back-and-forth.

I did learn my lesson: If you sales team doesn’t know how to sell software projects, and before you given them any estimates, add even more buffer on top of things learnt from the previous story! ;-).

Estimation as an organizational smell

Certainly there are many organizations in the world that do not go through similar stories as these. However, many still do! For those organizations that are affected by stories like these I suggest that we look at different ways to manage software projects. #NoEstimates provides a clear alternative that has proved very successful in the past. For example, in the paper below I mention a story where a team would have given an estimate within 4% of the actual time it took the project to complete, if they had used #NoEstimates!

The Silver Lining

The story continues, however. Here's what my friend added at the end of his email:

Well, after all having successfully changed the mindset of many in the company and in our team. We are already doing quite well.

Now I do things differently. The first thing I always discuss with the customer is how much money they would like to spend for the work they want to have done. This already gives me a good picture if we are even close in understanding of the amount of work that is necessary or more in the direction of “insane”.

Today I don't negotiate traditional contracts with my customers. Every job is now build on trust, communication and transparency.

Agile Eastern Europe announced, I'll be visiting

It is now official, I'll be speaking at Agile Eastern Europe. Hope to see you there!

I'll be hosting a workshop and a talk. Both are about the business view of software development. One will be specifically about Agile:

• Business Agility. How to take advantage of an Agile R&D?
  
  Many companies have jumped on the Agile bandwagon. That's good, but what for? In this talk we explore the consequences and possible benefits of adopting Agile for your Business. It's not enough to benefit your R&D, we need to learn how Agile can help our whole company.
  

The other talk will be more generic, but about something that is critical for Agile teams: a workshop about creating a Vision that speaks to the team and can therefore guide the software development:

• Workshop "From an idea to a Vision you can implement"
  
  You've been there. You are tasked with implementing a product that someone else cooked up. What do do next? Follow the spec you say? Wrong!
  
  Developing a product without this Vision is not just waste, it is bad business for you and for your customer.
  
  Before we start implementing any product we must explore it's reason to exist, what customers it benefits and ultimately how it can help your customers (not you!) make money.
  
  In this workshop we will take an example and go through a simple process that helps us explore a product idea to the point that a spec is just a reference, but the product comes alive in the minds of the team members.
  

On the decay of old media: Helsingin Sanomat goes back to the 1990's

I was skimming twitter this morning when I saw a link to an article/blog post in HS site about how they plan to "revolutionize" the way people interact/work with HS today.

First let me get this out of my chest: whoever was tasked with creating this project is either a complete noob in this thing we call the internet or their task was to create a good video for a concept that has no chance in hell to succeed! What a large #FAIL!

Now for the actual content. HS misses the whole internet thing completely.

1. First, they created this video which is about creating a platform that transfers their "paper" product to your screen. Really? Don't you have a little bit of imagination? Or at least look around to see what is happening! People don't want the "old" media in a new format. They want and need a new media, that is a more integral part of their life, not another format to learn!
   


2. Second, they completely ignore that in the age of the internet when the competition is just a click away the media business model has to be about producing good content first and only later about the medium to deliver it. The medium will evolve forever! (remeber sites from 1999? not so successful if launched today!) Having a concept that revolves around the way that media is organized *on the screen* is like trying to build a car by 1800's standards: you know with a 1000 bhp engine on a model T. Totally clueless if you ask me
   


3. Third, the video seems to show that they want to use all kinds of media formats in their content: videos, more photos, more ads, etc. Well, I don't see how a decreasing revenue ecosystem (print media) can really work with a future model that will exponentially increase their production costs (more video report teams, more photographers and photo editors, more expensive content handling and management systems, etc.) I mean, look at Huffingtonpost.com. They are competing for attention with the big guys with a much smaller work-force!
   


4. Fourth (and last - although I could continue for ever), they completely forget that in the age of the Internet, access to content is the commodity, what they should be focusing on is either a content-niche (opinion, creating social dialogue, etc. -- where they have no competition) or then try to create a dialogue with their customers that cannot be replicated elsewhere (like user engagement through social media, etc.). Trying to just create another "way" to access their content, which is closed, limited and will be out-dated 6 months after they spend millions implementing it is not just dumb, it should be considered neglect by WSOY board!
   

Helsingin Sanomat: please wise up and truly innovate! Look around, there's lots of people in Finland with great ideas. Take a risk, listen to them. After all Finland is one of the most advanced countries in online/mobile services. Surely we must have people in this country who can give you great ideas!

Here's the sad, sad video. Well produced, but with the wrong content...




Photo credit: fireflythegreat @ flickr

Response to Lynda Bourne on Agile and the Business message

The following is a comment to Lynda that I was not able to submit to her blog. You can find Lynda's original post here.

@Lynda

Nice post. I'm glad that the conversation is continuing. Both PMI and Agile people need this dialogue.

A couple of comments on your post.

1. I agree that Agile Advocates need to learn to speak to the business people. However,

2. Agile adoption has been fast in many places, not slow.

It is true that many Agile Advocates (AAs as you call them -- like the analogy BTW, and think it fits! :) don't talk "business-ese". That's been a failure of the community, however that is far, very far from being a problem in adoption. Having been involved in two major organization change projects I can vouch for the lack of business-oriented communication, but I can also assure you that speed of adoption has been anything but slow. I know of a company that has several 10's of thousands of employees where Agile is a topic of discussion at all levels, from top management to the team level. This is very encouraging, especially because this company has not been shy in removing some of the building blocks that are in PMBOK (and inadequate to software development) and build new processes at all levels based on the principles of Agile Software Development.

3. Agile software development needs discipline

I agree with this, but the lack of discipline existed in software organizations when PMBOK was the prevailing framework for organizing projects. Lack of discipline is a problem that Agile encoutered when it started, not a problem created by adopting Agile software development methods. Indeed, it is my experience that agile software development builds a much higher level of discipline into the development process than was previously the norm.

4. Agile alignment with business objectives.

This is a topic for a post in itself, but I'll just refer one example of how agile brings focus and alignment with business goals. Check this post by Dean Leffingwell where he explains how requirements management in Agile can be done in a way that directly reflects the strategy of the company.

Let's continue the dialogue. I've learned a lot and hope to continue learning from it! :)

Pearls of Wisdom: Taiichi Ohno on management

It is rare that one reads something written in 1978 that feels relevant to today's world in so many ways that is hard to explain all of its consequences.

This is what happened today and I wanted to share that. The following is a couple of paragraphs from the "Toyota Product System" (TPS) by Taiichi Ohno. Ohno was one of the key people in the development of what we know now as TPS.

Cost Reduction is the Goal

Frequently we use thew word "efficiency" when talking about production, management, and business. "Efficiency," in modern industry and business in general, means cost reduction.

At Toyota, as in all manufacturing industries, profit can be obtained only by reducing costs. When we apply the cost principle selling price = profit + actual cost, we make the consumer responsible for every cost. This principle has no place in today's competitive automobile industry.

Our products are scrutinized by cool-headed consumers in free, competitive markets where the manufacturing cost of a product is of no consequence. The question is whether or not the product is of value to the buyer. If a high price is set because of the manufacturer's cost, consumers will simply turn away.

Cost reduction must be the goal of consumer products manufacturers trying to survive in today's marketplace. During a period of high economic growth, any manufacturer can achieve lower costs with higher production. But in today's low-growth period, to achieve any form of cost reduction is difficult.
There is no magic method. Rather, a total management system is needed that develops human ability to its fullest capacity to best enhance creativity and fruitfulness, to utilize facilities and machines well, and to eliminate all waste.

The Toyota production system, with its two pillars advocating the absolute elimination of waste, was born in Japan out of necessity. Today, in an era of slow economic growth worldwide, this production system represents a concept in management that will work for any type of business.(1)

It is hard to believe when you read this (emphasis added) that it was written in 1978 (or before). But that is the case. All of this applies almost directly to the software industry today.


(1) Excerpt from Toyota Production System, By Taiichi Ohno, Productivity Press

What is a bottleneck? (TOC)

When faced with a problem in a process, the first step you should take is to identify the process map and bottleneck.

This is nothing new, TOC people have been advocating this for a very long time as part of their "5 focusing steps". But if it is why don't we do it?

Well, because it is hard work. To identify the bottleneck you can't just sit around a table and have a meeting with the managers to define what is the bottleneck. You have to go to the Gemba and analyze the process "as is", not as "it should be" and make some actual work on data collection and analysis.

Once you have done that you are then equipped to start searching for the bottleneck, it's location is buried in the data and process map you just collected.

To find the bottleneck you have to work hard, but finding it is just the first step! Much more is to come...

The Law of Disproportional Consequences

Why this title? I wanted to describe this idea that in software it is far harder to develop the right things. Yet the impact of developing the right product is disproportionally higher than incremental improvements in the way you develop an existing product.

If you look at the history of software companies this is quite clear. Google, Microsoft, Apple and other giants in the software world today were created by a pair of kids in a garage (literally). This is the positive Black Swan that Nassim talks about in his books and a clear indication that in software it is "what" you develop that matters, not "how" you develop it!

The consequence of understanding and taking advantage of this law is that software companies should be spending a lot more money creating products that "fail" than they do now. Just think about it. If Google did not have it's "labs" would they be as successful as they are now? Almost all of their cool/hot products came from the labs.

How about Apple? If they did not have a crazy guy like Jobs at the helm would they have invested in such complete departures from their original business like iTunes or the iPod?

For a company like the one where I work now, this means that we should be creating ideas about new products at least at the same rate that we create incrementally new products. If we get a 10% success rate with this approach we can be 10/100 times better than what we are now!

In Software there is a law of disproportional consequences, period. The question for us is can we take advantage of it?