Don't stand there, just do it!

Brian Marick in his blog talks about the trend he is noticing at Agile 2006 of people talking about leadership as the "silver bullet" to implement Agile (my interpretation).

I could not agree more with this comment, that kind of leadership is not what Agile needs. Agile needs old-school managers to learn to stop wanting to command-and-control (and learn to love the bomb). Leadership is many times used to disguise "management" in the worst sense of the word: "leading intelligent people like if they were barely able to think and needed minute-by-minute directions."

People that believe in the "leadership silver bullet" tend to also believe that one person can come and change a team/company to the best (like introducing Agile Software Development methodologies ;). This is not true, an Agile Coach (I like "coach" much better than "leader") does not have any real material power, she only has the permission to influence: she is allowed by the team to bring in her own experience and expertise to the table, but ultimately it is the team (as a whole!) that decides what to do and how to do it.

Many people are (understandably) skeptical about this "coach" role. They are skeptical because they have never met a real coach that could so much influence their work life as to make it meaningful and pleasant. Never having experienced a successful "coach"-"team member" relationship is probably the single most important reason for people to, still today, believe that a "leader" can come and save the day.

They are locked in the "great man" theory and only an important shift of their mental model will help. How do you bring that shift about? Lead by example, or better yet, don't lead, just do it and show it works! ("Being led by example" is what people call the desire to replicate a successful effort - I just call it common sense...)

The top management myth

Another myth related to the "Leadership" smoke-screen is the one I often face when people tell me that Agile cannot be implemented without the consent and even sponsorship of top management, after all, "it is in their best interest to do so because it will increase productivity", etc., etc. ad infinitum.

Stop! Agile only works if people believe in it and are willing to go through the process of "failing" in order to learn (remember this is not the "silver bullet"). As any new piece of knowledge, knowing how to apply Agile methods comes through some study and a lot of experience.

If you believe Agile will work for your company start working that way, learn from your mistakes and improve, every day. If you do so, you will be able to implement Agile without ever talking about it to your top management. More, they will be in the happy (but hopelessly ignorant) belief that they were the ones that improved your company and did it without your help! BTW: don't let this affect you, this is actually a sign of success, not failure to take credit! ;)

Agile for our sake

Agile works and continues to thrive not because a bunch of well paid consultants have been spreading it (even though that is starting to happen) or because a group of clueless CIOs suddenly saw the light (even though that is also starting to happen). Agile works and thrives because more and more people experiment with it and see the benefits.

What are you waiting for?

We always tested/refactored, we just did not do it right or called it that...

I was browsing the software blogosphere and ran into this post by David Chelimsky about why he started doing TDD and refactoring.

I just could not help to see my own story in that post. Was it like that for you too?

Chaordic projects and the empirical quest

The scientific process relies heavily on empirical observations. We must, in the software development industry, also rely on empirical observations of successful software development projects and follow the learnings from those projects.

With software development projects the hard part is that most of what happens is not quantifiable and if it were - by decomposing its parts - we would loose of an important view: how all the parts interact together is often the reason why projects succeed or fail.

Successful software development projects are, in my empirical experience, Chaordic. Chaordic means, according to Dee Hock, the author of the word:

"1. The behavior of any self-governing organism, organization or system which harmoniously blends characteristics of order and chaos (...) 3. Characteristic of the fundamental organizing principles of evolution and nature."

In my experience, although I have been a project manager in many projects, the ones that I consider successful were in large measure not controlled by me or any other single person but rather a community of enrolled/engaged people.

Here's an example of the organization that emerged in one of the largest projects I've managed.

The team was divided into 2 large teams that developed architecturally/naturally different components (main teams). Each of these teams had a "project manager" or a person that was responsible to follow the development of each main team. For each component there was also a clear division of responsibilities in smaller groups (sub-component teams).

The project managers leading the 2 main teams met regularly with a larger set of stake holders, not just project related but also related to other important activities/functions in the company, this allowed the "management team" as it was called to build a holistic view of the project and how it fitted the company strategically (high level long term goals) and tactically (shorter term goals). This allowed the group to make decisions based on many parameters not only project-related ones.

As the main and sub-component teams interacted I could not - and did not try - to control how they worked together and interacted. The main teams worked largely on their own with a few synchronization points defined by all of the project participants in a kick-off meeting early in the project timeline.

The sub-component teams also worked independently and synchronized their work independently from the synchronization points that I mentioned earlier, this meant that the sub-component teams pretty much coordinated their own work in a self-organizing way.

What enabled the self-organization of sub-component teams was that each sub-component team member knew other sub-component team members quite well and almost all were co-located in one floor. Communication happened naturally and meetings were called only when needed without intervention from the main team project managers.

The items I want to call your attention to are:

1. The project was divided into "natural" divisions (those main teams had been working independently before);
   
2. The sub-component teams largely self-organized (they knew each other very well and the pace of new entries was slow enough to allow the new people to "absorb" the local culture and adapt quickly)
   

These characteristics have been present in other successful projects that I've managed or been involved with. The attentive reader will notice that the "natural", "self-organizing" and "organic growth" terms refer heavily to the "natural" or Chaordic organization I'm referring to in this post.

The Chaordic Age

In the "Birth of a Chaordic Age" Dee Hock describes through his own empirical observations and subsequent studies what he sees as a better way to look at how organizations (business or otherwise) work and should be organized. In this book, references to Nature and "natural" organizations are abundant. The author tries to convey the idea that organizations should also be "naturally" organized as opposed to "oppressed" into an unnatural structure through command-and-control approaches (more details in the book).

Natural team organization is important, and I still get surprised by how often I see "natural team organization" emerge in successful projects.

Hard value versus System value of Agile Methods

First a clarification. I've used "hard" value as opposed to "system" value in this article, and my definitions are:
"Hard value": any value that can be or is systematically quantified in terms of money or time and normally focus on details (e.g. the value of TDD, the value of incremental development).
"System value": any value that can be quantified after the fact but is mostly subjective and understood when the whole picture is taken into consideration (e.g. the value of Agile in a software development project/organization).
"System value" is characterized by the fact that it cannot be quantified by looking at any of items in a system without understanding how that item interacts with all other items in the same system, i.e. the "system value" approach assumes that you cannot understand the "system value" of TDD without considering how it interacts with other practices, principles and values in the Agile system (e.g. how TDD interacts with Unit Testing and Continuous Integration).

Now for the real post:
In a recent series of posts (1,2,3) Dave Nicolette argued for a roadmap to assess the hard value of Agile, even more recently he described some studies that try to measure TDD/Pair Programming in "money terms" like productivity.

I think that his effort is commendable and I've also been devoting a lot of time to the question "why does Agile work?" and scientific evidence regarding that question.

However, I'd like to caution ourselves. Agile represents a new way of developing software that builds on a lot of shared experiences (failed and successful) in our community. Trying to pin down "every cent" of productivity for each of the different practices in Agile should not be the main goal for us as a community, even for those that want to "cross the chasm".

Agile Methods inherit an ecosystem (I like that word!) of values, principles and practices. Values and principles should always be present and guide the adoption of practices.

The practices per se do not yield any more value than any other point-improvement in a waterfall project. Indeed some of the practices advocated by the Agile community are being used in waterfall projects, with TDD and pair-programming being some of the most used, but certainly not the only ones.

The "system" value of Agile comes from much more than just practices, and we should not get blinded by the focus on justifying every "cent" of productivity and forget that without living the values and implementing the principles of Agile we are missing the big picture.

As a systems thinking apprentice I must also emphasize that the search for the answer to the question "why does Agile work?" in the details will only yield crumbles and small anecdotal evidence (albeit scientifically anecdotal) of something that is much bigger, the System of Agile values, principles and practices.

More anecdotal evidence

In my company we've had a few pilot projects to introduce Agile software development, some more successful than others. One of the learnings we took from one of the projects was that introducing the Agile practices without providing the full understanding of the values to the people working with the practices led to big people problems, even though the project was successful in monetary terms (on time, very few quality problems, with less resources than a third party estimated, etc.) it left behind a trail of problems that later on had to be tackled by the organization and eventually led to some of the developers getting back to the old habits of software development waterfall-style.

In another project the approach was totally different. No practices were forced on the team, the team was constantly challenged to follow the Agile values and principles but given the freedom to find their own solutions (which the Agile practices being presented to them as possible solutions, but not forced).
After that project the people were motivated to learn more about Agile and Agile practices and the project itself was a success in terms of schedule and controlled costs - but there was no third party estimate to compare it with.

When comparing these two projects in my mind I would say that the first was focused on the "hard" value of Agile software development, and delivered a successful outcome if you follow "hard" value only.
The second project was focused on "system" value, which at first did not necessarily include "hard" (money) value as a goal, but eventually also delivered "hard" value.

The point I'm trying to make is this: Agile is a system of Values (with principles and practices) not a collection of practices that you can apply separately in search of some "hard" value here and there. You can of course apply those practices separately but you will not be able to rip the larger benefit of applying the Agile system of values, principles and practices.

XP 2006 Conference, Day 2 highlightsDuring the second day in the conference I attended the scientific paper session in the morning. Of the three papers presented the most interesting was the one on TDD and the effects it has on how we develop code.The gist of the paper was that in order to be able to do TDD you will develop code with higher assignment controllability than if you don't use TDD. Assignment controllability was defined as the control over all the elements in any assignment statement within a method.Let's take an example: in the statement A = B + C, the assignment is controllable if the method that does the assignment is passed all the elements in the assignment.public int addition( int B, int C){ return B + C;}An assignment is less controllable if some of the elements in the assignment are not passed in:public int addToC( int B ){ C = new Integer(5); return B + C;}The author of the paper went on to detail a process to measure the controllability in any method.The implications of the theory that the author presented are interesting:- by having assignments in methods that cannot be controlled you cannot reliably use TDD for those methods.- Assignemnt controlability can be used to see if a project is TDD or not (the threshold is not clear, but you can see a "trend")- You can use the Assignment Controllability(AC) metric in legacy code to see where it would be easier/harder to insert the tests first. Start testing where AC is close or equal to Zero if you want to tackle the more problematic areas of the code first, and start where AC is close to one if you want to add tests quickly.XP 2006 Conference, Day 3 highlightsOn the third and last day of the conference Kent Beck had the Keynote and he used it to describe what he called "responsible development".He defined the current state of Agile/XP as:1- people believe that Agile/XP works;2- people understand and want to profit from the benefits it offers;3- people undestand Agile/XP as being practice focused, not value focused.He went on to explain that Agil/XP should be value focused and called that new approach "Responsible Development", based on the following pillars:1- accountability;2- responsibility;3- integrity;4- avoiding blame.Kent described what each of these values means for him and why he thinks people in the software industry should follow these values. He compared each of the concepts with past experiences and tried to justify the business sense that they make for software companies.One interesting proposal from him was that software companies start reporting on the number of bugs/problems they have open for each of the products they sell. This, he stated, would contribute to transparency in the business through a wide audience form of accountability, as in accountability to all users of our software and shareholders of our companies.Very interesting outline, I have also given these ideas some thought and went to the point of in a recent project ask the team to report the number of open bugs to all the stakeholders during the iteration reviews. This, in my opinion, brings transparency to the software development process and makes people think hard about the quality they are providing along with the functionality.All in all a very interesting talk. We as an industry need to pay attention to these concepts, they make very much business sense! Posted by

XP 2006 Conference, Day 2 highlights

During the second day in the conference I attended the scientific paper session in the morning. Of the three papers presented the most interesting was the one on TDD and the effects it has on how we develop code.

The gist of the paper was that in order to be able to do TDD you will develop code with higher assignment controllability than if you don't use TDD.
Assignment controllability was defined as the control over all the elements in any assignment statement within a method.

Let's take an example: in the statement A = B + C, the assignment is controllable if the method that does the assignment is passed all the elements in the assignment.

public int addition( int B, int C)
{
return B + C;
}

An assignment is less controllable if some of the elements in the assignment are not passed in:

public int addToC( int B )
{
C = new Integer(5);
return B + C;
}

The author of the paper went on to detail a process to measure the controllability in any method.

The implications of the theory that the author presented are interesting:
- by having assignments in methods that cannot be controlled you cannot reliably use TDD for those methods.
- Assignemnt controlability can be used to see if a project is TDD or not (the threshold is not clear, but you can see a "trend")
- You can use the Assignment Controllability(AC) metric in legacy code to see where it would be easier/harder to insert the tests first. Start testing where AC is close or equal to Zero if you want to tackle the more problematic areas of the code first, and start where AC is close to one if you want to add tests quickly.

XP 2006 Conference, Day 3 highlights

On the third and last day of the conference Kent Beck had the Keynote and he used it to describe what he called "responsible development".

He defined the current state of Agile/XP as:
1- people believe that Agile/XP works;
2- people understand and want to profit from the benefits it offers;
3- people undestand Agile/XP as being practice focused, not value focused.

He went on to explain that Agil/XP should be value focused and called that new approach "Responsible Development", based on the following pillars:
1- accountability;
2- responsibility;
3- integrity;
4- avoiding blame.

Kent described what each of these values means for him and why he thinks people in the software industry should follow these values. He compared each of the concepts with past experiences and tried to justify the business sense that they make for software companies.

One interesting proposal from him was that software companies start reporting on the number of bugs/problems they have open for each of the products they sell. This, he stated, would contribute to transparency in the business through a wide audience form of accountability, as in accountability to all users of our software and shareholders of our companies.

Very interesting outline, I have also given these ideas some thought and went to the point of in a recent project ask the team to report the number of open bugs to all the stakeholders during the iteration reviews. This, in my opinion, brings transparency to the software development process and makes people think hard about the quality they are providing along with the functionality.

All in all a very interesting talk. We as an industry need to pay attention to these concepts, they make very much business sense!