This blog has moved. Go to for the latest posts.

Friday, November 24, 2006

Egineering truths, how to make better engineering decisions

Often in an engineering problem software developers tend to try and find the perfect solution and quickly fall in love with one idea that may work or then shoot down ideas that could work but not in all hypothetical circumstances (even if they are realistically not possible).

Many discussions I’ve attended could have been cut short by just agreeing that we cannot come up with a perfect solution in a meeting and then proceeding to brainstorm -- during a limited time (say 1 day) -- 2 or 3 ideas that could possibly work and would be further analysed for validation.

Instead of following the recipe outlined above, many times engineers will debate for long hours what the perfect solution could be, only to see each of the ideas shot down due to an hypothetical case in which they may not work. This discussions create a large ammount of lost time (waste) in the software development process. Let’s take a simple example.

Imagine that you are in a team of 3 people that are faced with a technical/engineering problem. You set-up a meeting in the hope of finding the solution quickly, you call all stakeholders in and start the meeting:

The normal way a meeting would go would be:

  1. 1. Short introduction to the problem and the considered restrictions so far (5 minutes)

  2. 2. Presentation of one or more proposals for a solution (15 minutes)

  3. 3. Discussion about which solution to take (40 minutes or the remaining of the one-hour meeting)

Now, it may not be obvious but when you reached point 3 the meeting should have taken a different direction.
When you have two or more solutions to evaluate you cannot go on with the discussion before you clearly define evaluation criteria. If you had only one solution you probably should first brainstorm a while to come up with at least 1 more idea, but then you should follow that with a discussion about the evaluation criteria, not the solution to implement.
There’s a number of reasons for this, but at least two of these are crucial for a good quality decision: a) discussing the evaluation criteria will give all of the meeting participants a better understanding of the tradeoffs involved in the decision making process and b) you will also most likely find other factors to consider that you have not considered in the first proposed solutions.

So, the point 3 should be changed to:

  • 3. Discuss evaluation criteria to follow when making a decision (30 minutes)

What about point 4? Well, there’s none! At least not within the meeting. Once you have discussed the environment of the decision with the key stakeholders and you have agreed on the evaluation criteria your next goal is to make the best decision possible, you cannot do that without good quality alternatives for your decision. Therefore the next step should be to generate those alternatives. Take (at least) 2 groups of people (groups of 2 with 1 from each stakeholder group for example) and brainstorm a limited set of possible solutions (I would say that you need to have at least 3, but that’s my rule-of-thumb), evaluate them according to the criteria decided in the meeting and propose them to the stakeholders. Then call another meeting. This process should not take more than 0.5-1 days for medium-sized problems. The longer you take the more time will be “wasted” by not implementing a solution

In the second meeting you should have the following agenda:

  1. 1. Prior to the meeting each participant must learn about all the proposals put forward and be prepared to discuss them – not being prepared will produce a bad quality decision.

  2. 2. Shortly review the problem (to refresh people’s memories: 2 minutes)

  3. 3. Present each proposal’s tradeoffs (the proposals were presented prior to the meeting via a proposal document) and answer possible questions (20 minutes)

  4. 4. Possible new proposals arising from the discussion. For example: merging two complementary proposals (10 minutes – this is possible because the people in the meeting are aware of the details of all proposals)

  5. 5. Final decision on which proposal to take forward (5 minutes).

The second method is a possible implementation of
Set-based engineering and even though it may not seem so, this approach often takes less time to converge to a final solution and additionally it yields better quality results that lead to less rework which means less waste.

Many people in the software world have forgotten some basic truths about engineering. In this method I’ve tried to explain how to use 2 of those truths for your benefit:

  1. 1. Failing or making mistakes helps you learn: make several proposals and learn from the shortcomings of the ones you did not select;

  2. 2. Critical engineering decisions should be made following a disciplined process to ensure that the needed factors are taken into consideration: define evaluation criteria and follow that when evaluating an idea.

What is important is not to find the right solution, but to generate several solutions in a limited time-frame and pick the best from those. If you do that you are likely to fine-tune and improve one of the original proposals and come up with a much better solution in the end.

at 16:33 | 0 comments
RSS link

Bookmark and Share

Monday, November 20, 2006

Big Design Upfront = Design-in-progress = Less Productivity = Lost Money

Henrik Mårtenson has an
excellent post on Design-in-progress (similar to work-in-progress in Lean Manufacturint).

Really worth a read!

at 20:51 | 0 comments
RSS link

Bookmark and Share

Monday, November 06, 2006

You cannot inspect quality into a document!

Inspections are a failed analogy.SEI writes in the page about software inspection:
Software inspections provide value in improving reliability, availability, and maintainability.

They don't mention however that inspections improve reliability, availability and maintainability of the inspected artifact. In most software companies the inspected artifacts are: documents. This means that instead of improving the software itself (what the customer pays for), many companies spend large amounts of money inspecting documents that add no value to the customer.

It could be argued that inspection of documents reduces errors in software, however history proves that that is not so. The Chaos report talks about all kinds of projects, and they include projects that have inspections!

You cannot inspect quality in, you have to build quality in, and inspections do not help you with that.

You cannot inspect quality in

Documents are written by people. Even if people follow a process thoroughly when writing documents, the actual writing process is a creative process. Creative processes are not good candidates to be in "statistical control". Only processes that are in statistical
can be improved by inspection and continuous improvement.

What does this mean? You must be asking. Well, it means that since writing a document is always a unique process (not repeatable) trying to inspect problems out of them will not yield any lasting results.

Consider this: even if you were to remove all defects from one document, the next document from the same person would still have defects. What did you gain from the inspection process? Nothing. You may have removed some defects (there's not even a way to tell if you removed the most important ones!) but you did nothing to improve the way you work! You are back to square 1 every time you write a document.

You cannot inspect code or ideas in a document the same way you inspect the size of some car part fits the appropriate measures. This is basic process quality.

Documents are not parts in a car

Documents are not written to specifications like parts in a car are built. There's no specification for documents (maybe a templates for documents, but not specifications). Documents are a result of creative effort and not considering this is a primary failure of software inspections. Software inspections try to find mistakes after the fact when what we should be doing is preventing mistakes from happening in the first place.

Don't despair yet, there's hope.

You can improve your software development process and save time at the same time. Instead of trying to find errors after writing the document you should build your process so that errors don't get in to the document in the first place.

A friend of mine did just that when he switched the review process on it's head and instead of planning the review after the document is written he started planning the review of the document before the document was written, this allowed for the stake holders of that document to talk about it's content and finally settle on a common understanding of the problem before the document was even started. That common understanding was then crystallized into the actual document. In this process the document is only storage for a common memory, it can be used for future reference when in doubt, but the real communication happened way before the document was written.

In this document-writing process the review process became a communication catalyst, that's the real value in a document, communication! Trying to find defects after the fact is a waste of time and effectively avoids effective communication.

If you want to institute reviews or inspections in your company consider doing them before you write the document, not after...

at 21:49 | 0 comments
RSS link

Bookmark and Share

(c) All rights reserved