The law of immediate action on productivity
In Software as in other activities quantity is not a synonym with quality. Here's what I mean:
It is very common for people to think that if they find a lot of bugs in a software before it is released that they know more about the quality level of their software. Normally that requires that you have a issue tracking system and that you collect the myriad of bugs you find to be able to produce regular reports that "assure" you that you are getting better quality (or at least "assure" you that you know more about the quality level). If you transform this into a syllogism it would go something like this:
Syllogism 1 - The data vs information missconception
1- I find lots of bugs
2- I collect lots of information about the many bugs I find
3- Therefore I know what is my quality level
Seems reasonable enough, right? Wrong! The more bugs you find the less you actually know about the quality of the software, why? This is simple to answer, bugs are "data points", i.e. data about the problems in your software. As any scientist will tell you "data" is useless unless you can synthesize it (co-relate it in order to derive hypothesis about the quality of the software that you can later verify).
By collecting lots of bugs (i.e. putting the emphasis on "finding" bugs) you will be collecting lots of data, but at the same time you will be taking the focus away from understanding what that "mass" of data really means (i.e. syntethizing possible hypothesis about the _real_ quality level of you software).
The solution? Well, very simple, instead of focusing on collecting bugs what you should be doing is making your testers work with your developers so that both develop a deep understanding of the quality of your software by doing root cause analysis for every bug found and fixing it before proceeding to the next task. Now, this last part (fixing it asap) is very important, here's another syllogism that is commonly used in the software industry:
Syllogism 2 - The bug "data" implication on productivity and quality
1- I find and fix very many of bugs
2- Finding bugs and fixing them improves the quality of the software
3- Therefore by finding and fixing many bugs I'm improving the quality of my software.
It is a bit harder to explain why this syllogism is wrong and the explanation is also counter-intuitive, however it is important. Read on.
When you find lots of bugs you have more data, but certainly not more information (as explained above). What this means in practice is that every time you fix a bug based on limited data (no root cause analysis, just focusing on fixing that one bug) you create another problem somewhere.
The point is this: every piece of buggy code is the basis for another piece of buggy code. By asking your testers to concentrate on finding bugs and not on helping your coders understand the root causes of the existing ones you are effectively asking them to leave the "buggy code" laying around for a while (while = the time it takes the developer to develop another "buggy code" and come across to the entry in the issue tracking system) and build another piece of "buggy code" on top of the previous "buggy code". Now, you see that for every "bug" you leave laying around (for a while) there will be a lot more created. The first ones will be created by building new functionality on the previous "buggy code", and the second ones will be created by actually fixing the initial "buggy code", which in turn will break the good code you had working.
In short what I mean is that every bug is compounded by pre-existing bugs, the more time you leave bugs laying around the more problems you will have in the future.
The counter argument I've received about this explanation before is: "If we focus on fixing bugs first/asap we will be losing a lot of time handling bugs and will never be able to proceed with actual new functionality".
I understand how it may seem so, but that is not how it happens in reality. In reality every bug you avoid from getting into the next feature saves you a lot more time than it takes to fix due to the "compounding bugs effect" of existing bugs on new functionality.
In practice fixing bugs immediately when they are found will be much more positive for productivity than leaving the bug laying around for while you develop new functionality.
This leads me to my own Syllogism: The law of immediate action on productivity
1- Fixing bugs immediately avoids the "compounding bugs effect" of old bugs on new functionality
2- The compounding bugs effect of a bug has a much bigger negative impact on productivity than the time used to fix that bug immediately
3- Therefore if you fix bugs immediately as they are found your productivity will improve
The implications of the syllogism are the following corollaries:
Corollary 1- Whenever you find a bug fix it immediately and your productivity will rise.
Corollary 2- You should not use issue tracking system for bugs in which the tester and developer can work together directly without needing a "hand-over" (the issue description in the issue tracking system), that hand-over will hamper your productivity and should be used only when needed
Corollary 3- If your testers are also developers and your developers are also testers your productivity will be at the highest possible level because anyone that finds a bug can analyze it and fix it, and the ones coding new features only get interrupted if the bug-fix code interferes with their code.
All that I wrote is based on knowledge acquired from looking at statistics of issue tracking systems, observing running projects and people's behavior and reaction to different methods/tools. However I have never conducted an experiment to prove my "Law of immediate action on productivity" or any of it's corollaries. Do you have an experiment or a possible environment where we could prove/disprove this law? If so please contact me (through the comments below) and I would be glad to cooperate with you in testing these hypothesis!
at
09:28
It is very common for people to think that if they find a lot of bugs in a software before it is released that they know more about the quality level of their software. Normally that requires that you have a issue tracking system and that you collect the myriad of bugs you find to be able to produce regular reports that "assure" you that you are getting better quality (or at least "assure" you that you know more about the quality level). If you transform this into a syllogism it would go something like this:
Syllogism 1 - The data vs information missconception
1- I find lots of bugs
2- I collect lots of information about the many bugs I find
3- Therefore I know what is my quality level
Seems reasonable enough, right? Wrong! The more bugs you find the less you actually know about the quality of the software, why? This is simple to answer, bugs are "data points", i.e. data about the problems in your software. As any scientist will tell you "data" is useless unless you can synthesize it (co-relate it in order to derive hypothesis about the quality of the software that you can later verify).
By collecting lots of bugs (i.e. putting the emphasis on "finding" bugs) you will be collecting lots of data, but at the same time you will be taking the focus away from understanding what that "mass" of data really means (i.e. syntethizing possible hypothesis about the _real_ quality level of you software).
The solution? Well, very simple, instead of focusing on collecting bugs what you should be doing is making your testers work with your developers so that both develop a deep understanding of the quality of your software by doing root cause analysis for every bug found and fixing it before proceeding to the next task. Now, this last part (fixing it asap) is very important, here's another syllogism that is commonly used in the software industry:
Syllogism 2 - The bug "data" implication on productivity and quality
1- I find and fix very many of bugs
2- Finding bugs and fixing them improves the quality of the software
3- Therefore by finding and fixing many bugs I'm improving the quality of my software.
It is a bit harder to explain why this syllogism is wrong and the explanation is also counter-intuitive, however it is important. Read on.
When you find lots of bugs you have more data, but certainly not more information (as explained above). What this means in practice is that every time you fix a bug based on limited data (no root cause analysis, just focusing on fixing that one bug) you create another problem somewhere.
The point is this: every piece of buggy code is the basis for another piece of buggy code. By asking your testers to concentrate on finding bugs and not on helping your coders understand the root causes of the existing ones you are effectively asking them to leave the "buggy code" laying around for a while (while = the time it takes the developer to develop another "buggy code" and come across to the entry in the issue tracking system) and build another piece of "buggy code" on top of the previous "buggy code". Now, you see that for every "bug" you leave laying around (for a while) there will be a lot more created. The first ones will be created by building new functionality on the previous "buggy code", and the second ones will be created by actually fixing the initial "buggy code", which in turn will break the good code you had working.
In short what I mean is that every bug is compounded by pre-existing bugs, the more time you leave bugs laying around the more problems you will have in the future.
The productivity angle
The counter argument I've received about this explanation before is: "If we focus on fixing bugs first/asap we will be losing a lot of time handling bugs and will never be able to proceed with actual new functionality".
I understand how it may seem so, but that is not how it happens in reality. In reality every bug you avoid from getting into the next feature saves you a lot more time than it takes to fix due to the "compounding bugs effect" of existing bugs on new functionality.
In practice fixing bugs immediately when they are found will be much more positive for productivity than leaving the bug laying around for while you develop new functionality.
This leads me to my own Syllogism: The law of immediate action on productivity
1- Fixing bugs immediately avoids the "compounding bugs effect" of old bugs on new functionality
2- The compounding bugs effect of a bug has a much bigger negative impact on productivity than the time used to fix that bug immediately
3- Therefore if you fix bugs immediately as they are found your productivity will improve
The implications of the syllogism are the following corollaries:
Corollary 1- Whenever you find a bug fix it immediately and your productivity will rise.
Corollary 2- You should not use issue tracking system for bugs in which the tester and developer can work together directly without needing a "hand-over" (the issue description in the issue tracking system), that hand-over will hamper your productivity and should be used only when needed
Corollary 3- If your testers are also developers and your developers are also testers your productivity will be at the highest possible level because anyone that finds a bug can analyze it and fix it, and the ones coding new features only get interrupted if the bug-fix code interferes with their code.
The Experiment
All that I wrote is based on knowledge acquired from looking at statistics of issue tracking systems, observing running projects and people's behavior and reaction to different methods/tools. However I have never conducted an experiment to prove my "Law of immediate action on productivity" or any of it's corollaries. Do you have an experiment or a possible environment where we could prove/disprove this law? If so please contact me (through the comments below) and I would be glad to cooperate with you in testing these hypothesis!
RSS link
5 Comments:
Pretty nice thinking. However, as a researcher, I'm obligated to argue about the set of three corollaries.
1) How much does it take to fix and retest a bug, on average? If you find a bug, it may take pretty long time to find the origin of the bug. In other words, "immediatelly" is not always possible.
2) Unfortenately, some testers need to log bugs to the database just because the big brother is measuring their productivity based on the number of logged bugs.
3) This issue is often argued. If developers are also testers do they find the nasty bugs, too. I mean, the effect of being blind...
Finally, the causal analysis rocks. However, is it enough to do the analysis between a tester and developer? What about business?
By
Anonymous, at March 21, 2007 10:51 AM
Here's my comments:
1- My experience and observations lead me to believe that "on average" the time it takes to investigate and fix a bug is several times shorter than what it would take to investigate that same bug when it manifests itself later on when more code is built based on that "buggy" behavior.
2- This is true, and we _must_ get away from it, as in my experience this actually "kills" productivity by focusing on "many bugs" instead of "fixed bugs"
3- The blindeness issue is indeed an issue that needs to be tackled, however the benefits of that arrangement could be so great as to justify investing in fixing the "blindness" issue"(as demonstrated by the jidoka method/practice in "lean" or TPS).
By
Unknown, at March 21, 2007 12:04 PM
The key here is that bugs that are found late are really not single bugs, but they are systems of bugs (bugs, on top of bugs, on top of bugs, on top of bugs, ... You get the picture). Systems of bugs are incredibly hard to fix, once you fix one you uncover 3-4 more because you have "working" code that is built on the bugs you just fixed...
By
Unknown, at June 29, 2007 12:27 PM
I think it's a beautiful thought, that every single bug could and even should be fixed immediately.
However, that is unfortunately not always the case. Someone has to pay for the effort of fixing bugs. It's all about priorization. And without proper issue tracking tool, it will be very difficult to set priorities between "must fix" and "better fixed than not fixed" bugs.
After all, there will always be bugs in every single piece of software.
By
Anonymous, at July 10, 2007 1:01 PM
@Anonymous who, then, pays for the effort of NOT fixing the bugs and living with the consequences?
It seems to me that you are forgetting one side of the equation completely...
By
Unknown, at January 28, 2009 9:47 AM
Post a Comment
<< Home