Another job opportunity for 31337 h4x0rs that love testing!

F-Secure is hiring! And they have some interesting positions as well. Check out this one:

People who have experience in “SW Engineer in Test” type of a role. This is a fairly new role and not well established in Finland. The discipline is young and top notch professionals rare. An excellent niche to put yourself into in the job market.

This role is key in the setup for continuous integration. At F-Secure we test 1000's of unique system builds in each sprint. Results show on radiators in each team room immediately. Top that, if you can. For instance, nothing prevents testers to check out the product code, find the problem and alone or with the team fix it. And this does happen!

This is a great way of developing software. The feeling of community is tremendous, the drive is inspiring and the results exceptional.

Hellow World, wake up, and smell the raw unrefined source code.

Photo credit: roland @ flickr

About when I stopped worrying and embraced Engineering!

Product Managers often forget that they can make or break a project. If your team is using Scrum that is even more true, because Product Managers are (or should be) involved regularly in the planning of the work with the team. Either as Product Manager or in the role of Product Owner.

It is therefore very important to carefully craft the messages that the Product Manager gives the team. It took me a while to understand this simple message, and this is my story.

As a Product Manager I was eager to give the team direction and clarity on the goals for the product. I communicated regularly with the team with more information on the market, the product, the customers and generic feedback on the direction they were taking.

Eager to achieve the goals we had set for ourselves the team was churning new features at a very good pace. Then the bomb hit. Every time we were supposed to go to production we faced major hurdles. The Retrospectives did not point to anything that could cause the quality problems we had, so I went on an investigation. Why was deployment to production so hard to this team that was delivering features at such a good pace (uncommon for other teams at that time)?

As a Product Manager I had to forget about what I wanted and had to concentrate on finding the root-cause for the problem. I interviewed the developers but nothing I found would explain the situation. The team was testing regularly in their environments and they were practicing Scrum "by the book".

It was then that it hit me. Having a conversation about the development process with one of the developers I understood that they were neglecting their unit and integration tests, which in turn led them to have a long feedback cycle for integration (some days only, but still too long).

After I heard that, it was fairly easy to trace the deployment problems to the lack of automated, fast-cycle integration testing. In their eagerness to deliver more features the developers would be developing up to the last day and would not have time to do the integration testing for those last changes. In turn, that led to many problems when it came to deploy.

Through that conversation I realized that the problem the team had was that they were being implicitly rewarded for delivering more features every time I praised them about the new feature they delivered. However, they were not being rewarded for building the unit and integration tests that would prevent the quality problems.

The end result was that quality-sustaining practices were being neglected. Having understood that, I changed my communication with the team. From that time on I started asking them if they had the integration and unit tests for every feature they delivered and started giving them praise for delivering tested features, not just any feature.

Before this happened to me I was not aware of how strong an influence the Product Manager's message can have on team behavior. "They are the engineers" - I thought - "they should know what they need to do". It's not that simple!

Photo credit: pcalcado @ Flickr

Continuous integration requires a good SCM

Lately I've been discussing SCM (Source Control Management) systems and how that impacts the continuous integration efforts of the teams and the projects where they work.

I could not have expressed this better than Dean Wampler, check his blog post out.

The cost of un-fixed bugs or Good Code on top of Bad Code

While discussing with a colleague we started talking about the costs related to not fixing a bug immediately (yes, this does mean that you have a way to immediately figure out if you introduced a bug).

The reason for fixing bugs immediately is not only the cost of content switching later on when you have to come back, but it is even more the cumulative cost of fixing all the code you have built on top of the bug you introduced.

Think about it this way. Bug = Bad code. If you have a bug and continue development you are developing working code (good code) on top of that bug (bad code).

When you later on go back to fix the bad code, you will then have to fix all of the good code that is now broken because of the bug fix you introduced.

Software is build in stacks, the lower level affects directly the upper levels (many of them), if you have a bug lower in the stack and then fix it you will have to change all of the upper levels! And that is where the real cost is.

So, if you find a bug, fix it now. If you did not find a bug, think of ways you could find it faster -- before you build good code on top of bad code.

Testers are developers! Stop breaking them apart

First one clarification. Testing is needed, required, critical and very much a MUST. With that out of the way let's go to the post.

Many people have mentioned the need to include testers in Scrum or agile, and planning your processes with a testing step somewhere (normally close to the end).

All of these ideas are OK. There's nothing wrong with testing, the problem is that testing is done too late. Would you buy a car which engine was "fixed" after it came out of the line and didn't start? -- I certainly wouldn't not!

In software, like in cars, problems should be found early, very early. In fact problems should be found so early that they never reach the (needed) testing phase at the end (of the day preferably). This is why we need to stop thinking about testing as something separate from development or even coding. Testing must be part of every step of the process.

I don't agree with the proposal in this article. The tester should not be the one developing the test cases in isolation, and certainly not after the team has planned! The reason is that the tester will find faulty assumptions and possible design problems while thinking about the test cases. That information is crucial for the team to avoid building problems into the software.

Testers should be involved from the start. In fact, for each story that the team is planning in the sprint planning meeting, they should at the same time plan the test cases, review the initial assumptions and only after that should they actually plan the tasks to accomplish that story.

The best way to develop quality software is not to test the bugs out, it is to develop quality in -- this is why testers should be involved in every step of the process (Scrum in this example).

As Deming said:

Build quality in, don't inspect quality in.

Oh and by the way, if the test cases are (really) written why would the coder wait for the tester to verify that his code change works? -- Just run the tests immediately! It seems to be that the coder not running the tests herself is just a way to avoid feedback, which is key to Agile!

Testing to script is waste, creative testing is extremely valuable

Testing is a hard job. Imagine this, you have to make sure an application of more than 2 million lines of code is ready to ship. It all depends on you, and your team of 2 testers.

How do you do it? Well, one way to do it is to make sure you cover all the possible use cases and test for those. But that can go into the thousands. There's no hope you can test all of those cases in a short period of time (let's say 3 months or so...). Well, now we just made it even more difficult: we have to release every 4 weeks. Oh, and did we tell you that we are changing the architecture as we go? Incrementally of course, but nevertheless.

How would you cope with a situation like this? Yes, the answer is you would not (just setting up my answer... wait for it). The answer is that you must make sure you are never in the this position!

How to avoid being in a position to have to test a large piece of code with large code changes ongoing and still release every 4 weeks? Test automation. All tests that can be automated should be so, and at all levels: unit, integration, system, performance, reliability, you name it.

The point is this, testers brain power is wasted if all they can/are allowed to do is to test against a static specification with a few tests added every 4 weeks. That's not the best way to get the most out of the smart people you have in your company. If you are not a tester, just imagine yourself having to go over the same 40-50 pages of tests every single iteration, month-in, month-out. How long would it take you to quit? I suspect not too much...

Additionally, if you consider the effect of familiarity (reading the same test cases 2-3 times a month for several months) on the quality of the testing you quickly realize that manual testing against a script over and over again is the best way to get problems to escape even the most dedicated tester's eyes.

So, what next? Well, test automation is one solution. The next step is to train your testers to be expert "breakers", their goal should be to find more and more ways in which to break your software. Specifically ways you have not thought about!

The message is: testers are way too valuable and way to smart to have them spend their work-hours going over a brainless set of tests-to-spec, you will get a lot more from your test team if you automate the repetitive tasks and let the loose on your code.

This is, BTW, what Richard Feynman advocated when he reviewed the Challenger disaster in the 80's:
"(...) take an adversary attitude to the software development group, and tests and verifies the software as if it were a customer of the delivered product."