Arkisto: June 2010

A Requirement for Midsummer

24. Juneta, 2010 | Kirjoittaja: Antti Niittyviita

People have developed for themselves and for other people a huge amount of Midsummer traditions. Despite the circumstances, one must, or at the very least should, follow them. This is one:

Happy Midsummer for everyone, despite the circumstances.

How Much Does a Bug Cost?

23. Juneta, 2010 | Kirjoittaja: Antti Niittyviita

Testing is an investment. The purpose of the investment is to find bugs during a stage, where fixing them is as cheap as possible.

I spotted a link on LinkedIn’s discussion forum, which led me to an interesting article. The article goes through 10 most expensive software bugs of all time. The text is pretty rough to read, here are a few juicy ones from it:

A probe was sent to Mars. NASA had done its specs in SI-units, but the developer had used imperial units. The landing on the surface got effed up and the probe was destroyed. Cost: 125 million dollars.

CIA claimed that the USSR was stealing the guidance technology of USA’s oil lines. CIA sneaked in a tiny bug to be stolen. As a result, an oil pipe exploded in the USSR, which Wikipedia states is “the most monumental non-nuclear explosion ever witnessed from space.” No one died, but it must have taken a long while to repair the damage.

A supplier made a software to the public government sector in the UK. The new software was not compatible with the systems used by the client. Expenses so far: 1000 million pounds.

Radiation machine for medical treatment gave too much radiation. Five people died.

Rest of the bugs and better details can be found here.

It’s actually quite wonderful how those bugs and expenses condense pretty much every problem, which we have discussed here in this blog. The CIA stunt is by itself quite ingenious, but otherwise. Specifications problems, lack of buyer know-how, rush etc.

Which is why I cannot help but repeat:

Testing is an investment. The purpose of the investment is to find bugs during a stage, where fixing them is as cheap as possible.

They Fit the Specifications, so What?

16. Juneta, 2010 | Kirjoittaja: Antti Niittyviita
Comments: 1

Some time ago, we made two test cycles with an Asia-based team, just for the hell of it. The first test cycle was given a very clearly written test specs based on the product’s specifications.

The testing was conducted in India and the results were as expected. 18 defects were found among 40 test cases – As many as had been found in the design phase already (*). The next test cycle’s mission briefing was clearly different. They had as much time for the testing as they had had the previous time.

For this assignment, they were only sent the product to be tested, and a list of things that are worth checking during the testing. That is to say, no specifications, nor test cases.

Test this product to your best knowledge and report all issues in a way you find best suited

As a result, we received 25 reports. To report the defects, the test team got creative and used video capturing to show them, so everyone could see what they were and how they happened. The software developers also gave thumbs up!

Of the time spend in executing testing work, the result received was 38% higher when the testers’ hands were not tied with having to closely follow specifications or test cases. The tester put on the end user hat and reported defects to the best of their experience.

The chasm in specifications between functional and genuinely functional in this case was pretty intense. The underlying problem in this case is that requirements engineering is conducted with almost no exceptions before building the product. The specifications are an abstract entity inside the designer’s head, formed prior to tell how the end product should function. Keep this in mind when you design the testing.

Remember that a functional product according to specifications might not be a genuinely functional product. Only when the testing has been done with an open mind can you tell how it really is!

(*) During the design phase of the testing we had to familiarize ourselves with both functional requirements and investigate the product itself, in other words: To conduct investigative testing. This design phase therefore took a lot of time.

Test on Time!

8. Juneta, 2010 | Kirjoittaja: Antti Niittyviita

The sooner the bugs are discovered, the cheaper it becomes to fix them. To discover the bugs on time, the testing should be involved early on in the development cycle.

Of course, one must balance the quality and purpose of testing to its stage in the development cycle. When designing the architecture, one cannot simply check the placement and the colour of a button. Yet, one should by all means ensure, that the architecture makes it possible to place buttons later on in the cycle in a desirable way.

The projects are under continuous rush and the budget is on a tight leash. Despite that, ensure that the direction it is going is the right one.