Arkisto: November 2010

Did the Stress Test Tell the Whole Truth?

23. Novemberta, 2010 | Kirjoittaja: Antti Niittyviita

Ecofin commissioned an EU-wide stress test for the banking sector in June. The purpose of this testing was to find out how European banks could face an economic growth that was significantly smaller than expected. A total of 91 banks in the EU-region were selected to be tested. The tests encompassed a total of 65% of EU’s banking sector’s total balance.

The tests were performed quickly, and at the end of July the testers published their results. The Pass-rate was quite good. About 92%. Only seven banks in Spain, Germany and Greece failed the tests. In addition, the testers were convinced that the tests were reliable and harsh. That should make a good stress test.

Yet, there wasn’t a single Irish bank that received a fail-score in the tests held during summer. What the hell? From a tester’s point of view, this makes me think.

Despite any media attention, no one is asking what went wrong with the tests. That should be the most relevant question when evaluating whether or not the test was good. That is the most relevant question if we really want to improve the reliability of testing.

Regardless, it is important to note that people only ask that question if there is a belief that the tests failed. Which is why I feel that what failed was not the test directly, but the goals set for the test. The test result was exactly what they were set to find!

  • Gaining confidence
  • Finding defects

Which goal would you have rather placed for the stress testing of the European banks?

“Be careful what you wish for, lest it come true” – A widespread proverb

Delusions of Quality Ruin Even the Best Projects

16. Novemberta, 2010 | Kirjoittaja: Antti Niittyviita

“A splendid idea!” thought Päivi the Project Leader. It is without a doubt a sensible idea to divide the budget of development in such a way that testing can be bought as a service when it is needed. First, we have a test cycle for the first release candidate. After we get the results, we consult them and then fix the bugs, and the second cycle will be used to verify that everything went as planned. Two cycles are enough!

A surprising number of businesses that handle software projects and software products use Päivi’s style. The approach is ultra sensible as long as:

  • There are no new tests or use cases,
  • The product does not change in any way other than bugs getting fixed and
  • The bug fixes do not break other functionalities.

In projects such as this it inevitable so happens that the scheduling fails when the testing enters the picture. The original expectations just happen to be very unrealistic. An unexpected amount of bugs always get discovered during the first cycle. The deadlines become unmatchable, because fixing the bugs takes unexpected amount of time. Usually, on the first cycle we also meet the tests which get slapped with a ‘cannot run’-stamp. This is caused by the fact that some of the bugs prevent reliably testing the functionality they infest.

During the verification cycle we again discover new bugs simply because of the fact that the product has become more familiar for the testers. In addition, the probability that coders can flawlessly fix the central problems is negligibly small. With Päivi’s style one of the central strengths of testing becomes unattainable by the entire team. That strength is learning.

Don’t ruin a good project with unrealistic delusions of quality.

A Simple Question Educates

1. Novemberta, 2010 | Kirjoittaja: Antti Niittyviita

Jari Parantainen of Ediste returned me back to basics. The power of a simple question is actually quite crushing. This noticed Ted Kennedy as well, who had strived to be the President of the United States. Kennedy’s race for the office was put to a stop during the pre-election by a simple question.

Why do you want to be the President?

Simple questions have a habit of returning the respondent’s feet to the ground. A simple question demands a simple answer.

In software projects, the usual case is that people do hell of a lot of work and most often in a hurry. They rush, to attend a meeting. They rush, to compile the reports. They rush, to examine the defect control process. They rush, to make the test specs. The rushing and hurrying is constant. Why?

I claim that it is about the lack of simple questions. Everyone focuses to look diligent and productive. No one understands to stop and ponder the reasons and goals behind their actions. No one poses the questions.

Meetings. There are often a lot of people attending them, and your calendar might be full of meetings. They include a lot of discussions and a lot of note-taking. Sometimes they even make decisions in them. Most of the time, the majority of the people attending the meetings could very well be someplace else – to do something else entirely. A lot of wrong people attend the meetings. The next time you hold a meeting, ask each participant: Why are you here? Send those who don’t know how to respond back to work.

Processes. They are keenly defined at the start of the project. A lot of time is spent to document them. Meetings and even training sessions are brought about because of them. Finally, when people get working, they forget the process descriptions. The tea m finds shortcuts and skips phases. Often the right work methods are discovered only when people actually start to work. The next time you start a process planning, ask first: Why this way?

Documents. People generate a lot of documents. Writing them takes a hell of a lot of time and even more time is spent to examine them. Examiners need to be pestered for comments and finding an agreeable time for the examination meeting is pure pain. Finally, when the document is wrapped, no one reads it and it gets forgotten into the depths of version control until the expiration date is passed. So, the next time you decide to write a document…

You know the drill. You’ll find the simple question as long as you give it some thought!

Finding a simple answer to a simple question can also be easy. When this is the case, working is sensible, productive and useful. If providing an answer is like case Kennedy, it is very likely that you are about to do the wrong things.

Now you have a good opportunity to stop and think the first simple question:

Why do you test?

Quality, Reputation, and Social Media

1. Novemberta, 2010 | Kirjoittaja: Antti Niittyviita

You are about to acquire a cell phone or let us say, a new television set. You might be about to order a new CRM for your firm or about to change your laptop brand. Have you ever given thought to how your decision of what to buy is born?

Earlier, business was conducted face to face. By speaking with the expert, the seller. The seller’s knowledge had a lot of weight when it came to acquisitions. The seller was a person to trust when it came down to making a purchase. Comparing prices and products was difficult and that if anything was the dream situation for the producers. If you lucked out, the product you were considering had been reviewed in a magazine or someone you knew had already tried it. Decisions to purchase were also quite strongly influenced by advertising and marketing. That is to say, the traditional way of attracting customers.

These days, business is born from entirely different beginnings. What does the customer do?

Marketing is no longer traditional broadcasting about the product’s awesomeness on the media. Modern marketing is about ensuring that the products themselves are actually just that awesome. The products are supposed to offer good purchase- and user experiences. It is the users that they need to spread the sweet words of success.

What do you think: Can these days a successful business be born if the product is inferior? Or should they after all spend a little bit more effort towards testing?