Arkisto: March 2010

The Pitfall of Competitive Bidding

29. Marchta, 2010 | Kirjoittaja: Antti Niittyviita
Comments: 1

Niina had transferred to work in the public sector. To IT, to be more exact. The mission of IT was to primarily maintain the systems, keep them up to date, handle the training of new users, and most importantly, to handle implementation projects. The local authority was starting up a project for renovating the hour accounting software and the proper competitive bidding had already started.

The plan looked like this: Contract notice includes testing and reporting in completing the project. It would be one of the requirements for the supplier. They would do it like this, because they have done it always like this before. Typically, in the competitive bidding around local authority they use a process of elimination to bring down the bidders to a few that fit the criteria best. Finally, they choose the cheapest one of those few.

Niina was bothered by this already at the beginning of the project, since in the private sector she had faced projects with tight budgets time after time. It was in common with these projects that they had to compromise in some places. The products always need to be finished on time, so they can hardly ever compromise the coding parts. According to Niina’s personal experience, the projects that strive for a general settlement always divide resources from the testing first.

So, what is in common with projects working on a tight budget is that the testing is the first place to have budget cuts, and the one that gets cut the most.

Because of Niina’s strong demands the local authority decided to take a risk. For the first time they attempted a new approach: Niina’s idea was that they have a competitive bidding without testing and then get the testing from a third party. This solution felt like more expensive one to the authorities, though, since there were now two competitive biddings and the combined cost was probably therefore higher than with one case of bidding. Niina nevertheless demanded that they try this approach once and bury it if it turned out to be more expensive.

As a result, the competitive bidding turned out a combined cost higher than a single competitive bidding would have done. The authorities were grumbling that ‘We shouldn’t have listened to that Niina, this attempt turned out costly!’

As the project went on, there were differences to the old process model. The testing gave reports regularly and clear information of the system’s development. The bug database accumulated three times the information the projects normally collected, and the local authority had a straight access to that data. An objective perspective to the product produced more fruit than before. Yet, an advantage in expenses during the project could not be seen.

According to the contract, after the day the local authority took up the system they had 30 days to notify and report bugs. All bugs discovered and repaired after that date would go to be billed expensively by the hour. During those 30 days of grace period, the professional testers discovered a bunch of bugs and of course, bugs that had not yet been fixed even though they had been discovered during the project. All the central problems could then be fixed during the grace period, and of course, without the added expenses to the local authority!

In addition to the straight repair expenses a new advantage in expenses in the project was discovered later on, when the local authority started to renovate their payroll system. There hadn’t been a functional integration between hour accounting system and payroll system before, but in Niina’s model of competitive bidding the tester had been able to account for the system’s further development requirements. The testing had watched over the local authority’s interests by demanding that the supplied hour accounting system’s interfaces had been open for further development. This way, the local authority dodged huge expenses that would have risen from opening the closed interface. Thanks to the quality-cop.

Niina received thanks for her courage to demand changes. This attempt became a good custom in the local authority, which they had no problem to advertise to the municipal manager of the neighboring municipality.

Having everything bid competitively in one go causes unavoidably prioritization between the work. Next time you have a competitive bidding, think carefully are you willing to compromise testing for the benefit of development.

P.S. TestausOSY’s Oulu group had a mini seminar last week about outsourcing testing, which sparked this blog entry to begin with. A member or not, feel welcome to participate in future events. Follow our event schedule here.

When Do You Trust An Expert?

16. Marchta, 2010 | Kirjoittaja: Antti Niittyviita

Working in an average-sized IT-business as a project manager, Pekka is a sensible guy. The work takes a sufficient amount of Pekka’s time, but in addition to spending time with his family, Pekka does sports and watches movies.

Pekka meets some bad luck in a football exercise. A knee twists badly and Pekka faces a trip to a hospital. Pekka complains to the doctor that his knee is hurting. The doctor inspects Pekka’s knee and says that the lateral collateral ligament is torn and it needs to be operated. The leg undergoes surgery and Pekka starts a diligent rehabilitation. The knee did not become as good as new, but the leg will heal to a degree that allows playing football again in a few months.

Pekka needs to use his car pretty much. The children are taken to their hobbies and even work demands Pekka to drive often enough. Driving home from work, Pekka notices that the handling of the car is inaccurate and the steering wheel occasionally mishandled. Pekka drives straight to a servicing station near his home, realizing that something is amiss. The familiar car repairman tells him that the head of the outer tie rod is loose. It needs to be replaced, so that driving becomes safe again. After replacing the head, the toes need to be adjusted. This costs in total around 100-150 euro. Pekka rationalizes that it is a small cost to pay for the safety of his children and his own. Luckily, professionals are there to help him.

At work Pekka receives the responsibility of one of the largest projects in his company’s history. The client is a globally working company of the financial side, who is expanding strongly to European market. Pekka’s assignment is to execute a fitting client portal according to European laws in harmony with European culture. Pekka’s company has done such projects often, but never on this scale and with as high demands for data security and reliability.

Pekka realizes that the quality assurance of this project has an enormous significance for the future of his company. If successful, the reference will definitely attract more work. Pekka thinks over the questions relating to testing data security. Pekka’s bosses have lined that the test group working as a part of Pekka’s team focuses on coming up with test cases and that the actual testing is outsourced to an expert testing company. Pekka knows that his company might not have the right level of know how in data security.

Project leaders demand Pekka to give good reasons for altering the company’s plan for data security testing. Pekka tries to tell that there is not enough experience and that they cannot use this project as an exercise. This does not convince the bosses. Then Pekka comes up with a simile: “If our knee hurts, we ask a doctor to inspect it and tell us how to fix it. If our car has a problem, we ask the car mechanics to investigate it and give a price-estimate for fixing it. Why would we want to restrict experts working on a job so important for our company? Would we not get a right diagnosis of the problem if the experts would make the testing plan from the get-go?”

The project leaders did not need long to reflect on Pekka’s statement. A company specializing in data security testing was contacted later on that day.

When do you trust an expert?

It Reduces Future Expenses

3. Marchta, 2010 | Kirjoittaja: Antti Niittyviita

Hi! When was the last time you took a vaccine? I myself took one previously just before my trip to China.

Becoming vaccinated might have come across you previously during the swine flu wave of autumn or let’s say, when you took a business trip to India. Having access to vaccines in Finland is taken for granted. It is a part of the citizens’ health. Have you ever stopped to think why people here support vaccination so generously?

The real reason, when looked from the perspective of a payer, is naturally the same as when testing software. Expenses.

According to an expenses-analysis made in USA, every dollar invested in vaccination now reduces later expenses by 2-27 dollars in the health care. Have you ever stopped to think that a properly executed testing of an end product can save as much from the future expenses caused by reclamations, scheduling problems, warranty and critical patches?

When we are approached and asked to help with testing, it happens confusingly often during the final phase of the product development. The development might have been ongoing for two years, but now three months before it hits the market people want to be assured of the quality. Pretty daring, I say. According to research, discovering a bug at a later phase can, compared to the requirements specification phase, cost you:

90 times more when the bug is discovered during the system testing phase

440 times more when the bug is discovered during the acceptance testing phase, and

470-880 times more when the bug is discovered during the production phase

These multipliers sound pretty high, but they might be your average day in, for example, Toyota, who has had to call in one million cars to be serviced. Makes you think.

Testing is always an investment. It costs now, but is sure to save you money in the future.