Once upon a time there was a man named Andy. Andy was a normal guy like any one of us, with strengths and weaknesses. His strength was that he did a great job at work, but his weakness was that he was a procrastinator, leaving everything for the last minute.
It was the beginning of December, and Andy realized that he had accumulated all his vacation days for the year and had to use them or lose them. Luckily, he wasn’t working on anything urgent, so he immediately got approval for a two-week vacation, and booked a flight to Hawaii. He went home, woke up the next morning and started to pack because his flight was that afternoon.
As soon as he took out his suitcase, he realized that he had no clothes to pack, so he ran out to the department store to pick us some vacation-wear, sunglasses, etc. He becomes hungry on his way back home and in a rush, stops in a fruit store to pick up something to eat. He wants to buy a banana, but the storekeeper will only sell him a bunch, not a single banana. They get into a small argument but in the end Andy gives in and buys whole bunch because he’s hungry and in a rush.
When Andy gets home he drops the bananas on the windowsill, throws his department store bags into the open suitcase, zips it up and runs out of the house to the airport.
Two weeks go by and Andy returns home. As soon as he opens the door, he is greeted by a strange smell. He walks around the house, and he sees something black on the windowsill that’s oozing and dripping, soggy and mushy. There are gnats and flies buzzing around it, and it smells fermented. He gets close, takes a good whiff, sticks his finger in it and tastes it, and declares “these must be rotten bananas” as he passes out.
Why did I tell you this story? To describe what software testing is. In its most generic form, there are three basic elements of testing.[1] They are:
Please see Comments for more........
It was the beginning of December, and Andy realized that he had accumulated all his vacation days for the year and had to use them or lose them. Luckily, he wasn’t working on anything urgent, so he immediately got approval for a two-week vacation, and booked a flight to Hawaii. He went home, woke up the next morning and started to pack because his flight was that afternoon.
As soon as he took out his suitcase, he realized that he had no clothes to pack, so he ran out to the department store to pick us some vacation-wear, sunglasses, etc. He becomes hungry on his way back home and in a rush, stops in a fruit store to pick up something to eat. He wants to buy a banana, but the storekeeper will only sell him a bunch, not a single banana. They get into a small argument but in the end Andy gives in and buys whole bunch because he’s hungry and in a rush.
When Andy gets home he drops the bananas on the windowsill, throws his department store bags into the open suitcase, zips it up and runs out of the house to the airport.
Two weeks go by and Andy returns home. As soon as he opens the door, he is greeted by a strange smell. He walks around the house, and he sees something black on the windowsill that’s oozing and dripping, soggy and mushy. There are gnats and flies buzzing around it, and it smells fermented. He gets close, takes a good whiff, sticks his finger in it and tastes it, and declares “these must be rotten bananas” as he passes out.
Why did I tell you this story? To describe what software testing is. In its most generic form, there are three basic elements of testing.[1] They are:
Please see Comments for more........
1 comments:
Observing a reality
· Contrasting the perception of that reality with an existing model[2]
· Reporting the differences
In the story, Andy observed the bananas, and “tested” them against a mental model of what he believed bananas to be. He contrasted the perception of his observation in different ways: By color, consistency, smell, state, and taste. He believed bananas to be yellow, firm, fragrant, and solid; yet what he observed was black, mushy, putrid, and liquefied. He then reported his findings and suspected that these are rotten, defective bananas.
At the moment he is done testing them, can he honestly say that those are quality bananas? Can he assure that there is quality in those bananas after observing them and contrasting them with a model? Of course not! On the contrary, his suspicion was that they were inferior bananas, not quality ones!
I like to define the word “quality” as “a measure of excellence”. Consequently, it seems that testing has little to do with assuring a measure of excellence. In fact, testing seems to have more to do with assuring a measure of deficiency!
So, what exactly is Software Testing, what is QA, and why are they both important?
There are two accepted definitions of software testing. It’s an activity that checks software against (a) its requirements; or (b) its users’ needs. Whether you believe one or the other, or both, one goal of software testing is to detect defects using observation and comparison techniques. Like Andy in the banana story, who used his different senses as particular methods of comparison, software testers also use numerous testing types such as functionality, data integrity, black-box/white-box, end-to-end, boundary, automation, field validation, performance, compatibility testing, and the list goes on and on[3].
There are other things that testers do in addition to testing, which can be categorized as Testing’s pre-, co-, and post-requisites. Test planning, scheduling, documenting, and configuring equipment are tasks to be done before testing can begin[4]; and writing defect reports, demonstrating bugs to developers and validating fixed bugs[5] usually occur while testing. Concluding the testing process by writing a test summary report is a typical post-testing task.
So if testing, on its own, doesn’t increase quality, what does? Can quality be assured, and if so, how? Let’s look at the story. What could Andy have done to prevent those bananas from turning rotten?
Andy would have been a more organized traveler had he use a checklist to remember the details to prepare for his trip. Items on his vacation checklist might have been to set the burglar alarm or motion sensors, check the oven for gas leaks, water the plants, and check the house for perishables. Had he been working off a checklist, he would have seen the bananas, wouldn’t have left them on the window sill, and thus assure their quality[6].
Checklists are also used in engineering quality software, and they might include items such as:
· Are version control standards being used?
· Is each software component properly labeled, versioned, commented, and backed up?
· Are work-products being inspected (formally or informally) throughout the development process?
· To what extent will popular process guidelines be implemented? (CMM, ISO9000, Six Sigma, etc.)
· Are there enough resources (interest, knowledge, budget, staff, time) to effectively use testing tools?
· Is communication among departments automated?
· Is there adequate documentation, both for project and process?
· Are defects traceable to testcases and requirements?
· Is the correct life-cycle model being used for this project?
· Are team members clear on what their roles and responsibilities are?
· Is the project’s staff properly trained in the technology in which they are working?
· Is management committed to continuous improvement?
· Are critical staff members being compensated adequately so that they won’t jump ship
There are several conclusions to consider:
1. Testing for its own sake is worthless. You can test an application all you want, and you can report millions of bugs, but if no one is fixing those bugs, you aren’t providing any value by testing. Instead, you are just heaping on evidence that the software is deficient. Testing, on its own won’t affect the quality of the product[7]. However as part of a process that also includes bug fixes and bug-fix verification, testing can (and often does) improve quality; and the degree to which we make this process effective determines what assurances we can make.
2. Testing and QA are two different endeavors, but one is not better than the other, and they shouldn’t compete with one another. In fact, testing plays a crucial role within traditional QA. Theoretically, one might think that a successful QA campaign would not include testing, because since the goal of QA is to prevent defects from occurring, there would be no need to detect them. This is incorrect. Testing serves an important self-correcting function to QA process. When a tester in a “high-process” organization finds a bug in an application it proves two things: a) that there is a problem with the product and it has to be reworked, and b) there is a problem with the process and it has to be reworked.
3. The only thing you can assure after testing an application is that it has been tested.
4. What should we do about the title “QA”? It usually stands for quality assurance, but:
· Testing on its own can’t assure quality.
· Testing as part of a process usually improves quality, but doesn’t necessarily assure it.
· Version control, workproduct inspections and other typical QA techniques usually improve quality, but don’t necessarily assure it.
A few alternatives are:
o In The Impossibility of Complete Testing, Cem Kaner suggests that QA could stand for Quality Assistance.
o The acronym QA could be redefined to mean Quality Assessment [8]
o I’d like to offer a third possibility – QA doesn’t have to stand for anything. Instead, it could take on its own meaning and be defined broadly enough to encompass the ongoing debate as to what it should be, such that any activity that promotes the reliability of software might be considered a QA activity.
One thing that is certain. No matter what you call it, the activities that people undertake to increase the measure of excellence in software is far from an exact science. There is no one golden way to approach testing software. There are no proven formulas that will work each and every time.[9] There is no QA=MC2. If assuring quality were as easy as following a formula, we would have mastered it long ago.
As Software Testing continues to evolve it becomes more recognized and respected as a mature specialty within the software engineering discipline. But even with the continued development of the field, I don’t know if there will be a time where quality will be easy to assure. I doubt it. But the time will come when explaining it will not have to depend on stories about rotten bananas.
Post a Comment