KLOC - What does it mean to Software Testing

Introduction to KLOC

Lines of Code (LOC) is one of the software metric that is used by most of the people for Software Measurement. Thousand lines of code is treated as KLOC. This metric helps us in knowing the size and complexity of the Software Application.

What does it mean to Software Testing

We do test applications with the intention to see if the promised functionality works fine or not. Any deviation here will be considered as a Bug. So each of these bugs must be originated from some line of code in the product.

So it’s understood that when the size of the code is more there is a chance for more number of bugs in the prodcut. Even most of the process to talk about some % of issues is fine or acceptable quality per KLOC(even though there is lot of subjectivity).

The Defect Density is arrived at Number of Bugs / KLOC per the product under test. The defect density is one of the metric used to measure the quality of the product. Most of the Quality Process does talk about this metric.

Concerns

The concern in this approach is that how these values are measured. The general bias with KLOC is that people try to see that only the excutable lines of code in the product.

Each every line in the product may not be code at all & we may not execute each and every one of them. So it’s not taken care, then the issues related to documentation, images, installation etc might be misleading.

Since we are looking at KLOC as the size of the product it’s better to include each and every entity that effects the same. Then it’s helpful for both development and test teams.

Use Cyclomatic Complexity to determine the Risk and Test Scenarios

Cyclomatic Complexity (CC) is a software metric (mostly code based metric) used to the number of independent path executions in the application. Introduced by Thomas McCabe in 1976, it measures the number of linearly-independent paths through a program module.

It helps the developers to determine the independent path executions and base line unit tests that they need to validate. Using this, the developers can assure that all the paths have been tested atleast once. It’s a great comfort for the developer and their respective managers.

It’s better to write JUnit Tests for all these linearly-independent paths and integrate it with any code coverage tool. These reports help to focus more on the un covered paths and improve the code coverage.

It also helps to evaluate the risk associated with the application. The following are the results published by SEI and they are being followed widely to determine the health of the code base.

Cyclomatic Complexity Risk Evaluation
1-10 A simple program, without much risk
11-20 More complex, moderate risk
21-50 Complex, high risk program
Greater than 50 Un testable program (very high risk)

Explore more at Cyclomatic Complexity in Software Technology Roadmap from SEI.

Use metrics to evaluate the risk early in the cycle & improve your test coverage

YouTube India is Launched

Source : TechLads

With News Corp's MySpace having launched in India, can Google-owned YouTube be far behind? The popular video-sharing Web site today launched its local Indian version at www.youtube.co.in.

YouTube India is different in that it features a localized home page plus search functions, allowing users create and share videos, discover the most popular/relevant videos in India, and generally connect with other Indian and global users. Over time, YouTube India is expected to have an entirely 'local' flavor and feature content and functionality that is most desired by Indian users.

YouTube India sports local features like promoted videos, featured videos, home page promotions, localized user interface and help center, user support and community features (video ratings, sharing, and content flagging), and intends making easier for the Indian YouTube community to search and view videos from India. In addition, content uploaded by users in India would show up as 'top favourites' and 'recommended content' on the local YouTube Web site. YouTube India also aims at facilitating exchange amongst the large Indian NRI community.

Meanwhile, YouTube India has already inked partnerships with the likes of UTV, NDTV, India TV, Zoom TV, Rajshri Films, Eros Entertainment, IIFA, the Ministry of Tourism, IIT Delhi, and KrishCricket to name a few, with the objective of bringing exclusive Indian content to users in newer ways. Enjoy guys

Opera browser now has its own alternative to Firefox' Firebug

Source : betanews

Recently, Opera's developers have been touting it as the most compliant browser with Web standards. Now they're using that as leverage to help introduce Dragonfly, a tool they hope will promote Opera as a kind of standards watchdog.

Easily among the most useful and well thought-out extensions to Mozilla Firefox has been Firebug, an add-on by independent developer Joe Hewitt which instantly converts any active Web site into a fully-fledged JavaScript/XHTML/CSS/DOM diagnosis studio. You can see why an element is parsed and laid out the way it is by pointing at it, and letting Firebug take you to the code in question. Up until now, no tool with similar functionality and reliability has existed within the browser context; Microsoft's Web development tools are centered around Visual Studio and Expression.
Today, it's Opera which is changing that picture, with the introduction of its own live development environment add-on called Dragonfly, whose alpha version was released yesterday. It's produced by the Opera team itself, is recommended for recent builds of version 9.5, and is being billed not only as a Web development environment but also as a standards conformance tool.

With Opera reportedly scoring very high or perfect in recent Web Standards Project conformance tests, its creators may see Dragonfly as a way to build Opera into more than just a browser, but a development tool in itself.

According to a blog post from Opera developer relationship manager Chris Mills yesterday, Dragonfly makes use of monitoring tools that are now actually built into Opera itself, called Scope. Like Firebug, Dragonfly includes a JavaScript debugger and CSS and DOM inspectors in a separate Developer Tools window, which also includes a command line that lets you interact with a JavaScript parser in immediate mode.

The script window permits line-by-line execution stepping through individual instructions or between procedures, with variable breakpoints. Once execution of JavaScript has reached a breakpoint and pauses, the window can give you a full report of the state of the page's DOM and the contents of any active JavaScript variables.

Perhaps most importantly, Dragonfly is capable of providing a very detailed error and warning report within its Console tab.

The early reviews are mixed, with some impressed by how quickly Dragonfly responds, others depressed with how slowly it can load. The developer of the Python Web site framework Django, Simon Willson, expressed on his personal blog a viewpoint that's currently somewhere in-between: "Out in alpha and it shows (slow to load and the interactive console leaves a lot to be desired)," Willson wrote, "but still looks incredibly promising, especially the remote debugging tools for working with Opera on phones and games consoles."

Dragonfly's Web site is also promising that, in order to keep it adherent to the latest standards at all times, its final release versions will be self-updating, without user intervention. It will be distributed freely under the BSD license.

JavaNCSS - A source metric suite for Java

What is JavaNCSS ?

JavaNCSS is a command line utility which measures some standard source code metrics for the Java programming language. The metrics are collected globally, for each class and/or for each function.

What are the advantages ?

The following are some of the advantages that i have seen

1. Support for Ant Tasks, so easy to integrate with build process
2. Reports can be in Text, XML, HTML etc
3. Support for Stylsheets and easy to get nice HTML reports
4. Metrics at each level Package / Class / Method
5. Cyclomatic Complexity Number
6. List the number of packages / classes / functions / LOC counter at each level

Further Reading:

1. JavaNCSS Home
2. LOC Counters for C++ / Java on Joel Software Discussion group

Peeling the Performance Onion

The Infernal Onion
When we run a performance testing scenario, we usually start with a light load and measure response times as the load increases. You would expect that the response time would increase as the load increases, but you might not anticipate the dreaded "knee" in the performance curve. Figure 1 shows the hockey stick shape of the typical performance curve.

Figure 1: The classic hockey stick

The knee is caused by non-linear effects related to resource exhaustion. For example, if you exhaust all physical memory, the operating system will start swapping memory to disk, which is much slower than physical memory. Sometimes a subsystem like a Java interpreter or application server might not be configured to use all available memory, so memory limitations can bite you even if you have plenty of free memory. If your CPU horsepower gets oversubscribed, threads will start to thrash as the operating system switches between them to give each a fair share of timeslices. If you have too many threads trying to access a disk, the disk cache may no longer give you the performance boost that it usually does. And if your network traffic approaches the maximum possible bandwidth, collisions may impact how effectively you can use that bandwidth.

Figure 2

When we tune the performance of the system, we try to move that knee to the right so we can handle increasing load as long as possible before the response time shoots off the scale. This tuning often happens near the scheduled end of a project when most of the system is functional enough to allow for system-level performance testing. When you improve the performance of the system, what should you anticipate to happen next? Sometimes you're still limited by the same kind of bottleneck, though the knee has moved and overall performance is better. Often, though, you'll uncover a new bottleneck that is now the limiting factor in your performance (shown in figure 2). It may be that you're now exhausting a different resource, or that a different part of the system is exhausting the same resource as before. Figure 2 shows a second bottleneck that was masked by the first one.

This is an application of "Rudy's Rutabaga Rule" from Jerry Weinberg's The Secrets of Consulting. The rule is "Once you eliminate your number one problem, number two gets a promotion." Maybe if you get enough bottlenecks out of the way, you can achieve your performance goals for your system. But don't get frustrated if each change to the system only improves performance by a small amount. Figure 3 illustrates why. (See below.)

If your system doesn't hog one resource significantly more than any other resource, then your bottlenecks will be stacked closely together. Removing each layer will only make a small improvement; you'll most likely slam into another bottleneck waiting nearby.


It Won't Go Fast if It Doesn't Go at All
Testing the system's performance tells us how fast each user can complete a task, and how many users it can support. A related concept is reliability, where we look at how long the system can operate before encountering a failure. You might want to devise a reliability test that doesn't step up the load the way a performance test often does. Not all projects do reliability testing, though, so you might be conducting performance testing before the system's reliability is solid. In that case, you'll usually find latent reliability issues in the system during performance testing. So, when you run a performance test, there's a chance that you'll encounter a failure that renders the rest of your test run invalid.

There is a random nature to most reliability issues: You will probably have more than one reliability issue in the same build of your software that can bite you during a performance test. Whether you encounter one of the latent reliability bugs and which one you see first depends on a roll of the dice. Also, be prepared for bug fixes to unmask hidden reliability bugs the same way performance bottlenecks can hide behind each other.

Another related system attribute that comes into play is robustness--the ability of the system to gracefully handle unexpected input. You can test the robustness of your software with a stress test, which may involve simply ramping up the load in a performance test until you encounter a failure. Robustness issues tend to be easier to consistently reproduce than reliability issues. If you keep hitting the same failure at the same point in your test, you probably have a robustness issue where the system doesn't respond with a reasonable error message when some system resource is completely exhausted. For example, if both your physical memory and swap space are exhausted, a request to allocate more memory will fail, and often the end user doesn't get a useful error explaining that the server is too busy to complete a requested task. Even if the system does fail gracefully, your performance test needs to watch for errors, because it's important to know if any of the simulated users didn't actually get the results they asked for.

Figure 3

Handling the Onion Without Tears
Here are a few tips to improve upon the typically slow progress of peeling off one bottleneck at a time.

• Consider designing and running different types of performance-test scenarios that may each identify different bottlenecks. This changes the onion-peeling process so it's somewhat more parallelized. This presumes that the system is stable enough to survive these different tests and give meaningful, correlatable results.
• Make sure your performance tests catch errors that indicate if the system isn't doing the work the test expects it to be doing (reliability or simple functional bugs). This takes a lot of extra work in most load-test tools, but it's important because failures can render your performance measurements invalid.
• Perform stress testing early in your project to identify where the hard limits are. Run the tests all the way up to system failure--i.e., run "tip-over tests."
• Balance reliability testing with performance testing. The less reliable the system, the more unpredictable your performance testing will be. If it crashes at any level of load, no matter how slight, your performance results are not meaningful. You are still on the outer skin of the onion.

The best approach is to do performance modeling hand-in-hand with performance tests that validate the model. Performance models identify bottlenecks before you even start coding. Do unit- and subsystem-level testing early in the project that covers resource allocation, performance, and reliability. Most performance issues should already be resolved before you start peeling the onion at the system level. Try using simple models like spreadsheets during design and then more sophisticated dynamic models, perhaps with the help of a commercial tool, (e.g., Hyperformix) to simulate and predict behaviors;. i.e., design for performance and reliability from the beginning to grow a smaller, fewer-layered onion. Make sure that resource allocation, performance, and reliability testing are part of unit and integration testing.

A Tester’s Tips for Dealing with Developers

When I started my career as a software tester, I was made aware of an ongoing antagonism between developers and testers. And it took me no time or effort to be convinced that this is all too common. I received the kind of unwelcome response from developers that I think all testers experience at some point during their careers.

From indifferent shrugs to downright hostility (sometimes cloaked as sympathetic smiles), a tester has to endure a lot from developers. It can be hard to keep a positive attitude. But it’s up to us to keep our priorities straight, and push toward a quality project.

I picked up a beautiful line from Cem Kaner’s Testing Computer Software: “The best tester is not the one who finds the most bugs or who embarrasses the most developers. The best tester is the one who gets the most bugs fixed.”

So how can we do that?

Be Cordial and Patient
As a tester you may find it more difficult to convince a developer about a defect you’ve found. Often, if a tester exposes one bug, the programmer will be ready with ten justifications. It’s sometimes difficult for developers to accept the fact that their code is defective—and someone else has detected it.

Developers need support from the testing team, who can assure them that finding new bugs is desirable, healthy, and important in making the product the best it can be. A humanistic approach will always help the tester know the programmer better. Believe me, in no time the same person could be sitting with you and laughing at mistakes that introduced bugs. Cordiality typically helps in getting the developer to say “yes” to your bug report. An important first step!

Be Diplomatic
Try presenting your findings tactfully, and explaining the bug without blame. “I am sure this is a minor bug that you could handle in no time. This is an excellent program so far.” Developers will jump and welcome it.

Take a psychological approach. Praise the developer’s job from time to time. The reason why most developers dislike our bug reports is very simple: They see us as tearing down their hard work. Some testers communicate with developers only when there is a problem. For most developers, the software is their own baby, and you are just an interfering outsider. I tell my developers that because of them I exist in the company and because of me their jobs are saved. It’s a symbiotic and profitable relationship between a tester and a developer.

Don’t Embarrass
Nobody likes mistakes to be pointed out. That’s human nature. Try explaining the big-picture need for fixing that particular bug rather than just firing bulky bug reports at developers. A deluge of defects not only irritates the developer, it makes your hard work useless for them.

Just as one can’t test a program completely, developers can’t design programs without mistakes, and they need to understand this before anything else. Errors are expected; they’re a natural part of the process.

You Win Some, You Lose Some
I know of testers who make their bug reports as rigid as possible. They won’t even listen to the developer’s explanations for not being able to fix a bug or implement a feature. Try making relaxed rules for yourself. Sit with the developer and analyze the priority and severity of a bug together. If the developer has a valid and sensible explanation behind her reluctance to change something, try to understand her. Just be sure to know where to draw the line in protecting the ultimate quality of your product.

Be Cautious
Diplomacy and flexibility do not replace the need to be cautious. Developers often find an excuse to say that they refused to fix a bug because they did not realize (or you did not tell them) how serious the problem was. Design your bug reports and test documents in a way that clearly lays out the risks and seriousness of issues. What’s even better is to conduct a meeting and explain the issues to them.

A smart tester is one who keeps a balance between listening and implementing. If a developer can’t convince you a bug shouldn’t be fixed, it’s your duty to convince him to fix it.