A “How To” for Sorting Software Bugs

cover image – rozelt/istockphoto

Recently I was asked by a client to give a 100% guarantee that the next software release would contain zero bugs? I didn’t have to think about it for too long before I promptly said “I can’t guarantee that I’m afraid.”

Did I feel silly or stupid having to say that? No. Not at all. Everyday software companies release software that contains bugs. This isn’t because software companies don’t care about their software or the perception of software releases, but the nature of adding more and more lines of code, to applications of ever increasing size, inadvertently bugs may be introduced.

Let us take a look at Microsoft for example. When they release a new version of Windows, is it bug free? The outstanding bugs in Windows reportedly reach a five figure number. If Microsoft were to make a real effort to fix all of these bugs then could they ever get the bug count down to zero? The other question we need to ask is, would they want to? Shock horror, but should we fix all software bugs?

Well, when it comes to fixing software bugs we need to evaluate the software bug by the following parameters;

  • Severity
  • Cost
  • Frequency
  • Risk

Severity

Quite simply put, how severe is the bug? Is it a spelling mistake in a user prompt, or is it something a bit more serious that causes the computer monitor to explode while permanently imprinting a BSOD in the users features?

Cost

What is the cost involved in fixing this bug. Not only must we assess what it will cost us in time but we must also assess the cost implications to us if we don’t fix it. Will our software bug get us in to hot water further down the line, especially if one of our clients loses money? Clearly such bugs must be fixed as soon as you find them. Everything else must stop, because otherwise you might end up with no software to develop in the long run, and your own personal reputation may be tarnished.

Frequency

How often does the bug occur, and in what circumstances. Is it occurring infrequently, or not at all for 99% of the users and only affecting 1 user quite frequently? Should you resolve this straight away, or put this down to PC build, hardware issues etc? If it happens a lot, but does not affect the running of the application then should you fix it? Again, the other factors for assessing what to do about this bug (cost, risk, etc) must be evaluated alongside frequency.

Risk

Risk is a broad term used to evaluate a software bug. We must first assess the risk of the bug re-occurring. If it was a particular set of events linked to a particular scenario, where the risk of this happening again, or with a wider audience is very low, then should we fix it? What is the risk to the software user with this bug in the software? Could they lose lots of critical data for example. Could it end up costing us as the provider of the software?

As well as assessing the above we must also come to an agreement on whether or not it is a bug in the first place? You might think that it is obvious, but this is not always true.

For example, testers will often find genuine bugs (and thanks for all your hard work doing that), but they may also find something that doesn’t quite work as intended. This is where it can be difficult to get an agreement on whether it is a bug or not. For example they might not like the look of the report you developed, or might want to apply different filtering options. Is this a bug or by design?

Let’s take another example. You have developed a piece of software functionality for your client. Hopefully you got paid for it…? But what do you do when they scope creep and report a missing feature as a bug? Usually this feature wasn’t specified in the first place, and as such should have been picked up in the Requirements Analysis and Requirements Definition. This doesn’t stop clients reporting it as a bug. And this is when we have to take a deep breath and start counting… 1, 2, 3, 4…

So what should we do when we have a genuine bug reported, we have evaluated it against Severity, Cost, Frequency and Risk and have agreed to fix it. Given a number of years developing software, having introduced many bugs myself, then I would suggest the following as a “10 commandments for fixing software bugs”.

Thou shalt ensure thy bug has been correctly logged

Don’t do anything until you have correctly logged the bug report in your internal systems. Whether it is a fully blown tracking system or simply a spreadsheet, then it needs to be logged. It will allow others to see what bugs are being reported and also what is being done about the resolution.

Thou shalt not start looking at thy bug until thee has collated as much information as is possible about it

To reduce the costs associated with fixing the bug then there is no point in trying to fix it unless you are 100% clear on what the issue is. I have seen many developers trying to fix bugs without adequate information from the reporter of the bug. Get screenshots, steps to re-create, error messages…… Do it!

Thou shalt endeavour to be the re-creator

Now you have all the information you need then try to re-create the bug. If you can re-create it then this will go a massive way to helping you diagnose and fix the bug. Maybe you will need to have the client re-create it for you on a different environment? Perhaps you can get remote access to their system so you can test on their infrastrucure and data?

You shalt not be proud and will ask around

Do you know if anyone has seen this bug before? Save yourself a lot of time by asking around initially. It’s not a sin to even “Google It”. If you can’t find any help initially then you’ll need to start diving in. But again, if you find you are struggling then ask for some help. It is better to ask someone for half an hour of their time than to waste a few hours of your own getting nowhere.

Thou shalt befriend the debugger

Nobody ever figured out what went wrong by staring at the outside of the black box. They had to get in there and get the answers. Well the debugger is your window into what is going wrong. Be familiar with how to use it, how to set breakpoints, how to watch certain variables. In no time you’ll have tracked down that line of code that is generating the bug.

Thou shalt take a step back

Pin-pointing the line of code that generates the error is only a part of the challenge. Take a step back from this line of code and then a few more steps back to start identifying the links in the chain that may have caused the error. Don’t start guessing.

Thou shalt not dive in

I have seen many developers start diving in and hunting around like rabid “alpha males or females” searching for the source of the bug. They make changes here, there and everywhere in a desperate attempt to apply a fix. In the meantime they are inadvertently adding more bugs to the software and generating more work for their fellow developers. And this is the kind of work none of us really want to have to look at. So make a plan. Stick to it. Logically work your way through your approach. Hell, why not even print out these 10 commandments and use them as an aid!

Time to understand is the virtue of a wise developer

You might not be looking at your own code or an area of the application with which you are familiar. Take the time to understand this area of the system and it’s intention before you start looking at the bug. It will save you a lot of time in the long run and give you a better grounding in your companies applications.

Lack of sleep is thy enemy

If you are having trouble getting to the bottom of a software bug, then don’t keep looking at the same lines of code over and over again. Take a break. Ok, don’t necessarily have a sleep at your desk, but walk away from your desk, stretch your legs, go to the toilet… anything but keep looking at “the screen”. I have seen developers adopt this approach and then suddenly the light switches on in their brain, and hey presto… Figured it out! I can recall a time once when I actually dreamt about a solution to a problem in my sleep. It’s only happened once but I wish it would happen more often!

Thou shalt learn and spread thy wisdom

Once you have figured out what caused the bug and subsequently fixed it then ensure you let others know. Learn from it yourself, but also help others to learn from your experience. Ensure you update your bug tracking system or knowledge base if you have one. You’ll be thanked in the long run if you save new developers or colleagues a lot of work looking for a similar problem. Not only that, but one day they may return the favour…

Right now we live in a world where we have to fix our bugs. But what would be the holy grail, the ultimate end state for fixing bugs in software. Well… It has to be self-fixing software..It’s not a crazy notion and is one that is already being researched and developed. How far away we are from bug fixing engines that truly learn different rules to evaluate software bugs I couldn’t say. However, this is on the way…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s