The Art of Bug Reporting

Introduction

Finding a bug has always meant that half the work is done. Validating the bug with the developers makes or breaks the effort taken in finding the bug. Usually, a lot of bugs are overlooked due to the fact that the explanation has been poor/incomplete/ under-defined. At the other end of the spectrum, the bug’s explanation has been extended to the point of exhaustion. The following blog will break down the factors that need to be considered while writing any bug.</strong >

Background

Since my entire career has spanned around the video game industry, I will be citing examples based on my experience. I have been in the industry for over 14 years and have witnessed a lot of trends changing in the approach. This may not be the most modern in comparison to the available technology and reporting tools. But this is the core learning that every tester should have and I feel that it is an important skillset to upgrade with the testing knowledge.

Any tester who has his core concepts set right in the testing industry will have the potential to excel in any form of QA. While the topic is very simple, mastering it is a challenge that is faced by many. This blog will not just explore into the base rules of writing a bug but will also help to map the tester’s thought process. This will help them articulate and convey what they want to say in real world scenarios.

Solution

Research on documentation  

This is probably the most important aspect before testing the game. The game documentation will contain names of objects used in the game world. Knowledge of these names creates the world of difference between the explanations being vague / commonly worded to specific instructions which are to the point. This does not just help to explain the bug but to show the developers that you are aware of the game and know all the intricate details while writing a bug. Most of the games also have zone markings in each level which is usually not shown in-game. These markings on the documentation will help shorten the bug and be very specific in the description.

Example: the sentence “Level 18: After crossing the barricade of cars, observe the abandoned bus near the wall. If the bus is to the left-hand side, continue straight till you find the 4^th house to the right. This is the gray house with the broken window on the first floor. Enter this house through the back, head to the living room and observe the missing texture above the fireplace.” Can be rewritten as “Level 18: Zone 3, House no 18. Enter the back door and head to the living room. Observe the missing texture above the fireplace.”

Also, with bug writing, it’s helpful to use the consistency of words while describing a bug. Words like user/player, input names, character names, game specific objects, weapons and other game factors need to follow the same game terminology. This consistency will help to describe the game better.

Usage of phrases/terminology – A lot of games use real-world objects. Even the game environment settings are taken from famous buildings/areas and some games even are set in famous cities. A back research of the place where the game is set and the culture research will help to not just test the game but also give inputs on things which do not align with the game.

The areas of research which can help in identifying the culture are:

1. Game environment – City/Famous places etc
2. The involved city’s background
3. The people’s way of life
4. Art/Music /Entertainment
5. Climate
6. Basic flora/fauna
7. Social festival/events

Steps to reproduce

The best way to write a bug is to write the STR before writing the rest of the bug. By focusing on the STR at first, the bug is easily divided into sections. This will help to break down the bug and explain it better.

The best way to write a bug is to have the STR condensed to 5 basic steps. This helps in keeping the main explanation of the bug focused and removes all unnecessary information. To make this easier, I have broken down how a typical bug needs to be described:

I find this practice not just useful while writing a bug but by putting it in any real life explanation, I find it very easy to explain multiple varieties of things. It helps you keep your mind object oriented and focus on the end issue you want to convey.

In the above 5 step analogy, there will be issues where step 3 and 4 may need a further breakdown. This is not a hard and fast rule but a basic interpretation of what needs to be conveyed. While there can be more or lesser steps, this is the optimal approach to writing a bug. This method will remove the unnecessary information and keep the focus on the main issue.

It is human nature to drift away when exposed to a long explanation. Keeping it short will definitely retain the focus and convey the point. With these points in mind, the maximum acceptable STR needs to be within 8 steps.

Proofreading

The best way to write a bug is with Word or any other formatting tool for documents. They not only help you with spelling errors but also assist you with grammatical mistakes.

One area which has been a concern with a writing of bugs is the usage of tenses. Many, who write bugs, do mess up when it comes to writing a bug while explaining the tense. The below link is a small example of how the sentence needs to be structured.

http://www.english-for-students.com/Tenses.html

Bug review

This is a personal growth progress. Testers need to keep a track on how many bugs they reported, the severity of the bug and the days when they reported it. This helps to track a pattern of their areas covered, the maximum productivity of the week and the amount of contribution they have to the project.

One major thing that they need to track is a number of bugs they reported versus a number of bugs that returned with ‘need more information’ or ‘moot’. This will help to track the quality of bugs reported. This is important in a tester life as it reveals two important things:

• The bug did not make sense
• The tester did not understand if it was a bug or a feature

The second part is a bigger worry because the tester has not had the knowledge transfer and hence, he/she are not empowered with the game. As testers, we are expected to be masters of the game and the tools we work with. Any issue in this area shows weakness and does not instill confidence of the developers.

Severity versus Priority

This has to be the most challenging debate with testers and developers. An A class bug need not be fixed even if it feels important. Now suppose the player entered a car, drove it to a port, took a boat, hooked the character to a plane, jumped off and stole a bike, then ran into the wall which crashed the game. This bug is an A class in severity. But, consider how many gamers will attempt the same stunt while reproducing this bug. This will be less than 8% of the gamers who find this issue by mistake. If they reload the game and they’re saving progress is still intact, they don’t even bother about this issue. Consider all the games that have a strict schedule or timeline. When the game is about to release, an issue with this probability is low, will be overlooked. On the other end of the spectrum, if the player chooses the left side of the bridge to cross, the screen goes blank for 5 seconds, there is still a 50% probability that gamers will face it. This is not an A class issue, but since the probability of gamers facing that issue is still high. This issue will feel important to the end user’s overall game experience.

Hence the priority of the issue does take a front step. The entire development team does focus on releasing the product on the issued date while maintaining the maximum quality. We need to respect the fact that everyone works on a certain timeline. As a QA member, we need to focus on the issues which will change end user experience. This does not mean a tester does not report all edge case issues. We as testers have our primary fulfillment to report any issue that we encounter.

Alternate bug repro steps

While we find a bug, it’s not necessarily the best way to find the bug. We may be testing some other function in the game while we encounter the bug. The tester needs to be intelligent enough of the end cause of the bug and then retrace the steps that actually caused it. In this method, we encounter several other possibilities which caused this issue. We can take the simplest form to reproduce the bug. This will also teach us the root cause of the bug.

Example: On a calculator app, if I did 2*5, it crashed. While this scenario might be correct, it may fix the bug only for this calculation. But had I done 10+0, 20-10, 200-190, 20/2, etc, I would have found the same issue. The end result proving that the app has an issue with 10 being the result rather than 2*5 being the error. Hence, research into the issue and come up with alternatives. This will always help to narrow down the issue.

Summary

This is vital not just to explain the bug in a short form but it helps other testers to check if a bug is logged. Most database search strings involve looking at summary first. If the summary is optimized, the tester will spend lesser time searching for an issue instead of logging one. The summary needs to be crisp, precise and hold all important information within a sentence.

While writing a summary, know that the issue always takes precedence over other factors like location, game mode, user level etc. This will help to narrow down the issue and create a referral point for any other tester who wants to look up the issue.

Reproduction rate

This has been ignored by most testers. The above point of having alternate repro rate is vital to me having to emphasize this. Just because you find an issue doesn’t mean it’s down to the code or art. There are several other factors involved. There can be issues with the hardware or any software in the background which is causing interference. This may not seem like the bug you wished for but it is worth its weight 10 times in gold. Any background app/hardware which causes an issue to the running game, the developers spend much more time fixing this because they wouldn’t want the user to be deprived of any other app to accommodate the game. This can be a big deal since most users will blame the game and leave it to accommodate the app instead. This can be a huge negative publicity to the game and most developers tend to avoid being in this state.

Conclusion

As this blog tends to dwell on the issues that need to be addressed while writing a bug, it isn’t necessarily a bible to enforce how a bug is written. The tester has full control on what needs to be conveyed in the bug. All this article wishes to cover is the fact that a bug can be so much more powerful when carrying the right message. Do follow these steps and believe in it for it can be a useful method to showcase the thought process. It will not just help you in writing a bug but also assist you in formulating your thoughts.