The Art of Bug Reporting
Introduction
Finding a bug has always meant that half the work is done. Validating the bug with the developers makes or breaks the effort taken in finding the bug. Usually, a lot of bugs are overlooked due to the fact that the explanation has been poor/incomplete/ under-defined. At the other end of the spectrum, the bug’s explanation has been extended to the point of exhaustion. The following blog will break down the factors that need to be considered while writing any bug.
Background
Since my entire career has spanned
around the video game industry, I will be citing examples based on my
experience. I have been in the industry for over 14 years and have
witnessed a lot of trends changing in the approach. This may not be the
most modern in comparison to the available technology and reporting
tools. But this is the core learning that every tester should have and I
feel that it is an important skillset to upgrade with the testing
knowledge.
Any tester who has his core concepts set
right in the testing industry will have the potential to excel in any
form of QA. While the topic is very simple, mastering it is a challenge
that is faced by many. This blog will not just explore into the base
rules of writing a bug but will also help to map the tester’s thought
process. This will help them articulate and convey what they want to say
in real world scenarios.
Solution
Research on documentation
This is probably the most important aspect before testing the game. The game documentation will contain names of objects used in the game world. Knowledge of these names creates the world of difference between the explanations being vague /
commonly
worded to specific instructions which are to the point. This does not
just help to explain the bug but to show the developers that you are
aware of the game and know all the intricate details while writing a
bug. Most of the games also have zone markings in each
level which is usually not shown in-game. These markings on the
documentation will help shorten the bug and be very specific in the
description.
Example: the sentence “Level 18: After
crossing the barricade of cars, observe the abandoned bus near the wall.
If the bus is to the left-hand side, continue straight till you find
the 4th house to the right. This is the gray house with the
broken window on the first floor. Enter this house through the back,
head to the living room and observe the missing texture above the
fireplace.” Can be rewritten as “Level 18: Zone 3, House no 18. Enter
the back door and head to the living room. Observe the missing texture
above the fireplace.”
Also, with bug writing, it’s helpful to use the consistency of words
while describing a bug. Words like user/player, input names, character
names, game specific objects, weapons and other game factors need to
follow the same game terminology. This consistency will help to describe
the game better.
Usage of phrases/terminology – A lot of
games use real-world objects. Even the game environment settings are
taken from famous buildings/areas and some games even are set in famous
cities. A back research of the place where the game is set and the culture research will help to not just test the game but also give inputs on things which do not align with the game.
The areas of research which can help in identifying the culture are:
- Game environment – City/Famous places etc
- The involved city’s background
- The people’s way of life
- Art/Music /Entertainment
- Climate
- Basic flora/fauna
- Social festival/events
Steps to reproduce
The best way to write a bug is to write the STR before writing the rest of the bug.
By focusing on the STR at first, the bug is easily divided into
sections. This will help to break down the bug and explain it better.
The best way to write a bug is to have the STR condensed to 5 basic steps.
This helps in keeping the main explanation of the bug focused and
removes all unnecessary information. To make this easier, I have broken
down how a typical bug needs to be described:
I find this practice not just useful
while writing a bug but by putting it in any real life explanation, I
find it very easy to explain multiple varieties of things. It helps you keep your mind object oriented and focus on the end issue you want to convey.
In the above 5 step analogy, there will
be issues where step 3 and 4 may need a further breakdown. This is not a
hard and fast rule but a basic interpretation of what needs to be
conveyed. While there can be more or lesser steps, this is the optimal
approach to writing a bug. This method will remove the unnecessary
information and keep the focus on the main issue.
It is human nature to drift away when exposed to a long explanation. Keeping it short
will definitely retain the focus and convey the point. With these
points in mind, the maximum acceptable STR needs to be within 8 steps.
Proofreading
The
best way to write a bug is with Word or any other formatting tool for
documents. They not only help you with spelling errors but also assist
you with grammatical mistakes.
One area which has been a concern with a writing of bugs is the usage of tenses.
Many, who write bugs, do mess up when it comes to writing a bug while
explaining the tense. The below link is a small example of how the
sentence needs to be structured.
Bug review
This is a personal growth progress. Testers need to keep a track on how many bugs they reported,
the severity of the bug and the days when they reported it. This helps
to track a pattern of their areas covered, the maximum productivity of
the week and the amount of contribution they have to the project.
One major thing that they need to track is a number of bugs they reported versus a number of bugs that returned
with ‘need more information’ or ‘moot’. This will help to track the
quality of bugs reported. This is important in a tester life as it
reveals two important things:
- The bug did not make sense
- The tester did not understand if it was a bug or a feature
The second part is a bigger worry
because the tester has not had the knowledge transfer and hence, he/she
are not empowered with the game. As testers, we are expected to be
masters of the game and the tools we work with. Any issue in this area
shows weakness and does not instill confidence of the developers.
Severity versus Priority
This
has to be the most challenging debate with testers and developers. An A
class bug need not be fixed even if it feels important. Now suppose the
player entered a car, drove it to a port, took a boat, hooked the
character to a plane, jumped off and stole a bike, then ran into the
wall which crashed the game. This bug is an A class in severity. But,
consider how many gamers will attempt the same stunt while reproducing
this bug. This will be less than 8% of the gamers who find this issue by
mistake. If they reload the game and they’re saving progress is still
intact, they don’t even bother about this issue. Consider all the games
that have a strict schedule or timeline. When the game is about to
release, an issue with this probability is low, will be overlooked. On
the other end of the spectrum, if the player chooses the left side of
the bridge to cross, the screen goes blank for 5 seconds, there is still
a 50% probability that gamers will face it. This is not an A class
issue, but since the probability of gamers facing that issue is still
high. This issue will feel important to the end user’s overall game
experience.
Hence the priority of the issue does
take a front step. The entire development team does focus on releasing
the product on the issued date while maintaining the maximum quality. We
need to respect the fact that everyone works on a certain timeline. As a
QA member, we need to focus on the issues which will change end user experience. This does not mean a tester does not report all edge case issues. We as testers have our primary fulfillment to report any issue that we encounter.
Alternate bug repro steps
While
we find a bug, it’s not necessarily the best way to find the bug. We
may be testing some other function in the game while we encounter the
bug. The tester needs to be intelligent enough of the end cause of the
bug and then retrace the steps that actually caused it.
In this method, we encounter several other possibilities which caused
this issue. We can take the simplest form to reproduce the bug. This
will also teach us the root cause of the bug.
Example: On a calculator app, if I did
2*5, it crashed. While this scenario might be correct, it may fix the
bug only for this calculation. But had I done 10+0, 20-10, 200-190,
20/2, etc, I would have found the same issue. The end result proving
that the app has an issue with 10 being the result rather than 2*5 being
the error. Hence, research into the issue and come up with
alternatives. This will always help to narrow down the issue.
Summary
This is vital not just to explain the bug in a short form but it helps other testers to check if a bug is logged. Most database search strings involve looking at summary first.
If the summary is optimized, the tester will spend lesser time
searching for an issue instead of logging one. The summary needs to be
crisp, precise and hold all important information within a sentence.
While writing a summary, know that the issue always takes precedence over other factors
like location, game mode, user level etc. This will help to narrow down
the issue and create a referral point for any other tester who wants to
look up the issue.
Reproduction rate
This has been ignored by most testers.
The above point of having alternate repro rate is vital to me having to
emphasize this. Just because you find an issue doesn’t mean it’s down to
the code or art. There are several other factors involved. There can be
issues with the hardware or any software in the background which is
causing interference. This may not seem like the bug you wished for but
it is worth its weight 10 times in gold. Any background app/hardware
which causes an issue to the running game, the developers spend much
more time fixing this because they wouldn’t want the user to be deprived
of any other app to accommodate the game. This can be a big deal since
most users will blame the game and leave it to accommodate the app
instead. This can be a huge negative publicity to the game and most
developers tend to avoid being in this state.
Conclusion
As this blog tends to dwell on the
issues that need to be addressed while writing a bug, it isn’t
necessarily a bible to enforce how a bug is written. The tester has full
control on what needs to be conveyed in the bug. All this article
wishes to cover is the fact that a bug can be so much more powerful when
carrying the right message. Do follow these steps and believe in it for
it can be a useful method to showcase the thought process. It will not
just help you in writing a bug but also assist you in formulating your
thoughts.
Comments
Post a Comment