What is your testing style?
Jomama
23 Nov 2011
Hey all,
I'm not an alpha tester yet, but I hope to be.
I've amassed a great deal of flight time across more than a dozen titles over my 20+ year Flight Sim career. As a result, I've seen a great many ways to solve various problems in game play, balance, mechanics, playability vs realism and a host of other "issues" the developers must grapple with to create a successful game. I'm played some winners, and I've played some "losers". As a software developer myself, I've come to appreciate the effort it takes to create compelling games, and look forward to helping Wargaming create a "winner".
As part of that effort, I know that it is crucial to not only point out what does not work, but to provide useful information to the developers, so they can evaluate a problem, understand its underlying causes, and ultimately, solve the problem.
As part of that effort, I try to follow a process for providing that information. I'll share mine here. What's yours?
1. System info (what hardware do you have, and what might be an item that could adversely interact with the program?
2. Description of problem. This is probably one of the most crucial aspects of a report. Clear, concise and devoid of "emotion". Simply saying it doesn't work doesn't help much. What doesn't work? Detail here is important, as the developers do not have "your experience" of the problem as a basis for reading the description
3. Steps to reproduce. VERY IMPORTANT! if you can't reproduce the bug, you cant find it. If you can break it consistently, then pass on the EXACT steps needed to break it.
4. Characterize the problem. Work around the problem and see if there are other ways to "make it work". describe these steps. This helps the developers to determine which code path may be causing the problem. Many times there are multiple ways to "get to that spot" in code. It may work one way but not another.
5. Note environmental factors. Things like time of day, server load, how long you have been playing that session, and any other thing that could possibly be a factor in the problem discovered.
I keep these things in my mind as I play, and always look for ways to "make it easier" on the developers. This is what I look for when I'm testing my code.
Share your thoughts. What do you do?
I'm not an alpha tester yet, but I hope to be.
I've amassed a great deal of flight time across more than a dozen titles over my 20+ year Flight Sim career. As a result, I've seen a great many ways to solve various problems in game play, balance, mechanics, playability vs realism and a host of other "issues" the developers must grapple with to create a successful game. I'm played some winners, and I've played some "losers". As a software developer myself, I've come to appreciate the effort it takes to create compelling games, and look forward to helping Wargaming create a "winner".
As part of that effort, I know that it is crucial to not only point out what does not work, but to provide useful information to the developers, so they can evaluate a problem, understand its underlying causes, and ultimately, solve the problem.
As part of that effort, I try to follow a process for providing that information. I'll share mine here. What's yours?
1. System info (what hardware do you have, and what might be an item that could adversely interact with the program?
2. Description of problem. This is probably one of the most crucial aspects of a report. Clear, concise and devoid of "emotion". Simply saying it doesn't work doesn't help much. What doesn't work? Detail here is important, as the developers do not have "your experience" of the problem as a basis for reading the description
3. Steps to reproduce. VERY IMPORTANT! if you can't reproduce the bug, you cant find it. If you can break it consistently, then pass on the EXACT steps needed to break it.
4. Characterize the problem. Work around the problem and see if there are other ways to "make it work". describe these steps. This helps the developers to determine which code path may be causing the problem. Many times there are multiple ways to "get to that spot" in code. It may work one way but not another.
5. Note environmental factors. Things like time of day, server load, how long you have been playing that session, and any other thing that could possibly be a factor in the problem discovered.
I keep these things in my mind as I play, and always look for ways to "make it easier" on the developers. This is what I look for when I'm testing my code.
Share your thoughts. What do you do?
shnbwmn
23 Nov 2011
I think this topic borders on info fishing - I definitely won't be someone's bait a 2nd time round ...
But basically, I would open a game and critique whatever I see ---> why is this here? ... could this interface item be placed better elsewhere? ... What's missing? ... etc.
I have no solid process or anything, I just open the game like I'm a new player each time and think: "What do I want to see?"
But basically, I would open a game and critique whatever I see ---> why is this here? ... could this interface item be placed better elsewhere? ... What's missing? ... etc.
I have no solid process or anything, I just open the game like I'm a new player each time and think: "What do I want to see?"
James_White
23 Nov 2011
shnbwmn, on 23 November 2011 - 11:31 AM, said:
I think this topic borders on info fishing ...
Jomama
23 Nov 2011
James_White, on 23 November 2011 - 12:11 PM, said:
I also think so
My apologies. Fishing?
Could someone explain?
Jomama
23 Nov 2011
Helskga, on 23 November 2011 - 03:59 PM, said:
Trying to get an Alpha Tester to slip up and reveal something. You were fishing for information, trying to find out stuff that isn't public yet.
It may not have been your intention but to me it looks like you were trying to get an AT to talk about a recent bug, most bugs would be something to do with ingame content so they would have revealed something.
It may not have been your intention but to me it looks like you were trying to get an AT to talk about a recent bug, most bugs would be something to do with ingame content so they would have revealed something.
Thanks for taking the time to explain that to me. I appreciate it very much.
shnbwmn
23 Nov 2011
Papa_Joe, on 23 November 2011 - 04:41 PM, said:
Ah. I understand now. It was not my intention, and I will avoid creating threads that would seem to "point" in that direction in the future.
Thanks for taking the time to explain that to me. I appreciate it very much.
Thanks for taking the time to explain that to me. I appreciate it very much.
Jomama
23 Nov 2011
Thanks for that.
I still feel the thread has value, in the sense that I was seeking to "raise the standard of testing" so to speak. The more we share in terms of technique (NOT content), the better the data is that the developers receive.
So if you have a general testing method or technique that you use and would like to share, I'd love to learn more. PLEASE, do not share case examples or specifics. I'm only looking at the testing process (which really should be game agnostic), not specific examples or content.
I still feel the thread has value, in the sense that I was seeking to "raise the standard of testing" so to speak. The more we share in terms of technique (NOT content), the better the data is that the developers receive.
So if you have a general testing method or technique that you use and would like to share, I'd love to learn more. PLEASE, do not share case examples or specifics. I'm only looking at the testing process (which really should be game agnostic), not specific examples or content.
Redjevel
23 Nov 2011
When i test game im always looking for glitches and that sort of things.
"Put into your inventory "somethink" and then disconect the game, stacking items/duping them." (but thats mostly in RPGS)
Ofc i always report all, i dont want players have this advantage just becouse they used glitch or somethink.
But the prior is gameplay, soo all sort of bugs or crashes.
"Put into your inventory "somethink" and then disconect the game, stacking items/duping them." (but thats mostly in RPGS)
Ofc i always report all, i dont want players have this advantage just becouse they used glitch or somethink.
But the prior is gameplay, soo all sort of bugs or crashes.
