ROBLOX NEWS!

ROBLOX Users: An Essential Part of Our Development Team

By Simon Kozlov - April 19, 2012

We recently explained our extensive testing infrastructure and showed how various systems are checking ROBLOX’s code for errors and regressions at just about all hours of every day.
But get this: that wasn’t the full picture.
ROBLOX also has an experienced community of user testers, who round out our otherwise mechanized testing platform with good, old-fashioned – but no less vital – human interaction, and usually learn a few things in the process.

User testing’s place in ROBLOX

User testing is a cornerstone of ROBLOX’s elaborate testing infrastructure. It pits a release against the widest range of real-life user behavior. It’s the last line of defense before a work-in-progress ROBLOX gets promoted to official, player-facing release.
Before any new version of ROBLOX is released to users at large, we launch it on gametest.roblox.com – an exact replica of the real site, but running on a smaller scale. The extent of a given test depends on the release. While we generally follow the rule of “at least 100 testers over the course of three to seven days,” a minor patch usually doesn’t require the full-bore testing operation.
We notify users when a new test build is available by placing a notice on ROBLOX.com and making an announcement in our official forums. Then, ROBLOX users play games the same way they would on the live site.
That is, until they encounter a problem.

User testing is a two-way street

User testers’ key obligation is to inform us of the problems they encounter with solid information. Screenshots and video are always useful – easily attainable using ROBLOX’s in-game video- and screen-capture tools – but most important are the descriptive, detailed steps we use to reproduce the error. Even seemingly arbitrary details, such as what your character was wearing, are important.

This is good, and the only thing that would have made it better is operating system information. Mac or PC? Windows XP or Windows 7? OS 10.6 or OS 10.7?	There’s a lack of information, here. We’d like to know the operating system, browser and steps we could take to reproduce the error. We don’t know whether it was the build button, the stamper tool or something else that failed.

ROBLOX users can also upload logs of their testing sessions by pressing F8. ROBLOX will record system information and more to help us find and fix bugs.
While users are searching for bugs, we watch our internal metrics – number of crashes, number of thumbnails failed, average time to join and more. For example, this is one of the reports we use:

The report shows what kind of crashes users experienced during the testing, how many people crashed the same way and whether crash happened on the user’s computer or the ROBLOX server. It also shows whether a crash happened for the first time; if so, we have a hint that it was caused by a recent change to ROBLOX.
We often reach out to users for more help. We might ask how, exactly, they got ROBLOX to crash and what they were doing when it crashed. When hunting for a bug, even the smallest detail might be the key to catching it.

Join the user-testing club

This is all fueled by the voluntary participation of ROBLOX users. It begs the question: Why become a tester?
Learn. Testing not only trains the critical-thinking part of your brain, but also gives you insight into the software development process, brings you into a passionate community of ROBLOX users and positions you as a leader in the greater ROBLOX world. In some cases, you’ll get to spend time chatting with ROBLOX developers, who would like to learn more about your bug report.
Early access. The reason you’re testing is to make sure the latest ROBLOX features are ready for prime-time. You see it first and know how it works, so you get the jump on other players and creators.
Rewards. It’s our tradition to give user testers in-game items as a token of appreciation for reporting bugs we missed.
Plus, you rightfully receive the intangible, but unending gratitude of ROBLOX’s developers. We value your time and effort, and do what we can to make it a worthwhile experience.
With an understanding of ROBLOX’s testing infrastructure, the only thing left for you to do is become a contributor. Head over to our Game Test forum and join the conversation, or check out our public testing environment at gametest.roblox.com. Help shape our platform.

 475  2  1