Professional games user researcher and former Playstation veteran Steve Bromley gives us the low-down on the coolest sounding research role out there.
Steve Bromley has one of the coolest sounding jobs out there. It’s one of those gigs like professional food taster, chocolate consultant, or tropical island caretaker—that as soon as we hear of its improbable existence, we’re immediately sent into a kind of despair. Like we’ve been hoodwinked by curmudgeonly career advisors into a lifetime of drudgery while someone out there is eating chocolate for a living.
Bromley doesn’t scoff Lindt in his day-to-day, but to many, his job—Games User Researcher—sounds just as sweet. And there’s good news. While Bromley may have one of those too-good-to-be-true-sounding roles, he isn’t gatekeeping it. The opposite—Bromley, the author of How to be a Games User Researcher, wants to tell you how to live the dream.
But before you start rearranging that CV, read on—we’re not only covering what it takes to be a games user researcher. We’ll also be taking a broad look at the state of user research in gaming and tackling some of the challenges researchers are facing in the industry today.
How to be a games user researcher
According to Bromley, there are three major areas you need to cover if you’re considering a career in the industry. You need to understand:
How games are made
How to run playtests
How to apply and interview
How games are made
While there are definite similarities in the way games and traditional products are made, the differences truly define the games user research profession.
First, a games designer differs quite dramatically from a product designer. “The term game designer covers a whole bunch of different disciplines,” says Bromley. “From people who are crafting the puzzles and crafting levels or activities players do to more systems designers who are thinking about the core loop of the game—which is understanding what the player has to do with every part of the game and how it all joins together.”
Depending on the specialty, games designers spend a lot of their time thinking about all the different moving parts in the game. In the product world, the closest parallel, says Bromley, is a service designer.
However, the primary stakeholder in a games user researcher’s life is a producer—akin to a product manager in the traditional product world. “Producers have a very good overview of the timeline, what things are coming on board at which point and when, and when are good times to test,” says Bromley. “And then, as user researchers, we need to find the relevant designers and ask them about the intended experience.” The goal, Bromley iterates to me, is to uncover the vision for that part of the game and understand how the designer expects the player to react. A games user researcher then does the research thing and turns this information into a series of hypotheses that can be tested.
The art of playtesting
Ensuring a player reacts in the intended way to a specific experience is paramount to the game’s success. To ensure a designer’s creation is hitting the mark, the industry employs something called playtesting.
Playtesting is what it sounds like: a real player will sit through some or all of a game and answer questions relating to its playability. If it sounds familiar, that’s because playtesting is the industry’s version of usability testing. But some very particular considerations make it interesting.
Unlike traditional products, games are released in one big bang. Tonnes of marketing spend is deployed in the lead up to the game’s release, and accordingly, there is a lot more pressure to get the game right first go. Incidentally, this is why Bromley believes so strongly in the part user research plays in the game’s success. Practically, this also means studios are very concerned about information leaking before the launch date.
“Some of the methods we might use in product user research feel too unsafe for many games, such as unmoderated testing,” says Bromley. “Someone playing at home can be difficult because you don’t know if they are recording their screen or something like that.”
“Because of those risks, when we want to answer quant things, we need to bring them to us,” Bromley states. “So it’s very typical in a playtest lab for the big publishers to have 40, sometimes 80 gaming seats where we can have 80 participants at once just playing through the game, going through long campaigns or doing individual levels and surveys and analytics generated by the game to answer questions.”
Unsurprisingly, with all that prep work, ResearchOps plays a significant role in games user research. Not only in the gargantuan effort it takes to get 80 participants in a room simultaneously (shout out to my ResearchOps people! If you know, you know!), but also in the screening of said participants.
“[Gamers are] a very engaged audience, and everyone wants to see secret things and talk about secret things. Lots of people would like to participate in the research,” says Bromley. A typical product research session isn’t likely to be the most exciting thing our participants have done that day. Usually, the reward is financial or social. But in games user research, taking part can be the highlight of the month or even year.
Participants falling over themselves to get access sounds like a dream—but Bromley aptly points out that extra rigor must be taken in the screening process to ensure no time wasters slip through.
Where’s the industry at?
While playtesting sounds like the obvious domain of an experienced user researcher, the industry hasn’t quite matured to the point where employing professional user researchers is widespread. According to Bromley, this means that playtesting exists as an idea, but it’s still in its nascent stages.
“Playtesting can mean a whole bunch of different things in different studios, especially because user research and UX isn’t that established in games,” Bromley reveals. “The definition does vary from, ‘I am a game designer, and I play my game, and I see if I like it. And if I do like it, I give it a thumbs up,’ which, obviously, coming from a research discipline, we know that’s subject to human biases. You’re playing your own thing. You haven’t defined your criteria. You’re just deciding, does this feel right.”
Getting widespread adoption research is one of Bromley’s strongest points of advocacy. He acknowledges that part of the difficulty is that “unlike product development, games are considered art.” To many games designers, the idea of a researcher coming in and telling them how their game should operate smacks of interference and a watering down of their creative vision.
Bromley approaches this barrier with candor. His response is an amazing template for anyone looking to further the cause of user research with skeptical stakeholders. “As user researchers, we make sure we’re aware that the games are art. And we say, we’re not here to change your vision. We’re not doing market research to tell you what game to make. Instead, we want to understand your vision and make sure it’s being understood by players correctly and that there isn’t an interpretation gap.
But I think the fear is user research will be getting a focus group together. And participants will say, ‘I don’t like this game,’ and lots of opinion-based things rather than behavioral measurements against the intended experience. This means game designers don’t engage with user research until quite late in the development period or when our ideal process would begin.”
The idea of games as art isn’t prevalent at all studios, says Bromley. In particular, the booming mobile “games as a service” industry seems to have a more product-forward approach to development. Nevertheless, getting buy-in early for research remains a challenge for many user researchers in the industry.
But Bromley seems confident in the headway he and other researchers have made over the years and remains optimistic about the future potential of user research in games. “We’re professionalizing research and bringing in the expertise that user researchers elsewhere have to help make tests more efficient, more reliable, and ultimately better for making game design decisions. There are also really interesting nuances of doing it in games that you don’t see in other industries. It’s a fascinating industry for a user researcher to work in.”
How to get the job
You’re already a good candidate if you understand playtesting and grasp how games are made. But there are a few more considerations for those who might be interested.
First, geography. No, not the subject, but rather where you live or would be willing to live. “[Games user research roles] are very geographically specific to hubs like London, San Francisco, or Canada,” Bromley tells me. “There can be a lot of competition for the roles because a lot of people want to work in games. It’s a very prestigious industry, and there aren’t many roles out there.”
Because of the competition, those who stand the best chance of entering would bring a fair bit of experience. While there are many junior roles available, there’s also a lot of demand for those roles. While at the top end, many senior games user researchers are drawn out of the industry into the tech world. This leaves room for experienced researchers to enter somewhere in the middle or top.
If you’re looking to enter as a junior, Bromley’s best advice? Work on your interviewing skills. “What are the skills you need to demonstrate? How can you get the experience, and how can you demonstrate those skills?” he posits.
User research and creativity
Our earlier conversation about game designers, their creative vision, and auteurial control over development has me thinking. Particularly regarding designers’ skepticism toward research—could they be right? Is user research even compatible with creativity? Or does it, in fact, lead to a kind of design by committee—a camel instead of a horse?
Bromley believes wholeheartedly that user research is compatible with creativity. “The critical part as user researchers is that we need to understand the creative vision. We need to put hard work into talking to designers and understand what they think they’re making and how players should receive it. And then treat that as the hypothesis for what we’re testing against,” states Bromley. Rather than changing the vision, he says it’s supporting it.
But for Bromley, it doesn’t end there. He believes the future of user research in games is to be true collaborators in the creative decision-making process. In Bromley’s words, “Helping people recognize how a better understanding of users can inform their creative vision—but that’s a battle for the future.”
As for the old chestnut about horses and camels? Bromley leaves me with a neat little rejoinder on the matter. “I think we need to find a way to communicate to designers, ‘Hey, we’re not going to tell you to make a camel. We understand you want to make a horse. We’re going to investigate: Where does that horse need to go, when does it need to get there, who’s going on the horse? All the things that would help you design the horse appropriately so that you’re still in control of it, but you understand the context in which that horse has to work.”