I dedicate this Podcast Summary to [url=http://i868.photobucket.com/albums/ab249/Hylebosaiden/Shitake_zps98d7eb32.png]Shitake[/url]. Rest eternally, my Manly Lion.
Another month, another Podcast. While my initial predictions for the subject matter of this Podcast turned out to be terribly inaccurate, this is still a very solid Podcast that can be enjoyed nicely alongside turkey. The subject this month is User Research and Localization, two important vectors from which Bungie can reach out to the community near and far.
Before we begin, please remember that as always this summary is no substitute for the real deal. If you can carve out time to listen to this 51 minute Podcast this break, you should definitely do so, but if you can't find the time or you want to review what was discussed, that is what this summary exists for. Keep in mind that I am Human and I can make errors or portray things incorrectly, so if you spot any feel free to point them out.
1) DeeJ, Urk, and Halcylon are bringing us the Bungie Podcast from deep within the basement of Bungie in the Foley lab.
2) Time-wise, this podcast was recorded some time shortly after the launch celebration for the PS4.
3) Today’s guests will help us understand how Bungie interacts with their community both near and far.
4) They are: [url=http://halo.bungie.net/News/content.aspx?type=news&cid=32234]Jennifer Ash[/url] from Bungie User Research and [url=http://halo.bungie.net/news/content.aspx?type=news&cid=32214]Tom Slattery[/url] from Localization.
[b][u]5:33 - Jennifer Ash[/u][/b]
5) Jennifer Ash is a User Researcher. In her own words, User Researchers look at the game, they look at the data going on behind the gameplay, a lot of it is analyzing what goes on in terms of actual interactions that people have but also in terms of the data going on behind the scenes and analyzing and visualizing that data to better inform game designers.
6) Her undergraduate degree was in Game Design and Psychology which led her to Human Computer Interaction which set her on the path towards User Research. In graduate school she did educational games which gave her a cognitive science background which helped her to understand statistics and view games from different perspectives.
7) A lot of people misunderstand what User Research is and in their minds they blend it together with what they think is focus group testing, what’s the difference? A lot of participants apparently mistake the User Research team for test, they’ll report bugs and the UR team will be like “That’s great… but that’s not what we’re going for today…”, they care more about your experience and how you play the game. Focus groups are more of a marketing angle, which is much more about how people perceive things.
8) There are two major perspectives from which they approach user research: usability and play testing. With Usability testing they pair one user with one researcher and they analyze “What did you do there? Talk out loud as you go through your subject area…”. With play testing, they have a group of people in the lab and they are just playing normally, and they want to know “How would you normally play? What is some gameplay data that we can get from this that would be useful to designers?”, things like balancing items and balancing terrain.
9) Is it more taking in the data and looking at it retrospectively, or more actively listening and actively interacting with the subjects? Usability is more in the moment, a lot of what the researcher is doing is taking notes as the subject plays and writing up a summary afterwards about their experience. With playtesting it’s more of mulling over the data after the fact because they cannot process 18 people at once. There are things they do to help, from recording in game data to recording their faces as they play which can be reviewed after the fact.
10) The Laboratorium has about 18 stations, each one has a computer, dev kits, and one of the stations is set up with an eye tracker which they can use to see what their eyes are looking at while they play the game, which is useful for things like cinematics, user interface, and the heads up display. Every station is wired internally so all data is kept within the walls. The user studies can also be streamed so designers can watch it live, often in the theatre which is directly adjacent to the User Research Labs.
11) There are some user research tools built directly into Grognok (Bungie’s world editing tool) which allows them to view the data in-engine; things like player deaths or kills. Designers can figure out where all the kills are happening, especially in PvP, it can inform design decisions like geometry changes and such. It can also be used to see which abilities are popular in which spaces. It’s like a heat map, but in a real space that you can walk through instead of a drawn image.
12) Is it the User Research team’s responsibility to draw conclusions for the Designers using the data? That responsibility is shared from the beginning to make sure the user research team is answering questions that the designers are interested in, and in the end they try to keep things very data-based instead of having subjective opinions. They try to give the data to the designers and let them draw their own inferences and offering recommendations, but mostly through past experiences. Most of the time the designers have to be on board to understand, they can’t force those sorts of things.
13) Halcylon points out that the User Research team isn’t exclusively working with the game, but also with Bungie.net on the mobile app and web companion.
14) User research can be difficult sometimes because you have to go up to people and tell them with data in hand that their baby is objectively ugly, maybe you should try and change it.
15) It can also be difficult to interact with players sometimes. You need to be careful not to ask leading questions and make things as objective as possible, like if you ask “What did you like?” they’ll focus on the positives when you were hoping to hear more about what they disliked and such. A lot of it involves asking questions in a way that doesn’t suggest an answer, which is especially difficult in usability testing where you have to think on your feet a lot, you have to take a step back and interact very broadly.
16) This entire “coloring of conversations” is one of the reasons they try to discourage research participants from talking to one another, especially when they are filling out surveys.
17) How do they choose people? Basically they do random selection, they have a database with a profile for everyone who has registered for beta testing, and depending on the study that they are running they’ll want different profile types and they’ll invite them to the studio. Being local gives you a better chance of being called into the studio, but ideally anyone worldwide can be given an email survey at any point. They bring in all sorts of players, it all depends on what the designers are interested in.
18) After undergraduate college Jennifer worked at IBM in various divisions. The game industry is very different from the software industry, both in terms of schedule, the variety of things you work with on a daily basis, dress code, working towards a single goal…
19) Previous experience for designers is super useful, but User Research can be super useful for informing them about emergent activities, which are very prevalent when you are doing things that have never been done before.
20) The user research team is a team of four, and they’ve split the game up into various areas that they’ve divided amongst the team to focus on, but they’re all working together in making the experience of the game as good as possible and help each other out when they run into problems.
21) There’s a few amusing anecdotes about various User Research tests. One where a guy didn’t realize that you couldn’t bring a gun into the tests and ran back out to his car to drop it off before proceeding through the metal detector. Another generalized one where users would input their antiquated and embarrassing usernames that they would try to explain away as the researcher and all the designers in the other room are all silently judging them.
22) On the subject of the NDA that all user researchers sign, you’re not allowed to talk about anything. You’re allowed to wear the free t-shirt you’ve been given, you’re allowed to say that you’ve done testing at Bungie, and that’s about it, and that lasts forever.
23) User researchers do get fed, but no beer. DeeJ and friends share an amusing anecdote about how the user research would be more applicable if the subjects were in their natural gaming environment: probably wasted and wearing no pants on a couch.
24) One of the more surprising data-points were the reactions shared social experiences that people had in the playtests. People responded very well to them, from recognizing that the person they were fighting with was someone in the room, to two guardians sharing a silent sunset with one another.
25) Jennifer was one of the Bungie Employees who went to Gamescom, there’s a few amusing anecdotes about that.
26) They’re always looking for more people to join the User Research Program. [url=http://www.bungie.net/en/Questionnaire/UserResearch]GO HERE TO SIGN UP.[/url]
[b][u]30:20 – Tom Slattery[/u][/b]
27) Tom Slattery is a localization manager at Bungie. He manages a full team of people who take content that others have made and convert that into written or spoken language. They’re not so much the sole translators but more so the localization editors, they don’t translate everything but they look over all the translations to make sure things look right.
28) You would think translation is simple, but when you throw slang, idioms, and metaphors into the mix it can be difficult to convey meaning. There’s an anecdote about trying to translate “Riding Shotgun” into German.
29) Tom’s job is more management than translating, manages the team, the schedule, meets with various other teams at Bungie, the people who do the website and the writing. This is the first time they’ve had an in-house localization team at Bungie, some of the benefits was quicker turn-around time which allows for more last-minute tweaks, and the localization team has direct access to the rest of the game and their developers for immediate questions.
30) Tom speaks Japanese and English. He likes English the most because he knows it very well.
31) Tom originally got a degree in Computer Science and went on to teach public middle school English. He only had a minor in Japanese, and he happened into the career field when he moved to Japan to teach more middle school English, and he became a Game Translator, translating Final Fantasy, Chrono Trigger for the DS, some Zelda games.
32) Building the localization team was a long and difficult process, they started with writing samples in their language, Activision helped to evaluate and test candidates before bringing them into the Bungie Interview.
33) These translators actually interface with the community, they’ve been coached on interacting with us voracious forum-goers.
34) These people are important, there’s an entire world out there that is cut off by a language barrier which makes it hard to talk about Destiny.
35) Localization can be tough because you go around and tell people “This won’t work, you need to do this a different way”, things like “use an icon instead of words” and so on and so forth. It can be a pain, but it often leads to better site design as a whole.
36) There’s an amusing anecdote about how the Bungie Employees going to Germany couldn’t use a certain English word because phonetically in German it sounds like a really terrible phrase that would result in getting punched in the face by the German fans.
37) It’s not just translating words into other words, you have to understand the cultures of the countries that you are translating for. Things like “German audiences are very stoic, don’t feel bad if you don’t get wild cheering like at E3”.
38) The biggest challenge is that they’re at the tail end of development, once everything is locked down they get ahold of it and they have to get it turned around as quickly as possible. Anticipating problems and trying to influence people before they lock things down is a huge part of making the job smoother, as is getting them to prioritize jobs that will be harder to localize or would benefit from more time being polished from a localization standpoint.
39) Bungie.net currently supports four full localizations, Spanish, French, German, and Italian, and one partial localization for Brazilian / Portuguese.
40) DeeJ’s American voice is hilarious.
41) Not every news article gets translated, for example, they don’t carve pumpkins in Europe, so they didn’t translate that on over to the European languages.
42) DeeJ’s puns make translation difficult.
43) Bungie.net translations might die down as the game nears completion, but it’s worth it because the game will reach out to sectors of the globe that they’ve never reached out to before.
44) German seems to be the most popular of the five localizations.
45) If anyone can find Halcylon and Tom out in the wild they have drinks and beta codes.