Future Job Posting: Interactive Game Analyst

I'm a data junkie. In my old job as a strategist/analyst for an advertising agency, I used to gleefully swim in data like Scrooge McDuck swam in his doubloons. I took pride when the data feeding my pivot tables broke Excel's 65,000 row limit. I savored the moments when my ANOVA statistical tests would show the tested F higher than Fsig. Why?

Data pulls back the curtain on the unknown. Unlike magicians, businesses can't make money off of mystery and clever prestidigitation. Businesses work when you find a formula that works and feed more resources into it. That's where data comes in. To sum it up rather simply, data lets you know if something worked.

Now let's talk about the game industry. Up until recently, the only data available to measure a game's success were sales and probably focus groups or surveys. Unless you sat with someone while they're playing the game, you couldn't really tell what worked or what didn't. This is very similar to how the advertising industry used to work. You made a commercial or print ad and you measured this by sales or focus groups; this stuff was way downstream of what actually happens when consumers see the ad (the sticklers out there will throw out a few more data points but the basic limitations are still there).

My old ad job was something new to the ad biz. Online gave us the opportunity to monitor how people were engaging and reacting to our ads right away. I could gather data about the effectiveness of our advertising in almost real time. That's powerful information that helped us improve our advertising when we channeled this to our creative teams.

Online games are starting to pick up on the value of this information. Yesterday I ran into this bit of news about the creators of Half-Life 2 patching their game and changing several gameplay elements because of data they analyzed. Check out these stats and graphs. Their data still pretty much descriptive statistics but I don't see any reason why they couldn't be more robust in their analysis.

I'm sure MMORPGs have been doing this for a while. They probably monitor how certain gameplay tweaks (nerfs) affect usage and/or balance. However, I question whether someone really digs into the data or whether their analysis is just based off of observation and qualitative analysis. To me, the real insights occur when you take incorruptible information and extrapolate true behavior. For example, the folks at PlayOn have been discovering great insights gleamed from quantitative data on WoW player activity. Some of the stuff they've found just can't be surveyed or manually observed.

Let's imagine the possibilities if an MMORPG utilizes some robust analysis (such as the stuff I did in advertising). You can monitor gameplay styles of every player and combine that with demographic data such as age or gender. You can then group players by profile such as "healer/builder", "high-use PvPer", "socializer", etc. The profiles then form a foundation for you to analyze usage. You then crosstabulate the users with their actual activities. Perhaps you find that "socializers" are actually very profitable because they help retain players to the game. For instance, you find that gametime for a group increases when there's a "socializer" in the party. Unfortunately, you also find that "socializers" are struggling with a certain element of the game and start dropping off when they hit that element. Well then, you better do something about element.

The example I provide above is actually something that is frequently done in the broader market. Data mining marketing companies have placed all of us in some profile or another. We are then considered "targets" for certain products, promotions or content. (You'd be scared if you knew how much marketers knew about you. Don't ask, trust me, continue to live blissfully ignorant.) But in the realm of games, especially MMORPGs, this could provide you with some real value. Instead of just throwing out content to see what sticks, game designers could base their decisions on what works for you. It'd be a whole new way of thinking about game design.

Before some of you get your undies in a bunch, don't worry, I'm not advocating that data replace good old fashioned creativity. What I am saying is that we have the ability to do things with a little more intelligence (multiple definitions of that word in play here). Online is providing that opportunity and I'm glad to see it being taken advantage of by some innovative frontrunners out there. This is going to be important stuff, I can just feel it!