You try ignoring a train wreck

A research firm in the UK, Bunnyfoot, released a press release on a study on the effectiveness of in-game ads recently. To paraphrase, the firm concludes that in-game ads are a waste of time and that gamers don't engage with them. Well, you know I had to look into this one.

I found the press release. I read it. Before I begin, let's take a sidebar.

Yeah ok, so like, I know what you're thinking. You're like, dude, umm freak'n marketer, of course you hate this study. But it's not like that. I'm totally going at this from a objective perspective. I'm always ready to back down, even cave if you will, when presented with sound data. Hey, I might then shred the methodology but I'd stand corrected until more data was available.

Biggest problem with this press release: there's absolutely no data. The entire thing reads like a one guy's interpretation of a rorschach inkblot test. Straight from the press release (with my comments italicized and in blue):

Results showed an astonishing variation in consumer engagement (We were not able to definitively find results as our data was all over the place. However, we're going to tell a story anyways because we spent money on this puppy). Overall, SFI scores were comparatively low (compared to what?), especially when contrasted with the prevalence of brand placements (So the scores were low when compared to the number of ads shown? Shouldn't you like, compare scores to other scores? You know, instead of comparing scores to your own bias?). Highest SFI scores were found with NBA Live and Smackdown Vs Raw; however, recall and recognition figures were surprisingly low; (surprising to whom? Again, what's the benchmark, what are you comparing to?) a pattern evident across all titles. Moreover, PGR3 elicited no consumer engagement at all, resulting in 0% on all scores. (I stand corrected, this is the only data point in the entire press release)

Like I've said many times on this blog: beware of what you read in a press release. This release has gotten pick up on respectable pubs like Gamedaily, Next Gen and Gamasutra. None of those pubs have questioned the legitimacy of these findings.

Now I don't want to question anyone's journalistic integrity but perhaps the headline from this release was just too good to pass up. Maybe we don't really want to know how true it is, we just like hearing it. In-game ads have generally been received as warmly as a raging case of herpes (not that I know anything about that). Someone puts out a study that seemingly shows the ads to be impotent and guess what, that study will get headlines.

It's almost like they're saying: hey, let's forget about the methodology. Let's forget if the study was a one-off (play this game for 5 minutes while I watch you) or longitudinal (play this game for a few days). Forget about using a control or comparing results to other results. Forget about all that k? Just read the headline.