Counting Counts

A really good article yesterday on CNET regarding Second Life and how to measure its popularity. Some press coverage (with an assist from Linden Labs) has thrown around 2 million "residents" as a figure for Second Life's popularity. Skeptics point to the fact that that figure is inflated, since it counts everyone that's registered, regardless of whether or not they actually ever "play" or if they're alts of current players. They say the figure is more like 200k.

As long-time readers know, I'm not a fan of Second Life and said so back in July of last year (with a bit of pride [the bad kind], I'd like to point out that was when everyone was still ga-ga over this thing). My opinion stands and I'm glad to see others catching on. Hopefully, most people will see past the smoke and mirrors this year.

Anyways, enough of the shameless self-indulgence. This article brings up a really serious issue. How do we measure the success of products that don't have a tangible count (such as actual units sold)? Here's an overview of some metrics:

• Registered Users - Count each time someone registers an account
• Analysis: Registrations can give you a sense of how popular a game is in the sense of getting people to register. For free games, this number will obviously be inflated. This shows that a game was able to at least entice someone to register but says nothing about stickiness or profitability. In addition, current players almost always register more than once so that has to be factored in as well.
  
• Concurrent Users - Any given time, count every account that is actively playing. This is usually a count of the highest or average number of players logged in simultaneously.
• Analysis: This is a good figure to use since it shows how many people are actively playing at any given time. Unfortunately, most of the time, a concurrent user count represents the high point instead of the average. Maybe there are honest publishers that give high and average CC users but I haven't met one.
  
• Average Active Users - For a set period of time such as day, week or month/s, count every unique account that has logged in, average it out over longer period. This figures depends on what time period you select.
• Analysis: Since this is based on a time period and since there's rarely an agreement on time periods, this also has limitations
  
• Paid User - Total count of accounts that have paid for some service.
• Analysis: This may be a good indicator of how committed the player base is but it does not reflect too many other factors. What about non-paying users? What about games that derive revenue from ads?

Here's my suggestion: Let's set aside periods of time that an account is logged in, say 30 minutes and 2 hours. 30 minutes represent light play and 2 hours represent heavy play. We count users that reach 30 minutes but not 2 hours as light players. We count users that reach 2 hours as heavy players. We then count daily average sessions of light players and heavy players. By themselves, they give us an idea of casual versus hardcore usage. Combined, these two numbers become an Average Daily Engagement Metric.

The ADEM solves several problems. We are only counting real people playing rather than accounts that may not be active. We stop counting logins that don't matter, such as quick log ins on alts for inventory passing. We establish a daily count, which provides more accuracy. We establish an average count, which is more accurate than a high count. Finally, we count players and playtime rather than paid vs unpaid, allowing flexibility across games with different revenue models.

I reserve the right to refine this but I would trust this metric as a measure of success.