Support NeoGAF

[cvxfreak]

Sales-Age: Dengeki vs. Media Create vs. Famitsu for the past year

Okay, so I was looking through old sales threads to find stuff, and I observed the following with the pool of title in which people bothered to record the Dengeki numbers should they have been given within the last 12 months:

Game Title - Media Create - Dengeki - Famitsu - (Most, Least)

Mother 3 - 206,000 - 156,000 - Coming Soon
Dragon Quest Yangus - 127,000 - 148,000 - Coming Soon
Final Fantasy XI 2006 Expansion - 76,000 - 81,000 - Coming Soon

Final Fantasy XII - 1,764,000 - 1,817,000 - 1,840,000 - (Famitsu, Media Create)
Pokemon Ranger - 193,000 - 192,000 - 189,000 - (Media Create, Famitsu)
Ace Combat Zero: 109,000 - 124,000 - 117,000 - (Dengeki, Media Create)
Sengoku Musou 2 - 327,000 - 308,000 - 368,000 - (Famitsu, Dengeki)
Suikoden V - 135,000 - 124,000 - 122,000 - (Media Create, Famitsu)
Monster Hunter 2 - 362,000 - 313,000 - 368,000 - (Famitsu, Dengeki)
Final Fantasy VII DoC - 392,000 - 392,000 - 365,000 - (Media Create, Famitsu) / (Dengeki, Famitsu)
English Training DS - 242,000 - 261,000 - 232,000 -(Dengeki, Famitsu)
Shin Onimusha: Dawn of Dreams - 226,000 - 232,000 - 189,000 - (Dengeki, Media Create)
Kingdom Hearts II - 728,000 - 517,000 - 738,000 - (Dengeki, Media Create)
Naruto: Narutimate Hero 3 - 217,000 - 204,000 - 194,000 - (Media Create, Dengeki)
Tales of the Abyss - 405,000 - 355,000 - 368,000 - (Media Create, Dengeki)
Mario Kart DS - 219,000 - 220,000 - 224,000 - (Famitsu, Media Create)
Rogue Galaxy - 238,000 - 225,000 - 213,000 - (Media Create, Famitsu)
Biohazard 4 (PS2) - 243,000 - 242,000 - 234,000 - (Media Create, Famitsu)
Monster Hunter Portable - 114,000 - 129,000 - 118,000 - (Dengeki, Media Create)
Animal Crossing WW - 325,000 - 362,000 - 335,000 - (Dengeki, Media Create)
Pokemon Dungeon Blue - 124,000 - 153,000 - 135,000 - (Dengeki, Media Create)
Pokemon Dungeon Red - 113,000 - 130,000- 121,000 - (Dengeki, Media Create)
Daito Pachislot - 96,000 - 107,000 - 94,000 - (Dengeki, Famitsu)
Dare Demo Asobi Taizen - 58,000 - 58,000 - 67,000 - (Famitsu, Media Create) / (Famitsu, Dengeki)
Tales of Legendia - 242,000 - 246,000 - 242,000 - (Dengeki, Famitsu) / (Dengeki, Media Create)
Gundam CE - 118,000 - 133,000 - 124,000 - (Dengeki, Media Create)
Jump Superstars - 221,000 - 207,000 - 202,000 - (Media Create, Famitsu)
Winning Eleven 9 - 574,000 - 555,000 - 552,000 - (Media Create, Famitsu)
Pokemon XD - 103,000 - 114,000 - 101,000 - (Dengeki, Famitsu)
Grandia III - 122,000 - 121,000 - 134,000 - (Famitsu, Dengeki)
Summon Night Ecstacy - 95,000 - 102,000 - 100,000 - (Dengeki, Media Create)
Armored Core: Last Raven - 49,000 - 43,000 - 48,000 - (Famitsu, Dengeki)
Jikkyou Powerful Pro Baseball 12 - 277,000 - 277,000 - 249,000 - Famitsu lowest
Ape Escape 3 - 95,000 - 102,000 - 93,000 - (Dengeki, Famitsu)
Onimusha 3 Pachislot - 67,000 - 46,000 - 60,000 - (Media Create, Dengeki)
Naruto 2 RPG - 37,000 - 38,000 - 31,000 - (Dengeki, Famitsu)
DDR with Mario - 20,000 - 22,000 - 23,000 - (Famitsu, Media Create)
Gundam: True Odyssey - 86,000 - 92,000 - 64,000 - (Dengeki, Famitsu)
Genji - 70,000 - 86,000 - 73,000 - (Dengeki, Media Create)
Yawaraka Atamajuku - 73,000 - 60,000 - 53,000 - (Media Create, Famitsu)
Mushiking (GBA) - 208,000 - 234,000 - 222,000 - (Dengeki, Media Create)
Chibi Robo - 29,000 - 37,000 - 31,000 - (Dengeki, Media Create)
Killer 7 - 11,000 - 15,000 - 14,000 - (Dengeki, Media Create)
Nintendogs - 182,000 - 168,000 - 136,000 - (Media Create, Famitsu)
Kirby Canvas Curse - 75,000 - 97,000 - 72,000 - (Dengeki, Famitsu)

Stats: Company - Most - Middle - Fewest - Tie

Dengeki - 21 - 11 - 7 - 1
Famitsu - 7 - 16 - 17 - 0
Media Create - 12 - 11 - 14 - 2

Analysis: Assuming I counted correctly (someone doublecheck), in the last 12 months, Dengeki was the highest counter for the games polled in this list. Famitsu is the lowest, with Media Create in the middle. Famitsu had the most low counts, while Famitsu got the most middle range counts -- which means a lot when averaging between the three is done. Dengeki, however, is more prone to large errors - see Mother 3 (I know they fixed Kingdom Hearts II, but still.)

BTW, these are first four or so day sales -- the first times these games chart, in their release week.

 

[ahmad]

I love sales threads

 

[jamesinclair]

Whoever gives Nintendo the highest wins. Go and count that

 

[littlewig]

The lack of PSP software in the list can't be good news for Sony...

Anyway, it's good to see my suspicion on the matter being backed up with some hard numbers. I have always thought Famitsu lowballed their numbers, and Dengeki being off the charts on most titles.

 

[cvxfreak]

The lack of PSP software can't be good news for Sony...

Anyway, it's good to see my suspicion on the matter being backed up with some hard numbers. I have always thought Famitsu lowballed their numbers, and Dengeki off the charts on most titles.

Apparently not too many PSP games have charted high enough for Dengeki to say something about them in their summaries. All I see is Monster Hunter Portable, maybe another game.

Someone is free to group them by platform.

 

[Dalthien]

Interesting to see you collate the data in that way.

Your analysis matches my own observations from the past couple of years. I've always noticed that Dengeki was often on the high end compared with Media Create and Famitsu, which is why I've put more focus on the Media Create and Famitsu numbers. It's good to see that my observations are basically supported by your data. Good work.

 

[ioi]

Interesting to see you collate the data in that way.

Your analysis matches my own observations from the past couple of years. I've always noticed that Dengeki was often on the high end compared with Media Create and Famitsu, which is why I've put more focus on the Media Create and Famitsu numbers. It's good to see that my observations are basically supported by your data. Good work.

Why? You're saying that just because Dengeki are higher they are wrong?

I'm not necessarily saying I disagree by the way, but would be interested in your reasoning...

 

[lancubap]

I love sales threads

Me too. It will be interesting to know why there are these differences between Dengeki and Media Create. I mean, what changes in their methods ?

 

[Shard]

It must be nice to have that much free time on ones hands.

 

[Roders5]

It must be nice to have that much free time on ones hands.

You sound like my folks when I am gaming.

 

[cvxfreak]

Why? You're saying that just because Dengeki are higher they are wrong?

I'm not necessarily saying I disagree by the way, but would be interested in your reasoning...

I never remembered Dengeki being this high in the past. I specifically remember them lowballing the first two GC Biohazard games, and then there was the mistake with Kingdom Hearts II.

 

[Galactic Fork]

Is there any indication of company bias?

 

[cvxfreak]

Is there any indication of company bias?

I really don't think so. Each tracker has a Nintendo game in the list tracked highest (Media Create with Pokemon Ranger, Dengeki with English Training DS and Famitsu with Mario Kart DS). Same with Capcom, in which Biohazard 4, Shin Onimusha and Monster Hunter 2's highest trackers were the three companies respectively.

It just shows the volitility of Japanese tracking data. I'm all for a unified service someday, but it doesn't seem possible.

 

[ioi]

I never remembered Dengeki being this high in the past. I specifically remember them lowballing the first two GC Biohazard games, and then there was the mistake with Kingdom Hearts II.

Dengeki has always been much higher than Famitsu and Media Create of which MC usually slightly outdoes Famitsu in early weeks but then seems to fall behind in subsequent weeks. I don't know why these trends exists but they do, and they are the basis of much of what is done on Everything and Nothing.

 

[cvxfreak]

Dengeki has always been much higher than Famitsu and Media Create of which MC usually slightly outdoes Famitsu in early weeks but then seems to fall behind in subsequent weeks. I don't know why these trends exists but they do, and they are the basis of much of what is done on Everything and Nothing.

Always?

Other examples I can remember: Super Mario Sunshine less than 300K, and of course Wind Waker.

Maybe Dengeki just isn't as good in tracking popular titles?

 

[Dalthien]

Why? You're saying that just because Dengeki are higher they are wrong?

I'm not necessarily saying I disagree by the way, but would be interested in your reasoning...

Well, to be honest, I really have no idea which tracking firm is right or wrong the most. But from my own personal observations over the past couple of years, I've noticed that in general Famitsu and Media Create tend to track fairly closely, especially over the long-term, whereas whenever a sizable discrepancy tends to exist between the three, it usually tends to be Dengeki that is off from the other two (usually higher than the other two). Obviously, there are cases where each tracking firm is off from the others, but that has been my general observation in the past.

Given that Dengeki (of the three) usually seems to be the outlier, then I've sort of gone on the basis that it is probably more likely that the other two are correct, and that Dengeki is wrong, rather than assuming that Dengeki is right, and the other two are wrong. Just probabilities - if I go on the assumption that these firms are generally pretty good at tracking sales - then in a disputed case where one of the three firms is off from the other two, it seems reasonable to assume that two of the three would be correct and only one of the three would be incorrect. If I go on the assumption that two of the three were incorrect, then there's not much point in following the sales data anyway, because I am going on the assumption that these firms really aren't very good at tracking sales after all.

Anyway, as I said, these have been my observations in the past, and because of that, I tend not to pay much attention to Dengeki. If they are reporting data that is not available from Media Create or Famitsu, then I'll use it as a basis for that data, but I tend to give preference to MC or Famitsu.

The data set which cvxfreak used is obviously pretty small, and not necessarily indicative of the full picture, but his data pretty much confirms my own observations. This a just a quick and crude analysis, but just using his data, and assigning a value of 100 to the highest sales, 50 to the middle (and a tie), and 0 to the lowest - then the average for each would be

Dengeki - 21 - 11 - 7 - 1 = 2700/40 = 67.5
Famitsu - 7 - 16 - 17 - 0 = 1500/40 = 37.5
Media Create - 12 - 11 - 14 - 2 = 1850/39 = 47.4

As I said, this is not a real statistical analysis of the data, but I'm not about to spend the time to do that either. Also, I didn't double check cvxfreak's numbers either. There seems to be a data point missing for Media Create, and I noticed that Famitsu had a tie which wasn't represented, but that would only serve to bring Famitsu closer to the mean anyway. But it gives a rough idea of where the three firms tend to fall relative to each other. Dengeki is the farthest from the mean, whereas Media Create tends to fall almost exactly on the mean. In any case, if I am going to discount one of the three firms, it will be the one which seems to be the furthest away from the other two since I have to go on the assumption that these firms are fairly good at what they do.

Anyway, I kind of rambled a bit, but hopefully that helped to answer your question.

 

[ioi]

Well, to be honest, I really have no idea which tracking firm is right or wrong the most. But from my own personal observations over the past couple of years, I've noticed that in general Famitsu and Media Create tend to track fairly closely, especially over the long-term, whereas whenever a sizable discrepancy tends to exist between the three, it usually tends to be Dengeki that is off from the other two (usually higher than the other two). Obviously, there are cases where each tracking firm is off from the others, but that has been my general observation in the past.

Given that Dengeki (of the three) usually seems to be the outlier, then I've sort of gone on the basis that it is probably more likely that the other two are correct, and that Dengeki is wrong, rather than assuming that Dengeki is right, and the other two are wrong. Just probabilities - if I go on the assumption that these firms are generally pretty good at tracking sales - then in a disputed case where one of the three firms is off from the other two, it seems reasonable to assume that two of the three would be correct and only one of the three would be incorrect. If I go on the assumption that two of the three were incorrect, then there's not much point in following the sales data anyway, because I am going on the assumption that these firms really aren't very good at tracking sales after all.

Anyway, as I said, these have been my observations in the past, and because of that, I tend not to pay much attention to Dengeki. If they are reporting data that is not available from Media Create or Famitsu, then I'll use it as a basis for that data, but I tend to give preference to MC or Famitsu.

The data set which cvxfreak used is obviously pretty small, and not necessarily indicative of the full picture, but his data pretty much confirms my own observations. This a just a quick and crude analysis, but just using his data, and assigning a value of 100 to the highest sales, 50 to the middle (and a tie), and 0 to the lowest - then the average for each would be

Dengeki - 21 - 11 - 7 - 1 = 2700/40 = 67.5
Famitsu - 7 - 16 - 17 - 0 = 1500/40 = 37.5
Media Create - 12 - 11 - 14 - 2 = 1850/39 = 47.4

As I said, this is not a real statistical analysis of the data, but I'm not about to spend the time to do that either. Also, I didn't double check cvxfreak's numbers either. There seems to be a data point missing for Media Create, and I noticed that Famitsu had a tie which wasn't represented, but that would only serve to bring Famitsu closer to the mean anyway. But it gives a rough idea of where the three firms tend to fall relative to each other. Dengeki is the farthest from the mean, whereas Media Create tends to fall almost exactly on the mean. In any case, if I am going to discount one of the three firms, it will be the one which seems to be the furthest away from the other two since I have to go on the assumption that these firms are fairly good at what they do.

Anyway, I kind of rambled a bit, but hopefully that helped to answer your question.

Yes, I'm playing Devil's Advocate a bit I guess, as people do with the numbers I use often on my site and so forth.

It's hard to say "this value is furthest from the mean" when the mean is made up of three values of which that is one of them. For what it's worth I've said before that I think Media Create and Famitsu tend to track very closely to one another on the whole, but if I'm honest when there does seem to be a major difference between what a publisher says and what the tracking services say it tends to be Dengeki that are closest to getting it right.

 

[Dalthien]

Yes, I'm playing Devil's Advocate a bit I guess, as people do with the numbers I use often on my site and so forth.

It's hard to say "this value is furthest from the mean" when the mean is made up of three values of which that is one of them. For what it's worth I've said before that I think Media Create and Famitsu tend to track very closely to one another on the whole, but if I'm honest when there does seem to be a major difference between what a publisher says and what the tracking services say it tends to be Dengeki that are closest to getting it right.

Yeah - I can't argue what you're saying. There's no way of really knowing for sure. But if I assume that all three are pretty good at tracking sales, then it tends to follow that it is most likely that the worst of the three would be the one which strays the most from the other two. My analysis above was admittedly very crude. A true analysis would have to actually look at the actual differences from the mean for each data point, instead of just assigning a uniform value for each highest and lowest value, regardless of how far off each value actually is from the mean for that title. But to be honest, that is more work than I am really willing to do at the moment, and I suspect that it would tend to match the crude analysis that I already performed. But yeah, it could be possible that Dengeki is the most accurate of the three - I can't say for certainty either way. But given how closely Famitsu and Media Create tend to track most of the time, I tend to suspect that they are closer to the truth on most occasions. Plus, the fact that Nintendo tends to reference Media Create data from time-to-time suggests that Nintendo must have some confidence in the overall reliability of MC's data.

 

[cvxfreak]

Here are a few more 2001 year total sales between Famitsu and Dengeki... no Media Create, though. Famitsu Left, Dengeki Right.

Final Fantasy X - 2,264,000 - 2,434,000 (Dengeki)
Gran Turismo 3 - 1,438,000 - 1,657,000 (Dengeki)
Dragon Quest IV - 1,006,000 - 1,039,000 (Dengeki)
Onimusha - 928,000 - 983,000 (Dengeki)
Super Smash Bros. DX - 799,000 - 838,000 (Dengeki)
Minna no Golf 3 - 797,000 - 871,000 (Dengeki)
Metal Gear Solid 2 - 688,000 - 751,000 (Dengeki)
Devil May Cry - 555,000 - 617,000 (Dengeki)
Gundam: Fed. vs Zion DX - 585,000 - 654,000 (Dengeki)
Luigi Mansion - 272,000 - 307,000 (Dengeki)

Shin Sangoku Musou 2 - 716,000 - 727,000 (Famitsu)
Super Mario Advance - 772,000 - 748,000 (Famitsu)
Mario Kart Advance - 685,000 - 567,000 (Famitsu)
F-Zero For GBA - 334,000 - 244,000 (Famitsu)

Capcom vs. SNK 2 - 277,000 - 277,000 (tie)

Etc...

 

[Dalthien]

Game Title - Media Create - Dengeki - Famitsu - (Most, Least)

Kingdom Hearts II - 728,000 - 517,000 - 738,000 - (Dengeki, Media Create)

cvxfreak,

I don't remember - is that KHII number for Dengeki the corrected number, or is that the initial incorrect number? Or did they just correct it by adjusting their totals in the following week or two?

 

[Rock_Man]

Is there any indication of company bias?

Tear Ring Saga Series: Berwick Saga published by Enterbrain a year ago is obviously suspicious.
Famitsu (=Enterbrain), 108k
Media Create, 85k

 

[Dalthien]

Damn you ioi - now you went ahead and made me feel as though I had to finish my analysis from before.

Anyway, I doubt anyone will care about this except cvxfreak and ioi - and actually, they may not even care, but I went ahead and did a comparison of cvxfreak's data.

The ideal solution would have been to compare each tracking firm's data with the actual amount sold for each game to find out who is the most accurate, but since we don't know the actual sales numbers for the first week for any of these games, such a comparison is impossible.

The next best comparison that I could make was to compare the tracking numbers from each firm relative to each other. I calculated the mean (of the 3 tracking firms) sales numbers for each title, and then calculated the % difference of each tracking firm from the mean for each title. This method still doesn't provide any proof of which firm is the most accurate, but it does show (based upon cvxfreak's set of data) which firm tends to stay the closest to the mean, and which firm strays the farthest away from the other two firms.

Based upon the data,

The average % difference from the mean for the full set of data is:

Dengeki - 6.53%
Media Create - 5.85 %
Famitsu - 5.38 %

Note that this shows that Dengeki is the tracking firm which is the most different from the other two. It also shows that Famitsu is the closest to the mean, or that Famitsu deviates the least from the other firms. Note also that although these three percentages seem fairly close together, they really represent larger differences due to the fact that only 3 tracking firms are used for the calculation. Whenever one firm is significantly different from the other two firms, the percentage differences for the other two firms are also increased (albeit to a lesser degree) due to the mean being skewed due to the outlier data point. Therefore, the effect of the outlier data point is minimized in terms of a percentage difference, because all three firms have increased percentages, but the outlier firm will still have the largest percentage difference. Anyway, that all sounds very technical, but it basically just means that the difference between the three firms (relative to each other) isn't really as close as it appears to be.

Looking at the data another way, I also looked at the number of games that each firm was the farthest from the mean.

Farthest from the mean:

Dengeki - 17 games
Media Create - 13 games
Famitsu - 12 games

Again, this shows that Dengeki is the firm that is usually tracking differently from the other two.

 

[cvxfreak]

cvxfreak,

I don't remember - is that KHII number for Dengeki the corrected number, or is that the initial incorrect number? Or did they just correct it by adjusting their totals in the following week or two?

Initial incorrect number.

I can never find the correct one, sadly, but I think it was the lowest either way. The quotation should read "Famitsu, Dengeki". I was too tired to double check last night. >_<