• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Media Create Sales: 31 July - 6 August

CoolTrick

Banned
I really, seriously am not bothered by the work that ioi does and I'm happy to take his numbers when I just want some kind of rough estimate on something, but it's simply not accurate at all to say that his process, or final numbers, have the same legitimacy as those from the professional tracking firms. The simple fact that he generally uses their numbers as a starting point should, in itself, be enough to highlight this.

Agreed.
 
Barf_the_Mog said:
Please please please stop arguing over ioi's website. This fight is so ****ing annoying and uninteresting.

Amen. His figures are relatively accurate but far from precise. Deal with it. At least his methodology is transparent.
 
Ok, looks like I won't post in this jp sales threads anymore. Keep the lame discussion about an anonymous website spammed all over these threads, thank you all.
 
There IS a huge discrepancy between Media Create and Nintendo's sales numbers, especially when speaking of Brain Training. How could they be off by almost 400,000 units?

Famitsu is no better, unless they are tracking MORE stores, how could they ALWAYS have higher sales figures than Media Create? It's very upsetting because you never get the ACCURATE numbers. And if you use company announcements (like Nintendos for Brain Training for instance), you get a huge contrast between sales you've been tracking since it's release and what the producers say.

Blech. Keep up the averaging ioi, I like the charts, tho I can't look at them.
 
Hmm, MediaCreate only tracks down sales of games through retail chains, right? Is it possible that Nintendo is also selling their own games themselves (from their stores and online sites)?
 

CoolTrick

Banned
Please please please stop arguing over ioi's website. This fight is so ****ing annoying and uninteresting.

And you know what, personally I was tired of the retarded Monorojo spin offs because people seemed to think it was fun to create 5 page threads worth of pointless DS-PSP bickering that ultimately is among the most pointless discussion to be found on this forum. That was completely annoying and uninteresting after about the fifth Media Create thread.

You may not like the discussion over ioi's numbers, but...

If ioi's posts about his charts are on topic, there's no reason that further discussion on them also isn't on topic. I guess a mod will eventually have to clarify, but if vgcharts.org is allowed to be paraded around this forum as much as it is, I personally see no reason why we can't discuss them. Afterall, they are still sales numbers in a sales thread.

Ioi put a lot of time into his site which is great but just because he's able to pull long explanations for his figures out of a hat and spent a lot of time, doesn't mean we shouldn't be highly critical of the site's credibility if he's going to go around referencing it in every single sales topic.
 

Bo130

Member
What is Media-Create doing anyway? I want a new chart.

I guess they're still stunning by seeing a software for the PSP on top last week :lol
 
I remember reading that the first shipment of FFIII will only be 450k. Do people guessing 500k and up have an updated figure for shipment?
 

Nickel85

Member
And you know what, personally I was tired of the retarded Monorojo spin offs because people seemed to think it was fun to create 5 page threads worth of pointless DS-PSP bickering that ultimately is among the most pointless discussion to be found on this forum. That was completely annoying and uninteresting after about the fifth Media Create thread.

You may not like the discussion over ioi's numbers, but...

If ioi's posts about his charts are on topic, there's no reason that further discussion on them also isn't on topic. I guess a mod will eventually have to clarify, but if vgcharts.org is allowed to be paraded around this forum as much as it is, I personally see no reason why we can't discuss them. Afterall, they are still sales numbers in a sales thread.

Ioi put a lot of time into his site which is great but just because he's able to pull long explanations for his figures out of a hat and spent a lot of time, doesn't mean we shouldn't be highly critical of the site's credibility if he's going to go around referencing it in every single sales topic.

Thats totally true, buts its really annoying to read the same posts in every media create thread every weak! And its not that the discussions about his numbers get any further! There are the same people who have the same arguments! Either we find a solution or this will go week over week....
 
CoolTrick said:
If ioi's posts about his charts are on topic, there's no reason that further discussion on them also isn't on topic.

What if you wrote up a single, comprehensive post about why ioi is a baby-eater, and then when he posts his numbers every week he agreed to link to that in addition to the numbers on his website? Then could we stop having this argument every week?
 

CoolTrick

Banned
Are you kidding me? This happened in maybe 2-3 threads. If you can take Monorojo like bickering for months and months on end from him and other wannabes, I think this forum can handle some ioi criticalness.
 

CoolTrick

Banned
What if you wrote up a single, comprehensive post about why ioi is a baby-eater, and then when he posts his numbers every week he agreed to link to that in addition to the numbers on his website? Then could we stop having this argument every week

I'd stop if ioi would just have a damn disclaimer and stopped acting (and misleading people) that his numbers are as reliable as NPD/MC.
 

Yoshi

Headmaster of Console Warrior Jugendstrafanstalt
CoolTrick said:
And you know what, personally I was tired of the retarded Monorojo spin offs because people seemed to think it was fun to create 5 page threads worth of pointless DS-PSP bickering that ultimately is among the most pointless discussion to be found on this forum. That was completely annoying and uninteresting after about the fifth Media Create thread.

You may not like the discussion over ioi's numbers, but...

If ioi's posts about his charts are on topic, there's no reason that further discussion on them also isn't on topic. I guess a mod will eventually have to clarify, but if vgcharts.org is allowed to be paraded around this forum as much as it is, I personally see no reason why we can't discuss them. Afterall, they are still sales numbers in a sales thread.

Ioi put a lot of time into his site which is great but just because he's able to pull long explanations for his figures out of a hat and spent a lot of time, doesn't mean we shouldn't be highly critical of the site's credibility if he's going to go around referencing it in every single sales topic.
His numbers aren't off topic, as well as your discussion, but your point of view is clear, iois point of view is clear and you two will not change your opinion. So there are two possibilities:
1) Every week you post that you don't like ioi's numbers and want him to stop posting them, therefore creating a huge buzz about it, although you want to prevent that postings about his numbers are numerous in each thread
2) You just get over it, everone knows how ioi's numbers are made and so everone can decide on his or her own, whether to read them or not. I like his postings, because they give a good view on the general sales pattern in history. Ioi also has quite a huge set of numbers by Media Create and others and is helpful for questions like mine earlier in this thread.

I'd suggest calming down and choosing option number 2.
 
People don't just "get over things", it takes a while and a little more bickering AFTER they are told to calm down.

Anyways, isn't Final Fantasy III coming out next next monday? What a time this will be

Edit: IT is the 24th or 26th isn't it? I was thinking of SDSU...
 

Burnst

Member
donny2112 said:
I thought it was August 24th or 26th. The 24th is a Thursday (good), but the 26th sticks in my mind for some reason.
24th, will be interesting to see the FFIII numbers and what it does for the DSL numbers. Did Nintendo say how many DS's they where reserving for the release?
 

jesusraz

Member
parathod said:
I remember reading that the first shipment of FFIII will only be 450k. Do people guessing 500k and up have an updated figure for shipment?

No, but Nintendo quickly upped its shipment of NSMB to compensate for the extra demand and I have no doubt S-E will do the same :)
 

ioi

Banned
cvxfreak said:
Those Famitsu numbers are actual shipments from Nintendo and not estimates.

I'm not attempting to bring this up again, but for my own interest and so forth, I am pretty sure that they are not from the publisher.

I actually think that this week the conversation has been productive in so much that the top 50 famitsu document that someone referenced is very useful for my data, and actually supports the methods used very well.

I know a lot of people are fed up with these arguments, as am I, mainly because they are so lame and pathetic. If people actually want a serious discussion / critique about vgcharts then maybe we should create a new thread and discuss. As I think people like CoolTrick do it for no other reason than to troll and knock the site (probably out of jealousy) then I doubt this will happen. But for those who may want to discuss it properly then I'm happy to do so. I'm not going to repeat myself over and over making the same point though.
 

MrSardonic

The nerdiest nerd of all the nerds in nerdland
CoolTrick said:
I think people can admit that ioi and his little vgcharts.org are more than a single one-post presence in these threads.

stop being so obtuse, stop attempting to derail this thread, stop patronisingly over-using :lol in so many of your responses, and stop trying to belittle ioi's charts/numbers when many people are interested in their contribution and fully aware of the nature of the information.

no one needs you to "enlighten" us.
 

ioi

Banned
I'm not trying to derail this thread, and I don't want to cause a massive argument, but once and for all I want to use a real example with numbers to show how the data in VGCharts is produced, and anytime that someone tried to argue (Jonnyram, CoolTrick, A Link to the Past or whoever their successor may be) then someone can point them to this post.

Let's use Brain Training (Brain Age).


- Nintendo released a fancy press release about Brain Age that as of the 6th of August the game had sold 3m in Japan. They actually mean shipped most likely, and since other numbers (500k Europe, 600k NA) were quoted to nearest 100k, it's likely that the actual shipment figure was 2.95 - 3.05m as of the 6th August.

- As of the 6th August, total sales for the game stood at 2.65m according to Media Create and 2.61m according to Famitsu.

- The document referenced on the previous page shows that Famitsu's estimated shipped and sold numbers for Brain Training as of the 16th July were:

DS 東北大学未来科学技術共同研究センター2,561,530 2,506,497 97.9%

Showing a 97.9% sell through rate and 'stock' of around 55k, or between 1 and 2 weeks sales.

- Assuming a similar value for stock on the 6th August (totally reasonable looking at that document and seeing that all games seem to have around 1-2 weeks worth of sales difference between the shipped and sold figure) would give stock of around 60k (taking it a little on the high side).

- Now, only a fool would disagree that we have a number of discrepancies here. Using Nintendo's 2.95-3.05m shipped number and assuming the 60k stock we get from Famitsu, we get a sell through of 2.89-2.99m.

- Take even the lowest figure (as I'm sure you will all want to) and you have:

Nintendo shipped - 'stock': 2.89m
Media Create: 2.65m
Famitsu: 2.61m

- Nintendo's shipped number is ACTUAL. They will keep a record of each game that goes out of the door.

- Famitsu and Media Create's numbers are estimates. Their best guesses of how much the game has sold by covering a certain amount of retailers (around 50% each I think) and extrapolating their sampled sales up to estimate the sales across the whole country.

- Which one do you think is most accurate? Data that comes from 100% of all games, or data from 50% that is scaled up? Data that will probably miss online sales, non-gaming store sales etc and guess to try and account for them?

- Divide 2.89 by 2.63 (the Famitsu and Media Create average) and you get 1.099, so 10% higher. The same figure that I have been using for the last year to scale Brain Training up by. Magic.

VGCharts said:
Brain Training 35,250 / 2,896,000

As far as I am concerned, the data is as accurate a representation as physically possible of how many copies of a game have actually been sold. I completely stand by the methods and the reasoning behind them.

And when it comes to the question of an explanation on the site as to how they are arrived at:

1) I am working on one
2) It is difficult to explain the above without using numerical examples and unfortunately since every game is a slightly different case it is virtually impossible for me to produce a 'cover-all' explanation.
3) I don't, and never have had, any intention of misleading anyone or so forth. 99% of the data on the site comes from methods just like shown above, using a variety of sources to produce what I think is the most accurate data overall.
4) I'm not trying to 'compete' with NPD / Famitsu etc, my data is BASED ON theirs!! All I am doing is using multiple sources to come to the most accurate conclusions possible, since they all disagree slightly with one another.
5) If you don't understand this then please don't argue with me as it is obviously over your head and anyone saying my data is "estimated", "made up", "bullshit", "inaccurate" or otherwise has no idea of what they are talking about.

Thank you and goodnight! (it's 2am here and I need my beauty sleep)
 

Nutter

Member
MrSardonic said:
stop being so obtuse, stop attempting to derail this thread, stop patronisingly over-using :lol in so many of your responses, and stop trying to belittle ioi's charts/numbers when many people are interested in their contribution and fully aware of the nature of the information.

no one needs you to "enlighten" us.
IAWTP!

And if it really bothers you that much put him on ignore or something. There are people who enjoy his graphs and his site.

I dont really care for his numbers but you people are taking this shit to a new level of stupidity.
 

heidern

Junior Member
CoolTrick said:
I'd stop if ioi would just have a damn disclaimer and stopped acting (and misleading people) that his numbers are as reliable as NPD/MC.
You know, you should just stop anyway because of the fact that you simply don't understand his numbers.
 
ioi said:
I'm not trying to derail this thread, and I don't want to cause a massive argument, but once and for all I want to use a real example with numbers to show how the data in VGCharts is produced, and anytime that someone tried to argue (Jonnyram, CoolTrick, A Link to the Past or whoever their successor may be) then someone can point them to this post.

Let's use Brain Training (Brain Age).


- Nintendo released a fancy press release about Brain Age that as of the 6th of August the game had sold 3m in Japan. They actually mean shipped most likely, and since other numbers (500k Europe, 600k NA) were quoted to nearest 100k, it's likely that the actual shipment figure was 2.95 - 3.05m as of the 6th August.

- As of the 6th August, total sales for the game stood at 2.65m according to Media Create and 2.61m according to Famitsu.

- The document referenced on the previous page shows that Famitsu's estimated shipped and sold numbers for Brain Training as of the 16th July were:



Showing a 97.9% sell through rate and 'stock' of around 55k, or between 1 and 2 weeks sales.

- Assuming a similar value for stock on the 6th August (totally reasonable looking at that document and seeing that all games seem to have around 1-2 weeks worth of sales difference between the shipped and sold figure) would give stock of around 60k (taking it a little on the high side).

- Now, only a fool would disagree that we have a number of discrepancies here. Using Nintendo's 2.95-3.05m shipped number and assuming the 60k stock we get from Famitsu, we get a sell through of 2.89-2.99m.

- Take even the lowest figure (as I'm sure you will all want to) and you have:

Nintendo shipped - 'stock': 2.89m
Media Create: 2.65m
Famitsu: 2.61m

- Nintendo's shipped number is ACTUAL. They will keep a record of each game that goes out of the door.

- Famitsu and Media Create's numbers are estimates. Their best guesses of how much the game has sold by covering a certain amount of retailers (around 50% each I think) and extrapolating their sampled sales up to estimate the sales across the whole country.

- Which one do you think is most accurate? Data that comes from 100% of all games, or data from 50% that is scaled up? Data that will probably miss online sales, non-gaming store sales etc and guess to try and account for them?

- Divide 2.89 by 2.63 (the Famitsu and Media Create average) and you get 1.099, so 10% higher. The same figure that I have been using for the last year to scale Brain Training up by. Magic.

As questionable as the method may be to some, it is ingenious and brilliant.
 

cvxfreak

Member
ioi said:
I'm not attempting to bring this up again, but for my own interest and so forth, I am pretty sure that they are not from the publisher.

No, they are, in respect to Famitsu's coverage.

EDIT: Put CoolTrick on ignore.
 
CoolTrick said:
I'd stop if ioi would just have a damn disclaimer and stopped acting (and misleading people) that his numbers are as reliable as NPD/MC.
But say I had my own numbers which were the average of MC, Dengeki and Famitsu. Then they would be more accurate than at least one of them surely? If you knew how these groups worked then maybe you'd understand ioi's numbers are very good globally on accuracy, if a little less internally consistent than those of trackers.
 

Monk

Banned
So ioi's numbers try to get shipped numbers instead of actual sold to consumers numbers. Interesting.

ioi, Do you do this for all software regardless of which publisher?
 

Archie

Second-rate Anihawk
These weekly arguments about ioi's numbers makes me miss the good ole days when everybody laughed at Monorojo for trying so hard to spin PSP numbers.
 

Deku

Banned
LanceStern said:
As questionable as the method may be to some, it is ingenious and brilliant.


It's mathematically logical. What ioi is trying to capture is a variable in an equation by looking at the differentials between Nintendo's numbers and published industry total and constructing an equation as well as capturing a mystery variable 'x' which in term could be a counpound number of several other variables (a, b, c, etc). The equation and the variable are guesses based on published figuress.

He's basically building a kind of forecasting model. If his model is correct, we should be able to take data form any point in time (both the past and the future), plug it in, and the result should be pretty consistent with the real numbers throhg time. And we won't know how accurate his model is until we get more data in our datasets or until Nintendo reveals more data.

This isn't much different than what economists do the predict certain variables in the economy, like say, the interest rates on certain financial instruments. Granted we know a bit more about the financial markets due to the volumes of data we have for it, but its the same idea. There's no official LED display that keeps track of interest rates, they are calculated based on forumals/models which make certain assumptions about the markets.
 

cvxfreak

Member
Deku said:
It's mathematically logical. What ioi is trying to capture is a variable in an equation by looking at the differentials between Nintendo's numbers and published industry total and constructing an equation as well as capturing a mystery variable 'x' which in term could be a counpound number of several other variables (a, b, c, etc). The equation and the variable are guesses based on published figuress.

He's basically building a kind of forecasting model. If his model is correct, we should be able to take data form any point in time (both the past and the future), plug it in, and the result should be pretty consistent with the real numbers throhg time. And we won't know how accurate his model is until we get more data in our datasets or until Nintendo reveals more data.

This isn't much different than what economists do the predict certain variables in the economy, like say, the interest rates on certain financial instruments. Granted we know a bit more about the financial markets due to the volumes of data we have for it, but its the same idea. There's no official LED display that keeps track of interest rates, they are calculated based on forumals/models which make certain assumptions about the markets.

Pretty much. I'm fairly sure this goes on when coming up with the numbers ioi bases his calculations on, so there's double dipping going on. Still, people should just come to accept his conclusions and choose to use them or not.
 

nfreakct

Member
Deku said:
It's mathematically logical. What ioi is trying to capture is a variable in an equation by looking at the differentials between Nintendo's numbers and published industry total and constructing an equation as well as capturing a mystery variable 'x' which in term could be a counpound number of several other variables (a, b, c, etc). The equation and the variable are guesses based on published figuress.

He's basically building a kind of forecasting model. If his model is correct, we should be able to take data form any point in time (both the past and the future), plug it in, and the result should be pretty consistent with the real numbers throhg time. And we won't know how accurate his model is until we get more data in our datasets or until Nintendo reveals more data.

This isn't much different than what economists do the predict certain variables in the economy, like say, the interest rates on certain financial instruments. Granted we know a bit more about the financial markets due to the volumes of data we have for it, but its the same idea. There's no official LED display that keeps track of interest rates, they are calculated based on forumals/models which make certain assumptions about the markets.

If ioi's explanation is correct and not some greatly simplified version of what he's actually doing, then his numbers are just as good as made up. He has shown no real proof that his model is a predictive one and not a descriptive one. Essentially he's observing that shipped numbers from companies and numbers from the various tracking services display some discrepancies. He's attempting to look at the past (descriptive) and then take those same percentages and try to use them for future predictive modeling. Except nowhere has he shown in his explanation that this is a valid model for future use. It's a common fallacy in statistics (like baseball sabremetrics) that descriptive comparisons are equally valid in prediction, but rarely is that the case.

Additionally, it's far better to acknowledge the existance of noise in data (in this case the varying discrepancies between the tracking services) than attempt to create some "mystery variable x" and use that to eliminate the noise. The existence of noise is undesirable, but attempting to erase the statistical noise as if it never existed is even worse. In one case you're acknowledging that the "true" answer is unknown while in the other you're saying "this is the true answer" despite the fact that it may actually not be the true number. A false answer is worse than an unknown one since that false answer may lead to poor conclusions from the data.

This isn't even a rant about ioi, this is more about statistics in general. A larger sample size isn't necessarily better than a smaller sample size (in fact, in ioi's case he's using two seperate numbers that don't even track the same thing). Eliminating statistical noise without a valid model is less than worthless. And attempting to use descriptive statistics are predictive ones without evidence is just incorrect.
 

Fuzzy

I would bang a hot farmer!
I'm really glad I don't have the time to read all the posts going back and forth about ioi's numbers anymore.
 
nfreakct said:
If ioi's explanation is correct and not some greatly simplified version of what he's actually doing, then his numbers are just as good as made up. He has shown no real proof that his model is a predictive one and not a descriptive one. Essentially he's observing that shipped numbers from companies and numbers from the various tracking services display some discrepancies. He's attempting to look at the past (descriptive) and then take those same percentages and try to use them for future predictive modeling. Except nowhere has he shown in his explanation that this is a valid model for future use. It's a common fallacy in statistics (like baseball sabremetrics) that descriptive comparisons are equally valid in prediction, but rarely is that the case.

Additionally, it's far better to acknowledge the existance of noise in data (in this case the varying discrepancies between the tracking services) than attempt to create some "mystery variable x" and use that to eliminate the noise. The existence of noise is undesirable, but attempting to erase the statistical noise as if it never existed is even worse. In one case you're acknowledging that the "true" answer is unknown while in the other you're saying "this is the true answer" despite the fact that it may actually not be the true number. A false answer is worse than an unknown one since that false answer may lead to poor conclusions from the data.

This isn't even a rant about ioi, this is more about statistics in general. A larger sample size isn't necessarily better than a smaller sample size (in fact, in ioi's case he's using two seperate numbers that don't even track the same thing). Eliminating statistical noise without a valid model is less than worthless. And attempting to use descriptive statistics are predictive ones without evidence is just incorrect.
What? If you can show there is a consistent error aside from random noise (e.g. if Famitsu track 3% less than Dengeki consistently for a particular game), you think it's unreasonable to use that as a predictive measure? Bullshit, regardless of how obvious the reason for the discrepancy.
 

Deku

Banned
nfreakct said:
If ioi's explanation is correct and not some greatly simplified version of what he's actually doing, then his numbers are just as good as made up. He has shown no real proof that his model is a predictive one and not a descriptive one. Essentially he's observing that shipped numbers from companies and numbers from the various tracking services display some discrepancies. He's attempting to look at the past (descriptive) and then take those same percentages and try to use them for future predictive modeling. Except nowhere has he shown in his explanation that this is a valid model for future use. It's a common fallacy in statistics (like baseball sabremetrics) that descriptive comparisons are equally valid in prediction, but rarely is that the case.

Additionally, it's far better to acknowledge the existance of noise in data (in this case the varying discrepancies between the tracking services) than attempt to create some "mystery variable x" and use that to eliminate the noise. The existence of noise is undesirable, but attempting to erase the statistical noise as if it never existed is even worse. In one case you're acknowledging that the "true" answer is unknown while in the other you're saying "this is the true answer" despite the fact that it may actually not be the true number. A false answer is worse than an unknown one since that false answer may lead to poor conclusions from the data.

This isn't even a rant about ioi, this is more about statistics in general. A larger sample size isn't necessarily better than a smaller sample size (in fact, in ioi's case he's using two seperate numbers that don't even track the same thing). Eliminating statistical noise without a valid model is less than worthless. And attempting to use descriptive statistics are predictive ones without evidence is just incorrect.

Then refute his numbers with evidence. You don't have to undestand something completely to make reasonable predictions by simple observaiton. Economists does not understand many things in the financial markets and what exactly drives these markets, but it hasn't stopped them from making indexes and models to predict variables in them.

I'm not saying ioi is absolutely right, just that his methology isn't bunk. But if you have evidence his numbers are way off, then feel free to share with us.

I certainly reserve the right to say ioi's models are wrong or are bad models when I see evidence that support the claim. Heck, I think proving his numbers are wrong may be too hard. I'll lower the bar and ask people to simply prove his models are 'bad' models because he fails to take into account other variables. Otherwise this back and forth will never end and it just becomes a personal vandetta. Anyways I didn't realize I've been drawn into this long running discussion. So I'll make this my last on the matter.
 

cvxfreak

Member
The Friendly Monster said:
What? If you can show there is a consistent error aside from random noise (e.g. if Famitsu track 3% less than Dengeki consistently for a particular game), you think it's unreasonable to use that as a predictive measure? Bullshit, regardless of how obvious the reason for the discrepancy.

Here's something interesting:

Code:
[u]Shin Onimusha: Dawn of Dreams[/u]	
MC 226,000
F 189,000
[b]D 232,000[/b]

[u]Biohazard 4[/u]	
[b]MC 243,000[/b]
F 234,000
D 242,000

[u]Monster Hunter 2[/u]	
MC 362,000
[b]F 368,000[/b]
D 313,000

All three are Capcom games on the PS2 between December 2005 and Feburary 2006, so a span of three months. Notice the big winner among all of them.

ioi takes the game publisher in addition to the publisher into account.

Deku said:
I certainly reserve the right to say ioi's models are wrong or are bad models when I see evidence that support the claim. Heck, I think proving his numbers are wrong may be too hard. I'll lower the bar and ask people to simply prove his models are 'bad' models because he fails to take into account other variables. Otherwise this back and forth will never end and it just becomes a personal vandetta. Anyways I didn't realize I've been drawn into this long running discussion. So I'll make this my last on the matter.

Jonnyram explained in the last thread the hurdles ioi has to figure out/overcome/deal with in order to be perfect. And quite frankly no one can be perfect.
 

Monk

Banned
... If famitsu and media create use fact to extrapolate. And ioi uses the average of the two and scales up based on past discrepancies. Then why are his numbers less accurate? He doesnt say he is trying to eliminate noise. If the past can be considered a relatively accurate representationof how much is missed by famitsu and media create, then his number, more often than not are more accurate. He just has to revise the numbers every so often when accurate numbers by Nintendo themselves have been issued and continue from there.

His nintendo numbers at the very least will be accurate. Now if he used the same ratio to up other publishers games in which he has no idea how much the discrepancies are or how accurate famitsu and Media creates numbers are then it is wrong.


I think the best solution would be to put an asterisk beside Nintendos scaled up numbers and not use the scaled up numbers for game comparisons, just use the famitsu and media create averages.
 

ioi

Banned
nfreakct said:
If ioi's explanation is correct and not some greatly simplified version of what he's actually doing, then his numbers are just as good as made up. He has shown no real proof that his model is a predictive one and not a descriptive one. Essentially he's observing that shipped numbers from companies and numbers from the various tracking services display some discrepancies. He's attempting to look at the past (descriptive) and then take those same percentages and try to use them for future predictive modeling. Except nowhere has he shown in his explanation that this is a valid model for future use. It's a common fallacy in statistics (like baseball sabremetrics) that descriptive comparisons are equally valid in prediction, but rarely is that the case.

Additionally, it's far better to acknowledge the existance of noise in data (in this case the varying discrepancies between the tracking services) than attempt to create some "mystery variable x" and use that to eliminate the noise. The existence of noise is undesirable, but attempting to erase the statistical noise as if it never existed is even worse. In one case you're acknowledging that the "true" answer is unknown while in the other you're saying "this is the true answer" despite the fact that it may actually not be the true number. A false answer is worse than an unknown one since that false answer may lead to poor conclusions from the data.

This isn't even a rant about ioi, this is more about statistics in general. A larger sample size isn't necessarily better than a smaller sample size (in fact, in ioi's case he's using two seperate numbers that don't even track the same thing). Eliminating statistical noise without a valid model is less than worthless. And attempting to use descriptive statistics are predictive ones without evidence is just incorrect.


I appreciate your comments, but I think you've slightly misunderstood.

I'm not trying to 'eliminate noise' as you have put it. I am taking a number of individual cases and for each on seperately I'm adjusting the noisy and innacurate data using the accurate data we have. The only accurate data we have are the publishers shipped figures, so they are being used as a benchmark usually only in cases where we get the figures, or rarely in cases where a reasonable approximation can be made from which to adjust the original data. Errors are not accumulating, they are being eliminated.

And the final point is that I'm not saying my data is actual or perfect. Just that for a particular game, it is in my opinion the best possible guess that we have as to how much it sold.

When someone asks "is this the biggest week 12 ever?" then my best answer is yes. Not "well according to Famitsu yes, but according to Media Create no, but then according to shipped figures... blah" but given all the data and doing the best we can with it, my best answer is yes.
 
Top Bottom