You made a good post so I'll address your points in turn
Jonnyram said:
There's lots of good arguments being made both for and against ioi's theories. I also appreciate the effort that ioi puts into making all this happen, but my personal opinion is that his effort is wasted because of the inaccuracy of the data. This may be a clever way to avoid copyright issues, since his data doesn't match any of the research groups', but that's not my concern.
You've hit the nail on the head in some ways, avoiding copyright is obviously a major concern on a site such as VGCharts. And my personal feeling is that i don't want to just 'copy' other data, especially if in some cases I can see a flaw in it.
Jonnyram said:
The main cause of inaccuracy is the theory that shipped = sold. This is surely true for a number of games, but it is also totally untrue for a number of other games.
Agreed 100%. I think most games it is a fair assumption, but definitely not all.
Jonnyram said:
I'll show two opposite examples to make my point. One is the successful game with long legs that hangs around the bottom half of the top 50 for a few years. Shipped =/= sold in this case, because stores have a large amount of stock of said game. When the game does eventually stop selling, the stock remains in stores. You can go into a store in Japan and find multiple copies of games like Super Mario Sunshine, Super Smash Bros., Final Fantasy X, Dragon Quest VIII.
I don't fully agree with this one, since surely stores are in a better position to judge how many of a game to keep ordering in since it's sales are very predictable and uniform? I agree there will be some stock, but I can't image huge amounts as you suggest. 10-50k maybe? I'd also suggest that none of the games you mentioned (apart from perhaps Smash Bros) had particularly long legs. Mario Sunshine was a bit of a failure, FFX has had Greatest Hits releases and DQVIII isn't that old. Maybe they were just examples, but I was referring more to games like Mario Kart 64, Smash Bros 64, Gran Turismo etc. It's still a little too early to judge 'final sales' on titles like those you mentioned.
Jonnyram said:
The second example is the failure. Lots of orders are made by stores, lots of copies are pressed, and right now, the unsold software is sitting in storage somewhere, waiting to be either sold off at ridiculously cheap prices some day in the future, or disposed of entirely. If a 7000 yen game is sold off for 500 yen to get rid of stock, should its sales be counted at the same rate as a full price game? The research groups can't really track this data, as it's spurious, so it's not covered anyway, but equating shipped with sold will include junk like this, which demeans the value of the data.
I agree completely with this one, but to be honest we would be unlikely to get shipment figures released for a game like this (since it's a failure) and it's very unlikely I'd adjust a game like this from the standard averaged MC / Fam / Deng figures.
Jonnyram said:
With these points made, I focus again on ioi's data. He has stated himself, that his figures are an average of the different research groups' adjusted to be closer to shipment figures. This "adjustment" may work for companies that supply their shipment figures, but for those that don't, this "adjustment" is nothing more than a blind assumption. Not only is this adjustment based on an idea that is incorrect (that sales = shipments), but it's being applied uniform across the board for games that he doesn't have shipment data for.
The adjustements are only normally made for the big games which are selling well and that we get shipment data for. If I don't get shipment data for a game, how do I know what to adjust it by? There are some cases (again early SNES games) where we know that all other games are tracking, say, only 60% of the real figures so in cases like that I may scale any SNES games by some averaged scaling value, in cases where that game's shipments are not explicitly available.
Jonnyram said:
The people who don't put faith in ioi's data, the people who like to follow Media Create, or Enterbrain, or MediaWorks data do so because it is a single path of accuracy - any errors Media Create or whoever will make, will be relative only to that path, and can be spotted when compared with other researchers.
Well that's exactly what vgcharts does, smooths out these errors and discrepancies. It deos the comparison work for you and saves you from having to compare yourself.
Jonnyram said:
Significant figures are counted because we like the only data source we have to be as accurate as possible. Get sloppy and mistakes multiply, making bigger discrepancies in the data.
Indeed, but 10 copies on a scale of 2-3 million is insignificant and really not worth worrying about. The data is probably onlu accurate to the nearest 50k anyway.
Jonnyram said:
I personally just feel that ioi's efforts would be so much more useful if he actually directed them towards accuracy.
That's the whole point of adjusting and averaging the data- to ensure they are as accurate as possible.
Jonnyram said:
As it stands now, ioi makes his own numbers using his own theories and they cannot be relied upon.
Well of course you are more than fair in having that opinion, but as other people have said if you found out what 'theories' the
professional tracking firms use then you'd probably be shocked!! At least I am transparent with my methods, I spend a lot more time on here explaining what I do and how I do it, and I don't get paid to do so like Famitsu / MC.
Jonnyram said:
The DS life to date thread showed that even using his 5%+ mechanism, some of his data is still less than Media Create has on file. This is even after he's included his "adjustment" to bring sales closer to shipments.
Well I have yet to go back and include your Media Create data. The main reason for discrepancies is that most of the DS sales will be based on Famitsu data. We don't generally get MC data beyond the top 10, and until recently we had famitsu top 100s for the month. This means that once a game drops out of the top 10 (say Castlevania) the data i have is based purely on Famitsu. If MC have been tracking higher (as it seems they have) then I need to (and am in the process of) going back and add the mc data to the calculations. This, if anything, only bolsters my methods further and shows the differences we get between Famitsu and MC data.
Jonnyram said:
How can the data be relied upon when discrepancies like this occur.
Because as I explained, I am in the process of recalculating data we now have LTD MC numbers for, as is my commitment to having the most accurate data available on the site.
Jonnyram said:
He just has no way to track sales of games when they've fallen from the charts, so the adjustment is not compensating correctly. People who follow the raw data from Media Create simply wait until we have an opportunity to see the current data, rather than making blind assumptions.
All I have is data like that which you have posted. None of us can track when games drop off the charts, I interpolate from the data we get, fill in the blanks. This is where the 'estimates' come in. For example, we may know a game sold 5k in May in 4 weeks, so I'll just fit the weekly sales to that (1.5k, 1.25k, 1.25k, 1k for example), looking at how the game sells in June as well to make sure i have the drop-off about right and so on. If course none of those values are 'correct' but they are surely close enough, and the important thing is that the LTD at the end of May is correct surely, not how we got there.
Jonnyram said:
Long post, I know. I just felt like clarifying my stance on ioi's work.
His self-promotion is another thing entirely
I'm glad you did, and it is one of the more sensible 'criticisms' of the site. Hopefully I've addressed some of the points for everyone. The self-promotion is simply a kid who's done something he's proud of. I like the site, I think it's a very useful resource. I use it a lot myself and if I'm going to continue to put the work into it that I do, then I want as many people as possible to see it. There are always new people who say "wow, I've not seen that site before, it's cool" when I post a link, so this is why I do so, along with the fact that most of the links I post are actually contributing to the discussion and showing data that we cannot see any other way.