Jean-Luc
November 12th, 2009, 10:06 AM
This is something I've been wanting to discuss for quite some time now. Over the years, I've been looking at how video games are rated and I've realized that the system is not only biased in many ways (see Metacritic), but is also broken. I've spoilered each section as this post would otherwise be a brick wall of text.
Issue 1: The Faults of Modern Numerical Ratings
One of my biggest gripes with the modern rating system is a certain belief we've all gotten persuaded into and it's the notion that somehow any game with a rating below 80% is automatically worth our disdain. This truly hit home with me a few months back when I read a review of Halo: ODST, which on this particular website was given a 70%. After reading the review, I looked at the comments below. Quite frankly, it was horrifying. There were death threats, hate mail, cursing, even multiple calls for his firing; All these were being thrown at the reviewer, simply because he rated the game as “average.” I'll attribute part of the hostile response to the notorious Halo fanboy-ism that we have all experienced, but I feel it exposes a more pressing issue with our mentality as gamers.
So, now that we've identified the issue, how do we fix it? Ideally, no game should ever attain a 100% as no game is perfect, but getting the general public to accept that would be...difficult to say the least. If such a system could be implemented, professional reviewers need to stop with the rampant leniency they show games and start being truly honest. For those of you wondering what I mean by “leniency,” I'd like you to find a game that received a 100% on a review. Look through it, and I can, with 99% certainty, say that there will be a line, somewhere, that says something to the effect of “________ has its flaws, but ________ makes up for it.” That's exactly what needs to stop, and I mean completely. I don't care how minor the flaw is, the fact that it exists means the game isn't perfect and therefore doesn't deserve a perfect score. Now then, this isn't to say all reviewers need to be as aggressive as Yahtzee of Zero Punctuation...but flaws need to stop being overlooked, even if for no other reason than for the developer to improve.
The next step would be to give new significance to the number system of reviewing. No longer should 1-6 be reserved for the truly abysmal; let's make more use of these percentages! Now, what I'm going to present is entirely hypothetical and would need to be tested, but I am very confident that if we adopted a broader spectrum of percentages, games could be scored much more accurately.
1-29: Very bad: Games that fall within these percentages may occasionally have an interesting premise or gameplay idea, but they're completely buried under horrible game design, bugs, poor presentation, and a complete lack of replay value.
30-49: Poor: While more polished than the lowest category, there is nothing particularly memorable about these games, and they can often be frustrating due to some bad design choices, annoying bugs, and a mediocre audio/visual presentation.
50-69: Average: The majority of games will fall into this category. There isn't much inherently bad with these games, but they don't bring enough new ideas to the table either. There may occasionally be some poor design choices and a lack of polish in these titles, but you can still have an enjoyable experience.
70-79: Good: Games in this percentile are more refined than most. Sensible game design is apparent, but there isn't much innovation to be found, and there will be notable imperfections within the game. While none of these flaws will be severe, they noticeably detract from the game experience a little too often. Titles in this category do present above-average experiences however, so they're certainly worth a try.
80-89: Great: Brushing against excellence, games that fall into this category are highly recommended. Mostly refined and innovative game design is obvious, and the presentation is uniformly pleasing, but the occasional frustrating bug and flawed design decision holds these titles back from true gaming nirvana.
90-99: Exceptional: Games that fall into this category are shining examples of their genre. Exhibiting exceedingly polished gameplay and presented beautifully, the rare games that deserve this accolade bring high levels of innovation along with them or elevate an existing system to a new plateau. Extremely high replay value means that these games should not be missed.
Obviously the rating system I came up with assumes that all games are judged on a 1-99 percent scale and I know that doesn't exist everywhere currently. I personally prefer this scale because it allows the reviewer to more accurately rate the game, rather than generalizing on a x/10 or ABCDF scale.
Issue 2: Platforms Divided
The next issue to tackle is that of “platform separation.” Many of you have noticed that review websites will often use a technique that I call “blanket reviewing.” This is where they write one review and apply it to all platforms. While I can see this working decently with the Xbox 360 and PS3, PC reviews need to become completely separate from the consoles. This isn't to promote a division between gamers, but rather to review each platform fairly. As most of you know, PC gamers look for a substantially different experience than what is offered on the console and it's because of this that it's wrong to expect a game that works fine on consoles will automatically work on the PC as well. As much as I wanted to avoid dragging Modern Warfare 2 into this, it's probably the most perfect example of what I'm talking about.
For those of you who are unaware, Infinity Ward stated a while back that their goal with Modern Warfare 2 was to deliver an equal experience across all platform, but that's entirely under the assumption that PC gamers want the same experience as their console brethren. This is a very poor assumption, and it's easily refuted by a phrase that's seen a great deal of use recently: “If I wanted the console experience I would buy a console.” Despite how many people have reacted to that, there is absolutely nothing malicious in that phrase at all. It really comes down to the basic notion that PC gamers want a different experience. See, the thing about platform equality is that you can't expect the same version of a game to work on any platform you put it on. True equality would mean that the game is tailored SPECIFICALLY to the strengths of each system it's released on instead of forcing all to conform to the strengths of ONE.
Hopefully now you can see the importance of treating each system with the respect it deserves, rather than delivering 'blanket statements” under the assumption that they actually apply appropriately. So to aid in this issue, the simple step would be, as I said earlier, to have a separate review for PC games, and a separate review for console games. That would ensure a much more accurate rating and it would mean that the faults and benefits described in reviews would be tailored to match the appropriate platform. I could easily go more in depth with this system and suggest that we review games based upon their experiences offered (i.e. a game that is primarily singleplayer would have a different rating system than a primarily multiplayer game), but I'd rather not make things too complex right now.
Issue 3: Unfair and Irresponsible Bias
The final issue I want to tackle is that of bias. Now, I'm not talking about slight bias in reviews as that's understandable; reviews are opinions after all, but rather when bias is so strong that it has an unfair impact on the games score. I'm sure all of you can think of a review that made you go "That was way to opinionated for me to consider it valid." That's the point when you know there's trouble. Like I said before, reviews are opinions, but reviewers need to make sure they place at least as much importance on being objective as well. I know it's a lot to ask, but it is possible.
The other kind of bias is evidenced by the popular website Metacritic. For those of you who haven't used it, Metacritic is a website that complies professional reviews on games, movies, TV, etc and averages out the scores to give the viewer a fairly accurate assessment of what the product is truly like. However, if you click on “How are these scores calculated,” you will see that Metacritic openly admits to placing more importance on certain reviewers based upon their status. I feel that this is very wrong. Why are “less popular” review sites any less valid than a site like IGN? Reviews are and always have been subjective personal opinions, so to say that someone's opinion is more valid than another is borderline irresponsible. Everyone should have an equal right to say what they mean without worrying that their stance will be overshadowed by a larger entity. The only viable way to fix this problem is to treat each review site with equality, rather than placing weighted opinions on "designated" websites. On the off-chance that a site becomes biased to the point of absurdity, then additional action would be taken to either ensure they straighten up or discount that website as a valid source.
In conclusion, I hope the three issues I have just discussed have proved relevant to your past experiences. I do not expect you to agree with me on anything I have said here, but I hope you can treat it maturely. As for the solutions I provided, please do not take them as the only possible answer as I'm certain there are alternatives to what I was able to come up with. As a final note, I know I mentioned Modern Warfare 2 in some detail here, but please leave it in the context in which I mentioned it, and above all, try to refrain entirely from inciting any platform wars in this thread. We've had enough of that going around.
*NOTE* I wrote this after a night of zero sleep so there is the potential that I may need to revise some parts of this.
Issue 1: The Faults of Modern Numerical Ratings
One of my biggest gripes with the modern rating system is a certain belief we've all gotten persuaded into and it's the notion that somehow any game with a rating below 80% is automatically worth our disdain. This truly hit home with me a few months back when I read a review of Halo: ODST, which on this particular website was given a 70%. After reading the review, I looked at the comments below. Quite frankly, it was horrifying. There were death threats, hate mail, cursing, even multiple calls for his firing; All these were being thrown at the reviewer, simply because he rated the game as “average.” I'll attribute part of the hostile response to the notorious Halo fanboy-ism that we have all experienced, but I feel it exposes a more pressing issue with our mentality as gamers.
So, now that we've identified the issue, how do we fix it? Ideally, no game should ever attain a 100% as no game is perfect, but getting the general public to accept that would be...difficult to say the least. If such a system could be implemented, professional reviewers need to stop with the rampant leniency they show games and start being truly honest. For those of you wondering what I mean by “leniency,” I'd like you to find a game that received a 100% on a review. Look through it, and I can, with 99% certainty, say that there will be a line, somewhere, that says something to the effect of “________ has its flaws, but ________ makes up for it.” That's exactly what needs to stop, and I mean completely. I don't care how minor the flaw is, the fact that it exists means the game isn't perfect and therefore doesn't deserve a perfect score. Now then, this isn't to say all reviewers need to be as aggressive as Yahtzee of Zero Punctuation...but flaws need to stop being overlooked, even if for no other reason than for the developer to improve.
The next step would be to give new significance to the number system of reviewing. No longer should 1-6 be reserved for the truly abysmal; let's make more use of these percentages! Now, what I'm going to present is entirely hypothetical and would need to be tested, but I am very confident that if we adopted a broader spectrum of percentages, games could be scored much more accurately.
1-29: Very bad: Games that fall within these percentages may occasionally have an interesting premise or gameplay idea, but they're completely buried under horrible game design, bugs, poor presentation, and a complete lack of replay value.
30-49: Poor: While more polished than the lowest category, there is nothing particularly memorable about these games, and they can often be frustrating due to some bad design choices, annoying bugs, and a mediocre audio/visual presentation.
50-69: Average: The majority of games will fall into this category. There isn't much inherently bad with these games, but they don't bring enough new ideas to the table either. There may occasionally be some poor design choices and a lack of polish in these titles, but you can still have an enjoyable experience.
70-79: Good: Games in this percentile are more refined than most. Sensible game design is apparent, but there isn't much innovation to be found, and there will be notable imperfections within the game. While none of these flaws will be severe, they noticeably detract from the game experience a little too often. Titles in this category do present above-average experiences however, so they're certainly worth a try.
80-89: Great: Brushing against excellence, games that fall into this category are highly recommended. Mostly refined and innovative game design is obvious, and the presentation is uniformly pleasing, but the occasional frustrating bug and flawed design decision holds these titles back from true gaming nirvana.
90-99: Exceptional: Games that fall into this category are shining examples of their genre. Exhibiting exceedingly polished gameplay and presented beautifully, the rare games that deserve this accolade bring high levels of innovation along with them or elevate an existing system to a new plateau. Extremely high replay value means that these games should not be missed.
Obviously the rating system I came up with assumes that all games are judged on a 1-99 percent scale and I know that doesn't exist everywhere currently. I personally prefer this scale because it allows the reviewer to more accurately rate the game, rather than generalizing on a x/10 or ABCDF scale.
Issue 2: Platforms Divided
The next issue to tackle is that of “platform separation.” Many of you have noticed that review websites will often use a technique that I call “blanket reviewing.” This is where they write one review and apply it to all platforms. While I can see this working decently with the Xbox 360 and PS3, PC reviews need to become completely separate from the consoles. This isn't to promote a division between gamers, but rather to review each platform fairly. As most of you know, PC gamers look for a substantially different experience than what is offered on the console and it's because of this that it's wrong to expect a game that works fine on consoles will automatically work on the PC as well. As much as I wanted to avoid dragging Modern Warfare 2 into this, it's probably the most perfect example of what I'm talking about.
For those of you who are unaware, Infinity Ward stated a while back that their goal with Modern Warfare 2 was to deliver an equal experience across all platform, but that's entirely under the assumption that PC gamers want the same experience as their console brethren. This is a very poor assumption, and it's easily refuted by a phrase that's seen a great deal of use recently: “If I wanted the console experience I would buy a console.” Despite how many people have reacted to that, there is absolutely nothing malicious in that phrase at all. It really comes down to the basic notion that PC gamers want a different experience. See, the thing about platform equality is that you can't expect the same version of a game to work on any platform you put it on. True equality would mean that the game is tailored SPECIFICALLY to the strengths of each system it's released on instead of forcing all to conform to the strengths of ONE.
Hopefully now you can see the importance of treating each system with the respect it deserves, rather than delivering 'blanket statements” under the assumption that they actually apply appropriately. So to aid in this issue, the simple step would be, as I said earlier, to have a separate review for PC games, and a separate review for console games. That would ensure a much more accurate rating and it would mean that the faults and benefits described in reviews would be tailored to match the appropriate platform. I could easily go more in depth with this system and suggest that we review games based upon their experiences offered (i.e. a game that is primarily singleplayer would have a different rating system than a primarily multiplayer game), but I'd rather not make things too complex right now.
Issue 3: Unfair and Irresponsible Bias
The final issue I want to tackle is that of bias. Now, I'm not talking about slight bias in reviews as that's understandable; reviews are opinions after all, but rather when bias is so strong that it has an unfair impact on the games score. I'm sure all of you can think of a review that made you go "That was way to opinionated for me to consider it valid." That's the point when you know there's trouble. Like I said before, reviews are opinions, but reviewers need to make sure they place at least as much importance on being objective as well. I know it's a lot to ask, but it is possible.
The other kind of bias is evidenced by the popular website Metacritic. For those of you who haven't used it, Metacritic is a website that complies professional reviews on games, movies, TV, etc and averages out the scores to give the viewer a fairly accurate assessment of what the product is truly like. However, if you click on “How are these scores calculated,” you will see that Metacritic openly admits to placing more importance on certain reviewers based upon their status. I feel that this is very wrong. Why are “less popular” review sites any less valid than a site like IGN? Reviews are and always have been subjective personal opinions, so to say that someone's opinion is more valid than another is borderline irresponsible. Everyone should have an equal right to say what they mean without worrying that their stance will be overshadowed by a larger entity. The only viable way to fix this problem is to treat each review site with equality, rather than placing weighted opinions on "designated" websites. On the off-chance that a site becomes biased to the point of absurdity, then additional action would be taken to either ensure they straighten up or discount that website as a valid source.
In conclusion, I hope the three issues I have just discussed have proved relevant to your past experiences. I do not expect you to agree with me on anything I have said here, but I hope you can treat it maturely. As for the solutions I provided, please do not take them as the only possible answer as I'm certain there are alternatives to what I was able to come up with. As a final note, I know I mentioned Modern Warfare 2 in some detail here, but please leave it in the context in which I mentioned it, and above all, try to refrain entirely from inciting any platform wars in this thread. We've had enough of that going around.
*NOTE* I wrote this after a night of zero sleep so there is the potential that I may need to revise some parts of this.