How to Defy Review Aggregation

[Logik]

With the emergence of review aggregation sites such as GameRankings and RottenTomatoes, it’s becoming increasingly harder to get noticed as a game critic. GameDAILY Biz took a look at the situation and came up with 5 aggregation-defying tactics (http://biz.gamedaily.com/industry/media/?id=11632&rp=47) to help make the individual review worth reading again.

In an environment in which reviews are almost entirely interchangeable across all magazines, the core concept of a review section is rapidly nearing the end of its usefulness. If individual magazine reviews don't have something special that makes them stand out from the crowd, their primary role is reduced to serving as 1/100th of someone else's review aggregate.

However, individual reviews and review sections could certainly enjoy a resurgence in value if they were created with an eye towards originality and credibility. A key to making this happen is purposely breaking away from the review aggregate format.

[mightbe]

Interesting read. I personally use the aggregate in conjunction with individual review and the buzz on various forums.

I mean, it's worth it to do some research before you plunk down >50USD on a game.

[captainspankypants]

Yeah, the problem with the "aggregate" sites (I always thought "metareview" was a better term) is that they don't tell you what a game is like. They're good for the most general of impressions, but I doubt many people use them for purchasing decision. Even the most blind of fanboys don't buy a game solely because it scored a 90% at Game Rankings. They at least read ONE review to see if it's something they would personally like.

[mister_slim]

I generally disregard the scores. There's too much noise in the signal, from the magazine itself, to the reviewer quality, to the preferences of the reviewer, to what kind of a day the reviewer was having.

[AntB]

X-Men legends 2 sucked. Picked it up based on its reviews. A nice B game, sure. But it's boring as hell.

[Emabulator]

*cough* Black & White *cough*

[wudi]

If a magazine built a reputation for putting out a couple brilliant, comprehensive, and meaty reviews every month readers may turn to that publication to get the "full story" on their game. For instance, if everyone gives Halo 2 the core 600 words and a two page spread, most of the reviews are going to hit on the same topics. On the other hand, if one publication is known for its massive 6-8 page, 4000 word reviews of big games, readers automatically know that they're going to find out more when they read that publication.

Something like this might actually get me to buy more game magazines in general. Web sites are much better suited for reviews because they can reach their audience the day of a game's release, if not sooner, and deliver more depth than a typical magazine review. So, it stands to reason that I wouldn't plunk down 5 bucks to read a quarter of a page review of a game that's been out for 4 weeks already in EGM. However, if they slimmed down their roster, they could devote the kind of attention to a review that would in turn get them my attention (and money).

[jacktion]

Aggregate sites are great. They help get the overall picture in one look. Plus they collect all the writings about a particular game in one handy space. I read more reviews because of aggregate sites. If I am obssessing on one particular game I can just go down the list and read everything one after another. If reviewers can't see that then they are as blind as the music industry saying downloading Mp3s are bad for music.
Besides, most reviews are written like crap anyway so if a writer is blending into the chaff then maybe they should try to write in a better, more well-written way. Cream rises to the top, you know.

[dotbomb]

X-Men legends 2 sucked. Picked it up based on its reviews. A nice B game, sure. But it's boring as hell.

It is always a bummer to buy a game based on reviews and then find it just isn't your thing.

I was toying with the idea of purchasing that game based on reviews and then located the demo. I was completely underwhelmed by the demo and didn't purchase the game. Just goes to show you can't trust reviews since personal taste plays a large part in whether a title is a hit or not with you.

Aggregate sites are a godsend imo. I use rottentomatoes religiously for movies and have started doing so for games. Combining critic reviews with user reviews in one place is very handy.

I used to subscribe to magazines during the SNES generation. I learned quickly how craptastic the review system was. They can't do away with scoring but they do need to actually score games in a way to better communicate if the game would appeal to you or not.

[IIArchonII]

I remember when all i needed was a review from Gamespot, but now unless its a retarded football game or a GTA clone it doesn't score well.

I guess what we really need is a movie critic like system, where you can follow and trust a single critic

[51|RandoM]

Aggregate works good for me. I don't look at the game's score by itself, but compare its score to other games that are similar. That plus a video review from gamespot and I've got enough data to make a purchasing decision.

[Demo_Boy]

I think that the review classes are missing the point.

Should it not be
Innovation Index
SinglePlayer Replay Index
Initial Impact Index
Unique Multiplayer Index
Game Breaking Index (negative modifier)

So if you see a game that has innovation, high sp replay and no game breaking issues, then its worth a buy?

I dont know just loose thoughts.

[ElectricMonk]

I don't think critics add anything of value to this world so this news article makes me happy inside.

I usually don't read reviews because often they take away a lot of the surprise of the game, and makes the game worse. Like ninja gaiden, people slagged on the water level being terrible, so I kept waiting for this horrible water level, but once I got there the horridness of it was so overstated that I thought there was another water level somewhere.

edit: I do use the aggregate system

[Dag-Sabot]

I think the aggregate system is great. At the very least you get a feeling for which of the industry lapdogs are allowed to form their "own" opinions. These[reviews] while aggregated, stand in contrast to those of the publications that rubberstamp "100%" on even the most odious of crap.

[Spigot]

I do use the aggregate system but I also read the reviews. I tend to have certain game sites that I trust or at least am familiar with their general take on games. If I see the same problem cropping up across the board then I will be concerned. If only one publication mentions it, I might not be as worried.

As for the ideas presented in this article, I think that the best one is the crunching of the score curve. I'm sick of seeing very mediocre games scoring mid to high 7's. When did games deserving of a hearty "Meh" suddenly become an 8/10 game?

It's sad that people will turn down a game if it scores 7/10 now. That kind of game, on a real score sheet, should be in the almost great category. 8/10's should be great games, 9/10's AAA's and the 10/10 or perfect score should be something handed out to maybe one game per year, if that.

Famitsu (http://en.wikipedia.org/wiki/Famitsu) is a perfect example of what I think gaming criticism should be.

I wish more sites would use a percentage rating system as it allows a bit more flexibility in scores. A game scoring 81% would be a lesser game than one scoring 89%, but with a scale out of 5 or 10, that difference isn't shown.

[ElectricMonk]

It's sad that people will turn down a game if it scores 7/10 now. That kind of game, on a real score sheet, should be in the almost great category. 8/10's should be great games, 9/10's AAA's and the 10/10 or perfect score should be something handed out to maybe one game per year, if that.


I have no problem with the current 'top 40% of the curve' rating system. It's similar to a school grading system, the one people are used to getting graded at their entire life. A game is just a product made by people, and an average person gets things right 70% of the time. Most B-games have good ideas and flawed ones, and an 'average' game still has more enjoyable parts than flawed parts.

I think more review sites should start giving games 100% scores, because otherwise they get too focused on the problems of games and not the good parts.

[mister_slim]

Speaking of aggregation, do any of the sites do statistical correction of the review scales? That would be useful. I've only poked around gamerankings a bit, and they don't appear to do any normalizing (I love how even good Edge reviews drag averages down).

Also, and this is something I'd love to see EGM do, why don't more sites explicitly state the reviewer's perspective? EGM has three reviewers for each game, so why assign reviewers such that, when reviewing a FPS, for example, one reviewer loves the genre, one plays it casually, and one doesn't really care for it? And be explicit about each of those biases, so readers can easily figure out which reviewer best fits their preferences. EGM is already set up such that doing this would be fairly easy, but I'd love to see other review sources do the same.