March 24, 2013

Review systems reviewed

So, just how the hell do we review video games? What's the best way? Every method seems to suck, but there's got to be some kind of answer, right? Yes, I think there is. Let's figure this shit out once and for all.

The 'ol 100 point classic scale
This one is still more common than you think. Outlets like GameInformer, IGN, and GamesBeat still use this one. It usually consists of a 10-point scale with decimal 10th points (though GI will also go to hundredths like .75s). In the case of GamesBeat, they actually use a 0-100 scale. Metacritic uses this scale when it magically averages the scores from all systems together. Why is this system crap? The answer is very simple: It's just too many damn numbers. No one could possibly explain the difference between a 7.4, and a 7.5. Add in crap like 7.45, and you end up looking completely foolish. It's the most arbitrary of all the arbitrary numbering systems. There is no reason whatsoever reviewers can't just round up or down to, at least, the nearest half point. For some reason, this was the system back in the days of gaming magazines. Today, it's just ridiculous.

The 20-point new school standard
The most widely used scale today is the 10-point with halves, known here as the 20-point. Outlets like Destructoid, Polygon, and EGM use this system. It seems pretty good on paper, but suffers from a lot of problems in practice. Centrally, this one still uses too many points on the scale. Half points essentially mean "I can't decided between this score or this other score so I'm going to just go with something in the middle." It's the scale for indecisive folks and rarely works well. The real issue here is that things are still too arbitrary and there's a fuckton of inflation from it. Ever wonder why reviewers seem to use 7-10 (essentially six points) only? It's because there's just too many damn numbers to possibly use them all. The industry is calling out for a simpler system while we all either complain or accept 7 as a low score.

The real 10-point scale
Do not be fooled by stars! I can't think of a single major outlet that uses a 10-point scale without stars. At first, you may think sites like Joystiq or GamesRadar are using a 5-star scale, but they're really using a 10-point scale with stars because they use half points. The problem with this, of course, is that it's not easy to read. I always end up converting the stars into numbers in my head (one star equals two points, while a half equals one). This systems suffers from the same problem as the 20-point. It's got half as many numbers, but still that's still too many. Scores will still average on the high end, likely still 7-10. Except now you're really using a 3-point scale. If they chop off the other seven numbers, would that be a viable option?

The 3-point scale
A 3-point systems is pretty straightforward, but is probably too much so. It'd break down something like: good, average, and bad. That doesn't seem like enough wiggle room. What if the game isn't one of those three things but falls somewhere in between? This system is simply too restrictive. Many smaller sites (and, notably, Kotaku) use a variation on this where the verdict is something like "buy", "rent", or "skip". Using something like this is too definitive, and might actually insult readers. Rather than placing a score in front of you that you would (theoretically) interpret, this simply tells you what you should and should not be playing. A score should ideally reflect the reviewer's opinion and be used as a guide, along with the text, to help you make a purchasing decision. Under this system, the reviewer has made that decision for you.

Letter Grades
Although this system formerly used by EGM and 1up has gone the way of the dinosaur, it's still worthy to note. No one should ever use this again. Anyone who's ever gone to school knows that letter grading makes no sense anyway, and what's a good grade to one person may not be to someone else. Then there's the decision whether to use plusses and minuses or not. It can come out anywhere between 5-points (then why not just use numbers?) and 15. It's a mess.


No score
Let me be clear that I have absolutely no issue with this. I don't even like scores and wish everybody would just get rid of them. They're more trouble than they're worth. But there's this longstanding expectation readers have, not to mention the current (over)importance of the practice within the industry. Basically, I think Metacritic is the devil, but I have to accept that it's a thing and deal with it. This is the way to go in my book, though. It's the text that's important anyway. A score is just supposed to represent that with a simple glance.

5-point (star) scale
If there's got to be a review score, this system might as well be the industry standard. Almost nobody actually uses a 5-point scale (Rev3 is the only one I can think of), but it's clearly the most balanced. You've got an odd number, so there is a clear middle-ground. The 10-point also has this, but a 5 is never considered average. With fewer points, you can actually use them all because they all can clearly mean something. There's two points in each direction so you can have one that's highly good, one that's highly bad, and something in the middle both ways. The only problem with this is that it's hard to read because it's so alien. Without clearly defining what each point on the scale represents, most people would probably try to convert stars into numbers or else be very confused with just the numbers 1-5. Is a score of 2 supposed to be bad? How about 3?

The ideal scale
Ultimately, this scale would be 5-points, but would clearly define the meaning of those points. I would do one of two things: have a short description next to the score or replace the numbers altogether with words. The score would seek to sum up my experience playing the game. My scale would go like this:
  • 1- Terrible
  • 2- Playable
  • 3- Average
  • 4- Satisfying
  • 5- Superior
I thought about what word would best describe each level and I think I've reached a nice result. 1 and 5 are meant to be strongly good or bad. A score of 1 means that the game is significantly flawed and essentially unplayable and not fun. A score of 5 is the complete opposite. This is a game that exceeded expectations and is a blast to play. This does not necessarily mean it's perfect, but it has done something for the reviewer beyond just being good. A 3 is average. It may go up or down one space depending on the player. There is nothing extraordinary about one of these games either way. A 2 means the game isn't broken, but not particularly fun either. Something may have gone wrong in the design, but the game is technically sound. A 4 is reserved for games that are completely satisfying. They're about as good as you expected them to be. Fun to play, but not doing anything particularly revolutionary in the reviewer's mind.

The problem with using this scale, or course, is that it completely mindfucks Metacritic. What are they supposed to do with a game scored 1-5? Nothing, that's what. And that's why nobody uses this scale.

No comments:

Post a Comment