Quick: what are the top three smartphones that you can buy today?
Now score each of those smartphones on a ten-point scale. Tricky, isn’t it?
There are certain quantifiable metrics by which a smartphone, in this case, can be judged: screen resolution and size, battery life, or cell radio capabilities. But these metrics do not define the whole experience; you’re not just buying a screen or a battery, are you?
This necessitates the introduction of subjective scoring of other aspects, like hardware design, or the niceness of the onscreen interface. Each of these qualities is impossible to score objectively, or numerically, for that matter. What specific qualities separate “8” hardware from “7” hardware? Even aspects of a phone which are ostensibly measurable are no indication of their value to the end product. Processor frequency, for example, can be measured, but it is not the only factor that may influence performance or battery life.
All of these issues were combined recently with The Verge’s review of the Samsung Galaxy S4:
The Galaxy S4 is fast and impressive, but it’s also noisy and complex. The One is refined, quiet, comfortable, beautiful, and above all simply pleasant. I love using that phone, in a way I haven’t experienced with anything since the iPhone 5. That’s why, when my contract is up in June, I’ll probably be casting my lot with HTC instead of Samsung.
Given this summary paragraph alone, you would be forgiven for assuming that this is not a recommended product. It’s unpolished, and full of “noisy” features. But the phone itself received an 8.0 rating which, in a vacuum, you would be forgiven for assuming is a recommendation of it. Both cannot be true at the same time.
Ben Brooks has written about the ambiguity of The Verge before:
The problem is endemic of the industry as a whole. You can say all you want about my opinions — whether I am right or wrong — what you can’t say is that I don’t have one. I will take you disagreeing with me all day long over being a bland yes man.
I think this stems from the notion that subjective qualities can be ranked on an objective scale. This is completely absurd, yet it’s the basis for most popular review sites in the technology space and elsewhere. Pitchfork, for example, ranks albums on a 10.0 scale. What makes Cassie’s RockaByeBaby precisely 0.3 better than Phoenix’s Bankrupt!?
My objections aren’t simply that all of these rankings are opinions. It isn’t even the way these subjective rankings are masquerading as objective scores, though that is certainly part of my objection. Rather, it is the notion that there is some necessity in ranking or scoring things with numbers.
Make no mistake: there is a need to have reviews of the new products, new music, and new whatever that competes for our attention and money. But the idea that they need to be judged on a numerical scale is completely ridiculous. A much simpler and more honest approach would be to either “recommend” a product, or to “not recommend” it. Perhaps there could also be a “highly recommended” ranking, for particularly good things (and, for the pessimists out there, an “avoid” ranking, for truly terrible things). This system appears to be more vague, but it is no less accurate than an arbitrary number score.
Of course, this ranking system would require people to read reviews, rather than hopping to the comments to immediately complain about the numerical score. C’est la vie.