Consumer Reports’ iPhone 5 Review ⇥ consumerreports.org

In their review of the iPhone 5, Consumer Reports claimed the following:

The claimed improvements of the iPhone 5 in handling low-light shots were not apparent in our tests. In overall quality, both still and video images shot in low light on the iPhone 5 were of comparable quality to those shot on the iPhone 4S, though they did appear a little “cooler,” with a bluish hue.

To anyone who read any other iPhone 5 review (or has had the opportunity of shooting in low lighting conditions with one), this is baffling. The improvement from the 4S is noticeable.

I fired off a quick email to Consumer Reports asking if they would be willing to share the test photos. Their response was a clear “no”:

Consumer Reports does not share with the public any test data or criteria that is not published in our Reports. All information remains proprietary, and will only be shared with the manufacturer of the products we test (and only information relevant to their own product and not that of any competitor is shared.) In this way we remain free of any appearance of impropriety.

On the contrary, I think it would be proper to post comparison photos in what is purportedly a review. Marco Arment said it well a year ago:

A product as complex and multifaceted as a modern smartphone is beyond Consumer Reports’ ability to rate in a way that’s useful to most buyers.