Apple’s Statement on Consumer Reports’ MacBook Pro Battery Tests ⇥ loopinsight.com

Apple PR, in a statement provided to news outlets:

We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab.

The statement goes on a little longer, but the nutshell version comprises these three sentences. And I have issues with all of them.

Calling the Disable Caches setting “hidden” seems, at best, misleading. While it’s true that a user has to switch on the Develop menu in Safari’s preferences to expose this setting, that’s all done through Safari’s GUI. A “hidden” setting would be one that requires a Terminal command, wouldn’t it?

At any rate, arguably no battery test can truly reflect “real-world usage”, since all tests are — by definition — simulations of some kind of usage. Someone browsing the same three or four websites all day long with little else running would likely get very good battery life, while a user editing RAW photos that are synced to iCloud Photo Library would see pretty poor life. That’s just how it works. As the product becomes more targeted towards power users, the gap between the extremes of battery life will only get wider — you can bet that the number of users running Final Cut on a 12-inch MacBook is very, very low.

Consumer Reports’ browser-based battery test is, as Apple says, inconsistent with typical web browser usage. Most users will leave their cache on. But they’ll also probably browse more than ten web pages repeatedly, and might have iTunes, Messages, a couple of Finder windows, and Mail all running in the background.

We could argue about the validity of Consumer Reports’ test all day long. The third sentence in the excerpt I quoted above is the part where Apple admits that there is a flaw, but it seems pained and couched. Furthermore, it’s hard to see how a bug like this, when combined with a disabled cache, could lead to Consumer Reports seeing some test results with less than half that of Apple’s estimates, while other results were nearly double what Apple says. That’s a massive chasm, and I haven’t seen any MacBook Pro owner claiming to get battery life at the upper end of that spectrum.