Infinite Feedback Loop ⇥ eclecticlight.co

Howard Oakley:

Apple relies on bugs reported through its Feedback systems. As this Spotlight bug isn’t easy to recognise, users and third-party developers are only now realising the effects of Dave’s simple coding error. Without thorough testing, Apple is almost completely reliant on Feedback to detect and diagnose bugs.

This system is both flawed and woefully inefficient, as any expert in quality management will tell you. It’s like letting cars roll off the production line with no windows, and waiting for customers to bring them back to have them installed. By far the best choice is to build correctly the first time, or, as second best, to detect and rectify defects before shipping. So long as shipping updates remains relatively cheap, and your customers are happy to report all the defects which you didn’t fix, it appears to work, at least in the short term.

I’ve now reached the stage where I simply don’t have time to report all these bugs, nor should I have to. Indeed, I’ve realised that in doing so, I only help perpetuate Apple’s flawed engineering practices.

I was thinking about this piece earlier today as I filed a handful of pretty standard bug reports based on some visual problems I noticed in Big Sur.

For each one, Feedback Assistant automatically collected whole-system diagnostics, which wholly consumes system resources for a few minutes as it spits out a folder of logs totalling well over a gigabyte, plus the same folder as a compressed archive. The archive file is submitted to Apple and the uncompressed folder is locally cached for a little while — the oldest one on my drive is from January 4. It does not matter what the feedback is related to; this is a minimum requirement of all bug reports. If you are filing a report about many system features — Bluetooth or Time Machine, for example — it will also require you to collect separate diagnostics.¹

Often, I suspect, users will not attach all of the diagnostics needed for Apple’s developers to even find the bug. But I have to wonder how effective it is to be collecting so many system reports all of the time, and whether it is making a meaningful difference to the quality of software — particularly before it is shipped. I have hundreds of open bug reports, many of which are years old and associated with “more than ten” similar reports. How can any engineering team begin to triage all of this information to fix problems that have shipped?

To its credit, the quality of Apple’s software seems to have stabilized in the last year or so. But after the last several years, it feels more like the hole has stopped getting deeper and less like we are climbing out of it.