Diagnosing an Oft-Dropping AirPort Extreme Connection

As I noted last week, I purchased a MacBook Air to replace my ageing MacBook Pro. As part of the transition process, I’m trying to lose legacy, unnecessary files as much as possible. To make this process as easy as possible, I have turned on file sharing on my Pro to connect to it over WiFi for a la carte file transfers. Right now, it works great. But if you’d asked me a couple of weeks ago, I would have snapped.

After upgrading my Pro to Mountain Lion on July 25, I noticed that it dropped my otherwise perfectly stable WiFi connection every few hours, inconsistent with any previous task. Even worse, it wouldn’t automatically reconnect to the network. This is a behaviour I had not seen in any of the developer builds of Mountain Lion, nor any version of Mac OS X prior.

I kept track of each of the console messages for every incidence of this. Every single one looked similar to this¹:

2012-07-25 5:24:59.000 PM kernel[0]: AirPort: Link Down on en1. Reason 255 (unknown reason).
2012-07-25 5:24:59.000 PM kernel[0]: en1::IO80211Interface::postMessage bssid changed
2012-07-25 5:25:00.402 PM configd[17]: network changed: v4(en1-:10.0.1.40) DNS- Proxy- SMB
2012-07-25 5:25:00.440 PM com.apple.usbmuxd[23]: _SendDetachNotification (thread 0x[REMOVED]): sending detach for device [REMOVED]:5caa._apple-mobdev._tcp.local.: _BrowseReplyReceivedCallback got bonjour removal.
2012-07-25 5:25:00.442 PM iTunes[1364]: _AMDDeviceDetached (thread 0x11708a000): Device 'AMDevice 0x[REMOVED] {UDID = [REMOVED], device ID = 21, FullServiceName = [REMOVED]:[REMOVED]._apple-mobdev._tcp.local.}' detached.
2012-07-25 5:25:01.419 PM awacsd[58]: InnerStore GetWakeInfoForZone: zero address for [REMOVED].members.btmm.icloud.com.

[TRUNCATED]

2012-07-25 5:25:36.000 PM kernel[0]: en1: BSSID changed to [REMOVED]
2012-07-25 5:25:36.000 PM kernel[0]: en1::IO80211Interface::postMessage bssid changed
2012-07-25 5:25:36.000 PM kernel[0]: AirPort: Link Up on en1
2012-07-25 5:25:36.000 PM kernel[0]: en1: BSSID changed to [REMOVED]
2012-07-25 5:25:36.000 PM kernel[0]: en1::IO80211Interface::postMessage bssid changed
2012-07-25 5:25:36.000 PM kernel[0]: AirPort: RSN handshake complete on en1
2012-07-25 5:25:36.612 PM configd[17]: network changed: v4(en1+:10.0.1.40) DNS+ Proxy+ SMB
2012-07-25 5:25:36.660 PM UserEventAgent[11]: Captive: en1: Not probing 'Free Public WiFi' (protected network)
2012-07-25 5:25:36.681 PM configd[17]: network changed: v4(en1!:10.0.1.40) DNS Proxy SMB

The first half of that block represents my Mac instantly dropping its WiFi connection, and iTunes disconnecting my WiFi-synced iPad. Note that the iPad WiFi connection did not drop—it was just ejected from iTunes since the host WiFi connection was lost.

The second half of that block is my manual reconnection of the signal².

Since I intended to create a small local file server with my MacBook Pro, its WiFi connection needed to be as reliable as possible. I began with the basic troubleshooting steps: doing a quick Google search to see if anyone else saw the same message. I didn’t expect to find many results, but I thought there would be a few. I was shocked when I discovered that nobody else saw the same console readout for their dropped connection as I did (or, at least, nobody complained about it with that same “Reason 255” code)³.

Even though no other computers in my house were exhibiting the same behaviour, I thought it might be a good idea to examine the AirPort Extreme base station logs to see if there were any clues there. However, this was made more difficult than it needed to be because Apple removed much of the functionality of AirPort Utility with the more user-friendly version 6.0.

You would think that I would be able to download AirPort Utility 5.6 for Lion and install it on Mountain Lion, given that the latter is a newer OS. Alas, the installer for it has both a minimum and maximum OS version. I used the excellent Pacifist to pull the app out of the package and run it. To my dismay, there were no relevant log entries.

I rarely install beta software on my main partition, just in case something goes wrong. However, I decided to bite the bullet and install the then-latest build of 10.8.1, 12B17, just in case something was fixed. Comically, a minute into using it, it disconnected. Same reason. Obviously, it wasn’t going to be fixed in 10.8.1.

I decided that the best course of action would be to pull old copies of the drivers and compare them. From my Time Machine drive, I grabbed the IO80211Family kernel extension from a version of 10.7, an early developer preview of 10.8, the GM version of 10.8, and the 10.8.1 copy I had just installed. My hope is that something in the AirPortAtheros40 driver for my AirPort card would be relevant.

Jackpot.

Mac OS X 10.7.x displayed a bundle version of 504.64.2, and the bundle version for the 10.8 developer preview was 600.65.1. Neither of these copies were faulty. The connection dropping began with the gold master version of 10.8, and continued with the 10.8.1 preview. Not coincidentally, both of these operating systems showed an AirPortAtheros40 driver bundle version of 600.70.23.

I restored the kernel extension with a known working copy, and rebooted. My WiFi connection has been stable ever since.

As one should, I have filed two related bugs. Radar number 11960274 concerns the faulty driver, with a system configuration report and all referred-to kernel extensions enclosed, while radar number 12129328 notes that it is impossible to extract logs from an AirPort base station using AirPort Utility 6.0 or newer.

I posted this for a few reasons. The first is to gain at least one hit for this specific issue. As I mentioned, I haven’t seen it discussed anywhere else. The second reason is a reminder to keep a backup, just in case. Finally, I was surprised that this would continue to persist, despite filing that bug (as you’ll note by the console log above) the day Mountain Lion shipped. As far as I know, my radar hasn’t been looked at, and therefore this problem will not be fixed in 10.8.2. My old-ass MacBook Pro simply isn’t relevant enough any more.