Pixel Envy

Written by Nick Heer.

Archive for October, 2018

Concerns Linger About MacBook Pro Keyboards

Casey Johnston, the Outline:

[…] Every time I described the 2017 MacBook Pro I sold because I couldn’t stand its non-functional keyboard and asked an Apple store employee if the new one would screw me over the same way, each assured me that Apple had changed the keyboards so that that would never happen again. I described my issues with “dust” to one shop associate at the Apple Store at the World Trade Center and asked if the new computers were any better. “Yeah, yeah, they fixed that problem… it was a BIG problem,” she told me. “So it doesn’t happen at all?” I asked. “No, it shouldn’t happen,” she said. Maybe the bad days were finally over.

But checking around online, it appears the new keyboards have the same old issues. They may be delayed, but they happen nonetheless. The MacRumors forum has a long thread about the the “gen 3 butterfly keyboard” where users have been sharing their experiences since Apple updated the design. “How is everyone lse’s keyboard doing? I rplaced th first one because ‘E’ and ‘O’ gave double output. The replacment ither eats “E”, “O”, “I” and “T”, or doubles them,” wrote one poster. “I didn’t correct the typos above on purpose.”

It’s pretty wild that the Apple Store employee would admit to anyone that this was a “big problem”, given how often Apple has emphasized that it was a small percentage of users and that the silicone membrane in the 2018 models is just for quieter typing — though, in service documentation, they copped to its debris-fighting intention.

This is my favourite quoted response from that MacRumors thread:

“That’s just plain reckless,” responded a third. “I mean he took a laptop from a closed apartment to a balcony. It was probably an open balcony. Does he think that a laptop is a portable computer or what?!?”

The nature of online reviews and Mac enthusiast forum users, in general, tends to draw out negative experiences in a sort of shared commiseration experience. There aren’t loads of people who will chime in with their flawless keyboard experience. But, even if a smaller number of 2018 MacBook Pro owners are finding their computers susceptible to dust-induced keyboard failures compared to 2016 or 2017 model year users, these problems are still unique to the ultra low profile “butterfly” mechanism used in these models and are not present in previous generations of keyboards. This a serious regression of one of its single most critical components. These are not good keyboards.

Johnston’s thoughts on the current Apple notebook lineup echo my own:

[…] The MacBook is aesthetic but underpowered; the Air is an outdated design paradigm, a “thin and light” notebook that has the worst performance-to-weight-to-cost tradeoff of all the computers Apple makes, but the only one left with a decent keyboard; the MacBook Pro fails at being a Pro in a number of ways (a small number of ports that almost always require dongles, garbage battery life), not least of which is that the keyboard stops working after a couple of months for many people. Every laptop offering has serious tradeoffs, none of them are compellingly priced, and most are just old.

The MacBook today fills the same slot as the MacBook Air of 2008, and vice-versa. Neither represents a massive upgrade for me over my mid-2012 MacBook Air for my changed workflow. The MacBook Pro has a worrisome keyboard, and it’s extremely expensive: a base 15-inch PowerBook in 2004 cost $2,649 in Canada, the 2007 15-inch MacBook Pro started at $2,199, and the Retina 15-inch MacBook Pro started at $2,449 in 2015. But the new 15-inchers start at $3,199. That’s a big leap; Apple’s 15-inch portables haven’t been that expensive since the early 2000s.

More than anything I’m confused by the current Mac lineup. It feels all out of sorts — almost as if each model were handled by a separate team with its own shipping deadline and requirements. There isn’t a clear rubric. I don’t think the lineup needs to go back to the Jobs quadrant, but it ought to be easier to buy a computer than the current lineup permits.

Facebook Acknowledges That Contact Details of Twenty-Nine Million Users Were Stolen

Guy Rosen of Facebook followed up on their earlier disclosure of their security breach in a post euphemistically titled “An Update on the Security Issue”. They have to use the indefinite article “the security issue”, never “our security issue”.1 Anyway:

The attackers used a portion of these 400,000 people’s lists of friends to steal access tokens for about 30 million people. For 15 million people, attackers accessed two sets of information – name and contact details (phone number, email, or both, depending on what people had on their profiles). For 14 million people, the attackers accessed the same two sets of information, as well as other details people had on their profiles. This included username, gender, locale/language, relationship status, religion, hometown, self-reported current city, birthdate, device types used to access Facebook, education, work, the last 10 places they checked into or were tagged in, website, people or Pages they follow, and the 15 most recent searches. For 1 million people, the attackers did not access any information.

A portion of users have also had their Facebook Messenger conversation names and contacts compromised, and if they were an admin of a page, any messages to that page might also be compromised as well. Katie Notopoulos and Nicole Nguyen of Buzzfeed have put together a great article on how to tell if you’re one of the users impacted.

Earlier this week, Facebook launched an always-on microphone with an attached camera.


  1. I feel a little gross for interpolating Fight Club↩︎

Google Has Continued Its Growth in Europe Post-GDPR, While the Prevalence of Other Trackers Has Been Cut

Natasha Lomas, TechCrunch:

For the GDPR analysis, the team compared the prevalence of trackers one month before and one month after the introduction of the regulation, looking at the top 2,000 domains visited by EU or US residents.

On the tracker numbers front, they found that the average number of trackers per page dropped by almost 4% for EU web users from April to July.

Whereas the opposite was true in the US, with the average number of trackers per page rose by more than 8 percent over the same period.

[…]

Summing up their findings, Cliqz and Ghostery write: “For users this means that while the number of trackers asking for access to their data is decreasing, a tiny few (including Google) are getting even more of their data.”

This builds upon and somewhat echoes earlier reporting that GDPR would actually help Google and Facebook compared to their smaller competitors. That’s not surprising: GDPR requires individual companies to get an explicit opt-in from users for ad targeting and tracking, and that’s a lot easier to do when you’re Google or Facebook. It’s also something that can be addressed through greater antitrust enforcement, if the E.U. wishes to pursue more direct targeting of the mass surveillance business models of those two companies.

The Anatomy of a Click

As part of my morning review of news headlines, I like to read Charles Arthur’s excellent Overspill link roundup. In today’s edition, he linked to a fascinating-looking piece by James Ball in the Huffington Post called “The Anatomy of a Click” about programmatic advertising and all of the automated bidding that happens when you click. So I did.

I was greeted first by the burdensome opt-in advertising screen for Oath, the Huffington Post’s parent company. GDPR may require website owners to give visitors choices, but this is just egregious, and shows the scale of Oath’s operation. They don’t make it easy to simply opt-out of all targeting and tracking. This is why ad blockers are popular.

Then I noticed the URL, which now contained all sorts of referral information and tracking data.

The article itself is part of a section called “Digital Life”, which is sponsored by Microsoft — a company that runs a targeted programmatic advertising platform and allows Oath ads on its platforms, including in Windows. That what the people who make the big money call “synergy”, or “synchronicity”, or whatever.

If you look in your Web Inspector, you’ll notice that the article phones home to several trackers and contains loads of programmatic advertising. That makes it especially rich when you read to the bottom of what is generally a well-written explanation of how the market works:

The whole situation is summarised by data protection expert and privacy advocate Johnny Ryan.

“Every single time a person loads a page on a website that uses ‘programmatic’ advertising, information about what they are reading and the device they use is broadcast to a large number of adtech companies, who then do God knows what with it,” he explains.

[…]

“In GDPR terms, this “programmatic advertising” is a vast and ongoing data breach, and it means that everyone involved can be subject to an investigation by Elizabeth Denham, the Information Commissioner, and can be taken to court by Internet users.”

I’m not completely stupid; I understand why many websites — including this one — have analytics software and ads. But it is worth pointing out, and not solely to toot my own horn, that there is a vast difference between a “dumb” ad plus one or two analytics packages that do their best to anonymize traffic and respect Do Not Track, compared to the monstrosities created by companies like Oath and the Huffington Post that collect and distribute your browsing history on behalf of dozens of third parties in ways that are beyond your control.

You may, quite rightly, point out that the Huffington Post is not the pinnacle of journalism. But I would argue that the standards of the web should not be so low that we ought to tolerate privacy-invasive behaviour from anyone. And, for what it’s worth, practitioners of great journalism like the Washington Post and the Financial Times also have an egregious record when it comes to online tracking. It is their responsibility to give readers the best possible information, written as well as they can, and publish it on the safest and most reader-friendly platform available.

Two Angles on Apple Product Repairs

Joe Rossignol, MacRumors:

Due to advanced security features of the Apple T2 chip, iMac Pro and 2018 MacBook Pro models must pass Apple diagnostics for certain repairs to be completed, according to an internal document from Apple obtained by MacRumors.

For the 2018 MacBook Pro, the requirement applies to repairs involving the display, logic board, Touch ID, and top case, which includes the keyboard, battery, trackpad, and speakers, according to the document. For the iMac Pro, the requirement only applies to logic board and flash storage repairs.

If any of these parts are repaired in an iMac Pro or 2018 MacBook Pro, and the Apple diagnostics are not run, this will result in an inoperative system and an incomplete repair, according to Apple’s directive to service providers.

Apple’s diagnostic suite is limited to internal use by Apple Stores and Apple Authorized Service Providers, as part of what is called the Apple Service Toolkit. As a result, independent repair shops without Apple certification may be unable to repair certain parts on the iMac Pro and 2018 MacBook Pro.

Adam O’Camb of iFixit:

This service document certainly paints a grim picture, but ever the optimists, we headed down to our friendly local Apple Store and bought a brand new 2018 13” MacBook Pro Touch Bar unit. Then we disassembled it and traded displays with our teardown unit from this summer. To our surprise, the displays and MacBooks functioned normally in every combination we tried. We also updated to Mojave and swapped logic boards with the same results.

That’s a promising sign, and it means the sky isn’t quite falling — yet. But as we’ve learned, nothing is certain. Apple has a string of software-blocked repair scandals under its belt, including the device-disabling Error 53, a functionality-throttling Batterygate, and repeated feature-disabling incidents. It’s very possible that a future software update could render these “incomplete repairs” inoperative, and who knows when, or if, a fix will follow.

FUD aside, this is pretty good reporting: Apple’s repair guides say that, for security reasons, many of the components of the iMac Pro and 2018 MacBook Pro must pass a software diagnostics check after replacement; iFixit tested this and found it not to be the case that the product becomes inoperable, even though Apple’s guidance suggests that it will.

Maintaining the security of components like the keyboard, Touch ID sensor, and logic board seems completely fair to me. Even if Bloomberg’s recent report on compromised Supermicro servers from China turns out not to be exactly as described, it’s completely plausible for cheap parts to contain malicious components — HP’s laptops had a keylogger preinstalled, and there were reports last year that inexpensive replacement phone screens could track a user’s touch input.

But I also completely understand the value of right-to-repair legislation. Sometimes, a Genius Bar appointment is difficult to make either because they’re fully booked or there isn’t an Apple or Apple-certified store in your area. Other times, Apple’s retail staff may suggest needlessly expensive replacements when a simpler fix could be found by more experienced independent technicians.

Rather than compromising the security and privacy of their products, I’d like to see more progress made on certifying independent technicians and making Apple’s official tools more accessible. The security threat model isn’t the same as it once was; your phone probably has a lot more information on it than your computer of ten years ago. Yes, it’s more complicated to replace parts now, but it’s not entirely because companies like Apple want to lock out independent repair shops. Apple’s diagnostic tools could play a great role in this: imagine if you could take a printed report of a successful repair and type in a serial number on Apple’s website to verify that your device was serviced with genuine parts and passed Apple’s testing.

For a different story, Wayne Ma at the Information has a look inside the world of iPhone repair fraud in China. It’s paywalled, but Benjamin Mayo of 9to5Mac has a good summary. Ma:

Five years ago, Apple was forced to temporarily close what was then its only retail store in Shenzhen, China, after it was besieged by lines of hundreds of customers waiting to swap broken iPhones for new devices, according to two former Apple employees who were briefed about the matter. In May 2013, the Shenzhen store logged more than 2,000 warranty claims a week, more than any other Apple retail store in the world, one of those people said.

After some investigation, Apple discovered the skyrocketing requests for replacements was due to a highly sophisticated fraud scheme run by organized teams. Rings of thieves were buying or stealing iPhones and removing valuable components like CPUs, screens and logic boards, replacing them with fake components or even chewing gum wrappers, more than a half-dozen former employees familiar with the fraud said. The thieves would then return the iPhones, claiming they were broken, and receive replacements they could then resell, according to three of those people. The stolen components, meanwhile, were used in refurbished iPhones sold in smaller cities across China, two of the people said.

These criminals were so sophisticated that they resorted to bribing employees and acquiring the serial numbers of iPhones in China to support this scheme.

Ma’s report also helps explain my frustrating support experience at my local Apple Store:

To slow down fraud at its retail stores — a main point of vulnerability — Apple developed a reservation system, which required customers to make appointments online with proof of ownership before they could file claims, according to more than 10 former Apple employees. However, the system was soon swamped with hackers who exploited vulnerabilities in its website to snap up the time slots, one of the people said.

It’s unfortunate that many of the things that used to make Apple’s stores a completely different retail experience — the virtually untethered demo units, easy-to-access support, “surprise and delight”, and a comparatively relaxed staff presence — is being watered down either by crime or for what can often feel like financial reasons.

Assorted Updates Regarding Bloomberg’s ‘Big Hack’ Story

I was going to split these updates into several posts, but there are so many and they all fit around similar narratives that it makes more sense to bundle them together. Previously, I wrote a little about Bloomberg’s massive report and tech companies’ responses. After that came government corroboration of the companies’ statements, as well as a report from Buzzfeed that indicated that senior Apple executives were confused by Bloomberg’s findings.

Yesterday, George Stathakopoulos, Apple’s vice president of information security, sent a letter to congress once again reiterating their claim that they have not found malicious hardware planted in their servers, and that the FBI has not been contacted nor have they been contacted by the FBI about these concerns — this is clearly contrary to Bloomberg’s specific claim that “two of the senior Apple insiders say the company reported the incident to the FBI”. I cannot find any wiggle room in either statement on that matter.

One of the few sources in Bloomberg’s story that was willing to be named has now appeared on a podcast where he expresses concern over how his hypothetical ideas about how a piece of hardware like this might work have seemingly been entirely realized in the final article.

The team of Jordan Robertson and Michael Riley have a new article out today in Bloomberg that claims that a U.S. telecommunications company found manipulated Supermicro hardware in their possession two months ago:

The security expert, Yossi Appleboum, provided documents, analysis and other evidence of the discovery following the publication of an investigative report in Bloomberg Businessweek that detailed how China’s intelligence services had ordered subcontractors to plant malicious chips in Supermicro server motherboards over a two-year period ending in 2015.

Appleboum previously worked in the technology unit of the Israeli Army Intelligence Corps and is now co-chief executive officer of Sepio Systems in Gaithersburg, Maryland. His firm specializes in hardware security and was hired to scan several large data centers belonging to the telecommunications company. Bloomberg is not identifying the company due to Appleboum’s nondisclosure agreement with the client. Unusual communications from a Supermicro server and a subsequent physical inspection revealed an implant built into the server’s Ethernet connector, a component that’s used to attach network cables to the computer, Appleboum said.

Robertson and Riley stress that this is not an identical manipulation to the type described in their earlier story, but it tracks closely: hardware on a Supermicro board that could be used to siphon or reroute data.

However, Jason Koebler, Joseph Cox, and Lorenzo Franceschi-Bicchierai of Vice contacted American telecom companies and, so far, all are denying that Bloomberg’s report could possibly describe them. A source at Apple also told them that they launched another internal investigation after the story was published and they still can’t find any evidence of what Robertson and Riley are claiming.

For what it’s worth, I don’t want Robertson and Riley to have egg on their faces. I hope the story is not entirely as described because, if it is, it is truly one of the biggest security breaches in modern history — Supermicro has supplied a lot of servers to industry giants. But I don’t want the reporters to be wrong; Bloomberg has a great reputation for publishing rigorously-researched and fact-checked longform stories; I don’t want to have lingering doubts about their future reporting. And I’m not defending the biggest corporations in the world out of loyalty or denial — they have PR teams for that, and should absolutely be criticized when relevant. And I think the central point of the article — that the supply chain of a vast majority of the world’s goods is monopolized by an authoritarian and privacy-averse government is a staggering risk — is absolutely worth taking seriously.

But something about this story is not adding up. It doesn’t make sense as-is. I want to see more evidence and a corroborating third-party judgement. Bloomberg — and Michael Riley, in fact — appear to have gotten stories like this one wrong before. I hope that isn’t the case here, despite the terrifying reality if it is, indeed, completely true.

Update: Robert M. Lee was previously contacted by the same journalists regarding other stories while working at the NSA. He thought they were well-meaning, but duped by unsupported theories that didn’t withstand technical scrutiny.

Apple Releases iOS 12.0.1

This update fixes some WiFi, Bluetooth, and iPhone XS charging bugs; but, the best fix is this, documented by John Voorhees at MacStories:

iOS 12.0.1 includes a small design change on the iPad too. With the iOS 12 update, the ‘.?123’ key was moved. With version 12.0.1, that key has been restored to its previous position on the software keyboard.

For the first few days of running the iOS 12 beta, I didn’t notice this change. I did, however, notice the effects of this change. I couldn’t work out why I was suddenly inserting a lot more emoji into anything I was writing on my iPad until I looked at an old screenshot and figured out that the key for symbols and punctuation had been swapped with the emoji key. Presumably, this was changed for consistency with the 12.9-inch iPad Pro, but it upset seven years of iPad typing muscle memory.

Anyway, now that’s fixed and I can delete from my still-in-progress iOS 12 review the three paragraphs I spent pointing out what a terrible change this was.

Google Exposed Data of Half a Million Users Until March but Didn’t Disclose It Because They Feared ‘Regulatory Interest’

Douglas MacMillan and Robert MacMillan, Wall Street Journal:

Google exposed the private data of hundreds of thousands of users of the Google+ social network and then opted not to disclose the issue this past spring, in part because of fears that doing so would draw regulatory scrutiny and cause reputational damage, according to people briefed on the incident and documents reviewed by The Wall Street Journal.

As part of its response to the incident, the Alphabet Inc. unit plans to announce a sweeping set of data privacy measures that include permanently shutting down all consumer functionality of Google+, the people said. The move effectively puts the final nail in the coffin of a product that was launched in 2011 to challenge Facebook Inc. and is widely seen as one of Google’s biggest failures.

A software glitch in the social site gave outside developers potential access to private Google+ profile data between 2015 and March 2018, when internal investigators discovered and fixed the issue, according to the documents and people briefed on the incident. A memo reviewed by the Journal prepared by Google’s legal and policy staff and shared with senior executives warned that disclosing the incident would likely trigger “immediate regulatory interest” and invite comparisons to Facebook’s leak of user information to data firm Cambridge Analytica.

Chief Executive Sundar Pichai was briefed on the plan not to notify users after an internal committee had reached that decision, the people said.

That this disclosure wasn’t made until today — seven months after this breach was noticed — is unconscionable. But it is outrageous that the reason for not disclosing it in the first place was because they wanted to hide it from the law and that Pichai knew about it.

By the way, because Google tried so hard to make Google Plus work, it’s possible that your Google account — if you have one — is a Google Plus profile. You can disconnect it; Google calls it “downgrading”.

This is a fitting end to a bad product managed by people who were almost explicit in their intention for it to collect boatloads more information for advertisers.

Update: Brian McCullough:

Has anyone made this point yet? Pichai refused to testify to congress because he couldn’t. He would have either had to perjure himself or reveal this bug in real time before the committee.

I thought it was just strategic brilliance to let Facebook take all the heat. No, it was next level cowardice. One wonders if they really though they could whistle past the graveyard on this. In which case, also next level hubris.

Pichai is now scheduled to testify before Congress in November.

Update: Jack Wellborn:

I can’t help but think that by taking 7 months to publically disclose this breach, this incident makes Google seem somewhat hypocritical given their strict Project Zero policy to disclose vulnerabilities 90-days when patches aren’t released.

After a Year of Stories Confirming the Logical Consequences of Collecting All of Your Personal Information, Facebook Introduces an Always-Listening Assistant With a Video Camera

Nicole Nguyen, Buzzfeed:

Today, Facebook — which is still reeling from the fallout of the Cambridge Analytica data scandal and last month’s massive security breach — announced a voice-activated gadget with a screen, always-listening microphone, and camera designed for video chat called Facebook Portal. It’s like an Amazon Echo Show for Facebook Messenger.

There are two models: a small 10-inch Portal ($199) and a larger 15-inch Portal+ ($349), which can rotate to portrait or landscape orientations.

Saying a simple command, “Hey Portal,” and then the name of the person you’d like to call, starts a video chat. The camera has the ability to track people when they enter the room, and it can pan, widen, and zoom automatically. The devices also include the always-listening Alexa, Amazon’s voice assistant, and can be used to control smart home devices and offer weather information.

Nobody should buy this product. Moreover, it’s absurd that Facebook would think that now would be a terrific time to introduce an always-listening box with a camera — no matter how many reassuring bullet points they slap on a marketing webpage.

Apple Insiders Say Nobody Internally or at the FBI Knows What’s Going on With Bloomberg’s Story

John Paczkowski and Charlie Warzel, Buzzfeed:

Reached by BuzzFeed News multiple Apple sources — three of them very senior executives who work on the security and legal teams — said that they are at a loss as to how to explain the allegations. These people described a massive, granular, and siloed investigation into not just the claims made in the story, but into unrelated incidents that might have inspired them.

[…]

Equally puzzling to Apple execs is the assertion that it was party to an FBI investigation — Bloomberg wrote that Apple “reported the incident to the FBI.” A senior Apple legal official told BuzzFeed News the company had not contacted the FBI, nor had it been contacted by the FBI, the CIA, the NSA or any government agency in regards to the incidents described in the Bloomberg report. This person’s purview and responsibilities are of such a high level that it’s unlikely they would not have been aware of government outreach.

Guy Faulconbridge and Joseph Menn, Reuters:

Apple’s recently retired general counsel, Bruce Sewell, told Reuters he called the FBI’s then-general counsel James Baker last year after being told by Bloomberg of an open investigation into Super Micro Computer Inc , a hardware maker whose products Bloomberg said were implanted with malicious Chinese chips.

“I got on the phone with him personally and said, ‘Do you know anything about this?,” Sewell said of his conversation with Baker. “He said, ‘I’ve never heard of this, but give me 24 hours to make sure.’ He called me back 24 hours later and said ‘Nobody here knows what this story is about.’”

Reuters also reports that a division of GCHQ, Britain’s signals intelligence agency, does not presently doubt Apple and Amazon’s denials. Here’s the score so far:

  • Bloomberg is sticking by its reporting that modified circuit boards with potentially devastating security concerns were found by Apple and Amazon in servers of theirs supplied by and made for Supermicro. They also stand by the existence of cooperation between the tech companies and the FBI in an investigation that has been going on for years.

  • Apple and Amazon have both denied specific allegations in Bloomberg’s story, and have refuted its overall premise. Amazon’s chief information security officer and, now, Apple’s former senior-most legal counsel have put their names behind categorical denials of finding manipulated hardware in their data centres and having any knowledge of an FBI investigation, respectively.

  • Apple’s former legal representative has also said that a senior contact at the FBI told him that they didn’t know anything about this story.

  • British intelligence says that they believe Apple and Amazon’s statements at this time.

  • The U.S. administration has seized upon Bloomberg’s report to continue their campaign of criticism of the Chinese government.

That’s a lot of reputable organisations — and the American government — who have staked their credibility on widely varying accounts of the veracity of this story.

Update: Now the U.S. Department of Homeland Security is echoing the British viewpoint in support of the ostensibly affected companies’ statements, even while the Vice President is using Bloomberg’s report for political purposes.

Thinking About Bloomberg’s Report on Hardware Vulnerabilities in Servers Made in China

Jordan Robertson and Michael Riley of Bloomberg today published a startling report alleging that servers made in China for Supermicro and used by — amongst others — Apple, Amazon, and U.S. federal government agencies have been found to surreptitiously carry tiny chips, likely for backdoor access by the Chinese government, and installed without the knowledge of the companies through deep infiltration into the electronics supply chain. The report also states that individuals at Apple and Amazon discovered this several years ago, did not immediately make changes to their infrastructure, and are working with law enforcement and intelligence agencies, but none of this has been previously disclosed.

If these allegations are true, this would represent one of the most significant national security breaches in decades. Its effects could extend beyond current U.S. sanctions in place on Chinese-made electronic components to the entire electronics supply chain, the vast majority of which is based in China. It would also imply that massive amounts of Apple and Amazon customer data may have been at risk without public acknowledgement, though the report states that “[no] consumer data is known to have been stolen”.

Robertson and Riley:

As recently as 2016, according to DigiTimes, a news site specializing in supply chain research, Supermicro had three primary manufacturers constructing its motherboards, two headquartered in Taiwan and one in Shanghai. When such suppliers are choked with big orders, they sometimes parcel out work to subcontractors. In order to get further down the trail, U.S. spy agencies drew on the prodigious tools at their disposal. They sifted through communications intercepts, tapped informants in Taiwan and China, even tracked key individuals through their phones, according to the person briefed on evidence gathered during the probe. Eventually, that person says, they traced the malicious chips to four subcontracting factories that had been building Supermicro motherboards for at least two years.

As the agents monitored interactions among Chinese officials, motherboard manufacturers, and middlemen, they glimpsed how the seeding process worked. In some cases, plant managers were approached by people who claimed to represent Supermicro or who held positions suggesting a connection to the government. The middlemen would request changes to the motherboards’ original designs, initially offering bribes in conjunction with their unusual requests. If that didn’t work, they threatened factory managers with inspections that could shut down their plants. Once arrangements were in place, the middlemen would organize delivery of the chips to the factories.

The investigators concluded that this intricate scheme was the work of a People’s Liberation Army unit specializing in hardware attacks, according to two people briefed on its activities. The existence of this group has never been revealed before, but one official says, “We’ve been tracking these guys for longer than we’d like to admit.” The unit is believed to focus on high-priority targets, including advanced commercial technology and the computers of rival militaries. In past attacks, it targeted the designs for high-performance computer chips and computing systems of large U.S. internet providers.

These allegations are precise, comprehensive, and are clearly based on tremendous investigative reporting. However, the comments issued by Apple and Amazon have been uncharacteristically detailed as well.

Apple published their un-bylined responses to Bloomberg’s questions at various times throughout the reporting process:

On this we can be very clear: Apple has never found malicious chips, “hardware manipulations” or vulnerabilities purposely planted in any server. Apple never had any contact with the FBI or any other agency about such an incident. We are not aware of any investigation by the FBI, nor are our contacts in law enforcement.

In response to Bloomberg’s latest version of the narrative, we present the following facts: Siri and Topsy never shared servers; Siri has never been deployed on servers sold to us by Super Micro; and Topsy data was limited to approximately 2,000 Super Micro servers, not 7,000. None of those servers have ever been found to hold malicious chips.

This response unequivocally refutes specific allegations made in the Bloomberg report. This isn’t one of those stories where Apple’s PR team is being cagey or not commenting; they’re calling the story flat-out false. And the same is true for Amazon’s statement:

The article also claims that after learning of hardware modifications and malicious chips in Elemental servers, we conducted a network-wide audit of SuperMicro motherboards and discovered the malicious chips in a Beijing data center. This claim is similarly untrue. The first and most obvious reason is that we never found modified hardware or malicious chips in Elemental servers. Aside from that, we never found modified hardware or malicious chips in servers in any of our data centers. And, this notion that we sold off the hardware and datacenter in China to our partner Sinnet because we wanted to rid ourselves of SuperMicro servers is absurd. Sinnet had been running these data centers since we launched in China, they owned these data centers from the start, and the hardware we “sold” to them was a transfer-of-assets agreement mandated by new China regulations for non-Chinese cloud providers to continue to operate in China.

This statement was attributed to Steve Schmidt, Amazon’s chief information security officer and a former FBI section chief.

Supermicro and the Chinese government also issued denials of Bloomberg’s report. The cynical response is something like: of course these companies are denying an extremely sensitive report, whether because it’s embarrassing or due to a law enforcement requirement. But neither situation appears to be the case here. Apple confirmed in their statement that they are not under any sort of gag order that would prevent them from being able to comment on this.

Furthermore, Apple and Amazon are publicly-traded companies and, as a result, lying in public statements such as these would be an SEC violation. These aren’t the typical if-you-squint-it-could-be-seen-as-accurate statements that big companies’ PR teams typically release as damage control. They are wholesale rejections of key arguments in Bloomberg’s reporting: Bloomberg says that hardware modifications and malicious chips were found by Amazon and Apple in their servers; Amazon and Apple say that no hardware modifications or malicious chips were found in their servers. There’s not a lot of room for ambiguity.

This story has been rattling around my head all day today. My early thought was that perhaps the Bloomberg reporters did a Judith Miller. Maybe their government sources had a specific angle they wished to present to create a political case against China or in favour of further sanctions — or actions far more serious — and needed a credible third-party, like a news organization, to create a story like this. But Robertson and Riley’s seventeen sources include several individuals at Amazon and Apple with intimate knowledge of the apparent discovery of unauthorized hardware modifications, something they later confirmed in a statement to Alex Cranz of Gizmodo. This doesn’t seem likely.

Zack Whittaker in TechCrunch points to a couple of ways that these statements may technically be accurate, and how the reporting may be true as well:

Naturally, people are skeptical of this “spy chip” story. On one side you have Bloomberg’s decades-long stellar reputation and reporting acumen, a thoroughly researched story citing more than a dozen sources — some inside the government and out — and presenting enough evidence to present a convincing case.

On the other, the sources are anonymous — likely because the information they shared wasn’t theirs to share or it was classified, putting sources in risk of legal jeopardy. But that makes accountability difficult. No reporter wants to say “a source familiar with the matter” because it weakens the story. It’s the reason reporters will tag names to spokespeople or officials so that it holds the powers accountable for their words. And, the denials from the companies themselves — though transparently published in full by Bloomberg — are not bulletproof in outright rejection of the story’s claims. These statements go through legal counsel and are subject to government regulation. These statements become a counterbalance — turning the story from an evidence-based report into a “he said, she said” situation.

Indeed, Kieren McCarthy of the Register did a fine job parsing each company’s statements, albeit with his usual unique flair. But, though there is absolutely some wiggle-room in each denial, there are remarks made by each company that, were they found to be wrong, would be simple lies.

There are aspects of Robertson and Riley’s reporting that are consistent with previously-acknowledged problems and security concerns with Supermicro’s servers. Early last year, Amir Efrati of the Information reported that Apple was removing Supermicro’s servers from its data centres after a compromised firmware update the previous year. Robertson and Riley are reporting tonight that a Supermicro software update server was infiltrated in 2015; the same report also reiterates that Apple found hardware vulnerabilities on their servers.

This is a complicated story and apparently just the first in a series. My hope is that we’ll know more details soon, and a clearer picture of the truth will emerge. Right now, however, the credibility of a news organization and two trillion-dollar companies is on the line. But the nugget of this story — that outsourced and complex supply chains are prone to abuse due to bad actors and lack of oversight — is a known problem that isn’t taken anywhere near as seriously as it should be. In the garment industry, it’s at least partially responsible for deadly yet preventable incidents. In electronics, the prospect of compromised parts was once science fiction; it may now be reality.

Jason Koelber and Joseph Cox, Vice:

In 2005, the Pentagon warned in a report that outsourcing electronics manufacturing to China could become a problem for America, because of the risk of hardware “tampering.” America has largely lost the ability to create many of the electronics we use everyday — Donald Trump famously asked Apple CEO Tim Cook why the iPhone isn’t made in America, but it’s not clear that the United States is even capable of making iPhones in America at any sort of scale.

China’s cheap, skilled labor, manufacturing infrastructure, and vast rare Earth mineral-mining operations around the world have secured its spot as the high-tech manufacturing hub of the world. This of course has had many benefits for the United States and American companies, but it’s also a great risk.

There is a clear theoretical lesson in all of this, which is that monopolization of anything is extraordinarily risky and often self-destructive. Witness, for example, the ongoing debate over how much moderation power should be exerted by Facebook over posts made on the platform — it’s a difficult question to answer with any certainty in large part because it’s a decision that affects billions of users and a large chunk of worldwide communications. In the case of an apparently-compromised electronics supply chain with decades of highly-specialized knowledge and located in a country governed by an oppressive regime, any resolution is going to be painfully difficult. Outsourcing has deep flaws; even Bloomberg’s website is witness to that. Either manufacturing of these components becomes increasingly diversified or, more likely, far greater control and oversight is required by companies and end-client governments alike.

Mainstream Advertising is Still Showing Up on Conspiracy and Extremist Websites

Craig Timberg, Elizabeth Dwoskin and Andrew Ba Tran, Washington Post:

Jihadi rapists. Muslim invaders. Faked mass shootings. Pizzagate.

Somebody browsing highly partisan websites in recent weeks could have seen articles about all of these subjects — and on the same pages seen cheerful green ads for the Girl Scouts, bearing the slogan “Helping Girls Change the World!”

Such juxtapositions, documented by a Washington Post review of advertising on hundreds of websites, are more than simply jarring. They are products of online advertising systems that regularly put mainstream ads alongside content from the political fringes — and dollars in the pockets of those producing polarizing and politically charged headlines.

Because this is the Post, they use a rather mild description of the kind of horseshit they found mainstream advertisers implicitly supporting. Jeep, Hertz, and the Girl Scouts wouldn’t sponsor a Ku Klux Klan rally; if an ad agency supporting them put their banners up at an extremist’s event, they would be fired. Yet Google somehow has poor control over which websites may use AdSense, especially at the scale at which they operate:

Google says it does not serve ads on sites that feature hate speech, including bullying, harassment or content deemed derogatory or dangerous, and it prohibits publishers that misrepresent their identities. Last year, Google removed 320,000 publishers from the ad network for policy violations and blacklisted nearly 90,000 websites and 700,000 mobile apps, it said.

Those are huge numbers, but so are the numbers in their quarterly earnings report (PDF). I’m not suggesting that Google should be a non-profit, but they certainly can afford more moderators to review what websites are allowed to be in the AdSense program.

As it stands now, advertisers must manually blacklist websites and categories of sites that they don’t wish to see their ads on. If Steven Black’s hosts file is any guidance, that’s a lot of properties that must be blacklisted. Surely it would be more efficient for Google, instead, to quarantine every domain on that list that’s part of their AdSense program.

Update: The unwillingness for ad networks to be more judicious about where their ads may be used might have something to do with how hard it is for them to be held accountable by non-technical users. Think about how hard it is to know — without looking at a site’s markup — which ad network is supporting a website.

I imagine if every placement were required to have visible attribution, ad networks would be a lot more careful about which sites would be allowed. The first time “Powered by Google” appeared on some freelance propagandist’s website or a crank doctor’s bad advice on vaccinations, you know that users would notice.

MacOS Mojave Archaeology

“Uluroo” collected a series of examples of oddities and legacy support in MacOS. My favourite — other than the continued availability of a degauss function in Mojave — is their commentary on Dashboard:

Dashboard is still skeuomorphic. This surprises Uluroo a lot, given that iOS 7 killed skeuomorphism completely on the iPhone five years ago.

Many of Dashboard’s built-in widgets have a refreshingly retro, though inconsistent, aesthetic: Stocks, Dictionary, Weather, Calculator, Calendar, and more all look like they’ve gone untouched since the days of Scott Forstall. The World Clock widget’s second hand moves in the same way as a real clock, rather than moving in a smooth, uninterrupted motion like in iOS and watchOS. Apple still has a built-in “Tile Game” widget. Uluroo wonders if Dashboard will ever be updated to behave more like the Mac’s version of Control Center, or if Apple just doesn’t care much about it anymore.

The surprising thing, for me, about Dashboard is not that it continues to be skeuomorphic; it’s that it exists at all without a single update for years. What was once a top-line feature of Tiger has become abandonware.

(Via Michael Tsai.)

Increased Exclusivity Arrangements Correlates With the Reversal of a Downward Trend of File Sharing

Cam Cullen of Sandvine, a network management and analytics company:

In the first Global Internet Phenomena Report in 2011, file sharing was huge on fixed networks and tiny on mobile. In the Americas, for example, 52.01% of upstream traffic on fixed networks and 3.83% of all upstream mobile traffic was BitTorrent. In Europe, it was even more, with 59.68% of upstream on fixed and 17.03% on mobile. By 2015, those numbers had fallen significantly, with Americas being 26.83% on the upstream and Europe being 21.08% on just fixed networks. During the intervening year, traffic volume has grown drastically on the upstream, with more social sharing, video streaming, OTT messaging, and even gaming on it.

That trend appears to be reversing, especially outside of the Americas. In this edition of the Phenomena report, we will reveal how file sharing is back.

From the report (PDF):

We will talk quite a bit about video in this report, but it is important to highlight the diversity of video streaming traffic around the world. Although Netflix and YouTube are still the largest names in streaming (as you will see in the reports) there is an ever growing number of other streaming providers capturing consumer screen time.

This video diversity trend has led directly to the continued relevance of file sharing, which is still a major source of internet traffic. Consumers that cannot afford to subscribe to all of the different services turn to file sharing to get the latest content, even as governments attempt to shut down sharing sites.

At about $10 per month — give or take — per subscription, those costs begin to add up quickly, especially if users are only choosing a service or channel for one or two shows. This doesn’t seem realistic or sustainable as a long-term industry plan.

Vice News’ Interview With Tim Cook

Elle Reeve of Vice sat down with Tim Cook at Apple’s Grand Central Terminal store to discuss privacy, regulation, and the company’s decision to kick Alex Jones’ extremist fact-free fairy tales off its platforms. There’s one exchange I’d like to highlight, regarding Apple in China:

Reeve: In terms of privacy as a human right, does that apply to how you do business in China?

Cook: It absolutely does. Encryption, for us, is the same in every country in the world. We don’t design encryption […] for the U.S., and do it differently everywhere else. It’s the same. [So] if you send a message in China, it’s encrypted, [and] I can’t produce the content. I can’t produce it in the United States either. If you lock your phone in China, I can’t open it.

The thing in China that some people have confused is certain countries — and China is one of them — has a requirement that data from local citizens has to be kept in China. We worked with a Chinese company to provide iCloud. But the keys, which is the “key”, so to speak — pardon the pun — are ours.

Reeve: But haven’t they moved to China? Meaning: it’s much easier for the Chinese government to get to them.

Cook: Now, I wouldn’t get caught up in where’s the location of it?. I mean, we have servers located in many different countries in the world. They’re not easier to get data from being in one country versus the next. The key question is [sic]: how does the encryption process work? and who owns the keys, if anyone?. In most cases, for us, you and the receiver own the keys.

Apple’s executives are generally plainspoken and direct. Cook injects more corporate speak into his interview responses than, for example, Steve Jobs or Phil Schiller, but he still generally says what he means and avoids obfuscating. So it’s noticeable — and notable — when any Apple executive is cagey, as is the case here.

Cook’s response to Reeve’s second question sidesteps the comparative ease with which Chinese authorities can now demand access to users’ data because they no longer have to go through the stricter legal system of the United States. That appears to be a pretty significant concern to simply gloss over. Of similar concern is that the Chinese company that Apple partnered with to offer iCloud in the country is owned and operated by the Guizhou provincial government.

I don’t think it’s fair to say that Chinese users’ privacy is not subject to compromise. The actual method of encryption may not be any different or weaker than in other countries, but the requirement to store keys in the country behind weaker legal protections for users makes it, in practice, less strong. It is not a product of Apple’s own doing, and the only way they would be able to wipe their hands clean is to entirely discontinue iCloud and other internet services in China. I don’t know that it would be right — it’s likely that the replacement services chosen by users would be far worse for privacy — but it would mean that the company has no implicit connection to complying with a regime that has a piss-poor track record on human rights.

U.S. Justice Department Sues Hours After California Signs Strong Net Neutrality Law

Jazmine Ulloa, Los Angeles Times:

News that the governor signed the ambitious new law was swiftly met with an aggressive response from Justice Department officials, who announced soon afterward that they were suing California to block the regulations. The state law prohibits broadband and wireless companies from blocking, throttling or otherwise hindering access to internet content, and from favoring some websites over others by charging for faster speeds.

[…]

The bill’s August passage in the Legislature capped months of feuding between tech advocates and telecom industry lobbyists. Telecom giants such as AT&T and Verizon Communications poured millions into killing the legislation, while grass-roots activists fought back with crowdsourced funding and social media campaigns.

After Comcast and Verizon asked, the FCC was only too happy to prevent states from enacting their own net neutrality legislation. As far as I can tell, the DoJ hasn’t tried to block Washington’s similar law yet.

See Also: Jerri-Lynn Scofield’s summary and overview; Cecilia Kang’s reporting.

And Also: Karl Bode at TechDirt.

New Zealand Customs Authorities Can Now Demand Device Passwords, and May Copy and Review Data

Asha McLean, ZDNet:

The New Zealand Customs Service this week received new powers at the country’s borders, including the ability demand a password off a passenger to search their “electronic device”.

Customs officers have always been able to search a passenger’s laptop or phone, but the changes to the Customs and Excise Act 2018 now specifies that passengers must hand over their password.

[…]

Customs now also has the right to copy, in addition to review, the data stored on the device, and can also confiscate it to conduct a further search.

New Zealand isn’t the first place I’d think of as becoming a draconian country for visitors, but I was clearly myopic. If you’re travelling these days, it’s advisable — if you have the means — to travel with devices containing nothing more than their operating systems, and use a well-secured cloud service to store any files you might need while in transit, including your keychain. While New Zealand’s revised customs act does not permit them to download remote data, they could obtain a copy of your keychain which is typically encrypted with the same user account password you would have provided.

You can change your keychain password to be different if you wish, but you will likely need to reenter its password frequently, and it likely won’t protect you against legislation like this — but, alas, I am not a lawyer.

A Deep Exploration of the iPhone XS Camera System

Sebastiaan de With, writing on the Halide blog:

An iPhone XS will over- and underexpose the shot, get fast shots to freeze motion and retain sharpness across the frame and grab every best part of all these frames to create one image. That’s what you get out of the iPhone XS camera, and that’s what makes it so powerful at taking photos in situations where you usually lose details because of mixed light or strong contrast.

This isn’t the slight adjustment of Auto HDR on the iPhone X. This is a whole new look, a drastic departure from the “look” of every iPhone before it. In a sense, a whole new camera.

I don’t think this different look is a regression by any means — in fact, all of the photos I’ve seen from the iPhone XS indicate that this is a massive upgrade — but it is different. The rear cameras have large enough sensors and lenses that they are able to compensate for the higher noise created by faster shutter speeds through more intense noise reduction while preserving detail. When it comes to the front-facing camera’s much smaller sensor, though, it appears that the noise reduction is tuned to be a little more aggressive than expected, and it sounds like Apple is tweaking it.

One tip for RAW shooters:

To add insult to injury, iPhone XS sensor’s noise is just a bit stronger and more colorful than that of the iPhone X.

This isn’t the kind of noise we can easily remove in post-processing. This isn’t the gentle, film-like grain we previously saw in iPhone X and iPhone 8 RAW files.

As it stands today, if you shoot RAW with an iPhone XS, you need to go manual and under-expose. Otherwise you’ll end up with RAWs worse than Smart HDR JPEGs. All third-party camera apps are affected. Bizarrely, RAW files from the iPhone X are better than those from the iPhone XS.

With its bigger sensor, you should be able to get more detail out of an iPhone XS RAW image. But because this camera system is tuned to merge multiple exposures, it’s not quite as straightforward. This is a great piece for iPhone photographers.