Kashmir Hill, New York Times:
Mark noticed something amiss with his toddler. His son’s penis looked swollen and was hurting him. Mark, a stay-at-home dad in San Francisco, grabbed his Android smartphone and took photos to document the problem so he could track its progression.
[…]
After setting up a Gmail account in the mid-aughts, Mark, who is in his 40s, came to rely heavily on Google. He synced appointments with his wife on Google Calendar. His Android smartphone camera backed up his photos and videos to the Google cloud. He even had a phone plan with Google Fi.
Two days after taking the photos of his son, Mark’s phone made a blooping notification noise: His account had been disabled because of “harmful content” that was “a severe violation of Google’s policies and might be illegal.” A “learn more” link led to a list of possible reasons, including “child sexual abuse & exploitation.”
Hill is among the best reporters anywhere on these sensitive, nuanced topics, and this story is no exception. Through careful reporting, Hill writes about two people whose Google accounts were closed — and, at least in Mark’s case, deleted from its servers — because both had taken photos of their naked children for diagnostic purposes.
I think Hill’s article thoughtfully explores the difficult and often contradictory questions around CSAM policing — certainly in a more cogent way than I can write about it — and I think this article is worth reading in full. The part I am more able to comment on is Google’s final decision to lock these parents out of their accounts.
It is a stunning display of Google’s power — and a painful reminder about single points of failure — that it is able to eradicate someone’s connection to the world without warning or recourse. Ben Thompson went so far as to call it a civil liberties violation in spirit, if not in law.
The sad thing is how unsurprising this is to anyone who has tried to deal with Google in any capacity aside from being an ad buyer. A few years ago, Talking Points Memo’s Josh Marshall wrote about his frustration with Google’s demonetization of stories about right-wing terrorists:
With the events of recent months and years, Google is apparently now trying to weed out publishers that are using its money streams and architecture to publish hate speech. Certainly you’d probably be unhappy to hear that Stormfront was funded by ads run through Google. I’m not saying that’s happening. I’m just giving you a sense of what they are apparently trying to combat. Over the last several months we’ve gotten a few notifications from Google telling us that certain pages of ours were penalized for ‘violations’ of their ban for hate speech. When we looked at the pages they were talking about they were articles about white supremacist incidents. Most were tied to Dylann Roof’s mass murder in Charleston.
Now in practice all this meant was that two or three old stories about Dylann Roof could no longer run ads purchased through Google. I’d say it’s unlikely that loss to TPM amounted to even a cent a month. Totally meaningless. But here’s the catch. The way these warnings work and the way these particular warnings were worded, you get penalized enough times and then you’re blacklisted.
Now, certainly you’re figuring we could contact someone at Google and explain that we’re not publishing hate speech and racist violence. We’re reporting on it. Not really. We tried that. We got back a message from our rep not really understanding the distinction and cheerily telling us to try to operate within the no hate speech rules. And how many warnings until we’re blacklisted? Who knows?
TPM also faced a different issue where its main email account, a G Suite paid Gmail account, was blocked without notification because Google flagged it as a source of spam.
It is unfair to blame TPM for relying on Google’s email services, which are among the best options for a managed email product on a custom domain, or for its use of Google ads, which are ubiquitous. Similarly, heavy use of Google services like Mark and Cassio — the other dad in Hill’s story — is pretty normal and encouraged by the tight integration of these products.
Google’s response in each of these cases points to a lack of humanity. It reflects a complete absence of care about the context in which its products are being used, from a company that has a primarily American perspective and may miss real problems in other countries. It is a recipe for more stories like these, especially since a Google spokesperson told Hill the company stood behind its decision. Not only does Google think it did the right thing, it believes deleting Mark’s entire Google presence was the right outcome.
Someone more sympathetic than I am might point out that Google will always struggle to understand context because it is operating at a prohibitively massive scale. This is a cop-out or, at least, an incomplete thought. Google, like many other big businesses, has prioritized growth at the expense of caution because the incentives outweigh the risks. Some variation of this is true across industries, from banking to natural resources to food. Nestlé is practically synonymous with a jaw-dropping lack of ethics but people keep buying Perrier and Cheerios.
It is overly simplistic to say these problems would not exist if businesses were smaller, but I believe shrinking businesses would minimize these problems. And, when they do appear, they would have a smaller effect. I disagree with Thompson’s assessment that “it seems silly to argue that getting banned from a social media platform isn’t an infringement on individual free speech rights”; far from Thompson’s claim that “you can still say whatever you want on a street corner”, you can write whatever you want on a website untouched by Facebook or Twitter, as he did. We have never had so much freedom to speak our minds and broadcast it to an audience. But we have never placed so much of our identity in the hands of such a small number of private entities, often poorly regulated. Software and services need a warranty and they need something like a bill of rights; and, if those demands are untenable at scale, then vendors should be smaller.