Since Adobe will no longer be supporting Flash Player after December 31, 2020 and Adobe will block Flash content from running in Flash Player beginning January 12, 2021, Adobe strongly recommends all users immediately uninstall Flash Player to help protect their systems.
Flash, from the very beginning, was a transitional technology. It was a language that compiled into a binary executable. This made it consistent and performant, but was in conflict with how most of the web works. It was designed for a desktop world which wasn’t compatible with the emerging mobile web. Perhaps most importantly, it was developed by a single company. This allowed it to evolve more quickly for awhile, but goes against the very spirit of the entire internet. Long-term, we never want single companies — no matter who they may be — controlling the very building blocks of the web. The internet is a marketplace of technologies loosely tied together, each living and dying in rhythm with the utility it provides.
Most technology is transitional if your window is long enough. Cassette tapes showed us that taking our music with us was possible. Tapes served their purpose until compact discs and then MP3s came along. Then they took their rightful place in history alongside other evolutionary technologies. Flash showed us where we could go, without ever promising that it would be the long-term solution once we got there.
I am not as rosy-eyed about Flash as Davidson. Most of the Flash-based websites I remember loaded slowly, performed poorly, and were hard to use. I remain conflicted about a more interactive web and the entire notion of websites as applications, and I find it hard to be so kind to a plug-in that was responsible for so many security and stability problems.
But I do appreciate its place in history. Streaming video in the pre-Flash era was a particularly painful mix of codecs only supported by one of Real Player, Windows Media Player, or QuickTime. Flash video players allowed the web to standardize around H.264, eventually without requiring an SWF-based decoder.
It is impossible to know if we would have ended up with rich typography, streaming video players, full web applications, and online games without Flash — and, in the case of the latter two, Java. Regardless of my ambivalence, the web that we have today is rich, universal, and accessible, and much of that groundwork was catalyzed by Flash.
The next time you come across an Amazon product with hundreds or thousands of reviews, look closely. You might find some of them are for completely different products.
[…]
The mismatch between the product and the review may appear random, perhaps a glitch, but it’s a deliberate tactic Amazon sellers use to accumulate reviews. They take an existing product page, then update the photo and description to show an entirely different product. By retaining all the existing reviews, the new product looks more tested and legitimate to shoppers — and in the world of online reviews, quantity is key. More ratings make a product appear to be more well-reviewed and, ultimately, boosts sales.
When I sorted the reviews by date, I saw that the most recent reviewers actually had bought a drone and they were overwhelmingly not giving it five stars. “Bought this for my Grandson,” a customer wrote on December 26. “He played with it for 2 hours before it broke and is no longer working.” He gave the drone one star.
But the older reviews were for honey. Apparently, the manufacturer had tricked Amazon into displaying thousands of reviews for an unrelated product below its drone, helping the drone to unfairly rise to the top of Amazon’s search results.
The story was similar for the second and third results in my drone search. Both had thousands of reviews with five-star averages. In both cases, many of the five-star reviews were obviously for other products — including a bottle of vodka, a bracelet, and a box of Christmas cards.
This form of review fraud is so trivial for a seller to do but it is invaluable given the number of people who, according to Nguyen’s reporting, only look at the star rating. It is also something that Amazon should be able to guard against.
You may have heard that, several months ago, Glenn Greenwald was told by a senior editor at the Intercept, a publication he co-founded, to correct some factually-dubious claims in his work. He claimed this amounted to censorship and quit in a huff to start a new column on Substack with maybe a few friends. For months now, he has sent email newsletters to subscribers with a typically fervent and melodramatic flair. Consider this recent piece which posits that:
The outgoing administration was pretty typical for a U.S. presidency, and to imply that it flirted with authoritarian policies is an irresponsible exaggeration.
The real authoritarians are the executive teams of big tech companies and the incoming administration.
Like many of Greenwald’s columns, there are elements of truth to both of these statements, which have been shorn of context then, depending on the point he is attempting to make, either magnified or minimized for full effect.
I promise I won’t get too much into the weeds with my summary of the first half of Greenwald’s essay because it is the second half that interests me more.1 However, to substantiate the first argument, Greenwald says that the president’s rhetoric often did not match the actions of his administration. For example, though the president repeatedly floated the idea of banning followers of the world’s second-largest religion from entering the United States, the eventual policy amounted to banning travellers from some majority Muslim countries. That is certainly not fair, but the Supreme Court upheld the president’s right to control the borders and, to Greenwald, that means the outgoing president did not display any more of an authoritarian streak than his predecessors that did not attempt to ban entire religions:
Whether Trump secretly harbored despotic ambitions is both unknowable and irrelevant. If he did, he never exhibited the slightest ability to carry them out or orchestrate a sustained commitment to executing a democracy-subverting plot. And the most powerful U.S. institutions — the intelligence community and military brass, Silicon Valley, Wall Street, and the corporate media — opposed and subverted him from the start. In sum, U.S. democracy, in whatever form it existed when Trump ascended to the presidency, will endure more or less unchanged once he leaves office on January 20, 2021.
It is not “unknowable” whether he had “despotic ambitions” — it is right there in his speeches and actions, however limp, fact-free, and legally-dubious they were. In the paragraph that precedes this, Greenwald brushes aside the multiple lawsuits filed by this president’s failed reelection campaign because they were implausible.2 But they were real lawsuits because the president really does not want to leave this job. He likes power; you can tell by his, as ProPublica put it, “last-minute killing spree”. It is incorrect to call him an authoritarian. But it is completely accurate to say that he moved more explicitly in that direction than previous administrations, and it is only because of activists and journalists like, yes, Greenwald that checks and balances were mobilized to mollify some of the excesses of this administration.
It is also not irrelevant. This administration is a wake-up call to those in the United States and around the world living in what we like to consider stable advanced democracies that there are politicians with similar and greater ambitions of power. Those people are not as unelectable as we would like to believe. It is also a reminder that the press ought to subject the incoming administration to similar scrutiny. McKay Coppins, the Atlantic:
Yamiche Alcindor, a correspondent for PBS NewsHour, told me she hopes her colleagues will retain the lessons they’ve learned from covering Trump. The default skepticism toward government officials, the aversion to euphemism, the refusal to accept approved narratives—to Alcindor, these are features of a healthy press, not signs that something is amiss. She attributes this attitude to her background covering race and policing. “When something is racist, we should just say it’s racist,” she said. “When someone is lying, we should just say they’re lying.” (Trump has repeatedly singled Alcindor out at press conferences, calling her “threatening” and her questions “nasty.”)
It is Greenwald’s second point which is why I felt compelled to write this, though. I have written extensively about the worries I have about monopolistic companies — particularly in tech and tech-adjacent fields like telecommunications, because that is the kind of column this is — and also about platforms’ failed moderation policies. These are inherently related concepts: as platforms become bigger, their small-scale moderation failures also grow; and, as more communications pass through those platforms, any intervention can appear to be censorship, even when it is not:
As I told the online program Rising this week when asked what the worst media failings of 2020 are, I continue to view the brute censorship by Facebook of incriminating reporting about Joe Biden in the weeks before the election as one of the most significant, and menacing, political events of the last several years. That this censorship was announced by a Facebook corporate spokesman who had spent his career previously as a Democratic Party apparatchik provided the perfect symbolic expression of this evolving danger.
In the Rising clip, Greenwald goes further: first, by misrepresenting other reporting about the New York Post story in question, and also by claiming that Facebook and Twitter “censored the internet” because they algorithmically limited the story’s spread or prevented links to it from being posted. I am still uncertain about whether it was a good idea for either company to attempt to restrict the spread of that story. The closest analogy I can think of is when a stock market or a regulatory body suspends trading of a particular company’s shares because of breaking news.
Twitter, in particular, has become more assertive in labelling tweets that have the potential to spread misinformation. Most notably, it has labelled tweets from the president and other elected officials, as a sort of compromise between removing tweets and leaving these statements up to spread with impunity from figures of authority.
These are only symptom of a much wider problem. These platforms are built for engagement and have few controls to counter bad faith exploitation. There are parallels to this in live television coverage of the president’s rallies, which were often broadcast in full between 2015 and throughout his presidency. That meant that the president was free for ninety minutes to present blatant lies with unprecedented volume before a national audience, only to have news anchors struggle to rebut even a fraction of those claims. Print publications were better suited to contextualize the same statements because they are inherently slower. That does not mean they always — or even often — succeeded, however.
This pandemic brought new waves of misinformation that platforms struggled to control. It is one thing if it is about an election in one country; the stakes are much higher when public health is at risk. Taking a hands-off approach would be a callous display of irresponsibility.3 In a situation that requires nuanced expertise, there is not an open marketplace of ideas for everyone to participate in. Subject matter experts may get things wrong, but it is not because they lack fundamental knowledge. One person’s lightly-informed speculation is not a valid counterargument to an expert’s advice.
The biggest platforms — Facebook, Instagram, Twitter, and YouTube — have engaged in more public moderation of users’ posts this year than at any time before. The executives of these companies are able to influence which posts are promoted and which are demoted. But, contrary to Greenwald’s assertion, none of this can reasonably be called “censorship”, and it is a wild stretch to call executives “authoritarian” and equate it with the behaviour of governments. You may not have been able to share that Hunter Biden story in the New York Post for a few hours, but the Post still has one of the highest circulations of any newspaper in the English-speaking world. Your cousin’s tweet linking this pandemic to, of all things, cellphone towers may have a fact-checking label appended, but they can still post about it. And it seems that this more careful approach to moderation might improve the competitiveness of the social networking space as users flock to ostensibly “unmoderated” platforms — which, in turn, will step up their moderation efforts, just differently. Ironically, these concerns about platform “censorship” are instead creating more options for hosting and sharing. I welcome the narrowing focus of what is allowed on the biggest platforms to lessen their powerful catch-all nature.
But it also has the side effect of putting big public companies between users and the publication of their thoughts, appointing themselves as ultimate arbiters of what they want to see on their platforms. That is difficult and something they will get wrong from time to time, but it is also their prerogative, and it is not as though there are no alternative avenues for publishing. There are other social networks that are vastly smaller, and there are few obstacles to self-publishing. Greenwald writes on Substack, for example, which is an all-in-one product for getting text and images to people; my own website requires moderately more configuration, but I have more control than Greenwald. These alternatives are not as popular and require more promotion, often using bigger platforms. Still, none of this amounts to censorship.
As I have written before, I think the world would, in very general terms, be better served by smaller and more specialized companies. It is completely reasonable to be skeptical of the control held by conglomerates like Google — and Comcast, for that matter. But calling them and the incoming administration “authoritarian”, as Greenwald does in the closing paragraphs of this piece, is a ridiculous assertion, as much if not more so than the flirtations with fascist aspirations of the outgoing administration that Greenwald is so quick to wave away. There are many reasons to wish for greater intervention to reduce monopolization and concentration of power. But it is patently untrue that free expression is somehow more limited for Americans now than it was last year or ten years ago, and the most concerning threat to that has bipartisan support.
What I find so frustrating about this piece is that there is so much I would agree with in Greenwald’s article, if only the histrionics were dialled back by about fifty percent. A similar if more sober observation about these platforms’ gatekeeping characteristics was published by Ben Smith in the New York Times earlier this year. Among this pandemic’s more concerning long-term qualities is how much it increased dependence on big companies — partly because of the economies of scale, partly because of their supply chains, and partly because a large store operating at reduced capacity is still a large store that can fit many people. Working from home also means increased dependency on big communications companies. The outgoing U.S. administration tested the limits of existing powers already held by government. But Greenwald’s obsession with being a permanent contrarian obscures these quite reasonable points with alarmism and misdirection.
My head throbs when I read Greenwald’s screeds, even when our opinions overlap. For what it is worth, there is much to like about the substance of this piece. I only wish Greenwald employed the services of an editor. And, yes, I recognize the irony. ↥︎
Greenwald:
The last gasp for those clinging to the Trump-as-dictator fantasy (which was really hope masquerading as concern, since putting yourself on the front lines, bravely fighting domestic fascism, is more exciting and self-glorifying, not to mention more profitable, than the dreary, mediocre work of railing against an ordinary and largely weak one-term president) was the hysterical warning that Trump was mounting a coup in order to stay in office. Trump’s terrifying “coup” consisted of a series of failed court challenges based on claims of widespread voter fraud — virtually inevitable with new COVID-based voting rules never previously used — and lame attempts to persuade state officials to overturn certified vote totals. There was never a moment when it appeared even remotely plausible that it would succeed, let alone that he could secure the backing of the institutions he would need to do so, particularly senior military leaders.
One could point out that it is “hope masquerading as concern” and “exciting and self-glorifying” to find an excuse to leave a publication one co-founded because an editor wanted to stick to fact-based analysis instead of extending ultraprocessed grains of truth, only to begin a paid Substack newsletter gig that, by one’s own admission, was already in the works:
Prior to the extraordinary experience of being censored this week by my own news outlet, I had already been exploring the possibility of creating a new media outlet. I have spent a couple of months in active discussions with some of the most interesting, independent and vibrant journalists, writers and commentators across the political spectrum about the feasibility of securing financing for a new outlet that would be designed to combat these trends.
One of the examples Greenwald gives to minimize the president’s authoritarian tendencies was his delayed use of the Defense Production Act to redirect American manufacturing efforts for pandemic-related issues — if he really did have dictatorial aspirations, he would surely jump on that opportunity like a kid in a bouncy castle. Greenwald reframes this one instance where there was bipartisan pleading for the president to be more authoritarian and his refusal to take control over a serious situation as reason to believe he’s just like any other president which, you know, fine.
But this is a big problem with Greenwald’s simplified use of words like “censor” and “authoritarian”. In the case of a pandemic, we need expert figures that we can trust. You can superficially frame this as becoming “more authoritarian”, but a public health crisis is one instance where there must be candid and unified explanations of risk, prevention methods, and plans — especially since all of those things will change over time with new information.
This is one thing Greenwald gets absolutely right in his interview on Rising: popular media botched explanations for why political rallies were being cancelled and people were being encouraged to stay away from religious gatherings, yet participation in this summer’s widespread protests against systemic racial injustice was not acknowledged by some of the same outlets as a public health concern. There are, it turns out, key differences between rallies and marched protests that explain why many of the former have been considered “super spreader” events but not the latter, as Lawrence Wright writes in the overwhelming single-essay latest issue of the New Yorker:
Surprisingly, the marches did not appear to be significant drivers of transmission. “We tested thousands of people,” Michael Osterholm, the director of the Center for Infectious Disease Research and Policy, at the University of Minnesota, said. “We saw no appreciable impact.” One study found lower rates of infection among marchers than in their surrounding communities. Epidemiologists concluded that mask wearing and being outdoors protected the protesters. Moreover, demonstrators were on the move. Osterholm said that people in stationary crowds are more likely to become infected. In other words, joining a protest march is inherently less dangerous than attending a political rally.
This makes sense, and many of these facts were known at the time, but there was a lacking public dialogue to explain the difference between all of these things. Trusted figures of authority sure would have been helpful.
Maybe what I meant by “not getting into the weeds” is that the weeds would be entirely in footnotes. ↥︎
In introducing this series in August, Jason Snell defined his ranking criteria as based solely on notability:
Now, note my choice of words there: notable. I’m not saying these are my favorite Macs — a bunch of them I only knew in passing and never used myself. I’m not saying these are the best Macs ever — a difficult thing to measure, since (with a few obvious exceptions) the best Macs made are the most recent ones, otherwise we’d all still be using G3 iMacs.
Of course the iMac G3 is the most notable Mac. It is an icon with an impact many times greater than its five year production run and the now-modest five million or so that were sold:
People who didn’t live through it might not believe it, but the iMac took the product-design world by storm. Over the next few years, there would be very few consumer electronic products that had not offered a special, iMac-inspired translucent plastic edition. It started with USB accessories for the iMac, as printer and floppy-disk vendors quickly placed orders for translucent colored plastic pieces to replace their opaque beige ones. But it just kept going and going. Telephones. Toys. And my personal favorite, the George Foreman iGrill.
I was young but I remember this era so well. Everything you can think of was sold in a translucent jelly-coloured plastic variant. Gel pens were all the rage. IKEA decorated desks with fake iMac G3s. More than anything, it was the first time I remember people talking about a specific model of computer — even if they did have to replace the mouse.
Also, I think Snell did an excellent job with this series. I have enjoyed every piece and, even where I disagree — I think the SE/30 and the Power Mac G5 should have been much higher — these choices have been fair and comprehensive.
On Friday, I was reading… something… on my phone, and the above advertisement greeted me in the middle of it.
As you’ll see it, it’s for a company called IVROSE, and depicts a woman wearing long johns with the buttflap slightly open in a way that can most certainly be described as “cheeky.”
I posted about it on Instagram, and over the next couple days, others started tagging me because they were getting the ad too. Then, on Sunday night, a whole slew of people reading a new article on Elle.com about Martin Shkreli started reporting the same thing, though the woman is wearing a different pattern of long john than the one I had seen.
Zach Edwards of Victory Medium, an analytics and marketing firm, on Twitter [sic]:
When ppl say, “these butt PJ ads are following me all over the internet!” – translation @
1) A product w/ low margin ROI is burning cash across ad networks
2) The product likely owned by chinese astroturfing group
As Edwards pointed out in his Twitter thread, it’s not impossible to figure out what IVRose plans to do with what is almost assuredly boatloads of data from potential onesie customers. In fact, there’s a technical phrase for it: “cookie synching.” The easiest way to describe the process is something of a handshake between a set of partnering adtech platforms that lets folks on both sides of the arrangement swap specific sets of user data back and forth. But this is adtech we’re talking about — which means the process is needlessly complicated and probably the last thing any of us would want to talk about at a party.
At the same time, it’s also a deeply shitty tactic that promises our clicks on those assless pajamas will — no joke — likely keep on haunting us for the rest of our digital lives.
Edwards, in response to a question from Rand Fishkin:
Yup – users on Safari / Brave / Firefox or browsers who block these types of 3rd party cookies already (or anyone who locks down browser settings) won’t receive the userID syncs & makes these “audience ID honeypots” less valuable for shady SSPs & dropshippers.
Among the many complaints in the antitrust lawsuit filed against Google by the attorneys general of Colorado and Nebraska is that Google has the power to collect far more data about users than smaller rivals, and that its plan to restrict third-party cookies in Chrome will deepen that moat. Both of those things are almost certainly true. But it is imperative that the solution to these problems is not to allow or require more ad-based surveillance by smaller firms like those that powered this pyjama ad.
As state after state filed lawsuits against Google last week — plus the U.S. federal government earlier this year — I had to wonder about the timing. Why those states? Why launch this campaign now? Well, it turns out that those attorneys general heard from an oracle, whi— I’m sorry, I am being told now that they heard from Oracle, with a capital O.
Naomi Nix, Bloomberg:
Officials in more than a dozen of the states that sued Google received what has been called Oracle’s “black box” presentation showing how Google tracks users’ personal information, said Ken Glueck, Oracle’s top Washington lobbyist and the architect of the company’s antitrust campaign against Google. Glueck outlined for Bloomberg the presentation, which often entails putting an Android phone inside a black briefcase to show how Google collects users’ location details — even when the phones aren’t in use — and confirmed the contours of the pressure campaign.
“I couldn’t be happier,” said Glueck about the barrage of lawsuits. “As far as I can tell, there are more states suing Google than there are states.”
If you are lucky enough to get a few days’ downtime this year and you are looking for something to read, you could do a lot worse than picking through Bloomberg’s annual Jealousy List. It is full of great things published by other outlets, which is just as well because this year marked the second anniversary of the so-far unexplained “Big Hack” story. You know who I am jealous of? Whoever gets the full scoop on how that article was published, how criticism was handled internally, and why Bloomberg has remained entirely silent about it.
Late Tuesday night, Congress finally found the time to work out a modest covid-19 relief package that was tucked into a must-pass spending bill. Also included in the package was a pair of controversial copyright bills that will make the web a more dangerous place for sharing. What are you gonna do about it?
Incredibly, while the bill does have 2,000 pages of actual appropriations details, the other 3,000 pages are totally unrelated bills that Congress couldn’t pass through the rest of the year. Even if you like the bills, even if you are mad that Congress is gridlocked at other times, that’s no excuse to support this awful undemocratic process. Everything about it is bad.
Now, lots of people are still combing through the bill to find all the awful landmines that it’s too late to do anything about, but the two that we’ve been talking about here are the copyright provisions. I’ve already explained multiple times why the felony streaming bill and the CASE Act are extremely problematic, so I won’t go over either again. I will note that neither final provision is as bad as they were in earlier versions. Both were somewhat limited from truly terrible provisions to what is today merely awful. But that’s nothing to celebrate.
Stan Adams, of the Center for Democracy and Technology, wrote about the CASE Act when it was proposed last year:
The CASE Act would establish a quasi-judicial body within the Copyright Office (part of the legislative branch) empowered to hear a limited set of claims, make “determinations” about whether those claims are valid, and assign “limited” damages. The bill structures the process so that it is “voluntary” and lowers the barriers to filing claims so that plaintiffs can more easily defend their rights. Without the “quotes”, this description might sound like a reasonable approach, but that’s because we haven’t talked about the details. Let’s start at the top.
Yes, Congress approved a new bill Monday that would classify illegal streaming as a felony offense, but the feds won’t be going after your favorite Twitch streamers, YouTubers, or their subscribers. They’re more interested in services dedicated to streaming pirated content.
[…]
Still, it’s totally understandable that YouTubers and streamers would freak out about a bill whose final text was released only days before it was approved. Social video platforms haven’t done the best job in the past with copyright, Digital Millennium Copyright Act (DMCA) claims, and strikes. YouTube’s algorithm frequently flags content as violating copyright even when it doesn’t. For a long time, copyright owners like Universal were able to claim videos without expressly stating where the copyrighted material appeared in the content. Copyright is messy.
The whole point of copyright, if you believe the intellectual property argument, was to allow the creator of a work a short period of exclusivity with fair use exemptions, after which a work becomes part of the public domain. But automated tools suck at identifying works that fairly use copyrighted media, and the period of exclusivity has been unreasonably extended in many jurisdictions.
Laws like those snuck into the spending bill passed yesterday are a tremendous asset to the increasingly conglomerated major studios and publishers, and are worrisome for everybody else.
Last week, in two separate lawsuits, Google was sued on antitrust grounds — first by the attorney general of Texas and nine other states, then by the attorneys general of Colorado and Nebraska plus thirty-six other states. Both lawsuits are, I assure you, wild rides but I feel that it is important to warn you that none of these accusations have yet been tested in court and likely won’t be for another three years.
Let’s start in Texas. Fresh off blowing a “repugnant stunt” to undermine democracy, Ken Paxton, the attorney general, accused Google in a lawsuit of rigging the online ad marketplace and striking an illegal deal with Facebook. Gilad Edelman, Wired:
As described in the complaint, the scheme between Google and Facebook has its roots in 2017, when Facebook announced it would start supporting something called “header bidding.” The details are too wonky to get into here. Basically, Google, which runs the biggest online ad exchange, likes to make publishers give it first dibs on bidding to place an ad. (“Publisher” just means any website or app that runs ads.) Header bidding was a technical hack that allowed publishers to earn higher prices by soliciting bids from multiple exchanges at once. Google hated this, because it created more competition. When Facebook declared that it would work with publishers that used header bidding, it was seen as a provocation. The millions of businesses that advertise with Facebook don’t just advertise on Facebook; through the Facebook Audience Network, the company also places ads across the web, making it one of the biggest ad buyers on the internet. If it began supporting header bidding, that could cause Google’s ad platform to lose a lot of business.
Drawing on internal documents uncovered during its investigation, however, the Texas attorney general claims that Facebook’s leaders didn’t actually want to compete with Google; they wanted Google to buy them off. This seems to have worked. In September 2018, the companies cut a deal. Facebook, the complaint says, agreed to “curtail its header bidding initiatives” and send the millions of advertisers in its Facebook Audience Network to bid on Google’s platform. In return, Google would give the Facebook Audience Network special advantages in ad auctions, including setting aside a quota of ad placements to Facebook, even when the company didn’t make the highest bid. The agreement, the complaint says, “fixes prices and allocates markets between Google and Facebook.”
Paxton’s suit is heavily redacted, especially in its juiciest claims, so it is not an easy read. But enough is revealed about this allegedly illegal deal to indicate two things: it is alarming, and it appears to be at least partially based on an incorrect interpretation of backups. From the suit (PDF):
For instance, shortly after Facebook acquired WhatsApp, in 2015, Facebook signed an exclusive agreement with Google, granting Google access to millions of Americans’ end-to-end encrypted WhatsApp messages, photos, videos, and audio files.
This sounds like an alarmist way to write that you could back up WhatsApp to Google Drive beginning in 2015, but it also indicates that Google may be crawling WhatsApp backups for its own purposes. If that is the case, any supporting evidence in the lawsuit is currently redacted. However, according to a report in the Information, the agreement allowed Google to train its facial recognition software on users’ personal WhatsApp photos and video, which are not encrypted at rest in backups, if the user backed-up their WhatsApp account to their Google Drive account. At the very least, this strikes me as a violation of users’ expectations and trust over how the WhatsApp backup mechanism works. You might expect photos you drop into a cloud photos service to be used to improve that service; you probably don’t expect your backed-up conversations from a third-party app to be used for that.
Paxton’s suit also makes big claims about Google’s AMP page format: that it was developed in response to header bidding as a means of exercising greater control over publishers’ ads, and that AMP was just a precursor to total control. From page 84 of the suit:
Google’s control of publisher monetization will allow it to build a version of one giant walled garden that is particularly advantageous to Google. A walled garden where publishers own the property and bear the cost and risk of providing content but Google captures the benefit by extracting a high share of advertising revenues as the sole ad tech services provider. The following internal Google document summarizes Google’s future plans for the internet:
[redacted]
We are this close to seeing Larry Page’s plans for a hollowed-out volcano full of servers.
I have long maintained that AMP is a unique threat to an open web, and have been told by Reddit and Hacker News types — not to mention several people who work on AMP — that I am overreacting. If the allegations in this suit are correct, consider this a metaphorical bullhorn announcement that I told you so.
Let’s move on to the second suit; Shannon Bond, NPR:
The AGs accuse Google of giving its own products priority in search results over more specialized rivals, such as Yelp, which focuses on local businesses, and Tripadvisor, for travel listings. And they allege Google gives itself an unfair advantage over rival search engines, such as Microsoft’s Bing, in the ads that appear in search results.
The states’ complaint echoes one filed by the U.S. Justice Department in October. Both object to deals that, they allege, Google struck to ensconce itself in users’ lives. Notably, both suits point to a pact that made Google’s search engine the default on Apple devices.
The AGs go further, however, warning that Google is trying to lock in similar deals to dominate search on newer technologies such as smart speakers, voice assistants and connected cars. The states are asking for their complaint to be consolidated with the Justice Department’s suit and litigated together.
Google responded to this by saying that all it is doing is making it faster and easier to see relevant information, which it demonstrated by a search for “bread” returning recipes, local bakeries, and an information panel in addition to the standard ten blue links. It compares this to Bing, which looks pretty much the same.
But, notably, Google does not have a product that has anything to do with bread. If you search “music” with Google, the first result is YouTube Music; if you search “spreadsheet”, the first result is for Google Sheets; if you search “photos”, the first result is for Google Photos; and, if you search “domain registration”, the first non-ad result is Google Domains. Notably, even if you clicked on one of the ads to register a domain through a different company, Google would get paid for your click.
These factors combined — Google’s vertical integration, its prioritization of its own products, the positioning of advertising that displaces search results, and contracts with companies like Apple and Microsoft — are illegal, according to the Colorado and Nebraska attorneys general (PDF, page 36):
Google’s conduct has entrenched and solidified its monopoly positions against competition in three ways that individually and cumulatively harm competition. First, Google has put into place a series of artificially-restrictive contracts that have guaranteed it de facto exclusivity in the vast majority of distribution channels (like browsers and voice assistants), thus limiting the ability of consumers to reach general search competitors through search access points. Second, and notwithstanding Google’s pledge to operate its search advertising tool in a neutral fashion, Google operates its SA360 tool to harm advertisers by denying interoperability to important, competitive features, thereby harming what limited choice in general search services remains for advertisers in the wake of its exclusionary distribution contracts. Third, Google’s discriminatory conduct on its search results page has impaired the ability of specialized vertical providers to reach consumers, thereby thwarting their ability to lower barriers to expansion and entry for general search engine competitors.
These suits will undoubtably take years to be resolved. But they indicate a crumbling of trust in large tech companies with plenty of evidence from internal documents and analyses.
It would be a shame if this newfound zeal for investigations into possibly criminal anticompetitive behaviour were limited to tech companies.
Andrés Arrieta of the Electronic Frontier Foundation:
In reality, a number of studies have shown that most of the money made from targeted advertising does not reach the creators of the content—the app developers and the content they host. Instead, the majority of any extra money earned by targeted ads ends up in the pockets of thesedata brokers. Some names are very well-known, like Facebook and Google, but many more are shady companies that most users have never even heard of.
Bottom line: “The Association of National Advertisers estimates that, when the “ad tech tax” is taken into account, publishers are only taking home between 30 and 40 cents of every dollar [spent on ads].” The rest goes to third-party data brokers who keep the lights on by exploiting your information, and not to small businesses trying to work within a broken system to reach their customers.
Flawed and insufficient as current privacy legislation may be, this is one reason I remain a supporter of its principles — even though giants are virtually unaffected. The bright spotlight placed on Amazon, Facebook, and Google allows the proliferation of shadowy ad tech companies that are orders of magnitude less valuable and, therefore, are virtually invisible to most web users. The market for bulk surveillance should not be a legitimate one. These laws may make it harder for startups to compete with giants, but vigorous competition between providers of creepy tracking should not be the goal.
Taking on the giants requires a comparable giant. Most of the biggest companies in tech treat privacy violations as a core product offering for advertisers. Apple is an exception. It may be taking this stance because it does not affect its own business, but it is only able to be such an aggressive campaigner for user privacy because it does not build its business around being creepy. That is not an accident. Further legislation will take a while, and antitrust lawsuits will be batted between expensive lawyers for years before they reach trial. But this is something users can choose to opt into or out of starting next year because we are, at last, being given a choice. Good.
Overall, AppTrackingTransparency is a great step forward for Apple. When a company does the right thing for its users, EFF will stand with it, just as we will come down hard on companies that do the wrong thing. Here, Apple is right and Facebook is wrong. Next step: Android should follow with the same protections. Your move, Google.
Foreign hackers who pulled off a stealthy breach of at least a dozen federal agencies got caught after successfully logging in to a top cybersecurity firm’s network, tipping the company off to a broader hacking campaign targeting the U.S. government, according to officials from the firm and congressional aides briefed on the issue.
The suspicious log-in prompted the firm, FireEye, to begin investigating what it ultimately determined to be a highly damaging vulnerability in software used across the government and by many Fortune 500 companies.
It’s not clear how long it took FireEye to notice that it had been hacked, in a scheme that U.S. officials have linked to Russian intelligence. But the vulnerability, found in IT management software developed by a company called SolarWinds, had given the hackers months of access to internal email accounts in at least a dozen U.S. federal agencies, including the Treasury, Homeland Security and Commerce departments.
The Energy Department and its National Nuclear Security Administration, which maintains America’s nuclear stockpile, were targeted as part of the larger attack, according to a person familiar with the matter. An ongoing investigation has found the hack didn’t affect “mission-essential national security functions,” Shaylyn Hynes, a Department of Energy spokeswoman, said in a statement.
[…]
In addition, two people familiar with the broader government investigation into the attack said three state governments were breached, though they wouldn’t identify the states. A third person familiar with the probe confirmed that state governments were hacked but didn’t provide a number.
CISA is investigating incidents that exhibit adversary [tactics, techniques, and procedures] consistent with this activity, including some where victims either do not leverage SolarWinds Orion or where SolarWinds Orion was present but where there was no SolarWinds exploitation activity observed. Volexity has also reported publicly that they observed the APT using a secret key that the APT previously stole in order to generate a cookie to bypass the Duo multi-factor authentication protecting access to Outlook Web App (OWA). Volexity attributes this intrusion to the same activity as the SolarWinds Orion supply chain compromise, and the TTPs are consistent between the two. This observation indicates that there are other initial access vectors beyond SolarWinds Orion, and there may still be others that are not yet known.
Joseph Menn of Reuters is reporting that Microsoft’s systems were also breached, according to unnamed sources, and that product-level access was used to spread the attack. Microsoft denies the latter allegation.
SolarWinds was a ripe target, former employees and advisers say, not only for the breadth and depth of its software, but for its own dubious security precautions.
The company did not have a chief information security officer, and internal emails shared with The New York Times showed that employees’ passwords were leaking out on GitHub last year. Reuters earlier reported that a researcher informed the company last year that he had uncovered the password to SolarWinds’ update mechanism — the vehicle through which 18,000 of its customers were compromised. The password was “solarwinds123.”
This report carries the headline “Billions Spent on U.S. Defenses Failed to Detect Giant Russian Hack”. It is an embarrassing recitation of multiple layers of negligence and silence from U.S. agencies.
A key malicious domain name used to control potentially thousands of computer systems compromised via the months-long breach at network monitoring software vendor SolarWinds was commandeered by security experts and used as a “killswitch” designed to turn the sprawling cybercrime operation against itself, KrebsOnSecurity has learned.
This is an extraordinary breach with unprecedented scope. Its closest comparison is when files of millions of U.S. federal employees were stolen from the Office of Personnel Management, allegedly by Chinese intelligence. This series of intrusions has hit dozens of institutions at the federal, state, and local levels, as well as private companies, airports, and potentially organizations in Europe and Asia. News of this security compromise is less than a week old and new victims have been found at a steady clip. The consequences of this are going to reverberate for years.
Facebook Inc. lashed out at Apple Inc. in a series of full-page newspaper ads, claiming the iPhone maker’s coming mobile software changes around data gathering and targeted advertising are bad for small businesses.
The ads, which ran Wednesday in the New York Times, Wall Street Journal and Washington Post, carried the headline “We’re standing up to Apple for small businesses everywhere.” They home in on upcoming changes to Apple’s iOS 14 operating system that will curb the ability of companies like Facebook to gather data about users and ply them with targeted advertising.
The point of contention is a feature coming to iPhones in the new year that will require developers to ask for permission before they can track what users do across apps. Apple says the feature, which was originally slated for launch in October before being delayed in order to allow advertisers time to cope, is necessary to protect user privacy; it comes alongside a number of similar changes in new versions of iOS, such as a requirement that app developers provide a “nutritional label” for their software to explain what they do with user data.
Facebook objects – but seems keen to stress it is not doing so because it is defending its bottom line. According to its pitch, the real victims are “your neighbourhood coffee brewery, your friend who owns their own retail business, your cousin who started an event planning service and the game developers who build the apps you use for free”.
This affects not just app developers, but also small businesses that rely on personalized ads to grow. Here’s why. Small businesses have small budgets. For these small budgets to work, they have to be targeted at the customers that matter to small businesses. It doesn’t do a local wedding planner any good to reach people who aren’t planning a wedding. Likewise, it doesn’t do a small ecommerce outfit selling customized dog leashes any good to reach cat owners. Put simply, by dramatically limiting the effectiveness of personalized advertising, Apple’s policy will make it much harder for small businesses to reach their target audience, which will limit their growth and their ability to compete with big companies.
Levy deftly conflates “advertising” and “personalized advertising”, as if there are no ways to target people planning a wedding without surveilling their web browsing behaviour. Facebook’s campaign casually ignores decades of advertising targeted based on the current webpage or video instead of who those people are because it would impact Facebook’s primary business. Most people who are reading an article about great wedding venues are probably planning a wedding, but you don’t need quite as much of the ad tech stack to make that work.
The way to describe this feature coming to iOS devices next year is that all apps that wish to track users must ask permission. But Reed Albergotti of the Washington Post filed a credulous report:
Facebook took aim at Apple on Wednesday, criticizing the iPhone maker’s new policies limiting personalized ads on Apple products.
Facebook said changes Apple has made to how easily advertisers can track iPhone users will disproportionately harm small businesses that rely on personalized advertisements to reach customers and find new ones. Facebook said its internal research has found that small businesses earned 60 percent less in sales when they were not able to use the kind of targeted advertising that Apple aims to limit.
Dan Levy, Facebook’s vice president for ads and business products, blasted Apple, questioning the company’s motives for a move he said benefits Apple’s bottom line. “We believe Apple is behaving anti-competitively by using their control of the App Store to benefit their bottom line at the expense of app developers and small businesses,” he said during a call Wednesday. Facebook launched a new website and took out full-page ads in newspapers to try to drum up support.
At best, Albergotti’s report treats everything about this issue as an open debate where both parties’ claims are equally accurate. While I understand the inclination to avoid taking sides, it is possible to check their claims against the available evidence. I have quoted above the first three paragraphs of the story, only after which there is a comment from Apple clarifying that, no, this new policy does not inherently limit tracking or advertising:
“We believe that this is a simple matter of standing up for our users,” said Apple spokesman Fred Sainz in a statement. He said the new changes in iOS 14 don’t actually prohibit Facebook from continuing to offer the same tracking. Rather, “It simply requires they give users a choice,” he said. Apple has denied that it is making the changes for business reasons. Instead, Apple says, the changes, which require customers to specifically opt into personalized ad tracking, are meant to enhance its customers’ privacy, which the company has called a fundamental human right.
It would be more honest for Albergotti to describe this change for what it is: instead of requiring users to opt out of targeting later, it now requires users give explicit consent for each app to track them using the system identifier. There are parts of this that may be worth treating as an open discussion — the trade-offs for small businesses described by Facebook may be real to some extent, and this change may improve user privacy as Apple claims — but the mechanism itself is not a debate. These new policies are not, as Facebook and Albergotti say, explicitly about “limiting personalized ads”, only requiring that users meaningfully consent to them instead of burying that affirmation in a lengthy privacy policy.
If you read a lot of newsletters, you might find it challenging to keep track of all your subscriptions. We want Substack to be a distraction-free place for writers to connect directly with their readers, so we’ve created a new space, Substack Reader, that’s just for you and your newsletters.
If you’re logged in to Substack, Reader will become your new homepage. Use the “Discover” tab in your sidebar to find new writers to subscribe to. Reader also comes with RSS support, so you can really have all your subscriptions in one place.
This is clever. The problem with RSS is that it has always needed explaining, but everybody gets email. Adding RSS support to an email client makes more sense for a mainstream audience than an RSS client that can receive email. That is probably why system support for RSS historically landed in email applications like Outlook and Mail.
I still prefer the other way around, though. I do not want more email.
Let me give you a glimpse of my weekend: I like to get up just a little late, make myself a great cup of coffee, and sit down with the New York Times’ “Sunday Routine” column because it is hysterical. It is full of semi-famous wealthy New Yorkers sharing what they describe as a typical Sunday for them, and usually involves some fantasy combination of waking up at the crack of dawn, exercising for hours, eating a handful of nuts for breakfast, browsing boutiques, picking up some stuff at a farmer’s market, and cooking a big dinner with the family — and so scarcely resembles my own understanding of what a weekend ought to be that I find it hard to believe a single word.
Anyway, Paige Darrah interviewed John Foley, CEO of Peleton, for the series and this caught everyone’s eye for obvious reasons:
Twenty years ago a colleague told me the key to your day is to hydrate at much as you can, so the first thing I do is drink 40 sips of water from my hand at the upstairs bathroom sink. It’s efficient. I drink until I feel like I’m going to throw up water. Every day.
The Times article opens with an anecdote that Foley doesn’t like to wear branded Peloton gear for fear of looking, in his words, “like a tool.” I fear Foley has been pranked by his colleague and writer Paige Darrah, who have now made him explain in a fancy New York newspaper that he drinks water with his paw, like a cat, until he hurls. My cat doesn’t even drink water from the faucet, preferring instead to meow at me until I give her a pint glass.
The Times piece also mentions that Foley is touchy when it comes to being mocked. I don’t imagine this is going to help with that at all.
As we dug deeper into ProRAW, we realized it wasn’t just about making RAW more powerful. It’s about making RAW approachable. ProRAW could very well change how everyone shoots and edits photos, beginners and experts alike.
To understand what makes it so special, the first half of this post explains how a digital camera develops a photo. Then we go on to explain the strengths and weaknesses of traditional RAWs. Finally, we dive into what’s unique about ProRAW, how it changes the game, and its few remaining drawbacks.
Grab a coffee, because this is a long read.
As expected, this is much better than my attempt. You can read both, though. Are you really doing something better right now?
In news that surprises nobody, Goodreads last week quietly announced the deprecation of their public APIs. And I mean really quietly – the only people who were told about this were those unfortunate enough to have their existing API keys disabled without warning. Other than a small banner at the top of the API docs which mentions vague “plans to retire these tools”, nobody else appears to have heard anything from Goodreads, including those whose API keys remain active. So far it seems any key unused for 30 days has been disabled.
They’ve given notice that they will shut down their APIs, effectively shutting down outside interaction with Goodreads data. In addition, the developer terms and conditions don’t allow storing of any Goodreads data, so users won’t be able to pull their data from Goodreads for use in other arenas.
This sucks.
Seems pretty bold for Amazon, which owns Goodreads, to further silo user data as it is being investigated for criminal antitrust violations. I am not a lawyer so I have no idea what, if any, legal repercussions there may be, but it sure doesn’t look great.
Notifications remain a system feature that, at least on Apple’s platforms, asks a lot from users for such simple benefit. On iOS and MacOS, it is too easy to be zipped into a different app, especially if you are trying to dismiss the notification instead. And, if you indiscriminately opt into receiving notifications, there are few system-level ways to control the number and frequency from any app; it is up to developers to be judicious. And the preference pane for notifications remains too complex — but it is, at least, honestly representative of the messy features it controls.
Ben Smith, of the New York Times, reporting on how Tim Cook personally intervened to stop production of a show about a lightly-disguised Gawker:
But Hollywood is now firmly in the grip of giant companies with singular leaders — Mr. Cook and Apple; Amazon and its chief executive, Jeff Bezos; the Netflix C.E.O. Reed Hastings; and AT&T’s top executive, John Stankey — with big consumer brands and other pressing priorities, like their lucrative other businesses and their access to international markets.
So far, Apple TV+ is the only streaming studio to bluntly explain its corporate red lines to creators — though Disney, with its giant theme park business in China, shares Apple’s allergy to antagonizing China’s leader, Xi Jinping.
Eddy Cue, Apple’s senior vice president for internet software and services, who has been at the company since 1989, has told partners that “the two things we will never do are hard-core nudity and China,” one creative figure who has worked with Apple told me. (BuzzFeed News first reported last year that Mr. Cue had instructed creators to “avoid portraying China in a poor light.”)
This is an article that is purportedly about Apple’s streaming media ambitions, but is really about control in the broader world of mass media. And that is where, I think, Smith’s narrative becomes more complicated. Let’s start with that first part from the section I quoted:
But Hollywood is now firmly in the grip of giant companies with singular leaders […]
Is it true that these companies have “singular leaders”? Does Hastings frequently intervene in decisions about what shows and movies Netflix is producing? Quite the contrary, according to a recent article in the Economist that explores the company’s hands-off management style. What does this sentence mean — other than that these companies have well-known CEOs?
Furthermore, how can it be true that media companies have singular figureheads only “now”? Disney — a company named after the two brothers that founded it and defined its philosophy and style — was steered through its ’90s renaissance by the famously controlling Michael Eisner. The Fox media company is an extension of Rupert Murdoch. I can’t name any Universal Pictures executives off the top of my head, but Hollywood’s biggest studios are no stranger to the idea of a recognizable and involved figurehead.
Also, far from only Apple and Disney being worried about censorship by Chinese officials, it seems that most Hollywood studios are caving to demands. All studios have their own sensibilities about what they will and will not release, too; for example, that is why Disney created Touchstone Pictures.
But Smith posits that Cook’s intervention combined with the product placement of Apple’s products is unique:
And Apple’s willingness to sacrifice creative freedom for corporate risk management is still an outlier. None of my reporting suggests that Mr. Bezos is reaching into Amazon’s studio (or The Washington Post) to kill negative depictions of either e-commerce or the police, or that Mr. Stankey is ostentatiously slipping AT&T routers into “Lovecraft Country.” The question, of course, is how long, even at those companies, the old law will be suspended — that he who pays the piper calls the tune.
Even with the handful of things I enjoyed in its first year, I still think it is very strange for Apple to be making movies and television shows. I think I get the strategy, and media has long been funded by businesses with bigger pockets — beverage companies, telecommunications firms, and advertisers of all sorts. But it also means that Apple carefully walks a line between creative expression and business risks, and it will continue to face controversy from both angles as it builds its media library.
Still, Apple has built a studio with virtually unlimited budgets and an eye for detail. It may be exercising significant control over projects, but that seems to be broadly standard practice these days among the bigger studios, new and old. Apple’s efforts means that there is one more media company that could pass on a project; it also means there is one more media company that could say yes.
When it first became possible to capture unprocessed iPhone camera data with the API introduced in iOS 10, I remember being shocked by the images I saw. Here are the same unedited sample photos I showed in my review of the feature, both shot on an iPhone 6S:
Shot as a JPEG, 100% zoom.
Shot as a RAW image, 100% zoom.
Even on a camera from five years ago, you can see crisp edges on the windows of the building across the street, texture on the roof of the castle-like armoury in the bottom-centre, and more fidelity in the trees. Because RAW photos preserve the data straight off the sensor, they also allow for more editing flexibility. It is possible to precisely adjust the white balance, instead of simply making the image more orange or blue, and you can recover shocking amounts of detail in shadows and highlights in a way that simply is not possible with compressed and processed images.
Over time, the availability of this API has paid off as every new iPhone’s camera gets a little bit better. And I do mean little:
iPhone 6S
iPhone 12 Pro (Wide)
Megapixels
12
12
35mm Equivalent
28mm
26mm
Aperture
ƒ2.2
ƒ1.6
Pixel Size
1.22μm
1.4μm
These are literally microscopic changes, but they have a big impact with the tiny lens and sensor of a smartphone. As a result, I have been able to capture photos with better dynamic range and more detail than I would have thought possible for a phone using third-party apps like Halide.
The numbers above only tell part of the story, though. Today’s iPhone photography is only half about what the camera actually sees; the other half is about how that individual image and its contents are interpreted. Skies and skin tones are separated to be colour-corrected and have noise reduction applied specific to those typical image elements; in poor lighting, multiple images are combined so that noise can be reduced without compromising texture and detail. If you are shooting RAW images you are, by definition, not taking advantage of any of these computational photography advancements. But what if it were possible to combine the two?
The release of iOS 14.3 includes support for something Apple calls “ProRAW”, available on iPhone 12 Pro and 12 Pro Max models. Here’s how Apple describes it:
Apple ProRAW. For an absurd amount of creative control.
ProRAW gives you all the standard RAW information, along with the Apple image pipeline data. So you can get a head start on editing, with noise reduction and multiframe exposure adjustments already in place — and have more time to tweak color and white balance.
This is the first time RAW photography has been available in the first-party Camera app, so of course it has been done in a typically Apple way: it’s RAW, encapsulated in the industry-standard Digital Negative file format (DNG), and it has a bunch of little tricks that make it different from unprocessed RAW photographs. This is not going to be the kind of in-depth guide that the Lux team is able to put together. These are also not going to be particularly exciting images. These are merely some impressions after using and editing ProRAW for about a month.
The short version of my findings is that ProRAW bridges the gap between the fidelity and flexibility of a RAW photo and the finished product of a processed image — and whether this is what you want, as a photographer, is going to depend hugely on capture circumstance and what you are hoping to achieve. That sentence is pretty vague, so let me show you a few examples.1
Let’s start with this picture of Calgary’s hopelessly Soviet west side:
Camera app, HEIC [26mm “wide” lens, 1/1150s, ISO 32]
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32]
Halide, RAW [26mm “wide” lens, 1/1200s, ISO 32]
You can already see many differences, but I want to highlight a few specific areas. I’m going to start at McDougall Centre on the left-hand side of the image midway down:
Camera app, HEIC [26mm “wide” lens, 1/1150s, ISO 32], 100% crop
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32], 100% crop
Halide, RAW [26mm “wide” lens, 1/1200s, ISO 32], 100% crop
There’s a lot of fine detail in this area: trees, windows, flagpoles, sandstone, and scaffolding. The HEIC image gets to take advantage of HDR processing to better balance highlights and shadows, but there’s almost no fidelity left in this area. The Bob Rossian trees and Dali windows look fine when you’re looking at the whole image at a normal size, but as you zoom in, the illusion is revealed.
The two RAW files are interesting, though. Both images were taken with similar settings, but if I told you that one of those was taken with a faster shutter speed than the other, you might have assumed it was the one that is darker overall. It turns out that the Halide image is the one that was shot with a slightly shorter exposure time, yet it is the lightest of these two; the ProRAW file is handled differently, which is something I will get to a bit later. However, comparing unprocessed RAWs kind of defeats the point of the RAW format, so I edited both images to be more similar:
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32], 100% crop, processed to taste
Halide, RAW [26mm “wide” lens, 1/1200s, ISO 32], 100% crop, processed to taste
The ProRAW image stands out as being more defined and less noisy. There’s a black SUV on the rooftop parking lot at the bottom of the image and, in the ProRAW image, you can see its chrome door handles; in the Halide image, they simply don’t show up. No matter how much I fiddle with various contrast, definition, sharpness, and detail sliders in Lightroom, I just don’t see those handles in the Halide-captured image nor in the standard HEIC photo, but they are clearly present in the ProRAW one.
The ProRAW image is, overall, noticeably crisper without much editing, to the point where it almost looks like someone has artificially dialled up edge clarity. The Halide image is softer and noisier, but I noticed that I could push it further than the ProRAW file without it looking so aggressively retouched.
Let’s move to the concrete building in the foreground:
Camera app, HEIC [26mm “wide” lens, 1/1150s, ISO 32], 100% crop
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32], 100% crop
Halide, RAW [26mm “wide” lens, 1/1200s, ISO 32], 100% crop
The noise reduction applied to the HEIC image has completely erased the texture of the concrete, but in the ProRAW image, you can see how stippled it is. The Halide RAW image shows the difference between an unprocessed image and the adjustments applied by ProRAW: ProRAW appears simultaneously sharper and less noisy than the standard RAW file. How is this possible?
The readout from the DNG files gives us some clue. Here’s what Halide’s shows:
Minor differences aside, I want to draw your attention to three tags: BaselineExposure, BaselineSharpness, and NoiseReductionApplied. Adobe’s DNG spec (PDF) explains those tags in detail. As far as I can work out, those represent at least some of Apple’s special sauce: the RAW file already has corrected exposure, noise reduction, and sharpness, even before the photographer edits it. I found the latter two values to be consistent across images; it applies the same sharpness and noise reduction values regardless of the scene, but the BaselineExposure value varies.
One of the cool things about the DNG format is that, because these are tags applied as adjustment to the contained image data, we can edit those tags using exiftool. I was unable to change NoiseReductionApplied, but I modified a few other key tags, and this is the result compared to the base ProRAW:
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32]
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32], BaselineExposure set to 0, BaselineSharpness set to 0, and BlackLevel changed from 0 to 528, the same as the Halide RAW photo
That’s pretty dark. Let’s change one more thing — the WhiteLevel from 65,535 in the ProRAW file to the 4,095 value in the Halide file:
Camera app, ProRAW [26mm “wide” lens, 1/1000s, ISO 32], BaselineExposure set to 0, BaselineSharpness set to 0, BlackLevel changed from 0 to 528, the same as the Halide RAW photo, and WhiteLevel changed to 4,095 to match Halide
Lovely.
RAW image data that is kept in a DNG file might have different amounts of detail available to a third-party app based on the settings of that DNG file. These are differences that appear before the image is ingested into a RAW workflow. All of this is to say that at least some of the secret sauce of ProRAW appears to be about how its data is encapsulated. But wait — that’s not all. There is one more thing these images will not tell you, which is the size of the DNG files. The ones produced by Halide and Manual are in the 10–12 MB range, and Obscura’s tend to be around 13–16 MB — but ProRAW files are 20–30 MB. That is a big leap.
I am not sure why ProRAW files are twice as big or more compared to the RAW files from third-party apps. A big reason for this, I assume, is that the files are 12-bit, which accounts for about half that difference. I expect someone more capable will explain this better, because I am out of my depth here.
Let’s look at a different picture. This one was shot indoors and lit by diffuse daylight from a large window, and I tapped to focus on the nearest leaf.
Camera app, HEIC [26mm “wide” lens, 1/60s, ISO 500]
Camera app, ProRAW [26mm “wide” lens, 1/50s, ISO 500]
Halide, RAW [26mm “wide” lens, 1/55s, ISO 500]
We can see a few things pretty clearly here:
The images produced by the default Camera app look very similar.
Despite using near-identical camera settings, the Halide image is far brighter.
The HEIC image has more detail of the embarrassing amount of dust on the leaf, and better-balanced highlights. If you guessed that this is a Deep Fusion image, you’d be right.
The out-of-focus areas in this image also look less natural than those in the two RAW photos. I suspect this is a Deep Fusion side effect.
Because the Halide and ProRAW images are so different, I have processed the photos for these 100% crops to look more similar:
Camera app, ProRAW [26mm “wide” lens, 1/50s, ISO 500], 100% crop, RAW processed to taste
Halide, RAW [26mm “wide” lens, 1/55s, ISO 500], 100% crop, RAW processed to taste
These two images look pretty close to me. Parts of each are a little bit sharper than parts of the other. The slower shutter speed combined with a slightly moving leaf likely all contributed to that. And, while I braced my phone against a sturdy surface to take all of these photos, there may have been some camera movement as well. Alas, my phone tripod does not fit the iPhone 12.
Moving to the out-of-focus areas highlights a more obvious difference:
Camera app, ProRAW [26mm “wide” lens, 1/50s, ISO 500], 100% crop, RAW processed to taste
Halide, RAW [26mm “wide” lens, 1/55s, ISO 500], 100% crop, RAW processed to taste
I adjusted Lightroom’s noise reduction to the best of my abilities in that Halide image but, as the noise profile became closer to the smooth ProRAW image, I found that details in the foreground disappeared. I struggled to balance the two as successfully as they came out of the camera in the ProRAW image.
That made me think that ProRAW would be especially effective with low-light photography, so here is a series of test shots from a nighttime scene:
Camera app, HEIC [26mm “wide” lens, 1/40s, ISO 500]
Camera app, ProRAW [26mm “wide” lens, 1/40s, ISO 500]
Obscura, RAW [26mm “wide” lens, 1/40s, ISO 500]
Halide, RAW [26mm “wide” lens, 1/40s, ISO 500]
In this case, I decided to take four photos: an HEIC one, a ProRAW one, and then one RAW photo in each Halide and Obscura. I know that I have pointed out a few times in this piece that Halide’s images are grainier than the ProRAW samples. I want to be clear that I do not see this as a problem with Halide, nor am I bashing the app. I really like Halide. But, to make sure that I wasn’t missing anything, I wanted to see what the results would be from another third-party RAW-shooting camera app, and I also really like Obscura. Here are some things to know about the photos above:
The grain, lighting, and contrast all look very similar. Too similar. So I checked the DNG tags and they are virtually identical. I asked Obscura developer Ben Rice McCarthy about this and they said that the system takes care of those tags and that Obscura does not write them. That aligns with the documentation I found, which shows only a handful of options for processing RAW files compared to the long list of tags in Adobe’s DNG spec.
I braced the camera against a construction fence to take these pictures, but my efforts were not good enough and resulted in a blurry Halide image. I will not be using it for any detail comparisons.
As I mentioned earlier, it is more worthwhile to compare details if both of the RAW images are processed to look similar, so that is what you will see below. First, here are some leaves in the foreground:
Camera app, ProRAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
Obscura, RAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
In this case, I think the Obscura image looks far better. Yes, I know there’s more noise, but that cluster of leaves is actually recognizable as leaves. The smoothing in the ProRAW image creates an image with less fidelity and, critically, there does not appear to be a way to decrease the pre-applied noise reduction. You can see similar smearing in the train station in the background, but you can also see a somewhat demoired wire fence, which is nice:
Camera app, ProRAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
Obscura, RAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
But the ProRAW format’s aggressive noise reduction works out okay for the building on the right, as well as the sky:
Camera app, ProRAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
Obscura, RAW [26mm “wide” lens, 1/40s, ISO 500], 100% crop, RAW processed to taste
So, I am not sure I was correct — I don’t know that ProRAW is particularly good in low-light scenes. One of the advantages of RAW is that the photographer can choose their own adjustments, including balancing noise and detail. In the ProRAW world, that choice has been made and I cannot find a way to dial it back with the tools at my disposal. It isn’t like the Noise Reduction slider in Lightroom is at a preset amount; it is at zero, yet noise has been reduced within the file. The Obscura image is far grainier, but I think it is possible to bring the noise down to manageable levels while preserving many of the details that have been smeared together in the ProRAW example.
This particular example is quite challenging, with many fine lines and small details in a high-contrast setting at night. But I found similar results with fine detail in many ProRAW photos shot in the dark. This is a representative — albeit exaggerated — sample.
All of the example images so far have been made with the main “wide” camera on the iPhone 12 Pro, as it is the one with the biggest sensor, widest aperture, and creates the best quality images. But I have a few thoughts on ProRAW on the other three cameras:
The telephoto experience is very similar to the examples above, with one main exception: the first-party Camera app will continue to digitally zoom images from the “wide” camera in telephoto mode if it feels there is not enough light for the telephoto camera. This remains the case in ProRAW mode, and there is no immediate way of knowing when it is happening.
This is maybe the most irritating characteristic of something called “ProRAW” and explicitly marketed for its “absurd amount of creative control”. If you select the telephoto camera in third-party apps, you’re guaranteed to get the telephoto camera; in Apple’s app, you may or may not, and you have no way of knowing until you inspect the photo’s metadata.
ProRAW brings RAW photography to the iPhone’s ultrawide camera for the first time, and it is excellent. Its specs land it in a solid third place of the four cameras on an iPhone, but RAW brings more flexibility to that camera. There is still a surprising level of noise reduction, but I have found it to be more detailed and, naturally, more flexible than its HEIC cousin.
Apple has said that third-party app developers will be able to take advantage of the ProRAW pipeline, but it is unclear to me whether they will also be able to capture unprocessed image data from the ultrawide camera.
ProRAW is also available on the front-facing selfie camera. I am sure some photographers will be excited about the creative prospects of this, but it is the smallest sensor and lowest-quality camera of the four — and it shows.
I hope this gives you a sense of how ProRAW works and how it differs from the RAW capture that third-party apps have been offering for the past four years. As I mentioned, I have been trying out ProRAW for about a month now, and I am very pleased. It is another tool in the iPhone photography toolbox that, for me, does not entirely replace third-party camera apps. That is a good thing; I want to see exceptional independent apps like Halide and Obscura succeed on the iPhone. It is, however, a worthwhile addition that underscores how great of a camera this telephone really is.
ProRAW also largely accomplishes the pitch Apple makes for it. I can nitpick the amount of control a photographer has with ProRAW compared to other RAW capturing apps, but one thing I have been consistently impressed by is just how similar the HEIC and ProRAW versions of the same scene appear. It really does seem like Apple has managed to bridge the computational workflow of the standard camera app and the greater flexibility and quality of RAW images. I bet photographers will be pleased.
To create these test shots, I first photographed a scene in the first-party Camera app in standard HEIC mode. Everything was set to auto except the flash, which was switched off, and I tapped to set the focus and exposure. I then tapped the RAW toggle to capture a ProRAW file. I used the Exify extension to view the exposure settings, and attempted to replicate them as closely as I could to capture a standard RAW file in third-party camera apps like Halide and Obscura.
All photos were AirDropped to my Mac, imported into Lightroom Classic, and then exported without any adjustments except resizing to fit, unless otherwise noted. Images were exported as JPEG files with P3 colour space at quality setting 80. All metadata fields except location information were preserved. ↥︎
Earlier this week, the U.S. Federal Trade Commission sued Facebook for illegal monopolistic behaviour. They were joined by attorneys general from forty-six states plus the District of Columbia and Guam, but excluding Alabama, Georgia, South Carolina, and South Dakota, for whatever reason. There is a lot to go through in the suit and it will take years to play out in court.
The government is alleging that Facebook should not have acquired Instagram or WhatsApp and is demanding the divestment of both; it is also saying that the terms under which Facebook offered some APIs were illegally anticompetitive, which is a potentially worrisome complaint. Spinning off its biggest acquisitions will involve technical hurdles that Facebook began rolling out — very deliberately, you might say — earlier this year. Contrary to what Matt Stoller says, this is not a matter of “essentially copying and pasting some code”. But that does not mean it should not be done if the legal arguments are sound.
Because this is such a complex case, the reporting around it must be careful and well-sourced, but it is likely that much of it will not be. For example, here’s the Financial Times’ editorial board claiming the burden of proof that will be required to satisfy the government’s claims:
[…] There are significant hurdles for the case to clear, not least the fact that the FTC approved both of the acquisitions, that of Instagram in 2012 and WhatsApp in 2014. The courts will have to consider whether the acquisitions were lawful then, not whether they would be lawful today. […]
In 2012 and 2014, the FTC declined to take action against the acquisitions of Instagram and WhatsApp. As a legal matter — which matters, since we are speaking of a legal action — declining to take action has no binding effect on future enforcement actions. That is a well established principle not just in antitrust, but any area of law enforcement. Indeed, we applaud criminal prosecutors when they revisit cold cases with new evidence.
This is an important rule, because non-action might become action in the face of new evidence, and indeed that is what the FTC has in this case. In 2012, the durability of Facebook’s monopoly was unclear, and it was widely thought that Google+ would emerge as a major competitor to Facebook. But here in 2020 we know that the monopoly was durable, and that Google+ was not significant. Beyond that, more emails have come out, more third party testimony, and more evidence of a general, serial campaign to eliminate competitive threats, large and small. And the anticompetitive effects — less privacy, more ads, and so on — are now matters of fact rather than prediction.
The rest of the Times paragraph makes other arguments that Wu and Hemphill end up refuting in their piece: the government does not necessarily have to prove that acquisitions would have been significant competitors, nor should this case have a chilling effect on smaller acquisitions. It is worth reading as a preface for much of the reporting that will undoubtably be published over the next several years as this case is argued. That’s why I made it the main link for this post.
