Broken File in CrowdStrike Update Causes Worldwide Systems Outages ⇥ theverge.com

Emma Roth, the Verge:

On Friday morning, some of the biggest airlines, TV broadcasters, banks, and other essential services came to a standstill as a massive outage rippled across the globe. The outage, which has brought the Blue Screen of Death upon legions of Windows machines across the globe, is linked to just one software company: CrowdStrike.

[…]

“Our software is extremely interconnected and interdependent,” Lukasz Olejnik, an independent cybersecurity researcher, consultant, and author of the book Philosophy of Cybersecurity, tells *The Verge. “But in general, there are plenty of single points of failure, especially when software monoculture exists at an organization.”

Robert McMillan, Wall Street Journal:

Founded in 2011, CrowdStrike is widely used by Corporate America, supplying software that protects against cyberattacks to tens of thousands of customers, including 300 companies in the Fortune 500. The scale of the outage was compounded by the fact that cybersecurity software like CrowdStrike’s has access to the most fundamental elements of the operating system to ward off cyberattacks, security experts say.

This sounds like a terrible day for anyone affected. There are I.T. staff who were woken up in the middle of the night to see if there was anything they could do; while a workaround was posted within an hour and a half of CrowdStrike issuing this update, it requires tedious manual work on each impacted system. You can find countless stories online of hospitals, airports, government systems, broadcasters, and more severely interrupted by this one bad software update. A whole lot of people had a really terrible day today.

We keep seeing the ripple effects when just a handful of vendors are entrusted with the digital infrastructure on which our society runs. Bought tickets to a mainstream event in North America? It was probably facilitated by Ticketmaster, so your credit card was leaked. It and over a hundred other companies depended on Snowflake for data storage, which was breached. Do you live in the U.S. and own a phone? AT&T, T-Mobile, and Verizon have all suffered data breaches. Two years ago, Canadian ISP and cellular carrier Rogers was down for an entire day, “disrupting government services and payment systems”. Microsoft is busy convincing people it is taking security seriously after a series of embarrassing failures.

Even if there are serious financial and reputational consequences for these failures, the world is still no closer to being freed of its dependence on Windows or Ticketmaster or Snowflake or CrowdStrike. These seem to be incredibly fragile systems on which society rests with little accountability for their makers.

Update: Changed the word “driver” in my headline to “file” to reflect CrowdStrike’s technical analysis.