Thousands of websites went offline for as much as an hour on Tuesday morning, including several of the world’s largest news sites, streaming services, online retailers and even the UK government and the White House, disrupting millions of internet users.
Connectivity problems appeared to affect news sites including the The Irish Times, the BBC, the New York Times, CNN and the Financial Times, streaming services Spotify, Twitch, Hulu and HBO Max, payments service Stripe and message board Reddit. Together, the online services reach hundreds of millions of people every day.
Disruption to the UK government’s network of websites, gov.uk, may have caused problems for people booking Covid-19 vaccinations or reporting test results.
A Silicon Valley-based internet infrastructure provider, Fastly, admitted that a problem with its systems caused the outage. It blamed a configuration error but has not provided a detailed explanation of what went wrong.
The company operates a content delivery network, which is designed to facilitate faster loading times for web pages and larger files such as music and video. Other large CDN providers include Cloudflare, Akamai and Limelight.
Disruption
“We identified a service configuration that triggered disruptions across our POPs [points of presence] globally and have disabled that configuration. Our global network is coming back online,” said Fastly.
Though that would appear to rule out foul play, the UK’s National Cyber Security Centre, a branch of GCHQ, said: “We are aware of an issue affecting a number of websites and are working to understand the cause.”
CDNs store their corporate customers’ data on servers all over the world, easing the burden on the “backbones” of the internet by bringing content closer to consumers’ smartphones and PCs.
“It’s astonishing that one small piece of the immense jigsaw that powers the internet is able to cause such a massive outage,” said Ben Wood, analyst at CCS Insight, a tech consultancy. “On the one hand, it’s an unbelievably robust platform. On the flip side, these occasional blips underline the fragility of its fabric.”
Previous large-scale internet infrastructure outages have been triggered by problems at widely used cloud hosting services such as Amazon Web Services, as occurred in November last year, or the domain-name services that link a website's numerical internet code to their more user-friendly ".com" addresses, as experienced by Cloudflare DNS customers in mid-2020.
Fastly’s online status page started reporting problems with its CDN services at 09.58 UTC.
After 59 minutes, it said: “The issue has been identified and a fix has been applied. Customers may experience increased origin load as global services return.”
Infrastructure
Even an hour-long outage can cost millions of dollars in lost advertising revenues and online sales for the many companies, large and small, that rely on vital infrastructure providers such as Fastly.
“We were offline for a few minutes because the whole internet broke down,” tweeted Jitse Groen, chief executive of food delivery group Just Eat Takeaway.com, wrote on Twitter.
As well as several prominent media groups, Fastly’s website lists ecommerce platforms Shopify and Stripe, and retailers including Wayfair, Boots, Dunelm, Ticketmaster and Deliveroo among its customers.
Twitter accounts operated by technical teams at Shopify, Stripe and Spotify had all reported problems. Some PayPal users also reported difficulties accessing its website.
“It’s astonishing that so many high-profile sites were affected by the metaphorical flick of a switch,” said Wood. “It is a perfect example of how much society has become completely dependent on always-on, always-available connectivity.”
Shares in Fastly, which is listed on the New York stock exchange with a market capitalisation of just under $6 billion, dipped 5.5 per cent in pre-market trading on Tuesday as news of the outage broke. But they recovered that ground shortly after markets opened and were trading 4 per cent higher in late morning.
The 10-year-old company has headquarters in San Francisco. – Copyright The Financial Times Limited 2021
