Updated with Google comment 23/10/2019: 20:02 BST
AWS late yesterday was hit by a sustained DDoS attack, which appears to have lasted some eight hours. The incident hit its Route 53 DNS web offering, knocking down other services, and raises many questions about the nature of the attack and about AWS’s own DDoS mitigation service, “Shield Advanced”.
Google Cloud Platform (GCP) had a range of issues at a similar time. The two are not understood to be linked. In a status update GCP cited interruptions to “multiple Cloud products including Google Compute Engine, Cloud Memorystore, Google Kubernetes Engine, Cloud Bigtable and Google Cloud Storage” at a similar time. A Google spokesperson told us: “Our service disruptions were unrelated to any kind of DDoS attempt.”
The attack on AWS left many customers struggling to access AWS’s S3 services, with many AWS services relying on external DNS queries, including its Relational Database Service (RDS), and Elastic Load Balancing (ELB). The US East Coast appears to have been particularly severely hit. (AWS described the impact of the attack as only affecting a “small number of specific DNS names”).
AWS users on Reddit said they had found Aurora (a MySQL and PostgreSQL-compatible database) clusters also unreachable, with many complaining that their customers had been left unable to use cloud services for several hours.
AWS DDoS Attack
An AWS status update reads: “Between 10:30 AM and 6:30 PM PDT, we experienced intermittent errors with resolution of some AWS DNS names. Beginning at 5:16 PM, a very small number of specific DNS names experienced a higher error rate. These issues have been resolved.”
An email to customers pointed the finger at a Distributed Denial of Service (DDoS) attack. As widely shared on Reddit, Twitter, and reported by the Register, the email notes: “We are investigating reports of occasional DNS resolution errors. The AWS DNS servers are currently under a DDoS attack.
Content from our partners
“Our DDoS mitigations are absorbing the vast majority of this traffic, but these mitigations are also flagging some legitimate customer queries at this time.
It added: “We are actively working on additional mitigations, as well as tracking down the source of the attack to shut it down.”
Amazon’s own Shield Advanced DDoS mitigation offering dealt with much of the attack, but the mitigations were also flagging some legitimate customer queries as malicious, meaning they were unable to connect.
Given the sheer size of AWS and the traffic it handles at any given time, the attack must have been significant. It is not clear if a more detailed autopsy will be forthcoming. (Critics noted that AWS’s Route 53 Service Level Agreement (SLA) promises 100 percent uptime…
Fun fact: Route53 is the only AWS service with a 100% uptime SLA.
Unfun fact: AWS SLAs are mostly meaningless and Route53 is having issues right now. pic.twitter.com/PYE416pyNc
— Scott Piper (@0xdabbad00) October 22, 2019
AWS had not commented further nor answered specific questions from Computer Business Review about the attack as we published.
Customers were able to resolve the issue by updating the configuration of their clients accessing S3 to specify the specific region that their bucket is in when making requests to mitigate impact: e.g. specifying “mybucket.s3.us-west-2.amazonaws.com rather than “mybucket.s3.amazonaws.com”.
