Rewindex

RewindexOutage eventsSep 18, 2023

AWS outage, Sep 18, 2023

29 vendors posted 29 incidents between 18:00 and 21:00 UTC, against about 6 incidents from 5 vendors in the same hours on an ordinary week. 10 vendors named AWS in 10 incidents (6 of them blamed it). Vendors reported an AWS outage; AWS’s own incident for this date is not in the archive.

Observed, not verified. A line means a vendor’s incident named another vendor, or started in the same window. It is not a proven cause.

Times are UTC; lags are from when the first vendor named AWS.

    7. first vendor named AWS 18:07
Named AWSNamed another vendorDid not name anyoneEach column is 30 minutes

How it spread

Vendors that named AWS, grouped by when they posted, and vendors that blamed one of the vendors AWS took down. Solid lines: the vendor named the other. Dashed: same window, not named.

Swipe sideways to see the whole diagram.

AWSown incident not archivedWithin an hour3 vendors, 1 blamed AWSDagster, Snowflake, Vanta1 to 4 hours later7 vendors, 5 blamed AWSAmplitude, Temporal Cloud, Coralogix, Confluent +3
AWSown incident not archivedWithin an hour3 vendors, 1 blamed AWSDagster, Snowflake, Vanta1 to 4 hours later7 vendors, 5 blamed AWSAmplitude, Temporal Cloud,Coralogix, Confluent +3
Rendered on the server with pele from the Mermaid source below. Each box lists the first few vendors; every vendor is listed further down.
Mermaid source
flowchart LR
  root["AWS<br/>own incident not archived"]:::inferred
  h1["Within an hour<br/>3 vendors, 1 blamed AWS<br/>Dagster, Snowflake, Vanta"]:::blames
  root --> h1
  h4["1 to 4 hours later<br/>7 vendors, 5 blamed AWS<br/>Amplitude, Temporal Cloud, Coralogix, Confluent +3"]:::blames
  root --> h4

Vendors that named AWS

In their own words. “Blames” means a causal phrase sits next to the name (“due to”, “upstream provider”…).

Within an hour

3 vendors

  • +0 min

    Blames AWS

    Major incident

    DagsterServerless runs failing to launch

    …y, this hedges us against issues like capacity availability constraints. At 10:00 AM PST, AWS began experiencing elevated latencies for a networking subsystem in two of these availability zones. You c…

  • +2 min

    Names AWS

    Major incident

    Snowflake5 Deployments Affected: INC0092585

    …ed with our third-party service provider, who implemented a fix to restore service to the AWS - US West (Oregon) and AWS - US East (N. Virginia) regions. The environments have remained stable, which c…

  • +9 min

    Names AWS

    Minor incident

    VantaSubset of AWS related tests showing inaccurate test results

    …Subset of AWS related tests showing inaccurate test results This issue has been resolved and you should no longer see incorrect test results moving forward. If you have any questions or concerns, plea…

1 to 4 hours later

7 vendors

Same window, did not name anyone

19 incidents started during the event without naming AWS or another vendor. Some are part of it, some are a normal day’s background (about 6 in these hours).

Show all 19

Detected automatically from 13 status pages opening incidents in the peak hour (normally at most 6). Mentions are matched by name in each incident’s title and updates, then classified as cause, location or integration; quoted text is the vendor’s. Vendors that share one status page are shown once.