Rewindex

RewindexOutage eventsJul 30, 2024

AWS outage, Jul 30, 2024

12 vendors posted 12 incidents between 22:00 on Jul 30 and 08:00 on Jul 31 UTC, against about 14 incidents from 10 vendors in the same hours on an ordinary week. 7 vendors named AWS in 7 incidents (6 of them blamed it). Vendors reported an AWS outage; AWS’s own incident for this date is not in the archive.

Observed, not verified. A line means a vendor’s incident named another vendor, or started in the same window. It is not a proven cause.

Times are UTC; lags are from when the first vendor named AWS.

    21. first vendor named AWS 22:15
Named AWSNamed another vendorDid not name anyoneEach column is 30 minutes

How it spread

Vendors that named AWS, grouped by when they posted, and vendors that blamed one of the vendors AWS took down. Solid lines: the vendor named the other. Dashed: same window, not named.

Swipe sideways to see the whole diagram.

AWSown incident not archivedWithin an hour3 vendors, 3 blamed AWSXero, Substack, Mapbox1 to 4 hours later3 vendors, 2 blamed AWSSegment, Hootsuite, HoneycombLater1 vendor, 1 blamed AWSHeap
AWSown incident not archivedWithin an hour3 vendors, 3 blamed AWSXero, Substack, Mapbox1 to 4 hours later3 vendors, 2 blamed AWSSegment, Hootsuite,HoneycombLater1 vendor, 1 blamed AWSHeap
Rendered on the server with pele from the Mermaid source below. Each box lists the first few vendors; every vendor is listed further down.
Mermaid source
flowchart LR
  root["AWS<br/>own incident not archived"]:::inferred
  h1["Within an hour<br/>3 vendors, 3 blamed AWS<br/>Xero, Substack, Mapbox"]:::blames
  root --> h1
  h4["1 to 4 hours later<br/>3 vendors, 2 blamed AWS<br/>Segment, Hootsuite, Honeycomb"]:::blames
  root --> h4
  later["Later<br/>1 vendor, 1 blamed AWS<br/>Heap"]:::blames
  root --> later

Vendors that named AWS

In their own words. “Blames” means a causal phrase sits next to the name (“due to”, “upstream provider”…).

Within an hour

3 vendors

1 to 4 hours later

3 vendors

  • +1 h 13 min

    Blames AWS

    Minor incident

    SegmentHeap.io Partner Outage

    …Heap.io Partner Outage Heap.io confirmed that the AWS outage affecting data delivery operation is now resolved.…

  • +1 h 15 min

    Blames AWS

    Major incident

    HootsuiteScheduled messages, Media Upload and Inbox 2.0 issues

    …ages, Media Upload and Inbox 2.0 issues We are pleased to inform you that the issues with AWS services have been fully resolved. All areas of the Hootsuite dashboard, including Scheduled messages, Med…

  • +1 h 56 min

    Names AWS

    Minor incident

    HoneycombAWS CloudWatch integration ingest delays

    …AWS CloudWatch integration ingest delays Everything has remained stable for many hours so we are now closing this incident.…

Later

1 vendor

  • +9 h 44 min

    Blames AWS

    Major incident

    HeapData collection reload is being executed now

    …DT and 9:00 PM PDT (22:05 UTC - 01:00 UTC) on July 30-31 due to an underlying large-scale AWS service disruption. We have completed reloading the available data captured during that time. If you notic…

Same window, did not name anyone

5 incidents started during the event without naming AWS or another vendor. Some are part of it, some are a normal day’s background (about 14 in these hours).

Show all 5

Detected automatically from 8 status pages opening incidents in the peak hour (normally at most 6). Mentions are matched by name in each incident’s title and updates, then classified as cause, location or integration; quoted text is the vendor’s. Vendors that share one status page are shown once.