Rewindex

CompaniesFireboltIncidents

Firebolt: AWS Outage affecting Start/Stop Engines

MinorStatus: resolvedStarted Resolved (17 min)2 updatesstspg.io (opens in new tab)
Part of an outage event

AWS outage, Jun 13, 2023 · 56 vendors, 56 incidents

Firebolt’s incident blames AWS as the cause. (hop 1, posted +2 h 8 min vs the first vendor naming AWS)

“…AWS Outage affecting Start/Stop Engines AWS has announced all services are restored, although there is a backlog of requests in their services. We are seeing engines start and stop succesfully, althou…”

Swipe sideways to see the whole diagram.

AWSown incident not archivedWithin an hour13 vendors, 13 blamed AWSPipedream, Wiz, Temporal Cloud +101 to 4 hours later7 vendors, 6 blamed AWSCockroachDB, Vanta, Asana +4Firebolt (this incident)posted +2 h 8 min
AWSown incident not archivedWithin an hour13 vendors, 13 blamed AWSPipedream, Wiz, TemporalCloud +101 to 4 hours later7 vendors, 6 blamed AWSCockroachDB, Vanta, Asana +4Firebolt (this incident)posted +2 h 8 min
Firebolt is outlined in red. Groups show up to a few vendors each; the event page lists every incident.
Mermaid source
flowchart LR
  root["AWS<br/>own incident not archived"]:::inferred
  h1["Within an hour<br/>13 vendors, 13 blamed AWS<br/>Pipedream, Wiz, Temporal Cloud +10"]:::blames
  root --> h1
  h4["1 to 4 hours later<br/>7 vendors, 6 blamed AWS<br/>CockroachDB, Vanta, Asana +4"]:::blames
  root --> h4
  me["Firebolt (this incident)<br/>posted +2 h 8 min"]:::me
  h4 --- me

Full event: timeline, evidence and all vendors →

Detected automatically from incidents that started together and named the same provider. A link means one vendor’s incident named another, not a confirmed cause.

Affected components: EnginesFirebolt Components - Engines

Timeline of updates

  1. identified

    Several AWS services are experiencing issues. Because of this, Firebolt engines are not able to Start. Also, engines that are Stopped will be in Stopping status for a period of time, and then report Failed Stopping. They are effectively stopped and no longer running. We are monitoring updates from AWS.

  2. resolved

    AWS has announced all services are restored, although there is a backlog of requests in their services. We are seeing engines start and stop succesfully, although it may take longer than usual for a few hours.

Archive proof

Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves this record existed by then and has not been changed since, by anyone, Rewindex included.

How the archive is checked, and what it cannot prove.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Firebolt profileRecent Firebolt incidentsMarkdownJSON