Changes / Firebolt / Engines failing to start in us-east-1
Firebolt: Major incident: Engines failing to start in us-east-1
Summary
Firebolt reported a major outage affecting engine provisioning and startups in us-east-1 from 2022-04-19 15:18 UTC to 2022-04-19 18:04 UTC, lasting 2 h 45 min. Customers were told not to stop or restart existing engines. Firebolt attributed the disruption to an AWS issue; after its resolution, startups and restarts were permitted again, with stuck engines expected to start shortly.
Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.
What changed
Before: not present
After: Engines failing to start in us-east-1
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2022-04-19 15:18 UTC
Before: not present
After: 2022-04-19 18:04 UTC
Before: not present
After: Engines, Firebolt Components - Engines
Before: not present
After: Hello - we are experiencing issues provisioning engines in us-east-1. Please do not stop/restart existing engines during this time. We are actively investigating and will update with an ETA on the fix shortly.
Before: not present
After: AWS has issued a message as of 8:39am PDT stating they are having issues with all engines in us-east-1 and have identified and are working to fix the root of the issue. We will continue to monitor the AWS event and ensure we are operational asap.
Before: not present
After: The AWS issue has been resolved and any engine that was stuck will start up shortly. You may execute engine start ups and restarts in us-east-1 again. We appreciate your patience on this issue.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Engines failing to start in us-east-1 (this change) Firebolt reported a major outage affecting engine provisioning and startups in us-east-1 from 2022-04-19 15:18 UTC to 2022-04-19 18:04 UTC, lasting 2 h 45 min. Customers were told not to stop or restart existing engines. Firebolt attributed the disruption to an AWS issue; after its resolution, startups and restarts were permitted again, with stuck engines expected to start shortly. | Major incident |