Changes / Dagster / Cannot launch runs in Serverless
Dagster: Major incident: Cannot launch runs in Serverless
Summary
Dagster reported a major API incident preventing Serverless runs from launching, starting 2023-08-03 20:29 UTC and resolved at 2023-08-03 21:23 UTC (53 min). Runs were launching successfully as of 20:50 UTC; Dagster attributed the issue to AWS EC2 API errors and latency affecting AWS Fargate, used by Dagster Cloud Serverless.
Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.
What changed
Before: not present
After: Cannot launch runs in Serverless
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2023-08-03 20:29 UTC
Before: not present
After: 2023-08-03 21:23 UTC
Before: not present
After: API
Before: not present
After: We are currently investigating this issue.
Before: not present
After: We are seeing runs successfully launch as of 20:50 UTC. We are continuing to monitor the situation.
Before: not present
After: AWS was experiencing increased API error rates and latencies for their EC2 APIs, which impacted AWS services making use of these APIs. This included AWS Fargate, which is used in Dagster Cloud Serverless. Their incident was resolved as of 21:18 UTC.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Cannot launch runs in Serverless (this change) Dagster reported a major API incident preventing Serverless runs from launching, starting 2023-08-03 20:29 UTC and resolved at 2023-08-03 21:23 UTC (53 min). Runs were launching successfully as of 20:50 UTC; Dagster attributed the issue to AWS EC2 API errors and latency affecting AWS Fargate, used by Dagster Cloud Serverless. | Major incident |