Dagster: Cannot launch runs in Serverless

Impact: majorStatus: resolvedStarted 2023-08-03 20:29 UTCResolved 2023-08-03 21:23 UTC (54 min)3 updatesSource: stspg.io

Summary

Dagster reported a major API incident preventing Serverless runs from launching, starting 2023-08-03 20:29 UTC and resolved at 2023-08-03 21:23 UTC (53 min). Runs were launching successfully as of 20:50 UTC; Dagster attributed the issue to AWS EC2 API errors and latency affecting AWS Fargate, used by Dagster Cloud Serverless.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: API

Timeline of updates

investigating2023-08-03 20:29 UTC
We are currently investigating this issue.
monitoring2023-08-03 21:09 UTC
We are seeing runs successfully launch as of 20:50 UTC. We are continuing to monitor the situation.
resolved2023-08-03 21:23 UTC
AWS was experiencing increased API error rates and latencies for their EC2 APIs, which impacted AWS services making use of these APIs. This included AWS Fargate, which is used in Dagster Cloud Serverless. Their incident was resolved as of 21:18 UTC.

Rewindex changes for this incident

Major incident: Cannot launch runs in Serverless 2023-08-03 20:29 UTC

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Dagster profile · Recent Dagster incidents · Markdown · JSON