Changes / Dagster / Some Dagster Plus Serverless Runs Failing to Start
Dagster: Major incident: Some Dagster Plus Serverless Runs Failing to Start
Summary
Summary pending. The diff below shows exactly what changed.
What changed
Before: not present
After: Some Dagster Plus Serverless Runs Failing to Start
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-05-14 18:53 UTC
Before: not present
After: 2024-05-14 20:16 UTC
Before: not present
After: API
Before: not present
After: Some Dagster Plus Serverless runs are failing to start with an "Agent detected that run worker failed." error. We are working with our cloud provider to investigate the root cause of the problem as well as adding automatic retries for the underlying error that is causing the runs to fail to start.
Before: not present
After: We are continuing to investigate the root cause of this issue with our cloud provider. We are rolling out a change to add additional automatic retries to runs that are affected by this issue.
Before: not present
After: Our cloud provider has identified the root cause of the problem and is in the process of rolling back the change that caused it. We'll continue to provide updates as we work to resolve the issue.
Before: not present
After: The fix for this problem is now live.
First version of this incident archived by Rewindex. View the live incident.
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Some Dagster Plus Serverless Runs Failing to Start (this change) | Major incident |