Changes / Dagster / Some Dagster Cloud API Requests failing with an Internal Server Error
Dagster: Major incident: Some Dagster Cloud API Requests failing with an Internal Server Error
Summary
Some Dagster Cloud API requests and a small number of runs failed with Internal Server Error exceptions due to a storage-engine timeout. The vendor rated the incident major; it ran from 2024-04-11 17:02 UTC to 2024-04-11 21:25 UTC, lasting 4 h 23 min. A fix was implemented and the incident resolved.
Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.
What changed
Before: not present
After: Some Dagster Cloud API Requests failing with an Internal Server Error
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-04-11 17:02 UTC
Before: not present
After: 2024-04-11 21:25 UTC
Before: not present
After: API
Before: not present
After: Starting earlier this week, a small number of runs in Dagster Cloud started failing with an Internal Server Error exception due to a timeout in our storage engine. We've identified the source of the problem and are in the process of rolling out a fix.
Before: not present
After: We are continuing to work on a fix for this issue.
Before: not present
After: A fix has been implemented and we are monitoring the results.
Before: not present
After: This incident has been resolved.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Some Dagster Cloud API Requests failing with an Internal Server Error (this change) Some Dagster Cloud API requests and a small number of runs failed with Internal Server Error exceptions due to a storage-engine timeout. The vendor rated the incident major; it ran from 2024-04-11 17:02 UTC to 2024-04-11 21:25 UTC, lasting 4 h 23 min. A fix was implemented and the incident resolved. | Major incident |