Dagster: Degraded agent communication delayed schedules and automations
Affected components: APIDagster Cloud UI
Timeline of updates
- monitoring
Between 3:15 PM and 3:45 PM ET (12:15–12:45 PM PT, 19:15–19:45 UTC) on October 7, a backend issue delayed communication between Dagster+ and customer agents. During this window, some schedule, sensor and declarative automation ticks failed with "Unable to reach the user code server" errors, and some code location operations may have been slow or timed out. The issue was resolved by 3:45 PM ET (12:45 PM PT, 19:45 UTC), and ticks have run normally since. Schedule ticks that failed during this window may not have been retried automatically. If you're missing an expected run from that period, you can launch it manually from the Dagster+ UI. We're investigating the root cause and will take steps to prevent it from happening again.
- resolved
We've deployed several fixes and performance optimizations that have resolved this issue, and should prevent it in the future.
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).