Changes / Astronomer / Incorrect scheduler heartbeat metrics being reported

Astronomer: Major incident: Incorrect scheduler heartbeat metrics being reported

Summary

Astronomer reported a major degradation from 2026-08-18 18:51 UTC to 2026-09-10 22:19 UTC, lasting 23 days 3 h. A metrics reporting bug falsely flagged missed scheduler heartbeats, affecting Astro UI dashboard and monitoring visibility; task execution, scheduling, and workflow processing were unaffected. Astronomer reported rolling out a fix and later marked the issue resolved.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 100%)Changed Detected Source: status.astronomer.io

What changed

AddedTitle#1
Before: not present
After: Incorrect scheduler heartbeat metrics being reported
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-08-18 18:51 UTC
AddedResolved time#5
Before: not present
After: 2026-09-10 22:19 UTC
AddedUpdateinvestigating#6
Before: not present
After: Our metrics pipeline is falsely displaying spikes in scheduler heartbeat metrics, despite underlying component health.
AddedUpdateidentified#7
Before: not present
After: We've identified the cause of the false availability alerts that have been showing the Astro UI. This is a degradation affecting dashboard and monitoring visibility only. Task execution, scheduling, and workflow processing have not been impacted. The issue is the result of a metrics reporting bug that has been intermittently flagging scheduler heartbeats as missed when they were not. We are actively rolling out a fix that corrects how these metrics are reported.
AddedUpdateresolved#8
Before: not present
After: This issue is now resolved.

First version of this incident archived by Rewindex. View the live incident.

History of this incident

When (UTC)ChangeKind
Major incident: Incorrect scheduler heartbeat metrics being reported (this change)
Astronomer reported a major degradation from 2026-08-18 18:51 UTC to 2026-09-10 22:19 UTC, lasting 23 days 3 h. A metrics reporting bug falsely flagged missed scheduler heartbeats, affecting Astro UI dashboard and monitoring visibility; task execution, scheduling, and workflow processing were unaffected. Astronomer reported rolling out a fix and later marked the issue resolved.
Major incident

All Astronomer changes → · Astronomer profile · Markdown · JSON