rewindex

CompaniesGrafana CloudIncidents

Grafana Cloud: Alerting expressions pipeline failing when recovery settings

MinorStatus: resolvededitedStarted Resolved (0 min)1 updatestspg.io (opens in new tab)

Affected components: Alerting and SLO - GCP UK - prod-gb-south-0: Alertmanager and Rules Configuration APIGCP UK - prod-gb-south-0: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Canada - prod-ca-east-0: Alertmanager and Rules Configuration APIAWS Canada - prod-ca-east-0: Alertmanager and Rules Configuration APIAlerting and SLO - Azure US Central - us-central2: Rule EvaluationAzure US Central - us-central2: Rule EvaluationAlerting and SLO - AWS US West - prod-us-west-0: Rule EvaluationAWS US West - prod-us-west-0: Rule EvaluationAlerting and SLO - AWS US West - prod-us-west-0: Alertmanager and Rules Configuration APIAWS US West - prod-us-west-0: Alertmanager and Rules Configuration APIAlerting and SLO - AWS UAE - prod-me-central-1: AlertmanagerAWS UAE - prod-me-central-1: AlertmanagerAlerting and SLO - AWS US East - prod-us-east-0: Rule EvaluationAWS US East (OH) - prod-us-east-0: Rule EvaluationAlerting and SLO - AWS Brazil - prod-sa-east-1: AlertmanagerAWS Brazil - prod-sa-east-1: AlertmanagerAlerting and SLO - GCP Australia - prod-au-southeast-0: Alertmanager and Rules Configuration APIGCP Australia - prod-au-southeast-0: Alertmanager and Rules Configuration APIAlerting and SLO - GCP India - prod-ap-south-0: Rule EvaluationGCP India - prod-ap-south-0: Rule EvaluationAlerting and SLO - AWS Singapore - prod-ap-southeast-1: Rule EvaluationAWS Singapore - prod-ap-southeast-1: Rule EvaluationAlerting and SLO - AWS Sweden - prod-eu-north-0: AlertmanagerAWS Sweden - prod-eu-north-0: AlertmanagerAlerting and SLO - AWS US West - prod-us-west-0: AlertmanagerAWS US West - prod-us-west-0: AlertmanagerAlerting and SLO - AWS Sweden - prod-eu-north-0: Alertmanager and Rules Configuration APIAWS Sweden - prod-eu-north-0: Alertmanager and Rules Configuration APIAlerting and SLO - GCP Singapore - prod-ap-southeast-0: AlertmanagerGCP Singapore - prod-ap-southeast-0: AlertmanagerAlerting and SLO - Azure Netherlands - prod-eu-west-3: Rule EvaluationAzure Netherlands - prod-eu-west-3: Rule EvaluationAlerting and SLO - AWS Singapore - prod-ap-southeast-1: Alertmanager and Rules Configuration APIAWS Singapore - prod-ap-southeast-1: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Australia - prod-ap-southeast-2: Alertmanager and Rules Configuration APIAWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration APIAlerting and SLO - GCP UK - prod-gb-south-0: Rule EvaluationGCP UK - prod-gb-south-0: Rule EvaluationAlerting and SLO - AWS UAE - prod-me-central-1: Alertmanager and Rules Configuration APIAWS UAE - prod-me-central-1: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Germany - prod-eu-west-2: Alertmanager and Rules Configuration APIAWS Germany - prod-eu-west-2: Alertmanager and Rules Configuration APIAlerting and SLO - GCP Australia - prod-au-southeast-0: Rule EvaluationGCP Australia - prod-au-southeast-0: Rule EvaluationAlerting and SLO - AWS Switzerland - prod-eu-central-0AWS Switzerland - prod-eu-central-0Alerting and SLO - AWS Sweden - prod-eu-north-0: Rule EvaluationAWS Sweden - prod-eu-north-0: Rule EvaluationAlerting and SLO - GCP Brazil - prod-sa-east-0: Alertmanager and Rules Configuration APIGCP Brazil - prod-sa-east-0: Alertmanager and Rules Configuration APIAlerting and SLO - GCP Singapore - prod-ap-southeast-0: Alertmanager and Rules Configuration APIGCP Singapore - prod-ap-southeast-0: Alertmanager and Rules Configuration APIAlerting and SLO - AWS UAE - prod-me-central-1: Rule EvaluationAWS UAE - prod-me-central-1: Rule EvaluationAlerting and SLO - AWS India - prod-ap-south-1: Rule EvaluationAWS India - prod-ap-south-1: Rule EvaluationAlerting and SLO - AWS US East - prod-us-east-0: AlertmanagerAWS US East (OH) - prod-us-east-0: AlertmanagerAlerting and SLO - GCP Singapore - prod-ap-southeast-0: Rule EvaluationGCP Singapore - prod-ap-southeast-0: Rule EvaluationAlerting and SLO - AWS Canada - prod-ca-east-0: Rule EvaluationAWS Canada - prod-ca-east-0: Rule EvaluationAlerting and SLO - AWS Germany - prod-eu-west-2: Rule EvaluationAWS Germany - prod-eu-west-2: Rule EvaluationAlerting and SLO - GCP India - prod-ap-south-0: AlertmanagerGCP India - prod-ap-south-0: AlertmanagerAlerting and SLO - AWS US East - prod-us-east-0: Alertmanager and Rules Configuration APIAWS US East (OH) - prod-us-east-0: Alertmanager and Rules Configuration APIAlerting and SLO - GCP Belgium - prod-eu-west-0: Alertmanager and Rules Configuration APIGCP Belgium - prod-eu-west-0: Alertmanager and Rules Configuration APIAlerting and SLO - Azure Netherlands - prod-eu-west-3: Alertmanager and Rules Configuration APIAzure Netherlands - prod-eu-west-3: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Germany - prod-eu-west-2: AlertmanagerAWS Germany - prod-eu-west-2: AlertmanagerAlerting and SLO - Federal Cloud - AWS US Gov WestFederal Cloud - AWS US Gov WestAlerting and SLO - GCP Belgium - prod-eu-west-0: Rule EvaluationGCP Belgium - prod-eu-west-0: Rule EvaluationAlerting and SLO - AWS Japan - prod-ap-northeast-0: Rule EvaluationAWS Japan - prod-ap-northeast-0: Rule EvaluationAlerting and SLO - GCP Australia - prod-au-southeast-0: AlertmanagerGCP Australia - prod-au-southeast-0: AlertmanagerAlerting and SLO - GCP US Central - prod-us-central-0: Alertmanager and Rules Configuration APIGCP US Central - prod-us-central-0: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Brazil - prod-sa-east-1: Rule EvaluationAWS Brazil - prod-sa-east-1: Rule EvaluationAlerting and SLO - AWS Singapore - prod-ap-southeast-1: AlertmanagerAWS Singapore - prod-ap-southeast-1: AlertmanagerAlerting and SLO - GCP Brazil - prod-sa-east-0: AlertmanagerGCP Brazil - prod-sa-east-0: AlertmanagerAlerting and SLO - GCP Belgium - prod-eu-west-0: AlertmanagerGCP Belgium - prod-eu-west-0: AlertmanagerAlerting and SLO - Azure US Central - us-central2: Alertmanager and Rules Configuration APIAzure US Central - us-central2: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Australia - prod-ap-southeast-2: AlertmanagerAWS Australia - prod-au-southeast-1: AlertmanagerAlerting and SLO - GCP Brazil - prod-sa-east-0: Rule EvaluationGCP Brazil - prod-sa-east-0: Rule EvaluationAlerting and SLO - Azure Netherlands - prod-eu-west-3: AlertmanagerAzure Netherlands - prod-eu-west-3: AlertmanagerAlerting and SLO - AWS Ireland - prod-eu-west-6AWS Ireland - prod-eu-west-6Alerting and SLO - AWS Australia - prod-ap-southeast-2: Rule EvaluationAWS Australia - prod-au-southeast-1: Rule EvaluationAlerting and SLO - GCP US Central - prod-us-central-0: AlertmanagerGCP US Central - prod-us-central-0: AlertmanagerAlerting and SLO - GCP US Central - prod-us-central-0: Rule EvaluationGCP US Central - prod-us-central-0: Rule EvaluationAlerting and SLO - AWS Canada - prod-ca-east-0: AlertmanagerAWS Canada - prod-ca-east-0: AlertmanagerAlerting and SLO - Azure US Central - us-central2: AlertmanagerAzure US Central - us-central2: AlertmanagerAlerting and SLO - GCP UK - prod-gb-south-0: AlertmanagerGCP UK - prod-gb-south-0: AlertmanagerAlerting and SLO - AWS Brazil - prod-sa-east-1: Alertmanager and Rules Configuration APIAWS Brazil - prod-sa-east-1: Alertmanager and Rules Configuration APIAlerting and SLO - AWS Japan - prod-ap-northeast-0: Alertmanager and Rules Configuration APIAWS Japan - prod-ap-northeast-0: Alertmanager and Rules Configuration APIAlerting and SLO - AWS India - prod-ap-south-1: Alertmanager and Rules Configuration APIAWS India - prod-ap-south-1: Alertmanager and Rules Configuration APIAlerting and SLO - GCP India - prod-ap-south-0: Alertmanager and Rules Configuration APIGCP India - prod-ap-south-0: Alertmanager and Rules Configuration API

Timeline of updates

  1. resolved

    Due to a software bug the evaluation of some alert rules (primarily ones that have recovery threshold setting) were failing to be evaluated starting 14:00 UTC to 19:30 UTC today.

Changes Rewindex caught on this incident

Archive proof

Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves these records existed by then and have not been changed since, by anyone, Rewindex included.

How the archive is checked, and what it cannot prove.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Cite this

Status incident (vendor impact label: minor) posted by Grafana Cloud, started 2026-08-08 19:48 UTC, resolved after 0 min, 1 update(s); edited after publishing.

Title
Alerting expressions pipeline failing when recovery settings
Vendor
Grafana Cloud
Date
URL
https://rewindex.com/company/grafana-cloud/incidents/2tjkx7vn9nt0

Grafana Cloud profileRecent Grafana Cloud incidentsAll outages on August 8, 2026Grafana Cloud in August 2026MarkdownJSON