Grafana Cloud: Elevated Errors Ingesting Metrics in AWS US East (VA)

Impact: majorStatus: resolvedStarted 2026-10-12 09:45 UTCResolved 2026-10-12 16:42 UTC (7.0 h)8 updatesSource: stspg.io

Summary

Grafana Cloud reported a major metrics-ingestion incident in AWS US East (VA) - prod-us-east-3, causing failed or delayed writes and potentially affecting Frontend Observability and Synthetic Monitoring data. It started 2026-10-12 09:45 UTC and resolved 2026-10-12 16:42 UTC (6 h 56 min); Grafana isolated suspected contributing infrastructure and reported normal service restored.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: AWS US East (VA) - prod-us-east-3Metrics - AWS US East (VA) - prod-us-east-3

Timeline of updates

investigating2026-10-12 09:45 UTC
We are investigating an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes. Next update by 10:15 UTC.
investigating2026-10-12 10:08 UTC
We are continuing to investigate an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes. Next update by 11:00 UTC.
investigating2026-10-12 10:55 UTC
We are continuing to investigate an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. Next update by 12:00 UTC.
identified2026-10-12 11:56 UTC
We have identified the cause of an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA) - prod-us-east-3. Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. We are continuing to work on mitigations and are monitoring the impact to Grafana Cloud Metrics. Next update by 13:00 UTC.
identified2026-10-12 12:53 UTC
We have identified the cause of an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA) - prod-us-east-3. Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. We are continuing to work on mitigations and are monitoring the impact to Grafana Cloud Metrics. Next update by 14:00 UTC.
identified2026-10-12 14:01 UTC
We have isolated infrastructure suspected of contributing to the metrics ingestion issue in AWS US East (VA) - prod-us-east-3. We are validating recovery and completing mitigation steps. Customers may continue to experience failed or delayed metric writes while this work continues.
identified2026-10-12 15:01 UTC
We have isolated infrastructure suspected of contributing to the metrics ingestion issue in AWS US East (VA) - prod-us-east-3. We are validating recovery and completing mitigation steps. Customers may continue to experience failed or delayed metric writes while this work continues.
resolved2026-10-12 16:42 UTC
From 09:45 UTC on 2026-10-12, customers in AWS US East (VA) experienced failed or delayed metric writes to Grafana Cloud Metrics, which may have also affected Frontend Observability and Synthetic Monitoring data in that region. This has been fully resolved and all services are operating normally again.

Rewindex changes for this incident

Major incident: Elevated Errors Ingesting Metrics in AWS US East (VA) 2026-10-12 09:45 UTC

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Grafana Cloud profile · Recent Grafana Cloud incidents · Markdown · JSON