Changes / Grafana Cloud / Elevated Errors Ingesting Metrics in AWS US East (VA)
Grafana Cloud: Major incident: Elevated Errors Ingesting Metrics in AWS US East (VA)
Summary
Grafana Cloud reported a major metrics-ingestion incident in AWS US East (VA) - prod-us-east-3, causing failed or delayed writes and potentially affecting Frontend Observability and Synthetic Monitoring data. It started 2026-10-12 09:45 UTC and resolved 2026-10-12 16:42 UTC (6 h 56 min); Grafana isolated suspected contributing infrastructure and reported normal service restored.
Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.
What changed
Before: not present
After: Elevated Errors Ingesting Metrics in AWS US East (VA)
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2026-10-12 09:45 UTC
Before: not present
After: 2026-10-12 16:42 UTC
Before: not present
After: AWS US East (VA) - prod-us-east-3, Metrics - AWS US East (VA) - prod-us-east-3
Before: not present
After: We are investigating an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes. Next update by 10:15 UTC.
Before: not present
After: We are continuing to investigate an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes. Next update by 11:00 UTC.
Before: not present
After: We are continuing to investigate an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA). Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. Next update by 12:00 UTC.
Before: not present
After: We have identified the cause of an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA) - prod-us-east-3. Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. We are continuing to work on mitigations and are monitoring the impact to Grafana Cloud Metrics. Next update by 13:00 UTC.
Before: not present
After: We have identified the cause of an issue affecting metrics ingestion for Grafana Cloud Metrics customers in AWS US East (VA) - prod-us-east-3. Customers may see failed or delayed writes to Grafana Cloud Metrics, this may also impact Metrics written by Frontend Observability and/or Synthetic Monitoring. We are continuing to work on mitigations and are monitoring the impact to Grafana Cloud Metrics. Next update by 14:00 UTC.
Before: not present
After: We have isolated infrastructure suspected of contributing to the metrics ingestion issue in AWS US East (VA) - prod-us-east-3. We are validating recovery and completing mitigation steps. Customers may continue to experience failed or delayed metric writes while this work continues.
Before: not present
After: We have isolated infrastructure suspected of contributing to the metrics ingestion issue in AWS US East (VA) - prod-us-east-3. We are validating recovery and completing mitigation steps. Customers may continue to experience failed or delayed metric writes while this work continues.
Before: not present
After: From 09:45 UTC on 2026-10-12, customers in AWS US East (VA) experienced failed or delayed metric writes to Grafana Cloud Metrics, which may have also affected Frontend Observability and Synthetic Monitoring data in that region. This has been fully resolved and all services are operating normally again.
First version of this incident archived by Rewindex. View the live incident.
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Elevated Errors Ingesting Metrics in AWS US East (VA) (this change) Grafana Cloud reported a major metrics-ingestion incident in AWS US East (VA) - prod-us-east-3, causing failed or delayed writes and potentially affecting Frontend Observability and Synthetic Monitoring data. It started 2026-10-12 09:45 UTC and resolved 2026-10-12 16:42 UTC (6 h 56 min); Grafana isolated suspected contributing infrastructure and reported normal service restored. | Major incident |
All Grafana Cloud changes → · Grafana Cloud profile · Markdown · JSON