Changes / Datadog / Delayed data processing and errors in multiple products

Datadog: Major incident: Delayed data processing and errors in multiple products

Summary

Datadog reported a major US1 degradation causing data-processing delays and errors across multiple products, including CI Visibility, RUM Explorer, Database Monitoring, Error Tracking, and all Security products. It ran from 2026-06-30 17:51 UTC to 2026-06-30 20:32 UTC (2 h 40 min); a fix was implemented and monitored before resolution.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 64%)Changed Detected Source: stspg.io

What changed

AddedTitle#1
Before: not present
After: Delayed data processing and errors in multiple products
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-06-30 17:51 UTC
AddedResolved time#5
Before: not present
After: 2026-06-30 20:32 UTC
AddedAffected components#6
Before: not present
After: APM, CI Visibility, Code Coverage, Database Monitoring, Error Tracking, Observability Pipelines, RUM, Synthetics, Test Optimization
AddedUpdateinvestigating#7
Before: not present
After: We are currently investigating an issue affecting CI Visibility, Test Optimization, Code Coverage, Software Delivery, Preemptive Alerting, Session Replay, RUM Explorer, Agent Observability, Database Monitoring, Event Correlation, Error Tracking, and all Security products in our US1 datacenter. As a result, some users may experience delays in data processing and errors when using affected products. Our engineering teams have identified the issue and are actively working on a resolution. Our engineering teams are actively working to resolve the issue, and we'll follow up with more information shortly. Please let us know if you have any questions or need anything else in the meantime.
AddedUpdateinvestigating#8
Before: not present
After: We are continuing to investigate this issue.
AddedUpdateinvestigating#9
Before: not present
After: We are continuing to investigate this issue.
AddedUpdateinvestigating#10
Before: not present
After: We are continuing to investigate this issue.
AddedUpdateidentified#11
Before: not present
After: The issue has been identified and a fix is being implemented.
AddedUpdateidentified#12
Before: not present
After: We are continuing to work on a fix for this issue.
AddedUpdatemonitoring#13
Before: not present
After: A fix has been implemented and we are monitoring the results.
AddedUpdateresolved#14
Before: not present
After: This incident has been resolved.

First version of this incident archived by Rewindex. View the live incident.

History of this incident

When (UTC)ChangeKind
Major incident: Delayed data processing and errors in multiple products (this change)
Datadog reported a major US1 degradation causing data-processing delays and errors across multiple products, including CI Visibility, RUM Explorer, Database Monitoring, Error Tracking, and all Security products. It ran from 2026-06-30 17:51 UTC to 2026-06-30 20:32 UTC (2 h 40 min); a fix was implemented and monitored before resolution.
Major incident

All Datadog changes → · Datadog profile · Markdown · JSON