ChangesLiveKitObservability upload and Agent Simulations failures
LiveKit: Critical incident: Observability upload and Agent Simulations failures
Summary
Summary pending. The diff below shows exactly what changed.
Incident page: the full timeline of updates
What changed
Struck in red is what LiveKit removed; blue is what they added. Muted text is unchanged context.
Before: not present
After: Observability upload and Agent Simulations failures
Before: not present
After: critical
Before: not present
After: resolved
Before: not present
After: 2026-10-13 10:01 UTC
Before: not present
After: 2026-10-13 13:44 UTC
Before: not present
After: Cloud Dashboard (cloud.livekit.io)
Before: not present
After: We are investigating an issue with Agent Insights where agents are unable to upload their observability data at the end of their sessions. Some recent agent sessions may be missing from the dashboard's sessions list or shown as “In progress”. Agent sessions themselves are not affected. We will post an update shortly.
Before: not present
After: We have identified the cause of the Agent Insights issue and are working on a fix.
Before: not present
After: We are continuing to work on a fix for this issue.
Before: not present
After: Agent Insights uploads are now operational and session data has been storing normally since 12:40 UTC. We are still investigating an issue with Simulations. Simulation runs may complete without a transcript, summary, or verdict. We will post an update shortly.
Before: not present
After: This incident has been resolved. Between 08:15 and 12:35 UTC, agents were not able to upload agent observability data to LiveKit's servers in all regions. Agent sessions completed normally, but, unfortunately, any Agent Insights data which attempted to upload within that window (traces/transcripts/logs/recordings) is unrecoverable. The ability to download existing Agent Insights data was impaired until 13:15 UTC. Agent Simulations were unavailable due to a upstream dependency and came back online at 13:25 UTC.
Sincere apologies for the impact. We will follow up with a postmortem as soon as possible.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
Tuesday, October 13
1 incident change
- CriticalCritical incident
Observability upload and Agent Simulations failuresthis change