# LiveKit: Investigating reports of elevated egress & connector API errors

- Impact (vendor’s label): minor
- Status: postmortem
- Started: 2026-08-20 21:10 UTC · Resolved: 2026-08-20 21:47 UTC · Duration: 37 min
- Updates: 4 · Affected components: Global Egress
- [Vendor status page: https://stspg.io/2s0dm4mlxk95](https://stspg.io/2s0dm4mlxk95)
- [Company: LiveKit](https://rewindex.com/company/livekit.md)

## Timeline of updates

- **investigating** 2026-08-20 21:10 UTC: We are currently investigating reports of elevated API errors across the Egress service. We will post another update in 5 minutes.
- **investigating** 2026-08-20 21:24 UTC: We had a spike of errors between 20:52 and 21:01 UTC, and the errors have subsided to baseline as of 21:01 UTC. We are actively monitoring and will share more updates in 30 minutes.
- **resolved** 2026-08-20 21:47 UTC: This incident has been resolved. Between 20:52 and 21:01 UTC, we observed an increase in failed egress and connector API requests. Errors have returned to baseline as of 21:01 UTC. We will follow up with a postmortem with further details.
- **postmortem** 2026-09-10 19:08 UTC: On August 20, 2026, customers in our US East region experienced approximately 10 minutes of 503 errors on the LiveKit Ingress API and failures launching media track egress, between 20:52 and 21:01 UTC. All other regions were unaffected. ‌ \*\*Root Cause\*\*   An internal routing service lost one of its two instances, and a concurrent infrastructure issue with our cloud provider was preventing new compute nodes from provisioning in US East, so the replacement couldn't start. At 20:51 UTC, an automated maintenance process removed that last instance, making the service fully unavailable. It recovered at 21:01 UTC once the evicted instance freed capacity for a replacement. ‌ \*\*Timeline \\(UTC\\)\*\* \* Aug 20, 20:52 - Automated maintenance removes last instance in the pool; customer-facing errors begin \* Aug 20, 21:01 - Service restored \* Aug 21, 03:00 - Cloud provider resolves the underlying infrastructure issue; region fully healthy ‌ \*\*Mitigations & Follow-ups\*\* \* The service is restored with two instances guaranteed to run on separate physical nodes. \* We updated the automated maintenance process to avoid evicting the last healthy instance of a service. \* We added disruption protection for the affected service and are continuing broader resilience improvements. \* Our cloud provider resolved the infrastructure issue that had prevented new nodes from scaling up.


---
Canonical: https://rewindex.com/company/livekit/incidents/g3tvqpzhd9m0 · Markdown: https://rewindex.com/company/livekit/incidents/g3tvqpzhd9m0.md · JSON: https://rewindex.com/api/v1/companies/livekit/incidents/g3tvqpzhd9m0 · Generated 2026-10-19T23:15:31.931Z