# Lightdash: Intermittent API errors across US

- Impact (vendor’s label): major
- Status: postmortem
- Started: 2026-03-24 10:35 UTC · Resolved: 2026-03-24 12:19 UTC · Duration: 1.7 h
- Updates: 6 · Affected components: Lightdash Cloud - Lightdash Cloud (US), Lightdash Cloud (US)
- [Vendor status page: https://stspg.io/970vh708h991](https://stspg.io/970vh708h991)
- [Company: Lightdash](https://rewindex.com/company/lightdash.md)

## Summary

Lightdash Cloud (US) experienced intermittent API request failures with major impact from 2026-03-24 10:35 UTC to 2026-03-24 12:19 UTC, lasting 1 h 43 min. Outbound connections exceeded cloud networking capacity, dropping requests; Lightdash restructured its internal network architecture to reduce outbound connection load and reported all services healthy after deploying the fix. _(machine-written, sol)_
## Timeline of updates

- **identified** 2026-03-24 10:35 UTC: We have identified an issue in our network configuration that is leading to some requests failing. We've identified fix and are deploying to production.
- **identified** 2026-03-24 10:35 UTC: We are continuing to work on a fix for this issue.
- **identified** 2026-03-24 10:36 UTC: We are continuing to work on a fix for this issue.
- **monitoring** 2026-03-24 11:34 UTC: We have deployed a fix to our network configuration and all services are healthy. We will continue monitoring.
- **resolved** 2026-03-24 12:19 UTC: We've seen no errors related to network configuration.
- **postmortem** 2026-03-24 12:21 UTC: \*\*Root cause\*\*   Our cloud networking layer had insufficient capacity to handle the volume of outbound connections from our services. As traffic grew, we periodically exceeded the available connection limits, causing some requests to be dropped.      \*\*What we fixed\*\*   We restructured our internal network architecture, significantly reducing the load on our outbound connection layer. This freed up capacity. ‌ \*\*Ensuring it won’t happen again\*\* We’ve permanently re-architected to prevent this specific issue. We’ve added monitoring \\+ alerts to catch similar network errors.

## Rewindex changes for this incident

- 2026-03-24 10:35 UTC: [Major incident: Intermittent API errors across US](https://rewindex.com/change/i846c3ee6e873c6.md)


---
Canonical: https://rewindex.com/company/lightdash/incidents/b0j060x9z0nf · Markdown: https://rewindex.com/company/lightdash/incidents/b0j060x9z0nf.md · JSON: https://rewindex.com/api/v1/companies/lightdash/incidents/b0j060x9z0nf · Generated 2026-10-18T10:39:00.596Z