Companies / Lightdash / incidents
Lightdash: Intermittent API errors across US
Lightdash Cloud (US) experienced intermittent API request failures with major impact from 2026-03-24 10:35 UTC to 2026-03-24 12:19 UTC, lasting 1 h 43 min. Outbound connections exceeded cloud networking capacity, dropping requests; Lightdash restructured its internal network architecture to reduce outbound connection load and reported all services healthy after deploying the fix.
Machine-written from the archived incident (Sol). Check the timeline before relying on it.
Affected components: Lightdash Cloud - Lightdash Cloud (US)Lightdash Cloud (US)
Timeline of updates
- identified
We have identified an issue in our network configuration that is leading to some requests failing. We've identified fix and are deploying to production.
- identified
We are continuing to work on a fix for this issue.
- identified
We are continuing to work on a fix for this issue.
- monitoring
We have deployed a fix to our network configuration and all services are healthy. We will continue monitoring.
- resolved
We've seen no errors related to network configuration.
- postmortem
**Root cause** Our cloud networking layer had insufficient capacity to handle the volume of outbound connections from our services. As traffic grew, we periodically exceeded the available connection limits, causing some requests to be dropped. **What we fixed** We restructured our internal network architecture, significantly reducing the load on our outbound connection layer. This freed up capacity. **Ensuring it won’t happen again** We’ve permanently re-architected to prevent this specific issue. We’ve added monitoring \+ alerts to catch similar network errors.
Rewindex changes for this incident
- Major incident: Intermittent API errors across US 2026-03-24 10:35 UTC
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Lightdash profile · Recent Lightdash incidents · Markdown · JSON