Search

Companies / Expo / incidents

Expo: Website and services partially unavailable due to high database load

Impact: majorStatus: postmortemStarted Resolved (-27 min)2 updatesSource: stspg.io (opens in new tab)
Summary

Expo reported a major degradation starting 2026-10-05 20:38 UTC, with resolution recorded earlier at 2026-10-05 20:11 UTC; duration is unknown. High primary-database CPU load left the website and API at 80-90% availability, affecting builds, submissions, CI/CD jobs and publishing updates. Serving updates to end users was unaffected.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Timeline of updates

  1. resolved

    Due to high database load, the Expo website and several services including submitting CI/CD jobs, build jobs, and store submissions were partially unavailable. The issue has subsided as of 1:30PM PT and we are continuing to diagnose the root cause and monitor the health of the services.

  2. postmortem

    The primary database was under high CPU load from 1:11PM-1:29PM PT. The impact of this is that our website and API were partially unavailable \(more precisely, at 80-90% availability\) during that time. ### Scope What was affected: This affected most services including the website, builds, submissions, CI/CD jobs, and publishing updates. What wasn’t affected: Serving updates to end users maintained 100.000% availability. App performance measurements sent to Observe were also unaffected. ### Root cause analysis An internal orchestrator service used to update the state of CI/CD jobs, including build jobs and store submission jobs, was updated to retry upon application-level failures. We are investigating more thoroughly and currently believe this led to a cascade of retries that caused more failures once the database was under more load than it could handle. ### Remediation The commit that changed retry behavior was rolled back and we are clearing the queue of retries. For a more robust solution, the internal orchestrator must gracefully handle backpressure from the application server and database, and to apply more backoff.

Rewindex changes for this incident

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Expo profile · Recent Expo incidents · Markdown · JSON