# Expo: Major incident: Website and services partially unavailable due to high database load

**Summary:** Expo reported a major degradation starting 2026-10-05 20:38 UTC, with resolution recorded earlier at 2026-10-05 20:11 UTC; duration is unknown. High primary-database CPU load left the website and API at 80-90% availability, affecting builds, submissions, CI/CD jobs and publishing updates. Serving updates to end users was unaffected. _(machine-written, sol)_
- Kind: Major incident · Label: degradation (model (Jev))
- Changed: 2026-10-05 20:38 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/y0fcz8ysvmnh](https://stspg.io/y0fcz8ysvmnh)
- [Company: Expo](https://rewindex.com/company/expo.md)
- Versions: before (none: first archived) → after incident\_version:185/8mnxqntmv8zd@4fd8fc54f0a2
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Website and services partially unavailable due to high database load
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ postmortem
```

**4. Added: Start time**

```diff
+ 2026-10-05 20:38 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-10-05 20:11 UTC
```

**6. Added: Update** (resolved) · 2026-10-05 20:11 UTC

```diff
+ Due to high database load, the Expo website and several services including submitting CI/CD jobs, build jobs, and store submissions were partially unavailable. The issue has subsided as of 1:30PM PT and we are continuing to diagnose the root cause and monitor the health of the services.
```

**7. Added: Update** (postmortem) · 2026-10-05 20:40 UTC

```diff
+ The primary database was under high CPU load from 1:11PM-1:29PM PT. The impact of this is that our website and API were partially unavailable \(more precisely, at 80-90% availability\) during that time. ### Scope What was affected: This affected most services including the website, builds, submissions, CI/CD jobs, and publishing updates. What wasn’t affected: Serving updates to end users maintained 100.000% availability. App performance measurements sent to Observe were also unaffected. ### Root cause analysis An internal orchestrator service used to update the state of CI/CD jobs, including build jobs and store submission jobs, was updated to retry upon application-level failures. We are investigating more thoroughly and currently believe this led to a cascade of retries that caused more failures once the database was under more load than it could handle. ### Remediation The commit that changed retry behavior was rolled back and we are clearing the queue of retries. For a more robust solution, the internal orchestrator must gracefully handle backpressure from the application server and database, and to apply more backoff.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-10-05 20:38 UTC | Major incident: Website and services partially unavailable due to high database load (this change) | degradation |

All Expo changes: [timeline](https://rewindex.com/company/expo/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/ib631611537ca5f · Markdown: https://rewindex.com/change/ib631611537ca5f.md · JSON: https://rewindex.com/api/v1/changes/ib631611537ca5f · Generated 2026-10-18T01:53:19.882Z