# Upstash: Major incident: Fly.io Upstash Redis service distruption (FRA region)

**Summary:** Upstash reported a major degradation affecting some Fly.io Redis databases in FRA, with increased latency or timeouts. It started at 2026-05-11 15:05 UTC and resolved at 2026-05-11 18:22 UTC, lasting 3 h 16 min. The postmortem described Redis processes stalling inside a logging syscall; the supplied root-cause explanation is truncated. _(machine-written, sol)_
- Kind: Major incident · Label: degradation (model (Jev))
- Changed: 2026-05-11 15:05 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/xc9fkbk20d2w](https://stspg.io/xc9fkbk20d2w)
- [Company: Upstash](https://rewindex.com/company/upstash.md)
- Versions: before (none: first archived) → after incident\_version:171/47bg91tdj0hz@b3c71bc82de4
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Fly.io Upstash Redis service distruption (FRA region)
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ postmortem
```

**4. Added: Start time**

```diff
+ 2026-05-11 15:05 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-05-11 18:22 UTC
```

**6. Added: Update** (investigating) · 2026-05-11 15:05 UTC

```diff
+ Some databases may experience increased latency or timeouts in Fly.io’s FRA region.
```

**7. Added: Update** (investigating) · 2026-05-11 15:07 UTC

```diff
+ We are continuing to investigate the issue.
```

**8. Added: Update** (investigating) · 2026-05-11 17:19 UTC

```diff
+ We are working with Fly team to investigate the root cause.
```

**9. Added: Update** (resolved) · 2026-05-11 18:22 UTC

```diff
+ The incident has been resolved. We are working with Fly team on RCA.
```

**10. Added: Update** (postmortem) · 2026-05-15 12:49 UTC

```diff
+ On May 12th and 13th at various times, a subset of Upstash Redis instances on [Fly.io](http://Fly.io) experienced intermittent hangs and elevated error rates. The Redis process would stall inside a logging syscall — alive but not making progress — which made the issue hard to spot from our usual telemetry. After investigating with Fly's team, we identified the root cause as a bad interaction between a recent guest kernel update on Fly's newer machines and an upstream Cloud Hypervisor bug \([cloud-hypervisor#7672](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7672)\) affecting log writes from inside the VM. We mitigated by disabling the affected logging paths, and Fly has since rolled out a hypervisor-side patch, fully resolving the issue. No data was lost. Sorry for the disruption.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-05-11 15:05 UTC | Major incident: Fly.io Upstash Redis service distruption (FRA region) (this change) | degradation |

All Upstash changes: [timeline](https://rewindex.com/company/upstash/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/if87001a23cf3c8 · Markdown: https://rewindex.com/change/if87001a23cf3c8.md · JSON: https://rewindex.com/api/v1/changes/if87001a23cf3c8 · Generated 2026-10-18T02:55:17.215Z