# Grain: Critical incident: Zoom join failures

**Summary:** Grain reported a critical outage starting 2023-12-19 15:33 UTC: Zoom blocked Grain Recorder from joining approximately 75% of calls. A temporary mitigation appeared successful; Grain later worked with Zoom to deploy a proper fix, reporting no further issues. The incident was resolved 2024-01-19 17:40 UTC, with a recorded duration of 31 days 2 h. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2023-12-19 15:33 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/vwxyhpkrnp6b](https://stspg.io/vwxyhpkrnp6b)
- [Company: Grain](https://rewindex.com/company/grain.md)
- Versions: before (none: first archived) → after incident\_version:20/py2fdp73gxwq@fad339149fd3
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Zoom join failures
```

**2. Added: Impact**

```diff
+ critical
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2023-12-19 15:33 UTC
```

**5. Added: Resolved time**

```diff
+ 2024-01-19 17:40 UTC
```

**6. Added: Affected components**

```diff
+ Grain Recorder
```

**7. Added: Update** (investigating) · 2023-12-19 15:33 UTC

```diff
+ Zoom has started blocking our Recorder from joining calls. We are in discussion with Zoom and we are attempting to work around the issue.
```

**8. Added: Update** (investigating) · 2023-12-19 15:54 UTC

```diff
+ We are on a conference call with Zoom engineers trying to work around the issue
```

**9. Added: Update** (investigating) · 2023-12-19 15:56 UTC

```diff
+ We've rolled out a potential short term mitigation and we're monitoring the solution.
```

**10. Added: Update** (monitoring) · 2023-12-19 18:19 UTC

```diff
+ Our temporary mitigation continues to appear to be successful. Zoom is still investigating the source of the issue as it appears to be a bug on their end causing our bot to be rejected by ~75% of calls. They have proposed a fix that we will be testing over the next 24 hours.
```

**11. Added: Update** (resolved) · 2024-01-19 17:40 UTC

```diff
+ The team worked with Zoom over the holidays to deploy a proper fix. No further issues have been observed.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2023-12-19 15:33 UTC | Critical incident: Zoom join failures (this change) | outage |

All Grain changes: [timeline](https://rewindex.com/company/grain/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/i3c5311d1248de7 · Markdown: https://rewindex.com/change/i3c5311d1248de7.md · JSON: https://rewindex.com/api/v1/changes/i3c5311d1248de7 · Generated 2026-10-18T02:55:20.753Z