# Grain: Major incident: Bot failures

**Summary:** Grain reported a major Grain Recorder outage starting 2026-06-11 15:18 UTC, with scheduled bots delayed in joining and ad-hoc bot failures caused by an upstream provider outage. The incident lasted 6 h 6 min and was resolved at 2026-06-11 21:24 UTC, with no further bot-service degradation observed. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2026-06-11 15:18 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/fd8sv2py3911](https://stspg.io/fd8sv2py3911)
- [Company: Grain](https://rewindex.com/company/grain.md)
- Versions: before (none: first archived) → after incident\_version:20/hv63w8ksdsgg@a3cf6ee06339
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Bot failures
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2026-06-11 15:18 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-06-11 21:24 UTC
```

**6. Added: Affected components**

```diff
+ Grain Recorder
```

**7. Added: Update** (investigating) · 2026-06-11 15:18 UTC

```diff
+ An outage with an upstream provider is causing instability for bots including delays in scheduled bots joining and ad-hoc bot failures. The team is investigating potential mitigations.
```

**8. Added: Update** (monitoring) · 2026-06-11 15:40 UTC

```diff
+ At the half hour, the team observed minimal delays in scheduled bot and no ad-hoc failures. We are in communication with the upstream provider about any further potential issues.
```

**9. Added: Update** (resolved) · 2026-06-11 21:24 UTC

```diff
+ The team has not observed any further degradation in the bot service.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-06-11 15:18 UTC | Major incident: Bot failures (this change) | outage |

All Grain changes: [timeline](https://rewindex.com/company/grain/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/i2a189f875e46da · Markdown: https://rewindex.com/change/i2a189f875e46da.md · JSON: https://rewindex.com/api/v1/changes/i2a189f875e46da · Generated 2026-10-18T02:54:38.665Z