Changes / Grain / Bot failures
Grain: Major incident: Bot failures
Summary
Grain reported a major Grain Recorder outage starting 2026-06-11 15:18 UTC, with scheduled bots delayed in joining and ad-hoc bot failures caused by an upstream provider outage. The incident lasted 6 h 6 min and was resolved at 2026-06-11 21:24 UTC, with no further bot-service degradation observed.
Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.
What changed
Before: not present
After: Bot failures
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2026-06-11 15:18 UTC
Before: not present
After: 2026-06-11 21:24 UTC
Before: not present
After: Grain Recorder
Before: not present
After: An outage with an upstream provider is causing instability for bots including delays in scheduled bots joining and ad-hoc bot failures. The team is investigating potential mitigations.
Before: not present
After: At the half hour, the team observed minimal delays in scheduled bot and no ad-hoc failures. We are in communication with the upstream provider about any further potential issues.
Before: not present
After: The team has not observed any further degradation in the bot service.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Bot failures (this change) Grain reported a major Grain Recorder outage starting 2026-06-11 15:18 UTC, with scheduled bots delayed in joining and ad-hoc bot failures caused by an upstream provider outage. The incident lasted 6 h 6 min and was resolved at 2026-06-11 21:24 UTC, with no further bot-service degradation observed. | Major incident |