Grain: Bot Recording Failures
Grain Recorder had a major outage causing delayed bot joins and recording failures due to an upstream provider outage. It started at 2026-06-02 15:20 UTC and was resolved at 2026-06-03 16:49 UTC, lasting 25 h 29 min. Recordings stuck in processing required manual recovery; Grain confirmed all successfully captured recordings were recovered.
Machine-written from the archived incident (Sol). Check the timeline before relying on it.
Affected components: Grain Recorder
Timeline of updates
- investigating
Upstream provider is experiencing an outage causing out instability in our bot recorders including delayed joined times and recording failures. The team is investigating the scope and any potential mitigations.
- monitoring
The outage has subsided. The team has identified a set of recordings currently stuck in processing state that require manual recovery and will be working to recover them as quickly as possible.
- resolved
All recordings that were successfully captured have now been recovered. Future improvements around being more robust to upstream failures are being researched by the team.
Rewindex changes for this incident
- Major incident: Bot Recording Failures 2026-06-02 15:20 UTC
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Grain profile · Recent Grain incidents · Markdown · JSON