# Grain: Webhook consumers failing causing app failures

- Impact (vendor’s label): major
- Status: monitoring · edited after publishing
- Started: 2026-10-20 17:38 UTC · Resolved: not resolved · Duration: ongoing
- Updates: 3 · Affected components: Grain Recorder, Recording Processing
- [Vendor status page: https://stspg.io/979tbj5kzvjx](https://stspg.io/979tbj5kzvjx)
- [Company: Grain](https://rewindex.com/company/grain.md)

## Timeline of updates

- **investigating** 2026-10-20 17:38 UTC: The team is actively rolling back a failed update for its webhook consumers. Webhooks are currently failing to process leading to UI to fail to update for bots and recording processing. No data loss has occurred.
- **monitoring** 2026-10-20 18:05 UTC: The team has deployed a rollback and webhooks are correctly process. The team is determining if any recording process needs manual intervention.
- **monitoring** 2026-10-20 18:33 UTC: The team is manually reprocessing all dangling completed recordings as they become available. All new recordings are processing correctly.

## Rewindex changes for this incident

- 2026-10-20 17:38 UTC: [Major incident: Webhook consumers failing causing app failures](https://rewindex.com/change/i69aac0f11b710a.md)
- 2026-10-20 18:13 UTC: [Incident updated: Webhook consumers failing causing app failures](https://rewindex.com/change/r00a5e136b202ae.md)
- 2026-10-20 18:43 UTC: [Incident updated: Webhook consumers failing causing app failures](https://rewindex.com/change/r024917f19aff7e.md)


---
Canonical: https://rewindex.com/company/grain/incidents/7kpmkpwb04tr · Markdown: https://rewindex.com/company/grain/incidents/7kpmkpwb04tr.md · JSON: https://rewindex.com/api/v1/companies/grain/incidents/7kpmkpwb04tr · Generated 2026-10-20T23:37:28.078Z