# FireHydrant: Issue with Runbooks Attaching to Incidents

> FireHydrant status incident “Issue with Runbooks Attaching to Incidents” on 2024-07-23: impact critical (vendor’s label), 2.8 h, 3 update(s). Archived by Rewindex.

Canonical: https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87
Last updated: 2024-07-23T19:17:57Z

- Impact (vendor’s label): critical
- Status: resolved
- Started: 2024-07-23 16:30 UTC · Resolved: 2024-07-23 19:17 UTC · Duration: 2.8 h
- Updates: 3 · Affected components: Runbook Execution
- [Vendor status page: https://status.firehydrant.com/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87](https://status.firehydrant.com/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87)
- [Company: FireHydrant](https://rewindex.com/company/firehydrant.md)

## Timeline of updates

- **resolved** 2024-07-23 16:40 UTC: We're investigating an issue with Runbooks failing to attach to incidents
- **resolved** 2024-07-23 18:22 UTC: All backlogs have successfully cleared and Runbooks are functioning as expected
- **resolved** 2024-07-23 19:17 UTC: FireHydrant experienced a prolonged delay of runbook executions due to a sudden outlier amount of traffic to our services. This traffic caused a shared worker queue to flood and go into a "spin lock" preventing other work from executing, such as runbook attachment. Existing incidents were not impacted from this incident, and our systems have been scaled to handle traffic spikes like the one experienced in the future.

## Rewindex changes for this incident

- 2024-07-23 16:30 UTC: [Critical incident: Issue with Runbooks Attaching to Incidents](https://rewindex.com/change/i95078a3fda487f.md)

## Cite this

Status incident (vendor impact label: critical) posted by FireHydrant, started 2024-07-23 16:30 UTC, resolved after 2.8 h, 3 update(s).

- Canonical URL: https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87
- Citation: FireHydrant, “Issue with Runbooks Attaching to Incidents” (2024-07-23 16:30 UTC). Archived by Rewindex: https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87 (CC BY 4.0).
- Markdown link: `[FireHydrant: Issue with Runbooks Attaching to Incidents (2024-07-23)](https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87)`


---
Canonical: https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87 · Markdown: https://rewindex.com/company/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87.md · JSON: https://rewindex.com/api/v1/companies/firehydrant/incidents/d80cd6ef-4299-4ab6-a5a8-986045372d87 · Data licensed CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) · Generated 2026-11-04T16:57:02.324Z