ChangesFireHydrantFireHydrant integrations impacted
FireHydrant: Major incident: FireHydrant integrations impacted
Summary
Summary pending. The diff below shows exactly what changed.
Incident page: the full timeline of updates
What changed
Struck in red is what FireHydrant removed; blue is what they added. Muted text is unchanged context.
Before: not present
After: FireHydrant integrations impacted
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2025-10-20 13:10 UTC
Before: not present
After: 2025-10-21 13:09 UTC
Before: not present
After: Amazon Web Services (AWS) is currently experiencing an outage which is affecting our ability to connect with multiple third-party integrations.
The FireHydrant application is operational; however, runbook steps associate with multiple integration partners may fail or be delayed. This includes Atlassian (Jira, Confluence, Opsgenie), PagerDuty, Slack, Zendesk, and Zoom. Additionally, there may be delays with alert delivery via SMS and Whatsapp.
See https://health.aws.amazon.com/health/status for more details.
We will resolve this incident once we have received confirmation the AWS issue has been resolved.
Before: not present
After: We are continuing to monitor the AWS outage and it's affect on our upstream providers and integration partners.
Impacts to both Twilio and SendGrid are causing delays or failures in the ability to deliver alert notifications via SMS, Whatsapp, and email. Additionally, Slack messaging is affected.
**FireHydrant push notificaiton are unaffected by the outage and are the best option for you to continue receiving alerts.**
You can monitor the availability of our upstream providers as follows:
- Twilio: https://status.twilio.com/
- SendGrid: https://status.sendgrid.com/
- Slack: https://slack-status.com/2025-10/51d84290cf5c8e22
You can monitor the AWS outage here: https://health.aws.amazon.com/health/status
We will provide updates as they're available.
Before: not present
After: We are continuing to monitor the AWS outage and it's affect on our upstream providers and integration partners.
Impact to both Slack and Recall.ai is preventing or delaying the ability of the FireHydrant Scribe to join Zoom and Google Meet calls.
You can monitor the AWS outage here: https://health.aws.amazon.com/health/status
We will provide updates as they're available.
Before: not present
After: We are investigating the effect of the AWS outage on our feature flags.
This impacts users' ability to access features and settings in the FireHydrant UI, but we will provide another update once the full scope of the impact is understood.
Before: not present
After: We are investigating the effect of the AWS outage on our feature flags.
This impacts users' ability to access features and settings in the FireHydrant UI, but we will provide another update once the full scope of the impact is understood.
Before: not present
After: Our feature flag provider, LaunchDarkly, is experiencing an outage. This is causing a number of downstream issues, namely an inability to send ad-hoc alerts via /fh page or "Send an Alert" in the UI.
We are currently putting in a fix to bypass the Signals-related feature flags to restore manual paging.
Automated alert ingestion is still working so alerts generated from a team's alert rule or existing runbook steps remain operational.
You can follow LaunchDarkly's outage status here: https://status.launchdarkly.com/
Before: not present
After: Currently testing the fix to restore manual paging via Signals
Before: not present
After: LaunchDarkly is slowly recovering, but in the meantime we're simultaneously restoring functionality for flag-based features.
You can continue to monitor their status here: https://status.launchdarkly.com/
Before: not present
After: We've restored manual paging via Signals. Paging via alert rules and runbooks remained operational throughout this incident. Push notifications continue to be the best way to ensure alert notification delivery given continued service degradation for upstream providers effecting SMS, Whatsapp, and email.
Additionally, we patched an issue where users with a custom role were unable to access FireHydrant as expected.
AWS is also reporting that service recovery continues to improve. You can continue to monitor their status here: https://health.aws.amazon.com/health/status
Before: not present
After: LaunchDarkly is reporting significant improvements, and we are seeing our feature flags being successfully evaluated.
AWS is also reporting that they're recovering.
We're continuing to monitor the situation, as many of our upstream providers and integration partners are still impacted.
We'll provide material updates as they occur.
Before: not present
After: **AWS Outage: End-of-Day Update**
AWS has resolved their incident; however, we’re continuing to monitor upstream providers and integration partners that were affected by this outage. This includes, but is not limited to:
- Atlassian (Jira, Confluence, Opsgenie): https://status.atlassian.com/
- LaunchDarkly: https://status.launchdarkly.com/
- PagerDuty: https://status.pagerduty.com/posts/dashboard
- Recall.ai
- SendGrid: https://status.sendgrid.com/
- Slack: https://slack-status.com/
- Twilio: https://status.twilio.com/
Please report unexpected issues with FireHydrant via our shared Slack channel (preferred) or Support ticket (https://firehydrant.zendesk.com/hc/en-us/requests/new). We will be monitoring until all associated incidents have resolved.
Before: not present
After: We have confirmed all our upstream providers and integration partners have reported a full recovery from yesterday's outage. We will continue to monitor the situation and will post a full incident write-up on our blog later today.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
Archive proof
Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves this record existed by then and has not been changed since, by anyone, Rewindex included.
How the archive is checked, and what it cannot prove.
After captured
In the 2026-11-04 checkpoint, timestamped by FreeTSA and DigiCert on . Record hash
2fa5549b19025eee…
History of this incident
Monday, October 20, 2025
1 incident change
- OutageMajor incident
FireHydrant integrations impactedthis change
Cite this
Major incident: FireHydrant integrations impacted on FireHydrant’s status page, changed 2025-10-20 13:10 UTC (change 1 of 1).
- Title
- Major incident: FireHydrant integrations impacted
- Vendor
- FireHydrant
- Date
- URL
- https://rewindex.com/change/i4c97d6aeda457a