Search

Companies / Honeycomb / incidents

Honeycomb: Ingest issues for some customers

Impact: majorStatus: resolvedStarted Resolved (3.7 h)7 updatesSource: stspg.io (opens in new tab)
Summary

Honeycomb reported a major US1 Event Ingest outage affecting data ingestion for some teams on Classic environments, from 2026-04-10 13:41 UTC to 2026-04-10 17:24 UTC (3 h 42 min). Honeycomb applied a manual fix, then confirmed the fix covered all impacted and potentially impacted environments and related ingest errors should no longer occur.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: api.honeycomb.io - US1 Event Ingest

Timeline of updates

  1. identified

    We have confirmed reports of some teams getting issues ingesting their data. We have identified a probable source behind this behavior and are currently trying to correct it.

  2. identified

    We have further narrowed down that only teams on Classic environments are currently seeing issues. We are figuring out mitigation mechanisms at the moment.

  3. identified

    We've identified a manual fix and are deploying that by hand to each affected environment, while also investigating a systemic solution.

  4. monitoring

    We've applied the manual fix to nearly all affected datasets (less than 1% remaining). The systemic solution is still in progress.

  5. monitoring

    The manual fix has been applied to all environments.

  6. monitoring

    We are continuing to work on the systemic solution and monitor. We'll leave the incident open until we're able to confirm the issue won't recur.

  7. resolved

    We can confirm that the fix was applied uniformly to all impacted and potentially impacted environments; no teams should see ingest errors related to this issue anymore.

Rewindex changes for this incident

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Honeycomb profile · Recent Honeycomb incidents · Markdown · JSON