Changes / Keeper / API errors during maintenance operation

Keeper: Major incident: API errors during maintenance operation

Summary

Keeper reported major API errors during maintenance, attributing unexpected customer impact during an application restart to recent service health-check changes. The incident is marked resolved, but its recorded start (2026-04-01 01:40 UTC) is later than its resolution (2026-03-31 03:00 UTC), so duration is unknown.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 96%)Changed Detected Source: stspg.io

What changed

AddedTitle#1
Before: not present
After: API errors during maintenance operation
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-04-01 01:40 UTC
AddedResolved time#5
Before: not present
After: 2026-03-31 03:00 UTC
AddedUpdateresolved#6
Before: not present
After: On March 30 at 8:00 PM PST, Keeper performed a scheduled minor upgrade to our multi-region AWS RDS clusters as part of routine maintenance. The upgrade itself completed in approximately 3 minutes, followed by a standard application restart process that took about 15 minutes. While this process is typically non-disruptive, recent changes to service health checks caused unexpected customer impact during the restart window. This maintenance was not communicated in advance due to an internal miscommunication and the expectation of no impact. We apologize for the disruption and are taking steps to ensure all future maintenance—regardless of expected impact—is communicated ahead of time, as well as reviewing our health check configurations to prevent similar issues.

First version of this incident archived by Rewindex. View the live incident.

History of this incident

When (UTC)ChangeKind
Major incident: API errors during maintenance operation (this change)
Keeper reported major API errors during maintenance, attributing unexpected customer impact during an application restart to recent service health-check changes. The incident is marked resolved, but its recorded start (2026-04-01 01:40 UTC) is later than its resolution (2026-03-31 03:00 UTC), so duration is unknown.
Major incident

All Keeper changes → · Keeper profile · Markdown · JSON