Rewindex

CompaniesRed Hat (incl. Quay.io)Incidents

Red Hat (incl. Quay.io): Quay.io Push/Pull Degraded

MajorStatus: postmortemStarted Resolved (1.3 h)4 updatesstspg.io (opens in new tab)

Affected components: APIQuay.io - API

Timeline of updates

  1. investigating

    We are experiencing degraded performance on quay.io push/pull API. We are actively investigating.

  2. identified

    Pull API is working again. We are actively working on bringing push API back online.

  3. resolved

    Pull and Push APIs are fully up again. Service is working as expected.

  4. postmortem

    On April 21, a routine production deployment included a dependency upgrade that exposed a latent incompatibility, causing Quay.io application pods to crash on startup. Image pulls and pushes were fully unavailable for approximately 12 minutes before the team failed over to a secondary region, restoring pulls. Pushes remained unavailable for approximately 70 additional minutes while the deployment was reverted and traffic was incrementally shifted back to the primary region. We are enabling staging parity for all runtime features, improving automated rollback mechanisms, and isolating observability subsystems from the critical request path to prevent recurrence.

Changes Rewindex caught on this incident

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Red Hat (incl. Quay.io) profileRecent Red Hat (incl. Quay.io) incidentsMarkdownJSON