Convex: Small percentage of deployments experiencing outage

Impact: majorStatus: resolvedStarted 2026-04-09 05:42 UTCResolved 2026-04-09 06:24 UTC (43 min)3 updatesSource: stspg.io

Summary

A small percentage of Convex deployments experienced sporadic availability in a major-impact outage from 2026-04-09 05:42 UTC to 2026-04-09 06:24 UTC (42 min). During database primary failover, orchestration failed to provision a replica ready for new writes; Convex implemented a fix and reported resolution.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: BusinessLive Traffic - BusinessDevelopment Services - ProfessionalProfessionalDevelopment Services - Free & StarterFree & StarterBusinessDevelopment Services - BusinessLive Traffic - ProfessionalProfessionalFree & StarterLive Traffic - Free & Starter

Timeline of updates

investigating2026-04-09 05:42 UTC
Seems related to a problematic database failover. Updates soon.
monitoring2026-04-09 05:59 UTC
A fix has been implemented and we are monitoring the results.
resolved2026-04-09 06:24 UTC
This issue is considered resolved. Impact: A small percentage of Convex deployments experienced sporadic availability for approximately 17 minutes, from 22:42 to 22:59 PM PT. Root cause: one of our database clusters experienced a routine primary failover event. During the primary promotion process, the orchestration system failed to provision a replica that was ready to accept new database writes. Out of an abundance of caution, Convex will only use database clusters that support semi-sync replication, where at least two machines persist the writes before considering the write successful. Since the replicas were not ready, this cluster was not usable by Convex. Human operators were paged to intervene and fix the broken automation. We're working with our database partners to better understand what went wrong and what changes will prevent it in the future.

Rewindex changes for this incident

Major incident: Small percentage of deployments experiencing outage 2026-04-09 05:42 UTC

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Convex profile · Recent Convex incidents · Markdown · JSON