Companies / Census / incidents

Census: Census Full Outage

Impact: criticalStatus: resolvedStarted Resolved (5.1 h)7 updatesSource: stspg.io (opens in new tab)
Summary

A critical Census outage affected Census Sync Management UI and Sync Engine from 2026-01-05 15:17 UTC to 2026-01-05 20:24 UTC (5 h 7 min). Syncs could not run, and website access was disrupted. Census attributed the outage to an indexing change and said rebuilding previous indexes delayed recovery.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: Sync EngineCensus Sync Management UI

Timeline of updates

  1. identified

    Census is currently having issues running Syncs and rendering the Website. Engineers have identified the issue and are remediating.

  2. identified

    We have identified and triaged the affected system and systems are coming back up now. Web is operational. We will see Sync delays as we scale back up.

  3. identified

    We are still seeing degraded performance and in some cases failing syncs. We are continuing efforts to mitigate as the system repairs.

  4. identified

    Syncs are currently not operational. Scheduled syncs should still be queuing up but they won't start running at this time. We will provide an update once workers come back up.

  5. identified

    Syncs are now operational. Users may be experiencing issues loading Sync Configuration and Sync Run details on the web.

  6. monitoring

    The web is now operating normally, along with Sync Runs. We are monitoring the system for overall health.

  7. resolved

    This incident has been resolved. Syncs have been running normally as of approximately 1:00pm EST, and the web has been running at normal latencies as of 2:00pm EST. The issue was caused by an indexing change on our side that impacted our ability to run syncs reliably. Recovery was delayed because we needed to rebuild the previous indexes. To avoid any risk of data loss, we chose a safer recovery approach that allowed the system to remain temporarily degraded while repairs were being completed. To restore syncs as quickly as possible, we prioritized sync execution and applied targeted fixes to improve performance. During this time, access to the UI was temporarily down.

Rewindex changes for this incident

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Census profile · Recent Census incidents · Markdown · JSON