Census change history
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Sync Outage in azure-eastus region Census reported a major Sync Engine outage in azure-eastus starting 2026-10-01 14:20 UTC, with syncs failing amid spotty Azure Redis connectivity. Redis connectivity was restored and syncs were completing successfully by 2026-10-01 15:14 UTC; the incident was resolved at 2026-10-01 17:36 UTC, lasting 3 h 16 min. | Major incident | |
| Major incident: LinkedIn Sync Failures - server responding with status 426 Census reported a major incident affecting some customers: LinkedIn syncs failed with status 426, and new and existing LinkedIn Destination tests could fail. It started at 2026-09-16 11:42 UTC and was resolved at 2026-09-16 15:33 UTC, lasting 3 h 50 min. Impacted syncs should recover on their next scheduled or manual run; destination connections and tests should work again. | Major incident | |
| Critical incident: App failing to load Census reported a critical outage affecting the Census Sync Management UI from 2026-09-09 16:41 UTC to 2026-09-09 17:33 UTC (52 min). A backend change prevented Activation workspaces from loading; rolling it back restored access. Only UI components were impacted; sync runs were unaffected. | Major incident | |
| Major incident: Degraded UI performance Census reported a major Sync Management UI degradation from 2026-09-08 20:04 UTC to 2026-09-08 22:06 UTC, lasting 2 h 1 min. A recent change blocked some UI jobs, delaying notifications and preventing dataset previews, sync configuration fields, and connection tests from loading for some organizations. A fix was implemented, and the incident was resolved. | Major incident | |
| Major incident: Facebook (Meta) Sync Failures - API access disrupted Facebook (Meta) syncs failed with “API access disrupted” in a major incident affecting Census Sync Management UI and Sync Engine from 2026-08-26 13:09 UTC to 2026-08-26 14:11 UTC (1 h 2 min). Census identified an Activations Meta app issue and implemented a fix; all affected syncs will recover on their next triggered or scheduled run. | Major incident | |
| Critical incident: Service incident impacting app and sync execution Census reported a critical outage affecting the Census Sync Management UI and Sync Engine from 2026-07-07 15:45 UTC to 2026-07-07 18:22 UTC, lasting 2 h 36 min. Census linked the issue to a recent database migration; a fix was implemented, and all systems were operational at resolution. | Major incident | |
| Critical incident: Census Web Outage Census reported a critical outage affecting the Census Sync Management UI from 2026-02-26 00:28 UTC to 2026-02-26 00:32 UTC, lasting 4 min. The vendor attributed the issue to a configuration change and resolved it by reverting that change. | Major incident | |
| Critical incident: Census Full Outage A critical Census outage affected Census Sync Management UI and Sync Engine from 2026-01-05 15:17 UTC to 2026-01-05 20:24 UTC (5 h 7 min). Syncs could not run, and website access was disrupted. Census attributed the outage to an indexing change and said rebuilding previous indexes delayed recovery. | Major incident | |
| Major incident: Sync list UI not loading Census reported a major incident affecting the Census Sync Management UI: the sync list failed to load for some customers, while the sync engine was unaffected. It ran from 2025-06-04 20:11 UTC to 2025-06-04 20:54 UTC (43 min). Census identified a faulty deployment, began reverting it, and implemented a fix before resolving the incident. | Major incident | |
| Major incident: Sync might be failing with mismatch errors Census reported a major Sync Engine incident where some syncs failed with mismatch errors, starting 2025-04-04 22:40 UTC and resolved 2025-04-05 16:23 UTC (17 h 42 min). Census attributed failures to a performance issue, said errors could appear after the problem was resolved, and advised customers to rerun affected syncs. | Major incident | |
| Major incident: Some syncs failing with "Missing corresponding dataset file for prefix: record_deletes.0" Census reported a major Sync Engine incident starting 2025-02-05 22:38 UTC: some syncs failed with "Missing corresponding dataset file for prefix: record_deletes.0" after a code change introduced a bug. A revert deployed at 22:56 UTC; the incident was resolved at 2025-02-05 23:30 UTC after over 30 minutes without errors, with a computed duration of 51 min. | Major incident | |
| Critical incident: IP Address Change for Database Requests Census reported a critical Sync Engine outage starting 2025-01-30 02:59 UTC: a networking misconfiguration sent database requests from unadvertised IPs, causing syncs to fail for customers whose databases rejected those IPs. Census rolled back to a previous configuration and resolved the incident at 2025-01-30 03:31 UTC; the reported duration was 31 min. | Major incident | |
| Major incident: Many syncs failing for 30 minute period Census reported a major Sync Engine incident starting at 2025-01-25 02:11 UTC, resolved after 16 min. The vendor said a misconfiguration caused many syncs to fail approximately 2025-01-25 01:08 to 01:38 UTC; it deployed a fix and reported no further failures. | Major incident | |
| Major incident: Census unavailable. 503 Error page from the Web. Census reported a major outage affecting the Sync Management UI and Sync Engine, with a 503 error page, from 2025-01-07 20:53 UTC to 2025-01-07 21:28 UTC (35 min). An infrastructure upgrade inadvertently removed services; all services recovered, and queued syncs began starting at 2025-01-07 21:18 UTC. | Major incident | |
| Major incident: Sync Alerts missing as of last 2 days Census reported a major Sync Engine outage from 2024-09-20 20:18 UTC to 2024-09-20 22:55 UTC (2 h 37 min). Sync Alerts had been deleted from its database, preventing email alerts for qualifying sync runs over the prior two days. Alert data was restored, and Census said alerting should work again going forward. | Major incident | |
| Major incident: Partial Sync Outage Census reported a major Sync Engine outage starting 2024-09-11 19:00 UTC, lasting 5 h 5 min and resolved at 2024-09-12 00:06 UTC. Empty unload batches could fail with file or Parquet conversion errors; after a fix, some syncs encountered function errors, primarily advanced sync engine mirror syncs with no deletes. | Major incident | |
| Critical incident: Sync execution halted Census’s critical Sync Engine outage paused all production syncs starting 2024-08-23 17:41 UTC; many syncs failed after partial restoration. Census ruled out duplicate-record concerns and deployed a fix, restoring expected success rates with backlog delays. The incident was resolved at 2024-08-23 19:15 UTC after 1 h 33 min. | Major incident | |
| Major incident: Linkedin Syncs Failing Census reported a major Sync Engine outage: Linkedin syncs failed because its API version had been deprecated. The incident ran from 2024-08-16 16:57 UTC to 2024-08-16 18:36 UTC, lasting 1 h 39 min. Census fixed the issue by updating the API version and said affected syncs should succeed on their next run. | Major incident | |
| Major incident: Sync runs failing A subset of Census Sync Engine runs failed in a major-impact incident starting 2024-06-22 13:29 UTC and lasting 14 min, resolved at 2024-06-22 13:43 UTC. Census identified a storage issue affecting a core database and applied a fix, after which error rates decreased. | Major incident | |
| Major incident: Brief downtime for unscheduled database maintenance Census reported a major outage affecting the Census Sync Management UI and Sync Engine during unscheduled emergency maintenance of a primary database. Users could see UI and API downtime; failed syncs would be automatically retried. The incident started at 2024-06-03 19:37 UTC and was resolved at 2024-06-03 19:42 UTC, with a reported duration of 4 min. | Major incident | |
| Critical incident: Some users unable to log in to Census UI Some users could not log in to the Census Sync Management UI, seeing "Verify Your Email" and "Something went wrong with your session" errors. Census rated the incident critical; it ran from 2024-04-19 16:23 UTC to 2024-04-19 17:22 UTC (59 min). A fix was implemented at 16:53 UTC, and the incident was resolved. | Major incident | |
| Critical incident: Syncs not running Census reported a critical Sync Engine outage in which syncs were not running, starting at 2024-04-09 23:32 UTC and lasting 3 h 9 min until resolution at 2024-04-10 02:41 UTC. A fix was implemented by 04-09 23:50 UTC, followed by monitoring. | Major incident | |
| Major incident: UI slowdown across several areas Census reported a major slowdown in its Sync Management UI affecting Model Previews, Metadata Refreshes and Sync Alerts, starting 2024-03-26 21:53 UTC and resolved at 2024-03-26 22:47 UTC (53 min). Census attributed it to a faulty deploy and restored task processing by deploying the previous job processor version; metrics showed no impact on sync performance or scheduling. | Major incident |