# Census: Critical incident: Sync execution halted

**Summary:** Census’s critical Sync Engine outage paused all production syncs starting 2024-08-23 17:41 UTC; many syncs failed after partial restoration. Census ruled out duplicate-record concerns and deployed a fix, restoring expected success rates with backlog delays. The incident was resolved at 2024-08-23 19:15 UTC after 1 h 33 min. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2024-08-23 17:41 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/g9s40c48nzt2](https://stspg.io/g9s40c48nzt2)
- [Company: Census](https://rewindex.com/company/census.md)
- Versions: before (none: first archived) → after incident\_version:136/hz42nx4lgkv9@e9d60251a401
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Sync execution halted
```

**2. Added: Impact**

```diff
+ critical
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2024-08-23 17:41 UTC
```

**5. Added: Resolved time**

```diff
+ 2024-08-23 19:15 UTC
```

**6. Added: Affected components**

```diff
+ Sync Engine
```

**7. Added: Update** (identified) · 2024-08-23 17:41 UTC

```diff
+ Production sync execution has been paused while we investigate an issue
```

**8. Added: Update** (identified) · 2024-08-23 17:58 UTC

```diff
+ We have identified an issue which could cause records to be synced multiple times to destinations. Since this can affect the correctness of syncs for certain destinations, we have paused all syncs proactively until the issue can be diagnosed and resolved. We are continuing to investigate and are in the act of rolling back to a previous version of the sync engine.
```

**9. Added: Update** (identified) · 2024-08-23 18:42 UTC

```diff
+ We have determined that the bug does not affect all syncs, nor does it cause records to be sent multiple times - this was a red herring. A high percentage of syncs are still failing, but there is not a correctness issue, so we are reenabling the sync engine globally so that syncs that are not affected by the issue can return to operation.
```

**10. Added: Update** (identified) · 2024-08-23 18:48 UTC

```diff
+ We have identified the likely root cause and are deploying a fix at this time
```

**11. Added: Update** (monitoring) · 2024-08-23 19:04 UTC

```diff
+ Sync success rates have returned to expected levels. There is some delay in sync execution as we work through the backlog during the outage period. We are continuing to monitor
```

**12. Added: Update** (resolved) · 2024-08-23 19:15 UTC

```diff
+ This incident has been resolved.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2024-08-23 17:41 UTC | Critical incident: Sync execution halted (this change) | outage |

All Census changes: [timeline](https://rewindex.com/company/census/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/id33e013b4699ea · Markdown: https://rewindex.com/change/id33e013b4699ea.md · JSON: https://rewindex.com/api/v1/changes/id33e013b4699ea · Generated 2026-10-18T02:54:12.154Z