Changes / Census / Many syncs failing for 30 minute period

Census: Major incident: Many syncs failing for 30 minute period

Summary

Census reported a major Sync Engine incident starting at 2025-01-25 02:11 UTC, resolved after 16 min. The vendor said a misconfiguration caused many syncs to fail approximately 2025-01-25 01:08 to 01:38 UTC; it deployed a fix and reported no further failures.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 99%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Many syncs failing for 30 minute period
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2025-01-25 02:11 UTC
AddedResolved time#5
Before: not present
After: 2025-01-25 02:28 UTC
AddedAffected components#6
Before: not present
After: Sync Engine
AddedUpdatemonitoring#7
Before: not present
After: A misconfiguration of our sync engine resulted in many syncs failing from approximately 2025-01-25 01:08 to 01:38 UTC. We have identified the issue and deployed a fix.
AddedUpdateresolved#8
Before: not present
After: We have not seen any further failure and are declaring the incident resolved.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Many syncs failing for 30 minute period (this change)
Census reported a major Sync Engine incident starting at 2025-01-25 02:11 UTC, resolved after 16 min. The vendor said a misconfiguration caused many syncs to fail approximately 2025-01-25 01:08 to 01:38 UTC; it deployed a fix and reported no further failures.
Major incident

All Census changes → · Census profile · Markdown · JSON