Changes / Census / UI slowdown across several areas
Census: Major incident: UI slowdown across several areas
Summary
Census reported a major slowdown in its Sync Management UI affecting Model Previews, Metadata Refreshes and Sync Alerts, starting 2024-03-26 21:53 UTC and resolved at 2024-03-26 22:47 UTC (53 min). Census attributed it to a faulty deploy and restored task processing by deploying the previous job processor version; metrics showed no impact on sync performance or scheduling.
Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.
Major incidentLabel: degradation (model (Jev), 100%)Changed Detected Source: stspg.io (opens in new tab)
What changed
Before: not present
After: UI slowdown across several areas
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-03-26 21:53 UTC
Before: not present
After: 2024-03-26 22:47 UTC
Before: not present
After: Census Sync Management UI
Before: not present
After: We have identified slowdowns in the Census UI associated with several areas:
* Model Previews
* Metadata Refreshes
* Sync Alerts
These are related to a slowdown in our job processing engine. At this time, our metrics do not show that sync performance or scheduling is affected but we are investigating these as well.
Before: not present
After: We have identified a faulty deploy and we are rolling it back
Before: not present
After: A previous version of the job processor has been deployed and these tasks are once again being completed. We are continuing to monitor
Before: not present
After: This incident has been resolved.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: UI slowdown across several areas (this change) Census reported a major slowdown in its Sync Management UI affecting Model Previews, Metadata Refreshes and Sync Alerts, starting 2024-03-26 21:53 UTC and resolved at 2024-03-26 22:47 UTC (53 min). Census attributed it to a faulty deploy and restored task processing by deploying the previous job processor version; metrics showed no impact on sync performance or scheduling. | Major incident |