Changes / Omni Analytics / Slowness in Omni detected

Omni Analytics: Major incident: Slowness in Omni detected

Summary

Summary pending. The diff below shows exactly what changed.

Major incidentLabel: degradation (model (Jev), 100%)Changed Detected Source: stspg.io

What changed

AddedTitle#1
Before: not present
After: Slowness in Omni detected
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2025-12-04 09:15 UTC
AddedResolved time#5
Before: not present
After: 2025-12-04 17:12 UTC
AddedAffected components#6
Before: not present
After: Ireland
AddedUpdateinvestigating#7
Before: not present
After: We are aware of an internal error affecting access to Omni and are currently investigating
AddedUpdateinvestigating#8
Before: not present
After: We are continuing to investigate the root cause of the issue
AddedUpdatemonitoring#9
Before: not present
After: A fix has been implemented and we're monitoring the results
AddedUpdatemonitoring#10
Before: not present
After: We are still monitoring the results of the fix. Requests to our internal CPU continue to be high and we are working on bringing it down
AddedUpdatemonitoring#11
Before: not present
After: We have mitigated the issues for now and the app has stabilised with some continued slowness still expected. We are continuing to look into the root cause to identify a permanent fix.
AddedUpdatemonitoring#12
Before: not present
After: We are continuing to monitor for any further issues.
AddedUpdateresolved#13
Before: not present
After: This incident is now resolved. In certain configurations combined with a very large number of model extensions it was possible to issue a query to the internal database that exhausted resources. Our mitigation is stable and a permanent fix is in development.

First version of this incident archived by Rewindex. View the live incident.

History of this incident

When (UTC)ChangeKind
Major incident: Slowness in Omni detected (this change)Major incident

All Omni Analytics changes → · Omni Analytics profile · Markdown · JSON