Changes / Omni Analytics / Model and Query Loading Failures in East USA

Omni Analytics: Critical incident: Model and Query Loading Failures in East USA

Summary

Omni Analytics reported a critical East USA incident: reports and new analyses loaded slowly or failed to load from 2024-10-17 20:46 UTC until 2024-10-17 23:43 UTC, lasting 2 h 57 min. A fix was applied at 10-17 22:19 UTC; the vendor said it would perform a root cause analysis and take steps to prevent recurrence.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 85%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Model and Query Loading Failures in East USA
AddedImpact#2
Before: not present
After: critical
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2024-10-17 20:46 UTC
AddedResolved time#5
Before: not present
After: 2024-10-17 23:43 UTC
AddedAffected components#6
Before: not present
After: EastUsa
AddedUpdateinvestigating#7
Before: not present
After: Reports and new analyses are loading slowly or failing to load in our East USA environment, our team is investigating.
AddedUpdateinvestigating#8
Before: not present
After: We are continuing to investigate this issue.
AddedUpdatemonitoring#9
Before: not present
After: We have applied a fix and we are monitoring the impact.
AddedUpdateresolved#10
Before: not present
After: This incident has been resolved. Our team will be performing a root cause analysis and taking steps to prevent this issue from recurring.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Critical incident: Model and Query Loading Failures in East USA (this change)
Omni Analytics reported a critical East USA incident: reports and new analyses loaded slowly or failed to load from 2024-10-17 20:46 UTC until 2024-10-17 23:43 UTC, lasting 2 h 57 min. A fix was applied at 10-17 22:19 UTC; the vendor said it would perform a root cause analysis and take steps to prevent recurrence.
Major incident

All Omni Analytics changes → · Omni Analytics profile · Markdown · JSON