Changes / Omni Analytics / Slow Queries
Omni Analytics: Major incident: Slow Queries
Summary
Summary pending. The diff below shows exactly what changed.
What changed
Before: not present
After: Slow Queries
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-04-08 16:55 UTC
Before: not present
After: 2024-04-08 17:05 UTC
Before: not present
After: EastUsa
Before: not present
After: Queries in the EastUsa region are slow and/or delayed.
Before: not present
After: For approximately 10 minutes starting around 9:55am US/Pacific time, queries in the EastUsa region ran slower than normal. A spike of queries against an information schema where the cache had expired triggered a thundering herd effect. That combined with some missing logic in our auto scaling caused queues to back up. We will be working to address the issue in the auto scaling logic in the next few days and will also work to change our caching logic to avoid any further thundering herd effects.
First version of this incident archived by Rewindex. View the live incident.
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Slow Queries (this change) | Major incident |
All Omni Analytics changes → · Omni Analytics profile · Markdown · JSON