Search

Companies / Dremio / incidents

Dremio: Query engines not scaling up

Impact: majorStatus: resolvedStarted Resolved (1.3 h)5 updatesSource: stspg.io (opens in new tab)
Summary

Dremio Cloud - US had a major incident: query engines failed to scale up on demand, causing query failures; already-running engines remained unaffected. It started at 2022-05-10 05:32 UTC and was resolved at 2022-05-10 06:49 UTC, lasting 1 h 16 min. Dremio reported a successful mitigation and confirmed engine scaling was working normally.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: Dremio Cloud - US - Site AvailabilitySite Availability

Timeline of updates

  1. investigating

    Query engines are not scaling up on demand leading to query failures. Engines that are already running will continue to server queries without issue. We are currently investigating the engine scaling issue.

  2. investigating

    We are continuing to investigate the issue. One potential mitigation has been applied. We are evaluating the effectiveness of the change.

  3. monitoring

    A fix has been implemented and we are monitoring the results.

  4. monitoring

    A mitigation has been applied and appears successful. Monitoring for rebound.

  5. resolved

    This incident has been resolved. Engine scaling working normally.

Rewindex changes for this incident

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Dremio profile · Recent Dremio incidents · Markdown · JSON