Changes / Bitrise / Long waiting times on Mac stacks due to degraded VM startup

Bitrise: Major incident: Long waiting times on Mac stacks due to degraded VM startup

Summary

Bitrise reported major (SEV-1) degradation of Builds (CI), with long Mac stack queue waits and build start delays due to degraded VM startup. It started at 2025-11-10 11:10 UTC and resolved at 2025-11-10 11:14 UTC; the reported duration was 3 min. Engineering released a mitigating fix and monitored systems before resolution.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 100%)Changed Detected Source: stspg.io

What changed

AddedTitle#1
Before: not present
After: Long waiting times on Mac stacks due to degraded VM startup
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2025-11-10 11:10 UTC
AddedResolved time#5
Before: not present
After: 2025-11-10 11:14 UTC
AddedAffected components#6
Before: not present
After: ⚙️ Builds (CI)
AddedUpdatemonitoring#7
Before: not present
After: <b>We mitigated this incident and are currently monitoring our systems to resolve the elevated build queue times on Mac stacks.</b> <b>Services:</b> Build start times and Mac stack queue wait times <b>Scope:</b> Builds on Mac stacks <b>Severity:</b> SEV-1 <b>What we’re doing</b> Our Engineering teams released a fix to mitigate this incident and now actively monitoring our systems. <b>When will the next update be available?</b> You can expect a status update in 30 minutes.
AddedUpdateresolved#8
Before: not present
After: This incident has been resolved.

First version of this incident archived by Rewindex. View the live incident.

History of this incident

When (UTC)ChangeKind
Major incident: Long waiting times on Mac stacks due to degraded VM startup (this change)
Bitrise reported major (SEV-1) degradation of Builds (CI), with long Mac stack queue waits and build start delays due to degraded VM startup. It started at 2025-11-10 11:10 UTC and resolved at 2025-11-10 11:14 UTC; the reported duration was 3 min. Engineering released a mitigating fix and monitored systems before resolution.
Major incident

All Bitrise changes → · Bitrise profile · Markdown · JSON