Companies / GitLab / incidents
GitLab: Job delays on saas-linux-small-amd64 hosted runners
Affected components: CI/CDCI/CD - Hosted runners on Linux
Timeline of updates
- investigating
We are investigating increased queue times for CI/CD jobs using GitLab.com small Linux x86-64 hosted runners. Affected jobs may remain pending longer than usual before starting. We have identified errors and delays when provisioning runner capacity and are working to restore normal job start times. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23176
- identified
We have identified an ongoing Google Cloud Compute Engine issue affecting runner-capacity provisioning in us-east1. Some GitLab.com CI/CD jobs using small Linux x86-64 hosted runners may remain pending longer than usual before starting. Our engineering teams are working with Google Cloud to restore normal job start times. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23176
- monitoring
Job start times for CI/CD jobs on GitLab.com small Linux x86-64 hosted runners have returned to normal, and the backlog of pending jobs has been processed. The delays were caused by errors and capacity constraints from our cloud provider when provisioning runner capacity in one region, between approximately 13:17 and 19:15 UTC. Jobs that were delayed are now starting as usual. We are monitoring to make sure performance remains stable and will post an update when this incident is resolved.
- resolved
Job start times for CI/CD jobs on GitLab.com small Linux x86-64 hosted runners have returned to normal, and the backlog of pending jobs has been processed. The issue with the provider persists, but we are not seeing any impact on our systems. More details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23176
Impact, status and times are the vendor’s own, as last archived. View on status.gitlab.com (opens in new tab). GitLab profile · Recent GitLab incidents · Markdown · JSON