Companies / GitLab / incidents
GitLab: CI/CD pipelines intermittently stuck on running status
Affected components: CI/CD
Timeline of updates
- investigating
Some users are experiencing CI/CD pipelines where jobs and pipelines are stuck in a “running” state, even though earlier-stage jobs have completed successfully. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- investigating
No material change since the last update. The incident remains under active investigation. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- investigating
We’re still actively investigating and are currently collating data from affected pipelines to better understand the patterns and root cause. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- investigating
We are currently reviewing logs from our production infrastructure and specifically the status and behavior of the pipeline workers for potential causes. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- investigating
We’ve identified a potential issue with the job deferral mechanism that may be affecting pipeline transitions between stages. For details, please see https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462.
- identified
We have identified the cause of this incident as the recurrence of an existing defect in the pipeline job deferral mechanism. In this case, it was triggered by a period of temporary high pipeline concurrency. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- monitoring
At this time, we are confident that current pipelines are not being affected by this behavior. We are continuing to monitor new reports closely until the top of the hour to ensure there is no ongoing impact. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
- resolved
As there is no current impact, we are setting this incide to Resolved. We have identified the problematic behavior behind it and are working on a long-term fix. However, we know this defect can reoccur under certain high‑concurrency conditions. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.
Rewindex changes for this incident
- Major incident: CI/CD pipelines intermittently stuck on running status 2026-03-05 21:25 UTC
Impact, status and times are the vendor’s own, as last archived. View on status.gitlab.com (opens in new tab). GitLab profile · Recent GitLab incidents · Markdown · JSON