Changes / GitLab / Self-managed runners seeing 401 when picking jobs
GitLab: Major incident: Self-managed runners seeing 401 when picking jobs
Summary
Summary pending. The diff below shows exactly what changed.
Incident page: full timeline of updates →
What changed
Before: not present
After: Self-managed runners seeing 401 when picking jobs
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2025-09-29 04:00 UTC
Before: not present
After: 2025-09-29 07:17 UTC
Before: not present
After: CI/CD, CI/CD - Self-managed runners
Before: not present
After: We are investigating reports of self-managed runners receiving 401 errors when trying to pick up jobs. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.
Before: not present
After: We have identified the MR that caused GitLab Runner versions older than 18 to see 401s when picking up jobs from GitLab.com. We are rolling back the MR and will update here when rollback is complete. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.
Before: not present
After: The rollback has been completed, and we are observing a reduced rate of 401 errors. Please see https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.
Before: not present
After: The incident is now resolved, and we are no longer observing 401 errors. Please see https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Self-managed runners seeing 401 when picking jobs (this change) | Major incident |