Companies / GitLab / incidents

GitLab: Self-managed runners seeing 401 when picking jobs

Affected components: CI/CDCI/CD - Self-managed runners

Timeline of updates

1. investigating2025-09-29 04:00 UTC

   We are investigating reports of self-managed runners receiving 401 errors when trying to pick up jobs. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.

2. identified2025-09-29 04:38 UTC

   We have identified the MR that caused GitLab Runner versions older than 18 to see 401s when picking up jobs from GitLab.com. We are rolling back the MR and will update here when rollback is complete. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.

3. monitoring2025-09-29 05:40 UTC

   The rollback has been completed, and we are observing a reduced rate of 401 errors. Please see https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.

4. resolved2025-09-29 07:17 UTC

   The incident is now resolved, and we are no longer observing 401 errors. Please see https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20621 for more details.

Rewindex changes for this incident

• Major incident: Self-managed runners seeing 401 when picking jobs 2025-09-29 04:00 UTC

Impact, status and times are the vendor’s own, as last archived. View on status.gitlab.com (opens in new tab). GitLab profile · Recent GitLab incidents · Markdown · JSON