Search

Companies / GitLab / incidents

GitLab: Elevated error rates on GitLab.com

Impact: minorStatus: resolvedStarted Resolved (3.5 h)7 updatesSource: status.gitlab.com (opens in new tab)

Affected components: APIBackground ProcessingCI/CDCI/CD - Hosted runners for GitLab community contributionsCI/CD - Hosted runners on LinuxCI/CD - Hosted runners on macOSCI/CD - Hosted runners on WindowsCI/CD - Self-managed runnersContainer RegistryGitLab PagesGit OperationsPackage RegistrySAML SSO - GitLab SaaSWebsite

Timeline of updates

  1. investigating

    We're investigating elevated error rates on GitLab.com. Some users may experience intermittent failures or degraded performance while we work to identify the cause. We'll share another update as soon as we have more information.

  2. investigating

    We are seeing elevated error rates across GitLab.com and GitLab.com services. We are investigating the issue and will continue to update as we have more information.

  3. investigating

    We're investigating elevated error rates on GitLab.com. Some users may experience intermittent failures or degraded performance while we work to identify the cause. Users may experience delayed pipelines or pipelines not running. We'll share another update as soon as we have more information.

  4. identified

    We have identified the cause of the elevated error rates and are working to correct the issue. We will continue to post updates as we have them.

  5. identified

    We are continuing to investigate the issue. A major Redis Sidekiq cluster outage caused widespread elevated error rates and authentication failures for git, CI runners, and web services. We've performed targeted restarts and forced resets to restore service and are now seeing a drop in error rates. We are monitoring the recovery and health of the cluster. The issue is being tracked in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22103

  6. monitoring

    Error rates have started to lower and return to normal values and we are actively monitoring the issue. The issue is being tracked in gitlab.com/gitlab-com/gl-infra/production/-/work_items/22103

  7. resolved

    This issue has been resolved and the affected Redis sidekiq cluster has recovered. More information can be found in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22103

Impact, status and times are the vendor’s own, as last archived. View on status.gitlab.com (opens in new tab). GitLab profile · Recent GitLab incidents · Markdown · JSON