# GitLab: Major incident: CI/CD pipeline processing delays

**Summary:** _summary pending_
- Kind: Major incident · Label: degradation (model (Jev))
- Changed: 2026-03-09 15:59 UTC · Detected: 2026-10-18 05:08 UTC
- [Source: https://status.gitlab.com/pages/incident/5b36dc6502d06804c08349f7/69aeee6a90ff0a058537715d](https://status.gitlab.com/pages/incident/5b36dc6502d06804c08349f7/69aeee6a90ff0a058537715d)
- [Company: GitLab](https://rewindex.com/company/gitlab.md)
- Versions: before (none: first archived) → after incident\_version:188/69aeee6a90ff0a058537715d@b902e0fe9c36
- Change 1 of 1 for this incident

Incident page (full timeline): [CI/CD pipeline processing delays](https://rewindex.com/company/gitlab/incidents/69aeee6a90ff0a058537715d.md)
## What changed

**1. Added: Title**

```diff
+ CI/CD pipeline processing delays
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2026-03-09 15:59 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-03-09 18:15 UTC
```

**6. Added: Affected components**

```diff
+ CI/CD
```

**7. Added: Update** (investigating) · 2026-03-09 15:59 UTC

```diff
+ We are seeing delays in CI/CD pipeline processing. The cause for this problem is under investigation. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21484 for more details.
```

**8. Added: Update** (investigating) · 2026-03-09 16:19 UTC

```diff
+ Our team is still investigating this issue. Users might encounter some delays with processing CI/CD pipelines.
```

**9. Added: Update** (investigating) · 2026-03-09 16:33 UTC

```diff
+ Our team is considering increasing resources to address the issue with the Sidekiq backlog impacting CI/CD pipelines. We're working on a deployment fix to address this, though we are seeing some improvement overall with queue durations.
```

**10. Added: Update** (identified) · 2026-03-09 17:07 UTC

```diff
+ Our team has turned off a specific worker to help improve pipeline processing. There has been significant recovery, and our team has identified this as a resource issue. We are monitoring for stability in addition to investigating the root cause and determining a permanent fix.
```

**11. Added: Update** (identified) · 2026-03-09 17:45 UTC

```diff
+ We are continuing to monitor but the main backlog drivers are mitigated. Users should expect normal operations. Our team is currently deploying a configuration update to further address this. We'll provide an update once it is deployed.
```

**12. Added: Update** (resolved) · 2026-03-09 18:15 UTC

```diff
+ The configuration change has been deployed and we have seen stability with Sidekiq and pipeline processing. This incident is now resolved.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-03-09 15:59 UTC | Major incident: CI/CD pipeline processing delays (this change) | degradation |

All GitLab changes: [timeline](https://rewindex.com/company/gitlab/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/ia06e57fcd42485 · Markdown: https://rewindex.com/change/ia06e57fcd42485.md · JSON: https://rewindex.com/api/v1/changes/ia06e57fcd42485 · Generated 2026-10-18T12:19:05.580Z