# ClickHouse Cloud: Major incident: CDC ClickPipes stuck in maintenance state

**Summary:** CDC ClickPipes became stuck in maintenance during a weekly upgrade, initially in eu-central-1, with services in other regions also affected. ClickHouse Cloud rated the incident major; it ran from 2026-07-22 11:53 UTC to 2026-07-22 14:36 UTC, lasting 2 h 42 min. Services recovered, but the upgrade rollout remained on hold pending confirmation it could avoid the underlying pod networking failure mode. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2026-07-22 11:53 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://status.clickhouse.com/incidents/01KY4TRYA1FYRMZAHNCDPFCTC6](https://status.clickhouse.com/incidents/01KY4TRYA1FYRMZAHNCDPFCTC6)
- [Company: ClickHouse Cloud](https://rewindex.com/company/clickhouse-cloud.md)
- Versions: before (none: first archived) → after incident\_version:137/01KY4TRYA1FYRMZAHNCDPFCTC6@80e0e7ef904f
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ CDC ClickPipes stuck in maintenance state
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2026-07-22 11:53 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-07-22 14:36 UTC
```

**6. Added: Update** (investigating) · 2026-07-22 11:53 UTC

```diff
+ As part of our weekly upgrade process, CDC ClickPipes in eu-central-1 became stuck in maintenance state. The team is working on a mitigation.
```

**7. Added: Update** (identified) · 2026-07-22 12:36 UTC

```diff
+ Services in eu-central-1 have recovered. We're in the process of manually recovering ~20 services in other regions.
```

**8. Added: Update** (identified) · 2026-07-22 13:05 UTC

```diff
+ Most services have fully recovered. Continuing to roll out the fix to remaining 4 services (all in eu-central-1, us-east-2).
```

**9. Added: Update** (monitoring) · 2026-07-22 13:39 UTC

```diff
+ All services have recovered; we will continue monitoring the affected nodes and consolidating our findings. Please reach out to our support team to request access to the RCA, when available.
```

**10. Added: Update** (resolved) · 2026-07-22 14:36 UTC

```diff
+ The CDC ClickPipes upgrade rollout remains on hold while we confirm we can safely proceed without triggering the underlying pod networking failure mode that triggered the incident.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-07-22 11:53 UTC | Major incident: CDC ClickPipes stuck in maintenance state (this change) | outage |

All ClickHouse Cloud changes: [timeline](https://rewindex.com/company/clickhouse-cloud/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/i4376ccc7c44e2c · Markdown: https://rewindex.com/change/i4376ccc7c44e2c.md · JSON: https://rewindex.com/api/v1/changes/i4376ccc7c44e2c · Generated 2026-10-18T01:30:07.323Z