# Iterable: Critical incident: Ingestion Delay on Clusters 100 and Above

**Summary:** _summary pending_
- Kind: Major incident · Label: degradation (model (Jev))
- Changed: 2025-03-14 16:56 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/g6nmyb341kvp](https://stspg.io/g6nmyb341kvp)
- [Company: Iterable](https://rewindex.com/company/iterable.md)
- Versions: before (none: first archived) → after incident\_version:247/9n4l8nmwhv08@7fb90b2e3c3c
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Ingestion Delay on Clusters 100 and Above
```

**2. Added: Impact**

```diff
+ critical
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2025-03-14 16:56 UTC
```

**5. Added: Resolved time**

```diff
+ 2025-03-14 21:13 UTC
```

**6. Added: Affected components**

```diff
+ Global API Ingestion
```

**7. Added: Update** (investigating) · 2025-03-14 16:56 UTC

```diff
+ Our engineering team has identified an issue causing ingestion delay for all Clusters 100 and above. We are deploying the fix right now. Customers on these impacted clusters could be experiencing delays with user updates, list uploads, user deletion, and event processing.  Email sends and journey processing may see impact as well. Data is only being delayed.  It is not being dropped.  The engineering team is working to remediate these delays. Next update at 10:30 AM PDT or sooner.
```

**8. Added: Update** (investigating) · 2025-03-14 17:31 UTC

```diff
+ The engineering team is still working on the fix, and customers on Clusters 100 and above will still be experiencing some delays with user updates, list uploads, user deletion, and event processing.  Email sends and journey processing may see impact as well. Data is only being delayed.  It is not being dropped.  The engineering team is working to remediate these delays. Next update at 11:00 AM PDT or sooner.
```

**9. Added: Update** (identified) · 2025-03-14 18:07 UTC

```diff
+ Engineering has put in the fix. Some clusters have fully recovered, and we are actively working on the ones that have not. For those customers that are still impacted, you will still be experiencing some delays with user updates, list uploads, user deletion, and event processing.  Email sends and journey processing may see impact as well. Data is only being delayed.  It is not being dropped.  The engineering team is working to remediate these delays. Next update at 12:30 PM PDT or sooner.
```

**10. Added: Update** (monitoring) · 2025-03-14 19:30 UTC

```diff
+ Engineering has finished pushing the fix. Customers on Clusters 116, 120, and 132 may still see an ingestion lag greater than 1 hour. All other Clusters have recovered. Data was never dropped. The next update is at 3 PM PDT or sooner.
```

**11. Added: Update** (resolved) · 2025-03-14 21:13 UTC

```diff
+ Our engineering team has confirmed at 2:06 PDT that all ingestion should be back to normal and is stable. This incident is now resolved.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2025-03-14 16:56 UTC | Critical incident: Ingestion Delay on Clusters 100 and Above (this change) | degradation |

All Iterable changes: [timeline](https://rewindex.com/company/iterable/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/i3f94f1ad47a030 · Markdown: https://rewindex.com/change/i3f94f1ad47a030.md · JSON: https://rewindex.com/api/v1/changes/i3f94f1ad47a030 · Generated 2026-10-18T00:38:52.115Z