Companies / Iterable / incidents
Iterable: Ingestion Delay on Clusters 100 and Above
Iterable reported a critical Global API Ingestion delay on Clusters 100 and above from 2025-03-14 16:56 UTC to 2025-03-14 21:13 UTC, lasting 4 h 16 min. User updates, list uploads, user deletion and event processing were delayed; email sends and journey processing could also be affected. Engineering deployed a fix and confirmed recovery; no data was dropped.
Machine-written from the archived incident (Sol). Check the timeline before relying on it.
Affected components: Global API Ingestion
Timeline of updates
- investigating
Our engineering team has identified an issue causing ingestion delay for all Clusters 100 and above. We are deploying the fix right now. Customers on these impacted clusters could be experiencing delays with user updates, list uploads, user deletion, and event processing. Email sends and journey processing may see impact as well. Data is only being delayed. It is not being dropped. The engineering team is working to remediate these delays. Next update at 10:30 AM PDT or sooner.
- investigating
The engineering team is still working on the fix, and customers on Clusters 100 and above will still be experiencing some delays with user updates, list uploads, user deletion, and event processing. Email sends and journey processing may see impact as well. Data is only being delayed. It is not being dropped. The engineering team is working to remediate these delays. Next update at 11:00 AM PDT or sooner.
- identified
Engineering has put in the fix. Some clusters have fully recovered, and we are actively working on the ones that have not. For those customers that are still impacted, you will still be experiencing some delays with user updates, list uploads, user deletion, and event processing. Email sends and journey processing may see impact as well. Data is only being delayed. It is not being dropped. The engineering team is working to remediate these delays. Next update at 12:30 PM PDT or sooner.
- monitoring
Engineering has finished pushing the fix. Customers on Clusters 116, 120, and 132 may still see an ingestion lag greater than 1 hour. All other Clusters have recovered. Data was never dropped. The next update is at 3 PM PDT or sooner.
- resolved
Our engineering team has confirmed at 2:06 PDT that all ingestion should be back to normal and is stable. This incident is now resolved.
Rewindex changes for this incident
- Critical incident: Ingestion Delay on Clusters 100 and Above 2025-03-14 16:56 UTC
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Iterable profile · Recent Iterable incidents · Markdown · JSON