Changes / Iterable / Issues impacting Email Sending
Iterable: Major incident: Issues impacting Email Sending
Summary
Summary pending. The diff below shows exactly what changed.
What changed
Before: not present
After: Issues impacting Email Sending
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-10-15 16:06 UTC
Before: not present
After: 2024-10-15 21:32 UTC
Before: not present
After: Global Campaign Sends, Global Proof Sends
Before: not present
After: Starting around 7:27 AM PST, Iterable engineers began receiving alerts about elevated error rates on a 3rd-party ESP provider's API endpoints.
Iterable has been working directly with our 3rd party vendor and they have released a fix and as of 8:07 AM PST we are seeing successful email sends.
Customers using this ESP may experience issues impacting email sends, including elevated email campaign send skips during this impact window
Iterable Engineer will continue monitoring the issue. If you have any questions please reach out to support@iterable.com
Before: not present
After: The fix from our third party vendor has been stable and we're seeing email campaigns are sending successfully again. We are currently working to resend the emails that were impacted during the impact window.
Next update will be at 11:00 am pst or sooner.
Before: not present
After: Engineering is identifying alternative methods to identify affected campaigns and will contact customers to confirm whether redelivery is needed. Additionally, 3rd party vendor recovered as of 8:07 am PT and there are no issues with send. Next update at 12 pm PT.
Before: not present
After: We are continuing to monitor for any further issues.
Before: not present
After: Engineering is identifying alternative methods to identify affected campaigns and will contact customers to confirm whether redelivery is needed. Additionally, the 3rd party vendor recovered as of 8:07 am PT, and there are no issues with send. Next update at 12 pm PT.
Before: not present
After: Engineering is continuing to work on identifying alternative methods to determine affected campaigns. We will be reaching out to impacted customers to confirm whether redelivery is necessary. The 3rd party vendor recovered at 8:07 AM PT, and there are no ongoing issues with email sends.
Before: not present
After: As of 4:16 PM PST, all affected messages have been successfully processed, and the incident is now resolved. Our engineering team has confirmed that all campaigns have been reviewed, and there are no outstanding issues.
The 3rd party vendor's recovery at 8:07 AM PT resolved the initial sending issue, and no further delays or interruptions are expected.
We appreciate your patience during this time and apologize for any inconvenience caused. If you have any questions or need further assistance, please reach out to our support team at support@iterable.com
First version of this incident archived by Rewindex. View the live incident.
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Issues impacting Email Sending (this change) | Major incident |