Companies / Iterable / incidents

Iterable: Customers on c102 are experiencing degraded performance

Impact: minorStatus: resolvedStarted Resolved (1.2 h)4 updatesSource: stspg.io (opens in new tab)

Affected components: Cluster 102 - User DeletionsUser DeletionsCluster 102 - User UpdatesUser UpdatesCluster 102 - Email SendsEmail SendsCluster 102 - Journey ProcessingJourney ProcessingCluster 102 - Push SendsPush SendsCluster 102 - List UpdatesList UpdatesCluster 102 - SMS SendsSMS Sends

Timeline of updates

  1. investigating

    We are currently investigating an issue for one of ES clusters - c102 - The cluster is in a red state, leading to very slow ingestion and widespread API 500 errors. Customers are experiencing failed searches and errors on multiple API endpoints. Ingestion rates are near zero, with lag up to 20 minutes and increasing for internal, realtime, and bulk data pipelines.

  2. investigating

    Engineering is continuing to investigate this issue. At this time there is no improvement yet to any customers on C102. Next Update at 12:15 PDT or sooner.

  3. monitoring

    Engineering has restored C102 back to a "green" state and ingestion and API calls are succeeding again for this cluster. We will continue to monitor until we are fully through the accumulated backlog. Next update at 1 PM PDT or sooner.

  4. resolved

    We have fully caught up with any backlog on C102 and all queries and activity through C102 are functioning normally. If you continue to experience any issues please contact Support.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Iterable profile · Recent Iterable incidents · Markdown · JSON