Pipedream: HTTP Sources Outage

Impact: criticalStatus: resolvedStarted 2023-12-14 03:05 UTCResolved 2023-12-14 03:28 UTC (23 min)7 updatesSource: stspg.io

Summary

Pipedream reported a critical outage involving HTTP source timeouts, with temporary impact to its website, public API and platform; timer-based sources and workflows were later degraded by an event backlog. The incident started at 2023-12-14 03:05 UTC and was resolved at 2023-12-14 03:28 UTC, lasting 22 min; Pipedream reported mitigation but did not specify the root cause.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Affected components: Backend - GitHub SyncGitHub SyncBackend - Event Sources - HTTPEvent Sources - HTTPBackend - Pipedream destinationsPipedream destinationsBackend - Workflows - TimerWorkflows - TimerBackend - Event Sources - TimerEvent Sources - TimerBackend - Workflows - HTTPWorkflows - HTTPPublic APIs - REST APIREST APIBackend - Workflows - EmailWorkflows - EmailFrontend - https://pipedream.comPipedream

Timeline of updates

investigating2023-12-14 02:41 UTC
We are experiencing timeouts from HTTP sources, and we're investigating the causes of them.
investigating2023-12-14 02:51 UTC
We identified a potential root cause for the downtime, and are working towards mitigating its impact. This will temporarily affect our website, public API, and platform, for a potentially short period.
investigating2023-12-14 02:57 UTC
We identified a potential root cause for the downtime, and are working towards mitigating its impact. This will temporarily affect our website, public API, and platform, for a potentially short period.
investigating2023-12-14 03:03 UTC
The issue has been identified and we were able to mitigate its impact to a good extent. We will continue monitoring the issue until its full resolution.
investigating2023-12-14 03:05 UTC
The issue has been identified and we were able to mitigate its impact to a good extent. We will continue monitoring the issue until its full resolution.
monitoring2023-12-14 03:14 UTC
At this moment, timer-based sources and workflows are degraded due to a backlog of events that still need to be processed. We will continue monitoring the situation until it comes back to normal.
resolved2023-12-14 03:28 UTC
This incident has been resolved.

Rewindex changes for this incident

Critical incident: HTTP Sources Outage 2023-12-14 03:05 UTC

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Pipedream profile · Recent Pipedream incidents · Markdown · JSON