Changes / Pipedream / Timer-based triggers not running

Pipedream: Major incident: Timer-based triggers not running

Summary

Timer-based triggers, including Scheduler and scheduled app triggers, stopped around 21:30 UTC, affecting v1 workflows but not v2. Pipedream classified the outage as major, logged its start at 2023-11-03 21:49 UTC, and resolved it at 2023-11-03 22:00 UTC (computed duration: 10 min) after reverting code that introduced the bug.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 97%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Timer-based triggers not running
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2023-11-03 21:49 UTC
AddedResolved time#5
Before: not present
After: 2023-11-03 22:00 UTC
AddedAffected components#6
Before: not present
After: Backend - Event Sources - Timer, Backend - Workflows - Timer, Event Sources - Timer, Workflows - Timer
AddedUpdateinvestigating#7
Before: not present
After: Timer-based triggers — like the Scheduler trigger, and other app-based triggers that run on a schedule — stopped running around 21:30 UTC. Our team is investigating and will post updates here.
AddedUpdateidentified#8
Before: not present
After: We've reverted the code that introduced the bug. This also only affects v1 workflow triggers, not those using v2 (the latest version of workflows).
AddedUpdateresolved#9
Before: not present
After: This incident has been resolved.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Timer-based triggers not running (this change)
Timer-based triggers, including Scheduler and scheduled app triggers, stopped around 21:30 UTC, affecting v1 workflows but not v2. Pipedream classified the outage as major, logged its start at 2023-11-03 21:49 UTC, and resolved it at 2023-11-03 22:00 UTC (computed duration: 10 min) after reverting code that introduced the bug.
Major incident

All Pipedream changes → · Pipedream profile · Markdown · JSON