Changes / Vercel / Increase in Workflow runs stuck as pending

Vercel: Major incident: Increase in Workflow runs stuck as pending

Summary

Vercel reported a major Workflow degradation: some recently deployed projects had runs stuck as pending and queued messages were not delivered. The incident lasted 5 h 58 min, from 2026-06-25 07:45 UTC to 2026-06-25 13:43 UTC. A fix was fully rolled out; the interim workaround was rollback to a working deployment created before Jun 24 21:00 UTC.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 59%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Increase in Workflow runs stuck as pending
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-06-25 07:45 UTC
AddedResolved time#5
Before: not present
After: 2026-06-25 13:43 UTC
AddedAffected components#6
Before: not present
After: Workflow
AddedUpdateinvestigating#7
Before: not present
After: We are currently investigating an issue where some recently deployed Workflow projects are stuck as pending and not delivering queued messages as expected.
AddedUpdateidentified#8
Before: not present
After: The issue has been identified and a fix is being implemented.
AddedUpdateidentified#9
Before: not present
After: The issue has been identified. We are continuing to work on a fix for this issue.
AddedUpdateidentified#10
Before: not present
After: We are still working on a fix for this issue. While we fix the issue, users experiencing pending Workflow runs can trigger an instant rollback to a working deployment created before Jun 24 21:00 UTC to resume operation on new runs.
AddedUpdateidentified#11
Before: not present
After: We are currently rolling a fix for the issue. We are monitoring the results.
AddedUpdatemonitoring#12
Before: not present
After: The fix has been fully rolled out and we are monitoring the results.
AddedUpdateresolved#13
Before: not present
After: This incident has been resolved.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Increase in Workflow runs stuck as pending (this change)
Vercel reported a major Workflow degradation: some recently deployed projects had runs stuck as pending and queued messages were not delivered. The incident lasted 5 h 58 min, from 2026-06-25 07:45 UTC to 2026-06-25 13:43 UTC. A fix was fully rolled out; the interim workaround was rollback to a working deployment created before Jun 24 21:00 UTC.
Major incident

All Vercel changes → · Vercel profile · Markdown · JSON