ChangesCourierDelay in Message Processing
Courier: Major incident: Delay in Message Processing
Summary
Summary pending. The diff below shows exactly what changed.
Incident page: the full timeline of updates
What changed
Struck in red is what Courier removed; blue is what they added. Muted text is unchanged context.
Before: not present
After: Delay in Message Processing
Before: not present
After: major
Before: not present
After: resolved
Before: not present
After: 2024-08-19 18:32 UTC
Before: not present
After: 2024-08-19 21:35 UTC
Before: not present
After: API, Courier - API, Courier - Observability, Courier - Web Application, Observability, Web Application
Before: not present
After: The Courier team identified an issue in our health monitoring involving our message event processing. The issue has been identified and a revert is in place.
Before: not present
After: Our team has released a revert to address the regression and it's in the process of merging.
Before: not present
After: The release is live, and the team is monitoring it.
Before: not present
After: Release is published and building to production. ETA ~45 minutes.
Before: not present
After: Fix has been deployed, and enqueued messages have started to go through slowly. Once the bottleneck clears, messages should start to flow normally.
Before: not present
After: The general pipeline has recovered.
First version of this incident archived by Rewindex. View the live incident (opens in new tab).
History of this incident
Monday, August 19, 2024
1 incident change
- MajorMajor incident
Delay in Message Processingthis change