GitHub: Incident with Pull Requests, Actions and Issues
Timeline of updates
- resolved
On March 1, 2024, between 14:17 UTC and 15:54 UTC the service that sends messages from our event stream into our background job processing service was degraded and delayed the transmission of jobs for processing. No data or jobs were lost. From 14:17 to 14:41 UTC, there was a partial degradation, where customers would experience intermittent delays with PRs and Actions. From 14:41 to 15:24 UTC, 36% of PRs users saw stale data, and 100% of in progress Actions workflows did not see updates , even though the workflows were succeeding. At 15:24 UTC, we mitigated the incident by redeploying our service and jobs began to burn down, with full job catchup by 15:54 UTC. This was due to under provisioned memory and lack of memory based back pressure in the service, which overwhelmed consumers and led to OutOfMemory crashes. We have adjusted memory configurations to prevent this problem, and are analyzing and adjusting our alert sensitivity to reduce our time to detection of issues like this one in the future.
From GitHub’s published incident history: for incidents before Rewindex began polling this status page, only the final update text is available.
Archive proof
Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert.
How the archive is checked, and what it cannot prove.
This incident predates live tracking (October 17, 2026). It came from the vendor’s own published history, which is not covered by checkpoints.
Impact, status and times are the vendor’s own, as last archived. View on www.githubstatus.com (opens in new tab).
Cite this
Status incident (vendor impact label: minor) posted by GitHub, started 2024-03-01 14:39 UTC, resolved after 1.6 h, 1 update(s).
- Title
- Incident with Pull Requests, Actions and Issues
- Vendor
- GitHub
- Date
- URL
- https://rewindex.com/company/github/incidents/wcl1sw4mzg60
GitHub profileRecent GitHub incidentsAll outages on March 1, 2024GitHub in March 2024MarkdownJSON