Companies/GitHub/Incidents

GitHub: Incident with Actions

Timeline of updates

1. resolved2024-05-16 05:15 UTC

   On May 16, 2024, between 4:10 UTC and 5:02 UTC customers experienced various delays in background jobs, primarily UI updates for Actions. This issue was due to degradation in our background job service affecting 22.4% of total jobs. Across all affected services, the average job delay was 2m 22s. Actions jobs themselves were unaffected, this issue affected the timeliness of UI updates, with an average delay of 11m 40s and a maximum of 20m 14s. This incident was due to a performance problem on a single processing node, where Actions UI updates were being processed. Additionally, a misconfigured monitor did not alert immediately, resulting in a 25m late detection time and a 37m total increase in time to mitigate. We mitigated the incident by removing the problem node from the cluster and service was restored. No data was lost, and all jobs executed successfully. To reduce our time to detection and mitigation of issues like this one in the future, we have repaired our misconfigured monitor and added additional monitoring to this service.

From GitHub’s published incident history: for incidents before Rewindex began polling this status page, only the final update text is available.

Impact, status and times are the vendor’s own, as last archived. View on www.githubstatus.com (opens in new tab).

GitHub profileRecent GitHub incidentsAll outages on May 16, 2024GitHub in May 2024MarkdownJSON