Companies / GitLab / incidents

GitLab: Errors Opening Repositories

Impact: majorStatus: resolvedStarted Resolved (17 h)15 updatesSource: status.gitlab.com (opens in new tab)

Affected components: Git OperationsWebsite

Timeline of updates

  1. investigating

    We're investigating reports of customers unable to open projects, receiving error 500 instead.

  2. investigating

    No material updates are available at this time. Our team continues to investigate the root cause of the issue.

  3. identified

    We’ve identified an abuse vector as the root cause and begun mitigation efforts. Early signs indicate that the service is recovering.

  4. identified

    We’re continuing mitigation efforts and have observed some recovery. Customers may still encounter HTTP 500 errors in the UI and during Git operations when accessing certain projects.

  5. identified

    We are continuing mitigation and monitoring recovery. Customers may still encounter HTTP 500 errors in the UI and during Git operations for certain projects. We will provide a further update as mitigation progresses.

  6. investigating

    Mitigation has not yet fully restored the GitLab UI and Git operations in certain projects. We are still investigating HTTP 500 errors and will provide a further update as more information becomes available.

  7. investigating

    Investigation continues. We will shortly be restarting the affected Gitaly node after reconfiguration. Further updates will be provided accordingly.

  8. investigating

    The restart of the affected Gitaly node did not improve the issue; HTTP 500 errors continue affecting the GitLab UI and Git operations in certain projects. We are investigating and will provide further updates as the situation changes.

  9. investigating

    Further mitigation is being rolled out while we continue to investigate HTTP 500 errors affecting the GitLab UI and Git operations. We will provide further updates as the mitigation progresses.

  10. investigating

    A further mitigation change has been applied, and we are assessing its impact. HTTP 500 errors continue affecting the GitLab UI and Git operations in certain projects. Further updates will follow.

  11. investigating

    Further mitigation change are currently being applied, and we are continuing to investigate the issue. HTTP 500 errors may continue affecting the GitLab UI and Git operations in certain projects. Further updates will follow.

  12. investigating

    We are currently rolling back the deployment of Gitaly. We're waiting for the deploy to land, and we are continuing the investigation.

  13. monitoring

    A rollback release has been deployed to all Gitaly nodes. They are stable at present. However, we are continuing to monitor the situation.

  14. monitoring

    Gitaly nodes have stabilized following the completed rollback deployment. The incident has been mitigated and we remain focused on continued monitoring.

  15. resolved

    The incident has been resolved following the completion of the rollback deployment across the Gitaly nodes. All nodes have returned to a stable state, repository access has been restored. Incident team is currently investigating the root cause.

Rewindex changes for this incident

Impact, status and times are the vendor’s own, as last archived. View on status.gitlab.com (opens in new tab). GitLab profile · Recent GitLab incidents · Markdown · JSON