Changes / Brex / Severe latency in authz choking up most services

Brex: Critical incident: Severe latency in authz choking up most services

Summary

Brex reported a critical incident involving severe authz latency affecting most services, starting at 2026-02-24 20:21 UTC and resolved at 2026-02-24 21:16 UTC; the computed duration was 54 min. A fix was deployed at 20:25 UTC, after which errors appeared to recover.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 44%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Severe latency in authz choking up most services
AddedImpact#2
Before: not present
After: critical
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-02-24 20:21 UTC
AddedResolved time#5
Before: not present
After: 2026-02-24 21:16 UTC
AddedUpdateinvestigating#6
Before: not present
After: We are currently investigating this issue.
AddedUpdatemonitoring#7
Before: not present
After: We have identified the issue and a fix has been deployed. Errors seem to be recovering and we are continuing to monitor the situation.
AddedUpdateresolved#8
Before: not present
After: We have resolved the issue.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Critical incident: Severe latency in authz choking up most services (this change)
Brex reported a critical incident involving severe authz latency affecting most services, starting at 2026-02-24 20:21 UTC and resolved at 2026-02-24 21:16 UTC; the computed duration was 54 min. A fix was deployed at 20:25 UTC, after which errors appeared to recover.
Major incident

All Brex changes → · Brex profile · Markdown · JSON