Rewindex

CompaniesOrbIncidents

Orb: Processing delays

MinorStatus: resolvedStarted Resolved (16 h)19 updatesstspg.io (opens in new tab)
Part of an outage event

AWS us-east-1 outage, Oct 20, 2025 · 141 vendors, 181 incidents

Orb’s incident started in the same window without naming AWS (timing only, not evidence of a link). (timing only, posted +27 min vs AWS’s own post)

Swipe sideways to see the whole diagram.

blamed Twilioblamed Webflowblamed Basetensame windowAWS us-east-1posted 07:11 UTCPosted before AWS did5 vendors, 4 blamed AWSSalesforce, Descope, Ashby +2Within an hour18 vendors, 17 blamed AWSAsana, Firebolt, Monday.com +151 to 4 hours later30 vendors, 29 blamed AWSSingleStore, Loops, Buildkite +27Later45 vendors, 44 blamed AWSRudderStack, Airbyte, Unbounce +42Twilioposted +15 minBigCommerce+2 h 46 minWebflowposted +4 h 37 min, did not name AWSRootly+10 h 23 minBasetenposted +4 minZed+26 h 20 minOrb (this incident)posted +27 min
blamed Twilioblamed Webflowblamed Basetensame windowAWS us-east-1posted 07:11 UTCPosted before AWS did5 vendors, 4 blamed AWSSalesforce, Descope, Ashby +2Within an hour18 vendors, 17 blamed AWSAsana, Firebolt, Monday.com+151 to 4 hours later30 vendors, 29 blamed AWSSingleStore, Loops, Buildkite+27Later45 vendors, 44 blamed AWSRudderStack, Airbyte,Unbounce +42Twilioposted +15 minBigCommerce+2 h 46 minWebflowposted +4 h 37 min, did notname AWSRootly+10 h 23 minBasetenposted +4 minZed+26 h 20 minOrb (this incident)posted +27 min
Orb is outlined in red. Groups show up to a few vendors each; the event page lists every incident.
Mermaid source
flowchart LR
  root["AWS us-east-1<br/>posted 07:11 UTC"]:::root
  before["Posted before AWS did<br/>5 vendors, 4 blamed AWS<br/>Salesforce, Descope, Ashby +2"]:::blames
  root --> before
  h1["Within an hour<br/>18 vendors, 17 blamed AWS<br/>Asana, Firebolt, Monday.com +15"]:::blames
  root --> h1
  h4["1 to 4 hours later<br/>30 vendors, 29 blamed AWS<br/>SingleStore, Loops, Buildkite +27"]:::blames
  root --> h4
  later["Later<br/>45 vendors, 44 blamed AWS<br/>RudderStack, Airbyte, Unbounce +42"]:::blames
  root --> later
  p0["Twilio<br/>posted +15 min"]:::blames
  root --> p0
  k0["BigCommerce<br/>+2 h 46 min"]:::hop2
  p0 -->|"blamed Twilio"| k0
  p1["Webflow<br/>posted +4 h 37 min, did not name AWS"]:::timing
  root -.-> p1
  k1["Rootly<br/>+10 h 23 min"]:::hop2
  p1 -->|"blamed Webflow"| k1
  p2["Baseten<br/>posted +4 min"]:::blames
  root --> p2
  k2["Zed<br/>+26 h 20 min"]:::hop2
  p2 -->|"blamed Baseten"| k2
  me["Orb (this incident)<br/>posted +27 min"]:::me
  root -.->|"same window"| me

Full event: timeline, evidence and all vendors →

Detected automatically from incidents that started together and named the same provider. A link means one vendor’s incident named another, not a confirmed cause.

Affected components: AnalyticsAPI - AnalyticsAPI - ReadsReadsAPI - WritesWritesReal-time alerting

Timeline of updates

  1. investigating

    We are currently investigating API degradation. This seems to be related to an upstream cloud provider outage, and we are closely monitoring.

  2. investigating

    We are continuing to see and investigate errors.

  3. investigating

    We are continuing to investigate errors.

  4. investigating

    We are continuing to investigate this issue. We apologize for the disruption to impacted customers.

  5. investigating

    We are continuing to investigate this issue.

  6. investigating

    We are continuing to investigate the issue.

  7. investigating

    API services are broadly recovered; we're continuing to monitor the situation.

  8. investigating

    We are seeing recovery on most API endpoints - some analytics endpoints may still be degraded. We are continuing to work with upstream providers to resolve. Note also that consumption from payment processing may be delayed; we are working through the event backlog here.

  9. investigating

    We are failing over affected services gradually for our analytics APIs, which we expect to lead to errors in order to assist with full recovery.

  10. identified

    We're continuing to investigate, and working through usage calculation delays.

  11. identified

    We are working with our cloud provider to bring on more capacity to help with operational delays.

  12. identified

    We are continuing to investigate this issue. We are working with our cloud provider to bring on more capacity to help with operational delays.

  13. identified

    We are continuing to work with our cloud provider to bring on more capacity to help with operational delays.

  14. identified

    We are continuing to work with our cloud provider to bring on more capacity to help with operational delays. Asynchronous processing may be delayed, including the sending of webhooks.

  15. identified

    We are continuing to work with our cloud provider to bring on more capacity to help with operational delays. API calls may not reflect the latest usage data as we work through this. Asynchronous processing may be delayed, including the sending of webhooks and the receipt of information from your payment gateway connection. We are working diligently on our side to bring these up to date.

  16. identified

    We are continuing to work on a fix for this issue.

  17. monitoring

    We are seeing significant recovery as capacity is coming back online.

  18. monitoring

    Most async processing is operational, including webhooks and receipt of information from your payment gateway connection. We are continuing to monitor remaining recovery of usage calculations and alerting pipelines.

  19. resolved

    All systems are operational.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Orb profileRecent Orb incidentsMarkdownJSON