# Ashby: Major incident: Ashby Unavailable

**Summary:** Ashby reported a major outage starting 2026-06-01 06:33 UTC, resolved at 2026-06-01 09:04 UTC; the recorded duration was 2 h 30 min. Its postmortem says the main application was slow and approximately 50% of user actions displayed errors during the affected period. AI features were re-enabled and accumulated work was being processed. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2026-06-01 06:33 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/dxm8r9rr5cvq](https://stspg.io/dxm8r9rr5cvq)
- [Company: Ashby](https://rewindex.com/company/ashby.md)
- Versions: before (none: first archived) → after incident\_version:224/cmqhn7vxbn8d@a5a804b6ee5e
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Ashby Unavailable
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ postmortem
```

**4. Added: Start time**

```diff
+ 2026-06-01 06:33 UTC
```

**5. Added: Resolved time**

```diff
+ 2026-06-01 09:04 UTC
```

**6. Added: Affected components**

```diff
+ AI Notetaker, Analytics, Ashby API, Ashby APIs - Ashby API, Ashby APIs - Documentation, Ashby APIs - Job Post API, Ashby APIs - Reports API, Ashby Products - AI Notetaker, Ashby Products - Analytics, Ashby Products - Chrome Extension, Ashby Products - Hosted Job Boards, Ashby Products - Mobile, Ashby Products - Recruiting, Ashby Products - Scheduling, Assessments, ATS Sync, Authentication Services - Google, Authentication Services - Magic Link, Authentication Services - Office 365, Authentication Services - Single Sign On, AWS, Chrome Extension, Documentation, Dropbox Sign, Google, Google Calendar, Google Gmail, Google Meet, Help Center AI Chat, Hosted Job Boards, HRIS Integration, Infrastructure Providers - AWS, Infrastructure Providers - SendGrid API v3, Infrastructure Providers - SendGrid Parse API, Job Feed, Job Post API, Magic Link, Microsoft 365, Mobile, Office 365, Recruiting, Reports API, Scheduling, SendGrid API v3, SendGrid Parse API, Single Sign On, Third-Party Integrations - Assessments, Third-Party Integrations - ATS Sync, Third-Party Integrations - Dropbox Sign, Third-Party Integrations - Google Calendar, Third-Party Integrations - Google Gmail, Third-Party Integrations - Google Meet, Third-Party Integrations - Help Center AI Chat, Third-Party Integrations - HRIS Integration, Third-Party Integrations - Job Feed, Third-Party Integrations - Microsoft 365, Third-Party Integrations - Zoom, Zoom
```

**7. Added: Update** (investigating) · 2026-06-01 06:33 UTC

```diff
+ We are currently investigating this issue.
```

**8. Added: Update** (investigating) · 2026-06-01 06:33 UTC

```diff
+ We are continuing to investigate this issue.
```

**9. Added: Update** (investigating) · 2026-06-01 06:34 UTC

```diff
+ We are currently investigating the issue
```

**10. Added: Update** (investigating) · 2026-06-01 06:49 UTC

```diff
+ We are continuing to investigate this issue.
```

**11. Added: Update** (identified) · 2026-06-01 07:15 UTC

```diff
+ We are seeing signs of recovery, though there may be residual instability. AI features will be unavailable as we mitigate the remaining issues.
```

**12. Added: Update** (monitoring) · 2026-06-01 07:31 UTC

```diff
+ The application has recovered. AI features are still affected, and we are actively working to mitigate the issues.
```

**13. Added: Update** (monitoring) · 2026-06-01 08:07 UTC

```diff
+ AI features are enabled again, and we're processing the backlog of accumulated work.
```

**14. Added: Update** (monitoring) · 2026-06-01 08:29 UTC

```diff
+ All systems are now operational, we are continuing to monitor
```

**15. Added: Update** (resolved) · 2026-06-01 09:04 UTC

```diff
+ This incident has been resolved.
```

**16. Added: Update** (postmortem) · 2026-06-19 18:19 UTC

```diff
+ ## **Summary** On Sunday, May 31st, at 10pm PST, Ashby was slow and had a high error rate for approximately 1 hour, from 10pm to 11:10pm \(PST\). During this time, the Ashby main application would have been slow, and approximately 50% of user actions would have resulted in Ashby displaying an error. Some candidates may have noticed a slower response to their job applications, but all applications were successful and none were lost. ## **Why did this happen?** On Sunday, May 31st, at 4pm PST, an account with one of our AI providers became unavailable. As a result, calls to that provider started to error. These calls are configured to automatically retry periodically until they succeed, but the retries caused a steady increase in the number of such calls per minute. In turn, this drained the capacity of a shared subsystem that manages many different processes at Ashby. In particular, that shared subsystem is used for storing user session data. As the shared subsystem became overwhelmed, user sessions became slower to respond. This caused requests to our web servers to take significantly longer to process. Which then caused our web servers to begin running out of capacity. At 6am, capacity reached a critical low. At this point, our system began rejecting user actions. This is done to prevent complete system failure. ## **How did we resolve the situation?** At 10:04pm PST, our on-call engineers were notified. By 10:16pm PST, our on-call engineers were beginning an investigation. At 11:03pm PST, we identified the subsystem that had reached capacity. At 11:07pm PST, we stopped processing certain types of automated actions. At 11:09pm PST, we identified the issue with the external provider and rectified it immediately. At 11:10pm PST, error rates were down to 10%. At 11:20pm PST, error rates were down to normal levels \(~0%\). At 11:54pm PST, normal service was resumed, and no data was lost. ## **What have we put in place to prevent it from happening in the future?** We’ve done an internal postmortem on this incident and have implemented or plan to implement a variety of changes that achieve three things: 1. Reduced the likelihood of future failure 2. Faster incident response times 3. Lower impact on customers in the event of failure Specifically, we have done the following: * We were already in the process of replacing the subsystem at the center of the outage. We have accelerated that work. The replacement is not vulnerable to this particular kind of failure. * We’ve added additional monitoring around our AI providers to alert us to failures like this sooner. * We are prioritizing a fix for the retry mechanism that led to this incident. * We are migrating our user session storage to a new subsystem that is not vulnerable to this particular kind of failure.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2026-06-01 06:33 UTC | Major incident: Ashby Unavailable (this change) | outage |

All Ashby changes: [timeline](https://rewindex.com/company/ashby/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/i1d45beed2fcc6e · Markdown: https://rewindex.com/change/i1d45beed2fcc6e.md · JSON: https://rewindex.com/api/v1/changes/i1d45beed2fcc6e · Generated 2026-10-18T02:28:28.398Z