# Lever: Critical incident: Lever -hosted job site down

**Summary:** Lever-hosted job sites and the Postings API were down in a critical incident starting 2025-10-03 21:02 UTC and resolved at 2025-10-03 23:01 UTC (1 h 58 min). Lever attributed the outage to an unexpected pattern of incoming requests and reported implementing a fix at 10-03 22:09 UTC. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2025-10-03 21:02 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/hbjgzwvv1lsw](https://stspg.io/hbjgzwvv1lsw)
- [Company: Lever](https://rewindex.com/company/lever.md)
- Versions: before (none: first archived) → after incident\_version:230/2wczkspds18r@51531affc15e
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ Lever -hosted job site down
```

**2. Added: Impact**

```diff
+ critical
```

**3. Added: Status**

```diff
+ postmortem
```

**4. Added: Start time**

```diff
+ 2025-10-03 21:02 UTC
```

**5. Added: Resolved time**

```diff
+ 2025-10-03 23:01 UTC
```

**6. Added: Affected components**

```diff
+ Career Site, Global Data Center - LeverTRM - Career Site, Global Data Center - LeverTRM - Hire, Hire
```

**7. Added: Update** (identified) · 2025-10-03 21:02 UTC

```diff
+ At 2:04pm PDT we identified an issue causing Lever-hosted job sites and Postings API to be down. Users will notice a looping loading wheel and the page may timeout while loading. We are actively investigating the issue to implement a fix.  Additional updates will be provided as soon as a resolution is in place.
```

**8. Added: Update** (identified) · 2025-10-03 21:52 UTC

```diff
+ We are continuing to work on a fix for this issue.
```

**9. Added: Update** (identified) · 2025-10-03 22:09 UTC

```diff
+ We have implemented a fix for the issue. All Lever-hosted job site services should be available. We are continuing to monitor.
```

**10. Added: Update** (monitoring) · 2025-10-03 22:10 UTC

```diff
+ A fix has been implemented and we are monitoring the results.
```

**11. Added: Update** (resolved) · 2025-10-03 23:01 UTC

```diff
+ This incident has been resolved.
```

**12. Added: Update** (postmortem) · 2025-10-16 15:49 UTC

```diff
+ ## **Incident RCA: Job Site Outage** **Incident Date:** October 3, 2025   **Start Time:** 4:51 PM ET   **End Time:** 6:00 PM ET ### **Impact Summary** Between 4:51 PM and 6:00 PM ET on October 3, 2025, users experienced degraded performance and eventual unavailability across multiple job boards. ### **Root Cause** The outage was triggered by an unexpected pattern of incoming requests that, while not unusually high in volume, each initiated resource-intensive queries. This combination of request frequency and prolonged query execution overwhelmed backend systems, leading to degraded performance and service unavailability. The traffic resembled automated crawling behavior, which our systems were not optimized to handle at that level of query complexity and concurrency. ### **Resolution** To reduce server load and restore service availability, the impacted job site was temporarily disabled. This action immediately alleviated pressure on backend systems and allowed all job boards to recover. The site remained offline over the weekend and was re-enabled on October 6, 2025, after Engineering confirmed that the irregular traffic had subsided. ### **Preventive Actions** To prevent similar incidents in the future, the following measures have been implemented: * **Enhanced Caching:** Improvements to our job site caching mechanisms now limit the ability of similar request patterns to generate excessive server queries. * **Updated Crawling Guidelines:** We have revised our web crawling directives to encourage automated crawlers to reduce the frequency of their requests. This change helps maintain system stability by minimizing the impact of high-volume automated traffic from compliant sources.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2025-10-03 21:02 UTC | Critical incident: Lever -hosted job site down (this change) | outage |

All Lever changes: [timeline](https://rewindex.com/company/lever/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/ia3ebcdf8d270ca · Markdown: https://rewindex.com/change/ia3ebcdf8d270ca.md · JSON: https://rewindex.com/api/v1/changes/ia3ebcdf8d270ca · Generated 2026-10-18T02:54:36.649Z