# Crusoe: Major incident: Service Degradation in us-east1-a Region due to Power Disruption

**Summary:** _summary pending_
- Kind: Major incident · Label: major (vendor)
- Changed: 2025-08-20 17:09 UTC · Detected: 2026-10-19 17:33 UTC
- [Source: https://stspg.io/gf1tsgr0rtjt](https://stspg.io/gf1tsgr0rtjt)
- [Company: Crusoe](https://rewindex.com/company/crusoe.md)
- Versions: before (none: first archived) → after incident\_version:709/nm5r6txctfby@59746ea4bca5
- Change 1 of 1 for this incident

Incident page (full timeline): [Service Degradation in us-east1-a Region due to Power Disruption](https://rewindex.com/company/crusoe/incidents/nm5r6txctfby.md)
## What changed

**1. Added: Title**

```diff
+ Service Degradation in us-east1-a Region due to Power Disruption
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ resolved
```

**4. Added: Start time**

```diff
+ 2025-08-20 17:09 UTC
```

**5. Added: Resolved time**

```diff
+ 2025-08-21 23:35 UTC
```

**6. Added: Affected components**

```diff
+ GPU Virtual Machines - us-east1, Infiniband Networks, Networking - Infiniband Networks, us-east1
```

**7. Added: Update** (investigating) · 2025-08-20 17:09 UTC

```diff
+ We're investigating a service degradation in our us-east1-a region, triggered by a facility power disruption at our data center. The primary impact is to the Infiniband networking fabric, which may cause intermittent errors or failures for multi-node, distributed workloads. Some customers may also experience individual virtual machines becoming unavailable. Our teams are working to identify all affected resources. Our engineering teams are actively working to stabilize the affected systems and mitigate the risk of further disruption. We are coordinating with our data center provider to support their remediation efforts and restore full service resiliency as quickly as possible. We apologize for any impact this is causing.
```

**8. Added: Update** (investigating) · 2025-08-21 02:32 UTC

```diff
+ We are continuing to investigate and mitigate the service degradation affecting our us-east1-a region, following a facility power disruption at our data center. Our teams remain in close coordination with the data center provider as they work to fully restore services. Recovery of critical systems remains our top priority. We sincerely apologize for the ongoing impact and appreciate your continued patience as we work to resolve the issue.
```

**9. Added: Update** (monitoring) · 2025-08-21 19:59 UTC

```diff
+ We have successfully mitigated the issue affecting the us-east1-a region. The facility power disruption has been addressed, and the impacted Infiniband networking fabric and associated systems have returned to normal operation. All affected services are now stable, and full functionality has been restored. Our teams will continue to monitor the region closely to ensure continued stability. We appreciate your patience during this incident and apologize again for any disruption it may have caused.
```

**10. Added: Update** (resolved) · 2025-08-21 23:35 UTC

```diff
+ This incident has been resolved.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2025-08-20 17:09 UTC | Major incident: Service Degradation in us-east1-a Region due to Power Disruption (this change) | major |

All Crusoe changes: [timeline](https://rewindex.com/company/crusoe/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/iad2f5721669a20 · Markdown: https://rewindex.com/change/iad2f5721669a20.md · JSON: https://rewindex.com/api/v1/changes/iad2f5721669a20 · Generated 2026-10-20T00:48:44.091Z