# Keeper: Major incident: GovCloud Admin Console login with ARAM permissions

**Summary:** Keeper reported a major GovCloud Admin Console login outage affecting admins with Advanced Reporting & Alerts module (ARAM) permissions, from 2023-08-30 14:03 UTC to 2023-08-30 14:50 UTC (46 min). A subnet VPC routing change caused connectivity issues between backend services and a Redshift cluster; login was restored. _(machine-written, sol)_
- Kind: Major incident · Label: outage (model (Jev))
- Changed: 2023-08-30 14:03 UTC · Detected: 2026-10-17 18:22 UTC
- [Source: https://stspg.io/fsxzz8xvt0nt](https://stspg.io/fsxzz8xvt0nt)
- [Company: Keeper](https://rewindex.com/company/keeper.md)
- Versions: before (none: first archived) → after incident\_version:311/pwxhmw90t3y3@1f14729f16c4
- Change 1 of 1 for this incident

## What changed

**1. Added: Title**

```diff
+ GovCloud Admin Console login with ARAM permissions
```

**2. Added: Impact**

```diff
+ major
```

**3. Added: Status**

```diff
+ postmortem
```

**4. Added: Start time**

```diff
+ 2023-08-30 14:03 UTC
```

**5. Added: Resolved time**

```diff
+ 2023-08-30 14:50 UTC
```

**6. Added: Affected components**

```diff
+ Keeper Admin Console (GovCloud), Keeper Infrastructure (GovCloud) - Keeper Admin Console (GovCloud)
```

**7. Added: Update** (identified) · 2023-08-30 14:03 UTC

```diff
+ We have identified an issue affecting users on the Admin Console in the GovCloud region who have administrative rights over Advanced Reporting & Alerts module (ARAM).
```

**8. Added: Update** (identified) · 2023-08-30 14:14 UTC

```diff
+ We are continuing to work on a fix for this issue.
```

**9. Added: Update** (resolved) · 2023-08-30 14:50 UTC

```diff
+ The issue has been resolved. Console login to GovCloud for users with ARAM reporting administrative functions has been restored.
```

**10. Added: Update** (postmortem) · 2023-08-30 15:06 UTC

```diff
+ The GovCloud DevOps team uses automated infrastructure deployment systems via Terraform to manage all environments. A change was made to subnet VPC routing which inadvertently caused connectivity issues between certain backend services and a specific Redshift cluster. This issue affected admins on the Keeper Admin Console who have ARAM permissions applied to their role policies. Delegated admins without reporting permissions were not affected. Vault login and other client-side vault applications were not affected. Recent improvements to the Vault and Admin Console backend APIs masked the issue for the  automated detection systems which would have immediately flagged the error and alerted the DevOps team. We have resolved the infrastructure route and updated the monitoring systems to take this into account and will continue to monitor for any future issues.
```

## History of this incident

| When (UTC) | Change | Label |
| --- | --- | --- |
| 2023-08-30 14:03 UTC | Major incident: GovCloud Admin Console login with ARAM permissions (this change) | outage |

All Keeper changes: [timeline](https://rewindex.com/company/keeper/changes.md)
_Diff excerpts are shown on normalized text: dates, counters and tokens appear as ‹date›, ‹count›, ‹tok›; noise-only edits are hidden. Rewindex shows the changed passages, not a copy of the page._


---
Canonical: https://rewindex.com/change/ic5d42e03bcd6b9 · Markdown: https://rewindex.com/change/ic5d42e03bcd6b9.md · JSON: https://rewindex.com/api/v1/changes/ic5d42e03bcd6b9 · Generated 2026-10-18T02:28:23.954Z