rewindex

Changes1PasswordHigh error rate on 1Password.com

1Password: Incident updated: High error rate on 1Password.com

Incident updateNormal progress (rule, 100%)Changed stspg.io (opens in new tab)
Summary

Summary pending. The diff below shows exactly what changed.

Incident page: the full timeline of updates

What changed

Struck in red is what 1Password removed; blue is what they added. Muted text is unchanged context.

ChangedStatus#1
Before: resolved
After: postmortem
AddedUpdatepostmortem#2
Before: not present
After: Incident Postmortem - High error rates on 1Password.com when syncing data and provisioning users. Date of Incident: 16 October 2026 Time of Incident (UTC): 23:00 - 00:11 (Oct 17) Service(s) Affected: Admin Console, Item Syncing, Provisioning Impact Duration: 1hr10m Summary What happened: On Friday, October 16, 2026, from 4:00 to 5:10 p.m. Pacific (23:00–00:10 UTC), some requests to 1Password web services failed. Earlier that day, we deployed an infrastructure change meant to improve efficiency. The change caused one of our backend systems to hold on to resources longer than expected, and the system eventually ran out of capacity. Impact: About 2% of requests to 1Password web services in the US failed during the 70-minute window. The issue happened during a period of low traffic, which limited the number of people affected. Resolution: We increased the affected system’s capacity and applied configuration changes so resources are recycled more quickly. Impact on Customers Accessing vault contents. Users experienced intermittent failures when their 1Password clients tried to sync vault contents. Provisioning. Certain public endpoints used to provision users from customer identity providers were unavailable, leading to delayed user provisioning. Number of Affected Customers (approximate): 15 Geographic Regions Affected: United States What Happened? Timeline of Events (UTC): 23:00 (Oct 16): Error rates for 1Password web services begin climbing. ~23:20: Automated alerts notify on-call engineers. 23:27: Incident declared; engineers begin investigating. 23:45: Root cause identified; engineers begin preparing a fix. 00:10 (Oct 17): Affected systems recover, errors stop, and normal operations resume. 02:55 (Oct 17): Configuration changes applied to prevent the issue from recurring. 18:51 (Oct 20): Permanent fixes applied. Root Cause Analysis: On October 16, 2026, 1Password web services returned an elevated number of errors because certain data systems ran out of a type of memory used to process read-only requests. When this memory was full, those requests failed. Each open connection to these systems could hold a small piece of this memory in reserve, even while sitting idle. Earlier that day, we rolled out a new tool that keeps connections open after use so they can be reused, which normally improves performance. Its settings allowed too many idle connections to accumulate. Over the afternoon, the number of open connections grew until it exceeded the available memory, at which point requests began to fail. How Was It Resolved? Mitigation Steps: Engineers increased the amount of memory available to the affected systems, providing extra capacity to support the system’s projected traffic. Resolution Steps: After a period of observation, engineers applied further configuration adjustments and lowered the limit on the number of idle connections the connection management tool can keep open. Verification of Resolution: We improved monitoring to detect any recurrence of the issue. The fix remained in place and was observed throughout the weekend of October 17–18. What We Are Doing to Prevent Future Incidents We improved monitoring to monitor the specific type of memory exhausted during the incident. The monitor pages engineers well before 1Password services begin exhibiting errors. We updated our response procedures for handling this type resource exhaustion. We set stricter limits on our backend systems so they no longer hold on to memory longer than necessary. Next Steps and Communication No action is required from our customers at this time. We are committed to providing a reliable and stable service, and we are taking the necessary steps to learn from this event and prevent it from happening again. Thank you for your understanding. Sincerely, The 1Password Team

Compared with the previous archived version of this incident (10 to 30-minute polling). View the live incident (opens in new tab).

Archive proof

Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves these records existed by then and have not been changed since, by anyone, Rewindex included.

How the archive is checked, and what it cannot prove.

History of this incident

Thursday, October 29

2 incident changes

Tuesday, October 20

1 incident change

Monday, October 19

2 incident changes

Friday, October 16

1 incident change

Cite this

Incident updated: High error rate on 1Password.com on 1Password’s status page, changed 2026-10-29 19:53 UTC (change 6 of 6).

Title
Incident updated: High error rate on 1Password.com
Vendor
1Password
Date
URL
https://rewindex.com/change/r9e056bd8819215

All 1Password changes1Password profileMarkdownJSON