Changes / Close / Application impairment
Close: Major incident: Application impairment
Summary
Close reported a major Application UI outage in which parts of the application failed to load. The incident started at 2026-09-29 17:48 UTC and was resolved at 2026-09-29 18:15 UTC, lasting 26 min. Close said the issue was identified and reverted.
Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.
What changed
Before: not present
After: Application impairment
Before: not present
After: major
Before: not present
After: postmortem
Before: not present
After: 2026-09-29 17:48 UTC
Before: not present
After: 2026-09-29 18:15 UTC
Before: not present
After: Application UI
Before: not present
After: We are currently investigating issues with parts of the Close application not loading.
Before: not present
After: The issue was identified and reverted. We continue to monitor the application to ensure everything is back to normal.
Before: not present
After: This incident has been resolved.
Before: not present
After: Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring.
## Impact
Close systems were unavailable for 30 minutes from Sep 29, 17:40 UTC to Sep 29, 18:10 UTC.
## Root Cause and Resolution
During deployment of a new application version, we ran a multi-phased migration that added a permission to user roles and assigned it to existing custom roles. The timing of the phased migration wasn’t fully thought out; many application servers were still running the previous version which didn’t recognize the new permission in the database. As a result, error rates increased. We restored service by removing the permission from existing roles.
To prevent similar incidents, we’re updating our testing and deployment pipeline to identify migrations that require a phased rollout.
## Timeline
* 17:40 UTC - Migration was run and PR was merged.
* 17:41 UTC - Engineering team alerted of API failures and application unresponsiveness
* 17:43 UTC - Engineering team begins incident response, status page updated
* 17:44 UTC - Potential root cause identified \(backfill migration was run before code finished deploying\).
* 17:44 UTC - We thought we could fall forward, but realized that was going to take longer so we then decided to revert the migration. This added to the outage time.
* 18:10 UTC - Engineering team runs a script to drop the new permission from existing custom roles
* 18:10 UTC - Application loading again, error rates dropped. Customer impact ends.
First version of this incident archived by Rewindex. View the live incident.
History of this incident
| When (UTC) | Change | Kind |
|---|---|---|
| Major incident: Application impairment (this change) Close reported a major Application UI outage in which parts of the application failed to load. The incident started at 2026-09-29 17:48 UTC and was resolved at 2026-09-29 18:15 UTC, lasting 26 min. Close said the issue was identified and reverted. | Major incident |