Changes / Elastic Cloud / Replication bug causing slow recoveries

Elastic Cloud: Major incident: Replication bug causing slow recoveries

Summary

Elastic Cloud Hosted reported major degradation from 2026-09-01 08:16 UTC to 2026-09-30 20:44 UTC (29 days 12 h). In Elasticsearch 9.5.0, 9.5.1 and 9.5.2, bulk indexing could cause replica divergence, high disk usage and long-running recoveries. Elastic fixed the bug in 9.5.3 and advised affected customers to upgrade and follow workaround steps.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 97%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Replication bug causing slow recoveries
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2026-09-01 08:16 UTC
AddedResolved time#5
Before: not present
After: 2026-09-30 20:44 UTC
AddedAffected components#6
Before: not present
After: Elastic Cloud Hosted
AddedUpdateinvestigating#7
Before: not present
After: We are aware of a bug affecting customers on Elasticsearch versions 9.5.0, 9.5.1 and 9.5.2 where bulk index operations processed on the primary shard can result in divergence on the replicas resulting in high disk usage and long-running recoveries. The team is working on a fix. More information can be found on https://www.elastic.co/docs/release-notes/elasticsearch/known-issues#elasticsearch-9.5.2-known-issues The next update will be in the next 2 hours or sooner if needed.
AddedUpdateidentified#8
Before: not present
After: We have identified the root cause and the team is working on a fix scheduled to be released in Elasticsearch version 9.5.3 What you can do: - If you have not yet upgraded to 9.5.*, you are not affected as upgrades to 9.5 have been disabled until 9.5.3 is released - If you are already running 9.5 and experience the symptoms of this issue, please contact Elastic Support. More information can be found on https://www.elastic.co/docs/release-notes/elasticsearch/known-issues#elasticsearch-9.5.2-known-issues
AddedUpdateidentified#9
Before: not present
After: The fix has been identified and backported to the Elasticsearch 9.5 patch branch. The teams are performing the needed validations before scheduling and rolling out the 9.5.3 release.
AddedUpdatemonitoring#10
Before: not present
After: Elasticsearch 9.5.3 has been released and all customers currently running 9.5.0, 9.5.1 or 9.5.2 are advised to update to this new version as soon as possible.
AddedUpdatemonitoring#11
Before: not present
After: Elasticsearch 9.5.3 has been released and all customers currently running 9.5.0, 9.5.1 or 9.5.2 are advised to update to this new version as soon as possible. Impacted customers are advised to follow the workaround steps from https://github.com/elastic/elasticsearch/issues/158212 after the upgrade.
AddedUpdateresolved#12
Before: not present
After: This issue was resolved in the 9.5.3 release of Elasticsearch.  Customers running versions 9.5.0, 9.5.1, or 9.5.2 of Elasticsearch are advised to update to a newer version as soon as possible.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Replication bug causing slow recoveries (this change)
Elastic Cloud Hosted reported major degradation from 2026-09-01 08:16 UTC to 2026-09-30 20:44 UTC (29 days 12 h). In Elasticsearch 9.5.0, 9.5.1 and 9.5.2, bulk indexing could cause replica divergence, high disk usage and long-running recoveries. Elastic fixed the bug in 9.5.3 and advised affected customers to upgrade and follow workaround steps.
Major incident

All Elastic Cloud changes → · Elastic Cloud profile · Markdown · JSON