Changes / CockroachDB / Dedicated Cluster Operations Errors

CockroachDB: Major incident: Dedicated Cluster Operations Errors

Summary

CockroachDB reported a major outage preventing Dedicated cluster creation and edits and potentially affecting Cloud Console pages connecting to Serverless and Dedicated clusters; SQL connections worked normally. Starting 2024-07-18 01:56 UTC, it lasted 12 h 49 min and was resolved at 2024-07-18 14:45 UTC after the vendor addressed an unspecified underlying cause.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 100%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Dedicated Cluster Operations Errors
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2024-07-18 01:56 UTC
AddedResolved time#5
Before: not present
After: 2024-07-18 14:45 UTC
AddedAffected components#6
Before: not present
After: Cloud Console, CockroachDB Advanced, CockroachDB Basic/Standard, CockroachDB Cloud Console, CockroachDB Dedicated, CockroachDB Serverless
AddedUpdateidentified#7
Before: not present
After: We are investigating an issue that is preventing CockroachDB Cloud Dedicated cluster creation and edit operations from running successfully.
AddedUpdateidentified#8
Before: not present
After: We are continuing to work on a fix for this issue.
AddedUpdateidentified#9
Before: not present
After: We are continuing to work on a fix for this issue.
AddedUpdateidentified#10
Before: not present
After: We have further identified that this issue may impact pages within the Cloud Console that connect to Serverless and Dedicated clusters. SQL connections to both Serverless and Dedicated clusters are working normally. We are continuing to investigate the cause of this outage and working towards a solution.
AddedUpdateidentified#11
Before: not present
After: We are continuing to work on a fix for this issue.
AddedUpdatemonitoring#12
Before: not present
After: We have resolved the underlying cause of the outages, and are monitoring to ensure the system remains operational.
AddedUpdateresolved#13
Before: not present
After: Cluster operations and Cloud Console pages are restored to normal working order.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Dedicated Cluster Operations Errors (this change)
CockroachDB reported a major outage preventing Dedicated cluster creation and edits and potentially affecting Cloud Console pages connecting to Serverless and Dedicated clusters; SQL connections worked normally. Starting 2024-07-18 01:56 UTC, it lasted 12 h 49 min and was resolved at 2024-07-18 14:45 UTC after the vendor addressed an unspecified underlying cause.
Major incident

All CockroachDB changes → · CockroachDB profile · Markdown · JSON