Doppler: API unavailable due to contention over database locking

Impact: majorStatus: resolvedStarted 2021-10-01 19:34 UTCResolved 2021-10-01 09:00 UTC (-634 min)1 updateSource: stspg.io

Summary

Doppler reported a major API outage caused by database lock contention on its Workplace table. Recorded start: 2021-10-01 19:34 UTC; resolution: 2021-10-01 09:00 UTC; computed duration unknown (vendor reported 11 minutes). Service recovered when locks expired; Doppler refactored the affected component to avoid locking.

Machine-written from the archived incident (Sol). Check the timeline before relying on it.

Timeline of updates

resolved2021-10-01 09:00 UTC
Our API experienced an 11 minute outage due to database contention over locking our Workplace table. The issue resolved itself once the locks expired. We've refactored the affected component to achieve the desired result without locking. We'll be exploring a more holistic fix to prevent this issue from occurring in other components.

Rewindex changes for this incident

Major incident: API unavailable due to contention over database locking 2021-10-01 19:34 UTC

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). Doppler profile · Recent Doppler incidents · Markdown · JSON