Changes / Grain / Application Outage

Grain: Critical incident: Application Outage

Summary

Grain reported a critical outage starting 2023-05-10 17:12 UTC that lasted 1 h 1 min, with all functionality offline due to database issues, affecting Grain Recorder, Grain Web App and Recording Processing. Services were gradually re-enabled, and full functionality was restored at 2023-05-10 18:13 UTC.

Machine-written from the diff below (Sol; type: incident outage). Check the diff before relying on it.

Major incidentLabel: outage (model (Jev), 100%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Application Outage
AddedImpact#2
Before: not present
After: critical
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2023-05-10 17:12 UTC
AddedResolved time#5
Before: not present
After: 2023-05-10 18:13 UTC
AddedAffected components#6
Before: not present
After: Grain Recorder, Grain Web App, Recording Processing
AddedUpdateinvestigating#7
Before: not present
After: All Grain functionality is currently offline due to database issues. The team is actively working on a resolution.
AddedUpdateidentified#8
Before: not present
After: Grain apps are for the most part back online and functioning, though the team is slow re-enabling services to ensure health recovery.
AddedUpdateresolved#9
Before: not present
After: All services have been restored to full functionality and the team is continuing to monitor the health of the application.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Critical incident: Application Outage (this change)
Grain reported a critical outage starting 2023-05-10 17:12 UTC that lasted 1 h 1 min, with all functionality offline due to database issues, affecting Grain Recorder, Grain Web App and Recording Processing. Services were gradually re-enabled, and full functionality was restored at 2023-05-10 18:13 UTC.
Major incident

All Grain changes → · Grain profile · Markdown · JSON