Changes / RudderStack / Data loss of small percentage of warehouse events on free tier

RudderStack: Major incident: Data loss of small percentage of warehouse events on free tier

Summary

RudderStack reported major-impact loss of a small percentage of free-tier warehouse events from 2025-03-03 09:00 UTC to 2025-03-03 10:00 UTC, lasting 1 h 0 min. The vendor attributed degraded performance, failing health checks and increased service restarts to custom configurations not restored during an internal database migration; it reported implementing a fix and resolved the incident.

Machine-written from the diff below (Sol; type: incident degradation). Check the diff before relying on it.

Major incidentLabel: degradation (model (Jev), 76%)Changed Detected Source: stspg.io (opens in new tab)

What changed

AddedTitle#1
Before: not present
After: Data loss of small percentage of warehouse events on free tier
AddedImpact#2
Before: not present
After: major
AddedStatus#3
Before: not present
After: resolved
AddedStart time#4
Before: not present
After: 2025-03-03 09:00 UTC
AddedResolved time#5
Before: not present
After: 2025-03-03 10:00 UTC
AddedAffected components#6
Before: not present
After: Warehouses
AddedUpdateinvestigating#7
Before: not present
After: We are currently investigating this issue.
AddedUpdateidentified#8
Before: not present
After: The issue has been identified and a fix is being implemented.
AddedUpdateresolved#9
Before: not present
After: This incident has been resolved. During the migration of our internal database, certain custom configurations from the previous version were not restored in the new setup. This led to degraded service performance, causing failing health checks and an increase in service restarts.

First version of this incident archived by Rewindex. View the live incident (opens in new tab).

History of this incident

When (UTC)ChangeKind
Major incident: Data loss of small percentage of warehouse events on free tier (this change)
RudderStack reported major-impact loss of a small percentage of free-tier warehouse events from 2025-03-03 09:00 UTC to 2025-03-03 10:00 UTC, lasting 1 h 0 min. The vendor attributed degraded performance, failing health checks and increased service restarts to custom configurations not restored during an internal database migration; it reported implementing a fix and resolved the incident.
Major incident

All RudderStack changes → · RudderStack profile · Markdown · JSON