Companies/RevenueCat/Incidents

RevenueCat: [Scheduled] Database maintenance

Timeline of updates

1. resolved2024-04-10 07:30 UTC

   The maintenance is now finished. Our intention was to perform a minor upgrade to Aurora PostgreSQL 14.10 and upgrade the AWS instance type we use for our main database cluster. The version upgrade went as expected, but we ran into unexpected errors when trying to perform a failover to the new instance type. During the maintenance period there were three periods of elevated error rates: * ~5:45 UTC - there were about 4 minutes of elevated error rates while the upgrade was being performed. This was estimated to be about 30 seconds during our trial run, but ended up being much longer for the real upgrade. * ~6:00 UTC - there were about 2 minutes of elevated error rates from an unrelated database performance issue where where writes are blocked on IO:XactSync. We are working with AWS to find the root cause of this and believe it is unrelated to the maintenance. * ~7:25 - ~7:50 UTC - once the database was stable we ended up in a situation where a pgbouncer cluster for read queries was still pointing to an incorrect instance. This resulted in a more severe outage where about 30% of posted receipts were failing and a 1.5% error rate across all endpoints. All purchases that failed during the outage have been captured and are being replayed.

From RevenueCat’s published incident history: for incidents before Rewindex began polling this status page, only the final update text is available.

Impact, status and times are the vendor’s own, as last archived. View on status.revenuecat.com (opens in new tab).

RevenueCat profileRecent RevenueCat incidentsAll outages on April 10, 2024RevenueCat in April 2024MarkdownJSON