Companies / RevenueCat / incidents
RevenueCat: API partial outage, Scheduled Data Exports and Dashboard is down
Affected components: Dashboard - Paywalls EditorPaywalls EditorScheduled Data ExportsAPI UptimeRevenueCat Companion AppEvent DispatchingCustomer ListsDashboard - Customer ListsCollaboratorsDashboard - CollaboratorsDashboard - LoginLoginDashboard - OverviewOverviewChartsDashboard - Charts
Timeline of updates
- identified
A major outage in AWS is causing our Scheduled Data Exports and the data in our Charts to be delayed. Despite AWS services show some symptoms of recovery, there are big capacity problems and we are trying to bring more capacity as well as disabling non-critical features to handle the increasing load.
- identified
We are adding capacity by transferring resources from non-critical services, and we are managing to keep up with the load.
- identified
We have taken down our Charts temporarily to optimize traffic towards purchase handling, while AWS continues to have issues.
- identified
We are continuing to work on a fix for this issue, optimizing traffic towards purchase handling while AWS continues to have issues.
- identified
We are operating at elevated error rates on our API. We are continuing to add capacity and work to mitigate the issue.
- identified
We are operating at elevated error rates on our API. We are continuing to add capacity and work to mitigate the issue. New purchases are partially down, however, we have enabled RevenueCat Fortress which will allow new purchases to get processed.
- identified
We are operating at elevated error rates on our API. We are continuing to add capacity and work to mitigate the issue.
- identified
We are continuing to work on a fix for this issue.
- identified
Unfortunately our API backup service is also affected by the outage. A large percentage of new purchases being made through the SDK are still getting served with no visible issues due to our offline entitlements support: https://www.revenuecat.com/blog/engineering/introducing-offline-entitlements/ We continue our effort to bring our systems back to normal as soon as possible.
- identified
We are working on implementing a fallback for SDK purchases in another AWS region. We'll provide another update in 30 minutes.
- identified
We are continuing to work on the mitigations. We'll provide another update in 30 minutes.
- identified
We are continuing to work on mitigations. We are slowly ramping up traffic to some AWS servers and monitoring.
- identified
We're continuing to ramp up traffic to AWS servers and monitor performance. RevenueCat's dashboard and API will return errors as we work to mitigate them. Offline entitlements continue to work as before. We'll share another update in 30 minutes.
- identified
We've directed all traffic to AWS and are seeing some improvements. - Purchasing in the SDK is operational - RevenueCat V1 APIs and V2 non-charts APIs are operational. - RevenueCat customer history is operational. - RevenueCat charts, Overview metrics, and SDEs are not operational. We plan on focusing next on replaying purchases that happened during the outage to recover purchase activity. We're continuing to monitor the system and will provide another update in 30 minutes.
- identified
Our API's performance looks good, we are now working on replaying purchases from during the outage. We'll provide another update in 60 minutes.
- identified
We have completed replaying all purchases that happened during the outage. We also attempted to deliver events to integrations corresponding to these purchases. Some integrations might not accept these events depending on their specific policies. If you have questions about your integrations, please reach out to support@revenuecat.com. As of right now, charts are outdated and SDEs are disabled. We are working next on recalculating charts and SDEs. We will provide another update in 60 minutes.
- monitoring
Some charts have been recalculated and SDEs have been resumed. Customer lists are outdated and are being recalculated. We're continuing to recalculate charts. We'll provide another update in 60 minutes.
- monitoring
Most charts have been recalculated and the remaining ones will be recalculated over the course of a few hours. Scheduled data exports are resumed and are being delivered over the course of a few hours. Customer Lists have been recalculated and are fully up to date. We'll continue monitoring and send a final update when we confirm the incident is resolved.
- monitoring
Charts, scheduled data exports, and customer lists have fully recovered. All services are now operational. We will continue to monitor the situation closely.
- resolved
There haven't been any issues since the last update so we are considering this incident resolved. A postmortem will be published soon.
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab). RevenueCat profile · Recent RevenueCat incidents · Markdown · JSON