# Stream (GetStream): High error rates and timeouts

- Impact (vendor’s label): minor
- Status: postmortem
- Started: 2020-01-28 16:12 UTC · Resolved: 2020-01-28 16:58 UTC · Duration: 47 min
- Updates: 5 · Affected components: Chat API, us-east
- [Vendor status page: http://stspg.io/pfc05vy1jww7](http://stspg.io/pfc05vy1jww7)
- [Company: Stream (GetStream)](https://rewindex.com/company/getstream.md)

## Timeline of updates

- **monitoring** 2020-01-28 16:12 UTC: A recent released caused load increase on part of the chat infrastructure and caused degraded performance and timeout errors. Remediation is in progress.
- **monitoring** 2020-01-28 16:45 UTC: We are continuing to monitor for any further issues.
- **monitoring** 2020-01-28 16:57 UTC: We are continuing to monitor for any further issues.
- **resolved** 2020-01-28 16:58 UTC: This incident has been resolved.
- **postmortem** 2020-01-28 16:59 UTC: Between 4:05PM and 4:45PM UTC on January 28 2020 we had an API outage caused by performance degradation. The event was triggered by a new release to our Chat API servers; quickly after the new release was live, load on our database infrastructure increased and caused HTTP response times to spike and time-out in some cases. The event was detected by our latency and error monitoring. The team started working on the event by rolling back to the previous version at 4:20PM UTC. Unfortunately the rollback did not resolve the problem entirely. After another rollback attempt we realised there were still pending queries from the previous release running on our PostgreSQL database. We manually terminated all the pending tasks at 4:40PM UTC; after that the error rate dropped to 0% again. The outage affected 5% of HTTP requests at its peak \\(4:20PM to 4:27PM UTC\\).


---
Canonical: https://rewindex.com/company/getstream/incidents/r6dfv4qnvb4l · Markdown: https://rewindex.com/company/getstream/incidents/r6dfv4qnvb4l.md · JSON: https://rewindex.com/api/v1/companies/getstream/incidents/r6dfv4qnvb4l · Generated 2026-10-20T00:05:14.428Z