Svix: Elevated response times in the US region for a small percentage of the requests
Timeline of updates
- resolved
There were elevated response times in the US region for a small percentage of the requests. The issue was due to a memory leak in a new third party library we integrated with, which caused some nodes to operate with constrained resources (heavy CPU/MEM usage). This fixed itself automatically by killing the affected nodes (each one as it exhausted its resources over the span of a few minutes) and starting new ones instead. The team has also deployed a fix to revert the inclusion of the new library pending further investigation on why this was happening, and why it wasn't detected in the staging environment.
Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).