Rewindex

CompaniesimgixIncidents

imgix: Quality degradation for a small portion of rendered images in the EU

No impact listedStatus: postmortemStarted Resolved (-5218 min)2 updatesstspg.io (opens in new tab)

Timeline of updates

  1. resolved

    A small percentage of image requests served by a single machine in the EU region experienced quality degradation between December 9th, 11:08 AM UTC - December 10th, 6:38 AM UTC

  2. postmortem

    # Timeframe December 9th, 11:08 AM UTC - December 10th, 6:38 AM UTC # Impact A small percentage of image requests served by a single machine in the EU region experienced quality degradation. # Root Cause A hardware failure on one of the rendering machines resulted in images with artifacts being delivered to users. # Contributing Factors An absence of automated testing for faulty renders across active machines caused assets to be cached undetected. The lapse between when the issue was detected and when the issue began culminated in an unknown number of renders that needed to be purged from the cache. Along with inadequate cache tooling for purging affected images, these factors caused the issue to fully resolve in a longer timeframe than expected. # Corrective Actions To prevent this issue from happening in the future, we will: * Investigate and implement automated image quality testing for active rendering machines * Replace and refresh machines on a more frequent basis to minimize impact of single machine failures * Upgrade cache tooling for targeted issue resolution

Archive proof

Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves this record existed by then and has not been changed since, by anyone, Rewindex included.

How the archive is checked, and what it cannot prove.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

imgix profileRecent imgix incidentsMarkdownJSON