What vendors changed

CodaPartial outage across API request handling and doc processing

Coda reported a major partial outage degrading doc writes through the API server and doc processing, starting 2025-01-29 04:24 UTC. It lasted 1 h 26 min and was resolved at 2025-01-29 05:50 UTC; no cause or specific fix was stated.

OOrbElevated error rates on subscription related endpoints

UUnbounceApp.unbounce.com Currently Not Available

Unbounce reported a major outage of app.unbounce.com starting 2025-01-28 14:01 UTC, with visitors potentially seeing errors when accessing the app. A fix was implemented, and the app was reported up again at 14:30 UTC; the incident was resolved at 2025-01-28 15:10 UTC, with a recorded duration of 1 h 8 min.

OOmni AnalyticsIntermittent Errors in US Environment

Omni Analytics reported a major incident affecting EastUsa: intermittent US-region network errors caused error:500 "Something went wrong" messages; refreshing could help. It started at 2025-01-27 18:13 UTC and was resolved at 2025-01-28 01:11 UTC, lasting 6 h 57 min. The vendor identified the source but did not specify the cause or fix.

CCensusMany syncs failing for 30 minute period

Census reported a major Sync Engine incident starting at 2025-01-25 02:11 UTC, resolved after 16 min. The vendor said a misconfiguration caused many syncs to fail approximately 2025-01-25 01:08 to 01:38 UTC; it deployed a fix and reported no further failures.

PPipedreamGateway timeouts in HTTP workflow executions

Pipedream reported a major incident causing gateway timeouts in HTTP workflow executions, affecting Backend - Workflows - HTTP and Workflows - HTTP. It started at 2025-01-24 15:30 UTC and was resolved at 2025-01-24 17:37 UTC, lasting 2 h 7 min; no cause or specific fix was stated.

GrammarlyWe're experiencing a service interruption

Grammarly reported a critical service interruption affecting Grammarly Editor, with some users unable to log in or create accounts and slower loading. The incident ran from 2025-01-23 14:41 UTC to 2025-01-23 15:48 UTC, lasting 1 h 7 min. Grammarly implemented a fix and confirmed service was back to normal.

SSambaNovaRegional Login to SambaNova Cloud - Japan recieving errors

WarpRequests to warp server failing

GrammarlyGrammarly is experiencing a service interruption, some Grammarly users may be unable to log in

Grammarly reported a critical service interruption that left some users unable to log in, starting at 2025-01-22 19:35 UTC and resolved at 2025-01-22 21:58 UTC, lasting 2 h 23 min. The vendor identified the root cause without disclosing it and implemented a fix; affected components included Login, Writing Suggestions, Generative AI assistance, and Billing Service.

FFireboltUS-EAST-1 Engine Operations

Firebolt reported a major US-EAST-1 outage from 2025-01-22 19:22 UTC to 2025-01-22 20:58 UTC, lasting 1 h 36 min. Customers could not start engines or use USE ENGINE and could encounter Develop space access and engine/account listing API issues; queries on running engines were expected to work. Firebolt deployed a fix and confirmed full restoration.

SSynthesiaVideo Generation Issues

DropboxDocSend unavailable

DocSend was unavailable in a major-impact incident that started at 2025-01-22 15:45 UTC and was resolved at 2025-01-22 15:52 UTC; the computed duration was 6 min. Customers could have experienced errors or long load times. Dropbox did not state a cause or fix.

PrismaAccelerate issue provisioning new query engines in us-east-1

SubstackIncreased latency and errors

Substack reported a major incident causing increased latency and errors across its websites and applications, including custom domains, publications and writer dashboards. It started at 2025-01-21 23:44 UTC and was resolved at 2025-01-22 00:36 UTC, lasting 52 min; no specific cause or fix was stated.

AsanaPartial Outage

Asana reported a major partial outage affecting API, App, Automations and Background Actions, Mobile, and Webhooks and Event Streams, including US components. It started at 2025-01-17 21:50 UTC and was resolved at 2025-01-17 23:39 UTC (1 h 48 min); a fix was implemented at 01-17 22:05 UTC.

HHexHTML and styled dataframes not rendering

Hex reported a major Main site incident in which HTML and styled dataframes were not rendering. It started at 2025-01-17 15:12 UTC and was resolved at 2025-01-17 16:34 UTC, lasting 1 h 22 min; no cause or specific fix was stated.

QuickBooks OnlineQBO Create invoice forced logged out issue

QuickBooks Online reported a major incident in Canada and the United States that forced some customers out of their accounts when creating transactions or accessing the Sales Tax Center. It started at 2025-01-16 20:52 UTC and was resolved at 2025-01-17 15:08 UTC, lasting 18 h 16 min; an earlier update at 20:59 UTC had already reported service restored.

GrammarlySome Grammarly accounts are temporarily deactivated

Some Grammarly accounts were temporarily deactivated, affecting Login and prompting deactivation emails. Grammarly rated the incident major; it ran from 2025-01-16 01:57 UTC to 2025-01-16 03:54 UTC, lasting 1 h 57 min. All affected accounts were reactivated; Grammarly said it identified the cause but did not disclose it.

Jupyter (Anaconda)anaconda.org elevated error rates

Anaconda.org experienced elevated error rates affecting its front-end web service, channels and repository services, with major impact. The incident ran from 2025-01-15 13:08 UTC to 2025-01-15 14:35 UTC, lasting 1 h 26 min. Anaconda reported implementing a fix at 14:05 UTC and confirmed successful resolution after monitoring.

CodecovUnable to login to Codecov

Codecov reported a major outage preventing login using any service, affecting Codecov Backend and Codecov Frontend; uploads appeared unaffected. It started at 2025-01-15 00:45 UTC and was resolved at 2025-01-15 14:03 UTC, with a reported duration of 13 h 17 min. A fix was implemented by 01-15 13:50 UTC; no cause was stated.

JFrogJFrog services intermittent degradation affecting a subset of customers in AWS US West 2 region

JFrog reported major intermittent service disruptions affecting a subset of customers in AWS US West 2 (Oregon), starting 2025-01-13 18:34 UTC and resolved 2025-01-13 20:15 UTC, lasting 1 h 40 min. Affected components included Artifactory, Distribution, Pipelines, security services, Webhook and Web UI; JFrog reported implementing a fix at 19:52 UTC.

CodaAttachment and image processing delays

Coda reported a major outage preventing virus scanning of attachments and images, delaying their processing and availability to document collaborators; uploads were stored successfully. It started at 2025-01-10 04:31 UTC and was resolved at 2025-01-10 15:25 UTC, lasting 10 h 53 min.

TTeleportNew Tenant Onboarding Failures

MapboxTraffic Maps V1 Degraded Performance

Mapbox reported major degraded performance affecting the mapbox.mapbox-traffic-v1 tileset and Traffic Congestion Visual Layer from 2025-01-09 18:37 UTC to 2025-01-10 02:16 UTC, lasting 7 h 39 min. Mapbox identified the root cause without detailing it, implemented a fix, and confirmed resolution.

HHexProjects not loading

Hex projects were not loading on the Main site in a critical incident starting 2025-01-08 21:00 UTC and resolved at 2025-01-08 22:41 UTC, lasting 1 h 40 min. Hex reported implementing a fix at 01-08 21:08 UTC and monitoring the results.

CCriblCloud portal not accessible

CCensusCensus unavailable. 503 Error page from the Web.

Census reported a major outage affecting the Sync Management UI and Sync Engine, with a 503 error page, from 2025-01-07 20:53 UTC to 2025-01-07 21:28 UTC (35 min). An infrastructure upgrade inadvertently removed services; all services recovered, and queued syncs began starting at 2025-01-07 21:18 UTC.

IIterableIntermittent API errors

Iterable reported a major incident affecting Global API Success, with intermittent partial API outages lasting several minutes. The incident started at 2025-01-07 19:56 UTC and was resolved at 2025-01-07 22:10 UTC, lasting 2 h 13 min. Engineers implemented a fix and reported no further API outages.

Metabaseand MMetabase CloudMetabase Store unavailable

Metabase reported a major outage of the Metabase Store, starting and resolving at 2025-01-06 19:30 UTC (recorded duration: 0 min). Its resolution update said the store was intermittently unavailable for an hour due to a problem during a store update, conflicting with the recorded duration.

CCloseApp UI unavailable

Close’s critical Application UI incident ran from 2025-01-06 17:49 UTC to 2025-01-06 19:47 UTC (1 h 58 min). The postmortem reports Web UI unavailability from 2025-01-06 17:33 UTC to 2025-01-06 17:48 UTC (15 minutes). Close cited a deployment-process change and said the issue was reverted.

PrismaAccelerate partial disruption affecting new query engine starts

IIterableEvent-Triggered Journeys Delays Ingesting New Users

Iterable reported major delays ingesting new users into Event-Triggered Journeys from 2024-12-31 14:29 UTC to 2024-12-31 18:25 UTC, lasting 3 h 55 min. Errors in workflow-entrance-trigger pods caused a backlog; Iterable updated the service and scaled up pods, then confirmed the backlog was drained. Scheduled Journeys and API Triggered Journeys were unaffected.

JFrogJFrog platform services outage affecting a subset of customers in Azure US East region due to infrastructure issues

JFrog reported a critical platform services outage affecting a subset of customers in Azure US East (Virginia), attributed to infrastructure issues. It started at 2024-12-31 02:41 UTC and lasted 5 h 22 min, affecting Artifactory, Distribution, Pipelines, Security services, Webhook and Web UI. A fix was implemented; the incident was resolved at 2024-12-31 08:03 UTC.

RRudderStackHigh rate of 5xx in ingestion of events for starter and growth tier in US region

RudderStack reported a major incident with high rates of 5xx errors during event ingestion for starter and growth tiers in the US region. It started at 2024-12-27 22:55 UTC and was resolved at 2024-12-27 23:15 UTC, lasting 19 min; the vendor confirmed ingestion was back to normal.

SSynthesiaVideo Generation and Editor Issues

ContentstackContentstack - Azure NA - Network Outage

JFrogJFrog platform services outage affecting a subset of AWS regions due to AWS infrastructure issues

JFrog reported a critical platform services outage due to AWS infrastructure issues, affecting Artifactory, Distribution, Pipelines, security services, Webhook and Web UI in a subset of AWS regions. The outage started at 2024-12-22 13:08 UTC and was resolved at 2024-12-22 14:38 UTC.

SSambaNovaLlama 3.1 8B is having intermittent timeouts

JFrogJFrog Xray services outage affecting a subset of customers in Azure EU West

JFrog reported a major Xray services outage affecting a subset of customers in Azure EU West (Netherlands), starting 2024-12-20 03:24 UTC and resolved at 2024-12-20 03:51 UTC after 27 min. A fix was implemented by 2024-12-20 03:38 UTC; no cause was stated.

FFrontAutomated emails from Front are delayed - notifications, invites to your teammates, calendar events, etc.

Front reported a major incident delaying automated emails, including notifications, teammate invites and calendar events, from 2024-12-18 19:33 UTC to 2024-12-18 20:41 UTC (1 h 8 min). Front worked with its provider to remediate the issue and marked it resolved.

New RelicUI Irregularities

New Relic reported a major incident affecting US UI access, with errors or pages not loading; data ingest and alert notifications were unaffected. The incident started at 2024-12-18 09:18 UTC and was resolved at 2024-12-18 09:47 UTC, lasting 29 min. New Relic reported services returned to normal but stated no cause or specific fix.

npmPackage Install failures due to "304 Not Modified" responses in Mumbai, India region

PostmanFailures in Importing Collections and APIs in a workspace

Postman reported a major outage affecting collection and API imports in workspaces, including collection import flows and API Builder, on its browser and desktop platforms, with US components listed. It started at 2024-12-17 10:38 UTC and was resolved at 2024-12-17 12:13 UTC, lasting 1 h 35 min; no cause or fix was stated.

RRudderStackIncorrect MIME Type Headers for JavaScript SDK Plugin Artifacts

RudderStack reported a major incident: incorrect CDN MIME type headers (binary/octet-stream) caused modern browsers to reject JavaScript SDK plugin scripts, preventing plugin loading and losing instrumented events to RudderStack and device mode destinations. The incident started and was marked resolved at 2024-12-17 09:45 UTC, with a recorded duration of 0 min; impact varied across customers.

ngrokDashboard SSO logins are failing

SubstackElevated Error Rates

Substack reported a major outage with elevated error rates on its websites and mobile application, starting 2024-12-13 16:21 UTC and resolved at 2024-12-13 16:50 UTC (computed duration: 28 min). Services were fully available by 16:28 UTC; listed components included Email delivery, Push Notifications, Subscription invoice processing, Custom domains, Substack.com, Substack publications and Writer dashboards.

PyPI (Python Infrastructure)Infrastructure Outage

PulumiDegraded performance in Pulumi Deployments

Pulumi Deployments experienced increased error rates for deployments building docker images that rely on dockerhub, with impact rated major. The incident started at 2024-12-12 22:03 UTC and was resolved at 2024-12-12 23:57 UTC, lasting 1 h 53 min; no cause or fix was stated.