What vendors changed

OOmni AnalyticsModel and Query Loading Failures in East USA

Omni Analytics reported a critical East USA outage affecting model and query loading: reports and new analyses loaded slowly or failed. The incident ran from 2024-10-18 15:08 UTC to 2024-10-18 20:50 UTC, lasting 5 h 41 min. The issue was isolated and mitigated by 16:02 UTC, and the service subsequently remained stable.

OOmni AnalyticsModel and Query Loading Failures in East USA

Omni Analytics reported a critical East USA incident: reports and new analyses loaded slowly or failed to load from 2024-10-17 20:46 UTC until 2024-10-17 23:43 UTC, lasting 2 h 57 min. A fix was applied at 10-17 22:19 UTC; the vendor said it would perform a root cause analysis and take steps to prevent recurrence.

GustoAll customers are unable to use our chat option for support.

Gusto’s Chat System outage prevented all customers from using support chat, with critical impact. It started at 2024-10-17 14:35 UTC and was resolved at 2024-10-17 16:08 UTC, lasting 1 h 32 min. Gusto implemented a fix and directed customers to request a call through their account’s Support tab during the outage.

SingleStoreCapacity issues in Azure Southcentral

PPipedreamExperiencing significant delay with HTTP event sources

Pipedream reported a major degradation causing significant delays with HTTP event sources, starting 2024-10-16 17:53 UTC and resolved at 2024-10-16 18:43 UTC (49 min). The backend service powering HTTP events encountered an issue during maintenance failover; services returned online and event processing caught up.

ngrokNew endpoints failing to start in all regions

IIterableIssues impacting Email Sending

Iterable reported a major incident affecting Global Campaign Sends and Global Proof Sends, with email send errors and campaign send skips linked to a third-party ESP’s API. It started at 2024-10-15 16:06 UTC and resolved at 2024-10-15 21:32 UTC, lasting 5 h 25 min. The vendor released a fix, and all affected messages were successfully processed.

DDropbox SignWe are currently experiencing performance problems with Forms

Dropbox Sign reported major performance problems affecting form sending and form filling through Dropbox Forms, starting at 2024-10-14 18:26 UTC. The incident was resolved at 2024-10-14 18:38 UTC after 12 min; no cause or specific fix was stated.

IimgixUnable to connect to dashboard

CCourierCourier Inbox FetchMessages Connectivity Issues

AAptibleImpacted platform operation in us-east-2

TypeformTypeform Workspaces - Degraded - Unable to access Workspace "Sorry, there's a problem on our side"

Typeform reported a major incident preventing access to Workspaces, with users seeing “Sorry, there's a problem on our side.” It started at 2024-10-07 14:04 UTC and was resolved at 2024-10-07 15:19 UTC, lasting 1 h 15 min; Typeform implemented a fix and said it would investigate the cause.

TypeformTypeform Builder - Publish Button Unavailable - Publish button does not appear

Typeform reported a major Builder incident in which the Publish button did not appear, potentially preventing users editing forms from publishing. It started at 2024-10-07 13:54 UTC and was resolved at 2024-10-07 14:54 UTC, lasting 1 h 0 min. Typeform implemented a fix; no cause was stated.

TypeformTypeform Results/Summary Report - Unavailable

Typeform reported a major outage affecting Results panel access to Responses and Summary Report, starting 2024-10-07 13:28 UTC and lasting 18 h 33 min. A fix restored response access while response counters remained affected; the incident was resolved at 2024-10-08 08:02 UTC with counters restored.

SSambaNovaBrief Service Outage

SSpeechmaticsportal.speechmatics.com - Partial Outage

SubstackPosts not being delivered via email

Substack reported a major email delivery incident affecting some posts, starting 2024-10-01 14:04 UTC and resolved at 2024-10-01 15:06 UTC, lasting 1 h 1 min. At 14:32 UTC, Substack said delivery delays were resolved and previously scheduled posts were being delivered.

MMailjetLogin issues with Mailjet Application

DagsterDagster Cloud Service

Dagster reported a critical outage affecting its API and Dagster Cloud UI starting 2024-09-30 23:36 UTC, lasting 48 min and resolved at 2024-10-01 00:25 UTC. A storage-layer configuration error prevented service access; a fix was implemented, but some runs may have been delayed or failed.

QuickBooks OnlineQuickBooks Online - Unable to login

QuickBooks Online reported a critical login outage affecting United States components, starting 2024-09-30 22:19 UTC and resolved at 2024-09-30 22:22 UTC after 3 min. The vendor identified the issue and implemented a fix; customers still unable to reach the login page might need to clear browser cache and cookies.

SSvixIncreased latency and failures for non-create-message endpoints (create message not affected)

FFrontIncreased error rate sending and receiving messages on email-based channels

Front reported a major incident affecting Gmail, O365, and SMTP (non-Gmail / O365) message sending and receiving from 2024-09-27 20:25 UTC to 2024-09-28 02:51 UTC (6 h 26 min). A manually deployed fix circumvented a Docker outage; delivery recovered with residual backlog delays. us-west-2 and eu-west-1 recovered before us-west-1.

WasabiSystem Errors in US-WEST-1 Region

Wasabi reported a major incident in US-West-1 (Oregon), degrading performance and affecting customers’ S3 API calls. It started at 2024-09-25 13:51 UTC and was resolved at 2024-09-25 15:22 UTC (duration: 1 h 30 min). The postmortem attributed it to an upgrade misconfiguration that falsely marked a service ready; Operations corrected the issue.

RRudderStackIngestion got affected for customers in US enterprise tier

RudderStack reported a critical ingestion outage affecting US enterprise tier customers, starting at 2024-09-25 09:37 UTC and resolved at 2024-09-25 11:09 UTC, lasting 1 h 31 min. At 10:02 UTC, the vendor said a fix had been implemented, systems had recovered, and customers could expect ingestion back to normal.

OpsgenieReported issues with OEC functionality

Opsgenie reported a major OEC outage affecting Jira ticket creation and EU/US Outgoing Integration Flow, starting 2024-09-24 19:55 UTC and resolved at 2024-09-24 20:54 UTC (59 min). A routing configuration change sent OEC requests to the wrong service; Opsgenie reverted it and verified endpoints were back online.

Jupyter (Anaconda)Authenticated users receiving a 500 error

Anaconda.org reported a critical outage in which authenticated users received 500 errors, making the UI unusable, from 2024-09-24 16:09 UTC to 2024-09-24 16:49 UTC (40 min). The vendor identified the authentication issue’s root cause on the backend and confirmed anaconda.org was functioning correctly again.

KeeperKeeper API website load time and API errors in US region

Keeper reported a critical incident involving slow responses and HTTP 504 errors, listing Admin Console and Web Vault components in EU, GovCloud and US, plus the US website. It lasted from 2024-09-24 13:43 UTC to 2024-09-24 14:43 UTC (1 h 0 min); NGINX network traffic overload caused errors, and a resolution was rolled out.

CCensusSync Alerts missing as of last 2 days

Census reported a major Sync Engine outage from 2024-09-20 20:18 UTC to 2024-09-20 22:55 UTC (2 h 37 min). Sync Alerts had been deleted from its database, preventing email alerts for qualifying sync runs over the prior two days. Alert data was restored, and Census said alerting should work again going forward.

WarpRTC server downtime due to PGBouncer lacking db permissions.

CodecovReports failing

Codecov recorded a major report-processing outage starting and resolving at 2024-09-18 12:52 UTC (computed duration: 0 min). The vendor’s update reports failures from ~8:30 UTC to ~12:00 UTC and says affected reports must be uploaded again.

GustoSome customers are experiencing an error when accessing Gusto.

Some customers encountered errors accessing Gusto, affecting the API and Gusto.com website. Gusto classified the incident as major; it ran from 2024-09-17 01:31 UTC to 2024-09-17 04:50 UTC, lasting 3 h 19 min. A fix was implemented at 2024-09-17 03:05 UTC, followed by monitoring and resolution.

LoopsPrimary database outage

FFrontDelays in sending and receiving Gmail messages [gmail]

Front reported a major incident delaying Gmail message sending and receiving, initially in us-west-2 and later across multiple regions. It started at 2024-09-13 15:38 UTC and resolved at 2024-09-13 19:25 UTC, lasting 3 h 47 min. Front suspected Gmail rate limiting, drained the message backlog, and reported all channels back to normal.

CCloseApplication Loading Issue

Close reported a major incident affecting Application UI: intermittent loading issues began at 2024-09-12 19:29 UTC and were resolved at 2024-09-12 21:01 UTC, lasting 1 h 31 min. Its postmortem states that the app and API were severely degraded by a partial outage of its back end MongoDB database.

QuickBooks OnlineQBO Switch Company Issue

QuickBooks Online reported a major outage: some customers could not use Switch Company because the option was missing under the Gear Icon. It started at 2024-09-12 14:51 UTC and was resolved at 2024-09-12 16:09 UTC, lasting 1 h 18 min; a fix was implemented, and logging out and back in was offered as a workaround.

Opsgenie, Trelloand BitbucketUsers are experiencing reCaptcha errors while signing up

Opsgenie reported a critical incident preventing successful signup due to reCaptcha errors, affecting EU - Signup, Login & Authorization and Signup, Login & Authorization. It started at 2024-09-12 02:08 UTC and was resolved at 2024-09-12 03:28 UTC, lasting 1 h 20 min; Opsgenie said it identified the root cause but provided no details.

CCensusPartial Sync Outage

Census reported a major Sync Engine outage starting 2024-09-11 19:00 UTC, lasting 5 h 5 min and resolved at 2024-09-12 00:06 UTC. Empty unload batches could fail with file or Parquet conversion errors; after a fix, some syncs encountered function errors, primarily advanced sync engine mirror syncs with no deletes.

GustoSome Users are unable to access Gusto

Some users could not access Gusto in a major-impact outage affecting the Gusto.com website, Payroll, Benefits and HR. It started at 2024-09-10 20:57 UTC and lasted 37 min, resolving at 2024-09-10 21:34 UTC; a fix was implemented at 21:05 UTC, though some pages were still loading slowly at 21:17 UTC.

NNordLayerConnectivity issues with VPN servers in New York, USA

NordLayer reported a major incident affecting connectivity to dedicated VPN servers in New York, USA, impacting Dedicated Servers and Private Gateways (Servers with dedicated IPs). It lasted 3 h 21 min, from 2024-09-10 17:44 UTC to 2024-09-10 21:05 UTC; the service provider implemented a fix, and NordLayer reported no further connectivity issues.

FigmaService disruption

Figma reported a major disruption affecting the Real-time collaboration server, with intermittent errors that could cause file access permission errors. It started at 2024-09-10 07:57 UTC and was resolved at 2024-09-10 11:24 UTC, lasting 3 h 26 min. Figma reported implementing a fix at 09-10 10:17 UTC.

PPendoPendo Resource Center Missing

Pendo's Resource Center was missing from its own application, affecting Pendo UI in an incident rated major. It started at 2024-09-10 02:17 UTC and was resolved at 2024-09-10 02:42 UTC, lasting 24 min; no cause or fix was stated.

ContentstackContentstack - GCP NA - Content Management API and Web Application Outage

PusherWe are experiencing a major outage in our US2 cluster

Pusher’s critical US2 outage affected most traffic, including Channels presence channels, REST API, Webhooks and WebSocket client API. It started at 2024-09-08 18:00 UTC and was resolved at 2024-09-08 23:06 UTC, lasting 5 h 5 min. Pusher reported deploying a fix and restoring services; no cause was specified.

BitbucketBitbucket Cloud website performance degradation

Bitbucket Cloud website users experienced slower-than-expected response times in an incident rated critical by the vendor. It started at 2024-09-06 06:49 UTC and was resolved at 2024-09-06 07:05 UTC, lasting 16 min; a fix was implemented, but no cause was stated.

RRudderStackIngestion failure for a subset of webhook source events

RudderStack reported a major ingestion failure affecting a subset of webhook source events in EU and US Data Ingestion. The incident started at 2024-09-05 15:00 UTC and was resolved at 2024-09-05 15:30 UTC, lasting 30 min; RudderStack identified the issue and reported implementing a fix, without specifying the cause.

BitbucketBitbucket Webhooks and Pipelines on push and pull request not being triggered

Bitbucket reported a major outage: Connect Webhooks were not delivered, and Pipelines on push and pull request were not triggered; manual and scheduled pipelines still worked. It started at 2024-09-05 02:19 UTC and was resolved at 2024-09-05 04:23 UTC, lasting 2 h 3 min. A fix was implemented and monitored before resolution.

SSigma ComputingSnowflake tables not loading on AWS US West 2

Sigma Computing reported a major degradation affecting AWS US West 2: long-running Snowflake table queries in Sigma Workbooks could time out. Updates placed onset around 14:00 UTC on September 4, 2024; the incident record started at 14:57 UTC, with a computed duration of 52 min. Temporary mitigation proved effective; resolution was recorded at 15:50 UTC.

Jupyter (Anaconda)Errors for sign-in to anaconda.cloud

Anaconda reported a major incident causing HTTP 503 errors when signing in to anaconda.cloud, affecting AI Platform Application - anaconda.com/app and Anaconda.cloud - Anaconda.cloud Frontend Website. It started at 2024-08-30 14:32 UTC and was resolved at 2024-08-30 15:56 UTC, lasting 1 h 23 min; Anaconda said it identified the root cause and deployed a fix.