Companies/Google Workspace/Incidents

Google Workspace: **Summary:** Google Chat users are experiencing an issue where they receive notifications for new messages, but the actual messages fail to appear in their chat application unless the user refreshes their view. **Description:** We are experiencing an intermittent issue with Google Chat Friday, 2026-08-28 10:25 PDT. Our engineering team has investigated the issue and are currently working on mitigating the issue. We will provide more information by Friday, 2026-08-28 14:15 PDT with current details. We apologize to all who are affected by the disruption. **Customer Symptoms:** Users are experiencing an issue where they receive notifications for new messages, but the actual messages fail to appear in their chat application unless the user refreshes their view **Workaround:** Users can refresh to see new messages

Affected components: Google Chat

Timeline of updates

1. service information2026-08-28 19:25 UTC

   Summary:
   Google Chat users are experiencing an issue where they receive notifications for new messages, but the actual messages fail to appear in their chat application unless the user refreshes their view.
   Description:
   We are experiencing an intermittent issue with Google Chat Friday, 2026-08-28 10:25 PDT.
   Our engineering team has investigated the issue and are currently working on mitigating the issue.
   We will provide more information by Friday, 2026-08-28 14:15 PDT with current details.
   We apologize to all who are affected by the disruption.
   Customer Symptoms:
   Users are experiencing an issue where they receive notifications for new messages, but the actual messages fail to appear in their chat application unless the user refreshes their view
   Workaround:
   Users can refresh to see new messages

2. resolved2026-08-28 20:55 UTC

   Summary
   Google Chat users experienced an issue where they received notifications for new messages, but the actual messages failed to appear in their chat application unless the user refreshed their view.
   Description
   The issue with Google Chat has been resolved for all affected users as of Friday, 2026-08-28 13:45 PDT.
   From preliminary analysis the issue was triggered due to a recent change which was rolled back to mitigate the impact.
   We thank you for your patience while we worked on resolving the issue.
   Customer Symptoms
   Users were experiencing an issue where they received notifications for new messages, but the actual messages failed to appear in their chat application unless the user refreshed their view.
   Workaround
   None as the issue is resolved.

3. resolved2026-09-04 03:43 UTC

   Incident Report
   Summary
   On Friday, August 28, 2026, between 10:50 US/Pacific and 13:34 US/Pacific for a duration of 2 hours and 44 minutes, Google Chat users experienced delays in receiving real-time incoming messages within active conversation streams across web, desktop, and mobile applications. While users continued to receive notifications for new messages, the message contents did not automatically render in active conversation streams until the user manually refreshed the application or sent a reply message.
   To our valued Google Workspace customers, we are deeply sorry for the service interruption you experienced. We are taking immediate and decisive action to address the issue and strengthen our systems against future incidents.
   Root Cause
   The incident was triggered during a global database schema rollout. An incompatible database schema change caused database writes required for message delivery to start failing.
   Specifically, a mechanism for schema synchronization—designed to ensure high consistency between primary storage and live message delivery—encountered a rare race condition bug. This caused backend message dispatchers to attempt writes against an unready schema state, leading to write failures.
   Because live message delivery had an active dependency on these writes completing successfully before pushing events to user screens, the failure caused incoming messages to experience synchronization delays. Separate push notification services and core message storage operated independently and were unaffected, which is why users still received alerts for new messages while active conversation streams experienced delays.
   Remediation and Prevention
   The issue was detected through automated backend alerting on dispatch error rates, alongside corresponding customer reports. Google Engineering initiated parallel workstreams to restore service.
   * At 12:44 US/Pacific, Google Engineering directly mitigated the source of the issue by removing the dependency that live message delivery had on the failing database write. This allowed new incoming messages to bypass the failed write and immediately resume normal real-time delivery to user screens.
   * At 13:00 US/Pacific, Engineering halted and reverted the database schema change to its previous verified stable version, restoring full database compatibility.
   * By 13:34 US/Pacific, as queued messages were processed, real-time message synchronization delays returned to 0% across all customer domains. Engineering verified that full service health was restored with zero message loss.
   To prevent this from happening again, we are executing the following preventative actions:
   * Hardened internal detection: Strengthen automated validation and verification to cover this specific failure scenario across pre-production and internal rollout stages. This aims to catch and mitigate schema incompatibilities and synchronization edge cases internally before any customer traffic can be impacted in the future.
   * Resilience Against Race Conditions: Implement Architectural improvements to make the schema synchronization and message delivery architecture resilient to race conditions. This aims to ensure that background schema and config synchronization inconsistencies cannot disrupt real-time message delivery.

   Detailed Description of Impact
   The incident impact window occurred between 10:50 US/Pacific and 13:34 US/Pacific on Friday, 28 August 2026. This issue applied globally to all Google Chat environments.
   Google Chat:
   * Delayed message rendering: New messages did not appear in active conversation streams automatically. Users were required to manually refresh their client or send a new message/react to force the stream to synchronize.
   * Stalled read states and inbox sorting: Background actions, such as updating unread badges and moving active spaces to the top of the inbox, experienced heavy internal error rates and were significantly delayed for users.
   * Search indexing delays: New messages sent during the outage window stalled in the indexing pipeline, experiencing delays in becoming searchable.
   Scope Exclusions:
   * No Data Loss: At no point was customer data lost or compromised. All messages transmitted during the incident window were securely captured in primary storage and successfully delivered to conversation streams once the backlog cleared.

Impact, status and times are the vendor’s own, as last archived. View on www.google.com (opens in new tab).

Google Workspace profileRecent Google Workspace incidentsAll outages on August 28, 2026Google Workspace in August 2026MarkdownJSON