rewindex

CompaniesSalesforceIncidents

Salesforce: Feature Perf Degradation: Agentforce

MinorStatus: resolvededitedStarted Resolved (5.5 h)9 updatesstatus.salesforce.com (opens in new tab)

Affected components: Agentforce91 instances

Timeline of updates

  1. update

    We're investigating an issue affecting some Agentforce customers who use large language models hosted by a third-party provider. Affected requests may return errors or fail to complete.

    Customers in multiple regions may be affected when their requests use the impacted capacity. The issue currently appears limited to models hosted by this provider. We're working with our service provider and reviewing changes to how requests are routed to resolve the issue. We're investigating the issue and will provide an update in 30 minutes, or sooner if we have more information.

  2. update

    Investigations have highlighted that the impact radius is broader than initially understood. We have updated the posting to reflect the additional instances experiencing impact, and customers in these instances will receive communications related to this incident going forward.

  3. update

    We're working with the provider to get a recovery timeline and looking at options to route requests to alternative large language models. Because the issue affects multiple regions, we have limited rerouting options, and putting them in place may take more time. We'll provide an update in 30 minutes, or sooner if additional information becomes available.

  4. update

    Upon further investigation, we've determined that the start time of impact is different than initially understood. We've revised the start time of the Trust post to more accurately reflect the time customers may have begun to experience impact. We apologize for any confusion caused by this.

  5. update

    We're working with the third-party provider and looking at alternate large language model options. We don't have a time for resolution yet, as the provider continues to investigate the incident trigger. Contractual and data residency commitments limit where we can reroute requests. We'll provide an update in 30 minutes, or sooner if additional information becomes available.

  6. update

    The third-party provider has shared initial findings but doesn't yet know the definitive cause of the widespread service disruption. Their engineering team is actively investigating across the impacted regions. We'll provide an update in 30 minutes, or sooner if additional information becomes available.

  7. update

    The third-party provider has identified an update that may be causing the issue. The third-party provider will begin the rollback in one region to confirm it resolves the issue which is expected to take about two hours. We'll provide an update in 30 minutes, or sooner if additional information becomes available.

  8. update

    The third-party service provider is continuing to roll back the recent change to how it routes requests. After the rollback, the provider will also apply a follow-up fix. We're working with the provider to confirm recovery steps and expected recovery time. We'll provide an update in 30 minutes, or sooner if additional information becomes available.

  9. update

    We've resolved the issue affecting Agentforce requests that use certain third-party large language models. A third-party service provider made a change to how it routes requests. The provider rolled back the change and applied a follow-up fix. We confirmed that service has returned to normal, and customers no longer see errors.

    We apologize for how this incident may have affected our customers and their businesses. We'll investigate the incident to confirm the cause and the steps we'll take to prevent it from happening again.

Changes Rewindex caught on this incident

Archive proof

Rewindex commits each version of an incident it captures to a daily checkpoint, chains it to the day before, and has the checkpoint timestamped by FreeTSA and DigiCert. That proves these records existed by then and have not been changed since, by anyone, Rewindex included.

How the archive is checked, and what it cannot prove.

Impact, status and times are the vendor’s own, as last archived. View on status.salesforce.com (opens in new tab).

Salesforce profileRecent Salesforce incidentsMarkdownJSON