Companies / Salesforce / incidents

Salesforce: Feature Service Disruption: Field Service Scheduling

Impact: minorStatus: resolvedStarted Resolved (10 h)8 updatesSource: status.salesforce.com (opens in new tab)

Affected components: Field Service Scheduling595 instances

Timeline of updates

  1. update

    We are investigating a feature disruption impacting Field Service Scheduling across multiple regions. Customers may get 503 errors when trying to bundle or unbundle service appointments. This will stop scheduling actions from working. The issue appears to be related to the Bundle Manager Service, where certain service PODs are not responding as expected. A recent deployment was initially thought to be the trigger of the issue. A rollback of the deployment did not resolve the issue. We are exploring multiple paths to resolution. Investigations have also highlighted that the impact radius is broader than initially understood. We have updated the posting to reflect the additional instances experiencing impact, and customers in these instances will receive communications related to this incident going forward. We will provide an update within 30 minutes, or sooner if additional information becomes available.

  2. update

    We have identified that a configuration change implemented yesterday is the cause of the issue. This change prevented new service components from starting correctly and resulted in the bundling service becoming unavailable. We're working on reverting the configuration change and will closely monitor the progress. We will provide an update within 30 minutes, or sooner if additional information becomes available.

  3. update

    Investigations have highlighted that the impact radius is narrower than initially understood. We will now remove any instances that have not experienced an impact, and these will no longer receive communications about this incident going forward. Impacted instances will continue to be updated. We apologize for any inconvenience caused.

  4. update

    We have reverted the configuration change and have redeployed the change. However, post-deployment validation reveals that the service is still not fully restored, as the container services terminate shortly after startup. We are troubleshooting ‌deployment verification failures. Preparations are underway to redeploy once the cause of the failure is confirmed and addressed. Investigations have also highlighted that the impact radius is broader than initially understood. We have updated the posting to reflect the additional instances experiencing impact, and customers in these instances will receive communications related to this incident going forward. We will provide an update within 30 minutes, or sooner if additional information becomes available.

  5. update

    Upon further investigation, we have determined that the start time of impact is different than initially understood. We have now revised the start time to 06:59 UTC to more accurately reflect the time customers may have begun to experience the impact. We apologize for any confusion caused by this.

  6. update

    We are implementing an improvised configuration code change in the new permanent fix forward with a staggered deployment. As deployment of the fix progresses, immediate validation will take place to confirm that services are returning to normal. We will provide an update within 30 minutes, or sooner if additional information becomes available.

  7. update

    Upon further investigation, we have determined that the impacted experience is broader than initially understood. Customers may encounter HTTP 503 errors when attempting to use any service appointment bundling features (including but not limited to bundling, unbundling, status update bundle, or bundle member service appointment), rather than only impacting bundling and unbundling, as initially stated. We will provide an update within 30 minutes, or sooner if additional information becomes available.

  8. update

    As of 17:25 UTC, the staggered deployment has been performed across all impacted environments with confirmation that all services have returned to normal. We apologize for how this incident affected you and your business. We will undertake a full investigation to establish the technical trigger, the underlying cause, and preventive action to avoid a repeat in the future.

Impact, status and times are the vendor’s own, as last archived. View on status.salesforce.com (opens in new tab). Salesforce profile · Recent Salesforce incidents · Markdown · JSON