Companies/Lightstep (ServiceNow Cloud Observability)/Incidents

Lightstep (ServiceNow Cloud Observability): Lightstep experience major issues with web UI, query and ingest.

Affected components: Observability - Data Ingestion - Trace StatisticsUS - Trace StatisticsObservability - Third-Party Notification Methods - Slack MessagingSlack MessagingObservability - Third-Party Notification Methods - Slack ConnectionsSlack ConnectionsObservability - Web UI - DashboardsUS - DashboardsObservability - APIUS - Observability - APIObservability - Data Ingestion - MetricsUS - MetricsObservability - Web UI - Service DirectoryUS - Service DirectoryObservability - Web UI - NotebooksUS - NotebooksObservability - Third-Party Notification Methods - Slack Apps/IntegrationsSlack Apps/IntegrationsObservability - Third-Party Notification Methods - PagerDuty Events APIPagerDuty Events APIObservability - Data Ingestion - Trace AssemblyUS - Trace AssemblyObservability - Web UI - ExplorerUS - ExplorerObservability - Alerting - Metric alertingUS - Unified AlertingObservability - Alerting - Trace alertingUS - Streams AlertingObservability - Web UI - Change IntelligenceUS - Change Intelligence

Timeline of updates

1. investigating2023-07-11 01:20 UTC

   We are currently investigating this issue.

2. investigating2023-07-11 01:46 UTC

   We are continuing to investigate this issue.

3. investigating2023-07-11 02:09 UTC

   We are continuing to investigate this issue.

4. investigating2023-07-11 02:10 UTC

   Ingest of all data is currently recovering. The web UI is back up. Queries should be working.

5. investigating2023-07-11 03:21 UTC

   Most services should be operating normal. There may still be issues with alerting. We continue to investigate.

6. investigating2023-07-11 04:31 UTC

   We are continuing to investigate this issue. There has been a regression.

7. monitoring2023-07-11 05:12 UTC

   A fix has been implemented and we are monitoring the results.

8. resolved2023-07-11 12:48 UTC

   This incident has been resolved.

Changes Rewindex caught on this incident

• Jul 11, 2023Critical incident: Lightstep experience major issues with web UI, query and ingest.

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Lightstep (ServiceNow Cloud Observability) profileRecent Lightstep (ServiceNow Cloud Observability) incidentsMarkdownJSON