Companies/Lightstep (ServiceNow Cloud Observability)/Incidents

Lightstep (ServiceNow Cloud Observability): Span Ingestion & Trace Assembly Degraded

Affected components: Observability - Data Ingestion - Trace StatisticsUS - Trace StatisticsObservability - Data Ingestion - Trace AssemblyUS - Trace Assembly

Timeline of updates

1. investigating2023-02-16 16:39 UTC

   We are currently investigating an issue with span ingest into: ingest.lightstep.com.

2. investigating2023-02-16 17:05 UTC

   We have identified an issue ingesting spans for customers sending traffic directly to ingest.lightstep.com. Customers that are running microsatellites are not affected.

3. identified2023-02-16 17:05 UTC

   A suspected issue has been identified and a fix is being rolled out.

4. identified2023-02-16 17:06 UTC

   We have identified an issue ingesting spans for customers sending traffic directly to ingest.lightstep.com. Customers that are running microsatellites are not affected.

5. identified2023-02-16 17:31 UTC

   Span ingestion appears to be functional. Trace assembly is still impacted and actively being worked on.

6. identified2023-02-16 17:35 UTC

   We are continuing to work on a fix for this issue.

7. monitoring2023-02-16 17:43 UTC

   We have deployed the fixes and are actively monitoring the affected systems.

8. resolved2023-02-16 17:50 UTC

   All affected systems appear to be returned to a working state.

Changes Rewindex caught on this incident

• Feb 16, 2023Major incident: Span Ingestion & Trace Assembly Degraded

Impact, status and times are the vendor’s own, as last archived. View on stspg.io (opens in new tab).

Lightstep (ServiceNow Cloud Observability) profileRecent Lightstep (ServiceNow Cloud Observability) incidentsMarkdownJSON