# Harness: FME - Some customers are experiencing delays in scheduled exports of impressions

- Impact (vendor’s label): minor
- Status: postmortem
- Started: 2026-07-02 15:48 UTC · Resolved: 2026-07-03 16:11 UTC · Duration: 24 h
- Updates: 5 · Affected components: FME, Prod 1 - FME, FME, Prod 2 - FME, FME, Prod 3 - FME
- [Vendor status page: https://stspg.io/pydwzxnfsc2x](https://stspg.io/pydwzxnfsc2x)
- [Company: Harness](https://rewindex.com/company/harness.md)

## Timeline of updates

- **monitoring** 2026-07-02 15:48 UTC: A fix has been implemented and we are monitoring the results.
- **monitoring** 2026-07-02 20:10 UTC: We are seeing the backlogged items worked through and continuing to monitor results.
- **monitoring** 2026-07-02 23:31 UTC: We are continuing to monitor results while the backlog is worked through.
- **resolved** 2026-07-03 16:11 UTC: This incident has been resolved.
- **postmortem** 2026-07-09 19:24 UTC: \*\*Incident Summary\*\* Harness FME \\(Feature Management & Experimentation\\) faced substantial delays in scheduled impressions data exports on Jul 2, 2026. This issue stemmed from performance degradation within Tinybird's infrastructure, affecting query execution crucial to our export pipeline. \*\*Root Cause\*\* The core issue was a temporary \*\*performance degradation on the provider's infrastructure\*\*. This directly impacted the metadata retrieval endpoint, which failed to return job IDs intermittently. Consequently, completed export jobs underwent repeated retries, leading to a backlog in the export queue. \*\*Impact\*\* \* The export backlog accumulated, peaking at more than 6 hours behind the planned schedule. \* No data loss was detected during the incident. ‌ \*\*Mitigation\*\* 1. \*\*Adjusted Temporal workflows:\*\* Extended timeouts from ~30 to ~50 minutes and increased polling attempts from ~180 to ~300. 2. \*\*Expanded resource capacity.\*\* 3. \*\*Traffic redirection:\*\* Systematically shifted export workloads over to an optimized in-house cluster. 4. \*\*Manual intervention:\*\* Terminated stalling workflows stuck in retry loops from failed metadata lookups. ‌ \*\*Next Steps\*\* Complete the migration of all remaining export jobs to the internal analytics cluster \\(In progress\\).


---
Canonical: https://rewindex.com/company/harness/incidents/51nvd88ny5b1 · Markdown: https://rewindex.com/company/harness/incidents/51nvd88ny5b1.md · JSON: https://rewindex.com/api/v1/companies/harness/incidents/51nvd88ny5b1 · Generated 2026-10-20T02:21:32.717Z
