2026-01-16: Pipelines are failing to upload artifacts due to 502 Bad Gateway

Pipelines are failing to upload artifacts due to 502 Bad Gateway (Severity 2)

Problem: Pipelines were failing to upload artifacts due to 502 Bad Gateway errors, primarily when uploading large files.

Impact: Artifact uploads failed globally for both internal and customer CI/CD pipelines, with at least 12 customers affected. These failures disrupted pipeline execution and blocked feature flag rollouts for impacted jobs. Deployments were not blocked.

Causes: A recent deployment of a Cloudflare Worker in the HTTP router caused large artifact uploads to exceed the worker's memory buffer, triggering 'ReadableStream.tee() buffer limit exceeded' errors. This led Cloudflare to return 502 Bad Gateway errors for affected requests.

Response strategy: We rolled back the Cloudflare HTTP router deployment in both staging and production. After the rollback, Cloudflare dashboards showed a sharp drop to zero 502 errors for artifact endpoints. A test job was retried to confirm successful artifact uploads post-rollback. A follow-up was opened to investigate missing error reporting in Sentry.

This ticket was created to track INC-6665, by incident.io 🔥