2025-11-11: Docker API version mismatch blocking deployments

Docker API version mismatch blocking deployments (Severity 3)

Problem: Deployments and customer CI pipelines are failing due to a Docker API version mismatch. Errors include 'client version 1.52 is too new' and 'client version 1.41 is too old', depending on the images used.

Impact: Deployments are blocked for both internal teams and customers. At least 4 support tickets and multiple customer CI jobs have failed to run or start containers due to Docker API version errors. Some support tickets appear unrelated, but there is a consistent pattern of CI job failures with Docker errors.

Causes: A recent update to the Docker version in CI images caused the Docker client to be newer than the supported Docker-in-Docker server version, resulting in an API version mismatch. Reverting the Docker version update in CI images has not resolved the issue. The use of floating tags like ':latest' in both official and customer CI images has made it difficult to ensure the fix is applied, and some failures may be unrelated to the Docker version update.

Response strategy: We reverted the Docker version update in CI images, but new pipelines are still failing. We have filed a follow-up to pin specific Docker image versions instead of using floating tags like ':latest' to reduce future risk.


This ticket was created to track INC-5623, by incident.io 🔥