Release race condition in Konflux
We are facing release issue since a while now e.g: fedora-bootc-rawhide-standard have not been updated since January 12th.
So I took a closer look on what is going on.
AIUI we are facing release race conditions, below a timeline:
- a new commit is merged to the main branch, triggering the per-component
on-pushpipelines - once built, Konflux creates a new snapshot for each build e.g:
fedora-bootc-43-1,fedora-bootc-43-2,fedora-bootc-43-3. Each snapshot contains its updated component + the latest ones of the other components. - Konflux runs the ITS against each snapshot. Let's say
fedora-bootc-43-3finishes first, the updated component of this snapshot e.g:fedora-bootc-43-minimalis then released, hence a new image offedora-bootc-43-minimalin Quay. Thenfedora-bootc-43-1finishes, but as it's older thanfedora-bootc-43-3Konflux does not release it. e.g:
oc get snapshots fedora-bootc-43-m7m2t -o yaml
[...]
- lastTransitionTime: "2026-01-28T12:54:52Z"
message: Released in newer Snapshot
reason: AutoReleased
status: "True"
type: AutoReleased
To have all the snapshots released, we'd need to have the exact same order between the build (snapshot created) and the ITS (snaphot validated) phases.
e.g: build-1 -> build-2 -> build-1-validated -> build-2-validated
Note: retriggering a failed ITS does not help, because Konflux reuses the same Snapshot which might be older than another one from the same commit.
As the integration tests of -minimal and -minimal-plus are faster than the -standard and -iot ones. The snapshots of those components are, most of the time if not ITS errors, validated and released first, making the others skipped.
I double checked our config, I think setting singleComponentMode to False would fix the issue, as Konflux would release all the components of the Snapshot.
05/02/26 update:
- setting
singleComponentModeto False AND usingrenovate on-push pipelinesmakes Konflux releasing components built from unmerged commit c.f https://github.com/konflux-ci/support/issues/9 - a more accurate observation of the current issue is described in https://github.com/konflux-ci/integration-service/issues/1453