add checks to fail cleanly if the downloaded OCI artifact does not have the right size/digest

Thomas Morinrequested to merge
fail-on-size-or-digest-mismatch into main

We ran into cases where the script was failing with:

2025-07-04 07:35:11,234 INFO __main__ <module>: Pulling image: opensuse-15-6-hardened-rke2-1-32-5 from artifact uri: oci://172.20.136.39/proxy_cache_registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/opensuse-15-6-hardened-rke2-1-32-5:0.5.3 to /tmp/tmp5umxff_l
2025-07-04 08:05:13,179 INFO oras.logger text_handler: Successfully pulled /tmp/tmp5umxff_l/sha256:05665ae2fb910bfe3c0ea46ee3f574eb1dd92954cbd6f659a7d1ec1c99dec455.
2025-07-04 08:05:13,179 INFO __main__ <module>: Unzipping artifact...
EOFError: Compressed file ended before the end-of-stream marker was reached

This seems to happen in corner cases where the registry misbehaves and the downloaded artifact is truncated.

The underlying oraspy library unfortunately does not check the downloaded size and digest.

This MR adds such a check to ensure that our logs produce helpful errors on failures:

2025-07-04 09:38:25,140 INFO __main__ <module>: Pulling image: opensuse-15-6-hardened-rke2-1-32-5 from artifact uri: oci://172.20.136.39/proxy_cache_registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/opensuse-15-6-hardened-rke2-1-32-5:0.5.2 to /tmp/tmp2s2qocjs
2025-07-04 09:38:38,588 INFO oras.logger text_handler: Successfully pulled /tmp/tmp2s2qocjs/sha256:212219c937f0b73e4a9f1a5b3e37d08842e772c557626585792d1583a8f38522.
2025-07-04 09:38:38,590 ERROR __main__ pull_image: downloaded OCI artifact file size is 1681154267 which is not the expected size (1681154268)
2025-07-04 09:38:38,591 ERROR __main__ pull_image: Failed to pull image.
Exception: downloaded OCI artifact file size is 1681154267 which is not the expected size (1681154268)
Exception: downloaded OCI artifact file size is 1681154267 which is not the expected size (1681154268)
2025-07-04 09:41:38,013 INFO __main__ <module>: Pulling image: opensuse-15-6-hardened-rke2-1-32-5 from artifact uri: oci://172.20.136.39/proxy_cache_registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/opensuse-15-6-hardened-rke2-1-32-5:0.5.2 to /tmp/tmp6mkgsdte
2025-07-04 09:41:50,247 INFO oras.logger text_handler: Successfully pulled /tmp/tmp6mkgsdte/sha256:212219c937f0b73e4a9f1a5b3e37d08842e772c557626585792d1583a8f38522.
2025-07-04 09:42:00,208 ERROR __main__ pull_image: downloaded OCI artifact digest is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 which is not the expected size (212219c937f0b73e4a9f1a5b3e37d08842e772c557626585792d1583a8f38522)
2025-07-04 09:42:00,208 ERROR __main__ pull_image: Failed to pull image.
Exception: downloaded OCI artifact digest is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 which is not the expected size (212219c937f0b73e4a9f1a5b3e37d08842e772c557626585792d1583a8f38522)
Exception: downloaded OCI artifact digest is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 which is not the expected size (212219c937f0b73e4a9f1a5b3e37d08842e772c557626585792d1583a8f38522)

Test

This was tested in sylva-projects/sylva-core!4843 (closed), with a failure that happened in https://gitlab.com/sylva-projects/sylva-core/-/jobs/10566535980:

...
2025-07-04 10:15:53,065 INFO __main__ <module>: Pulling image: ubuntu-noble-plain-kubeadm-1-32-5 from artifact uri: oci://172.20.136.39/proxy_cache_registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/ubuntu-noble-plain-kubeadm-1-32-5:0.5.3 to /tmp/tmpyw3kc_ar
2025-07-04 10:38:13,949 INFO oras.logger text_handler: Successfully pulled /tmp/tmpyw3kc_ar/sha256:b7cd224f8a800762e021da85df80665d25fedb51a95c13aa02b12373ae46ae48.
2025-07-04 10:38:13,949 ERROR __main__ pull_image: downloaded OCI artifact file size is 453385520 which is not the expected size (1323635987)
2025-07-04 10:38:13,949 ERROR __main__ pull_image: Failed to pull image.
Exception: downloaded OCI artifact file size is 453385520 which is not the expected size (1323635987)
Exception: downloaded OCI artifact file size is 453385520 which is not the expected size (1323635987)
Edited by Thomas Morin

Merge request reports