Improve push-images-to-glance.py
What does this MR do and why?
Improve python code push-images-to-glance.py
to:
- apply PEP8 style (1st commit)
- move ENV var at beginning (2nd commit)
- improve directory cleaning to save disk space (3rd commit)
- upload image to glance with visibility=community + verify if image not already in progress "saving" (4th commit)
algorithm put in place to upload images to Openstack Glance:
- if image doesn't already exist in Glance (name + sylva-tag- + community)
- donwload image from OCI or web download (this step could be very long, so maybe another deployment is also trying to deploy images)
- after having retrieved the image locally, we verify if the same image is not already "in progress", it could have been started by another deployment ("saving" status in openstack), or it could be in stalled status ("queued status").
- we enter a loop to wait "saving" status to become "active"
- if "saving" status switch to "queued" (it means upload of another deployment failed), we delete the "queued" image and we continue algo (4)
- if image doesn't already exist in "saving" status, we upload it
- after uploading successful or not, we clean temporary directory to save disk space
Tests done
- PYTHON standalone tests:
python execution with a custom os-images-info.yaml file, here with an old image not anymore in falcon Glance
for instance ubuntu-jammy-plain-kubeadm-1-27-10:0.2.1
:
- test 1: straight case, image not in Glance
- scripts downloads OCI, extract it and push it into Glance without any issue
- manual verification in Gance to check community flag is well set
- test 2: image already exists in Glance (try to upload the same image than test 1)
- scripts detects image already exist in Glance and return the existing UUID
- test 3: queued image owned by same user
prerequisite:
- manually delete image uploaded in test 1 with
openstack image delete uuid
- upload image with script, but kill the script (Ctrl-C) during Glance upload
- manual verification:
openstack image list --community
, we have an image inqueued
with the corresponding name execute script: - script doesn't find existing "active" image (the image with same name is "queued" not "active")
- it downloads OCI, extracts it, and just before it tries to upload it to glance, detects there is a queued image
- wait a while and if image status is still queued, try to delete if current user is owner:
- because queued image owner is current user: we delete the image and we continue by upload the image
- manual verification to check image is well uploaded with community flag
- manually delete image uploaded in test 1 with
- test 4: saving image case
to simulate this case, we need to:
- remove image from glance,
- then launch the script twice with a short delay 30sec, the idea is let enough time to the 1st execution to start upload before the second and reach the case where the 2nd execution will find a "saving image" (this case is possible because temporary directories used to save OCI have uniq name)
- 1st execution is like test 1
- 2nd execution detects the "saving image" (processed by test 1), and wait for it to complete, then return the UUID of test1
- test 5: queued image owned by another user
prerequisite:
- manually delete image uploaded in test 1 with
openstack image delete uuid
- as Openstack admin create image with CLI
openstack image create --community --property "sylva/md5=614896890794700473dccbd832f2f8ef" --tag sylva-md5-614896890794700473dccbd832f2f8ef ubuntu-jammy-plain-kubeadm-1-27-10-sylva-diskimage-builder-0.2.1
(this create the image object without data, image.status is queued) - manual verification:
openstack image list --community
, we have an image inqueued
with the corresponding name execute script: - script doesn't find existing "active" image (the image with same name is "queued" not "active")
- it downloads OCI, extracts it, and just before it tries to upload it to glance, detects there is a queued image
- queued image owner (admin) is different from current user: script exits with error code(1)
- manually delete image uploaded in test 1 with
Related reference(s)
closes #1178 (closed) #1176 (closed)
Edited by Alexandre Seitz