Skip to content

Improve push-images-to-glance.py

Alexandre Seitz requested to merge aseitz/getopenstackimage into main

What does this MR do and why?

Improve python code push-images-to-glance.py to:

  • apply PEP8 style (1st commit)
  • move ENV var at beginning (2nd commit)
  • improve directory cleaning to save disk space (3rd commit)
  • upload image to glance with visibility=community + verify if image not already in progress "saving" (4th commit)

algorithm put in place to upload images to Openstack Glance:

  1. if image doesn't already exist in Glance (name + sylva-tag- + community)
  2. donwload image from OCI or web download (this step could be very long, so maybe another deployment is also trying to deploy images)
  3. after having retrieved the image locally, we verify if the same image is not already "in progress", it could have been started by another deployment ("saving" status in openstack), or it could be in stalled status ("queued status").
    1. we enter a loop to wait "saving" status to become "active"
    2. if "saving" status switch to "queued" (it means upload of another deployment failed), we delete the "queued" image and we continue algo (4)
  4. if image doesn't already exist in "saving" status, we upload it
  5. after uploading successful or not, we clean temporary directory to save disk space

Tests done

  • PYTHON standalone tests:

python execution with a custom os-images-info.yaml file, here with an old image not anymore in falcon Glance for instance ubuntu-jammy-plain-kubeadm-1-27-10:0.2.1:

  • test 1: straight case, image not in Glance
    • scripts downloads OCI, extract it and push it into Glance without any issue
    • manual verification in Gance to check community flag is well set
  • test 2: image already exists in Glance (try to upload the same image than test 1)
    • scripts detects image already exist in Glance and return the existing UUID
  • test 3: queued image owned by same user prerequisite:
    • manually delete image uploaded in test 1 with openstack image delete uuid
    • upload image with script, but kill the script (Ctrl-C) during Glance upload
    • manual verification: openstack image list --community , we have an image in queued with the corresponding name execute script:
    • script doesn't find existing "active" image (the image with same name is "queued" not "active")
    • it downloads OCI, extracts it, and just before it tries to upload it to glance, detects there is a queued image
    • wait a while and if image status is still queued, try to delete if current user is owner:
    • because queued image owner is current user: we delete the image and we continue by upload the image
    • manual verification to check image is well uploaded with community flag
  • test 4: saving image case to simulate this case, we need to:
    • remove image from glance,
    • then launch the script twice with a short delay 30sec, the idea is let enough time to the 1st execution to start upload before the second and reach the case where the 2nd execution will find a "saving image" (this case is possible because temporary directories used to save OCI have uniq name)
    • 1st execution is like test 1
    • 2nd execution detects the "saving image" (processed by test 1), and wait for it to complete, then return the UUID of test1
  • test 5: queued image owned by another user prerequisite:
    • manually delete image uploaded in test 1 with openstack image delete uuid
    • as Openstack admin create image with CLI openstack image create --community --property "sylva/md5=614896890794700473dccbd832f2f8ef" --tag sylva-md5-614896890794700473dccbd832f2f8ef ubuntu-jammy-plain-kubeadm-1-27-10-sylva-diskimage-builder-0.2.1 (this create the image object without data, image.status is queued)
    • manual verification: openstack image list --community , we have an image in queued with the corresponding name execute script:
    • script doesn't find existing "active" image (the image with same name is "queued" not "active")
    • it downloads OCI, extracts it, and just before it tries to upload it to glance, detects there is a queued image
    • queued image owner (admin) is different from current user: script exits with error code(1)

Related reference(s)

closes #1178 (closed) #1176 (closed)

Edited by Alexandre Seitz

Merge request reports