Skip to content

Fix cleanup.sh CI script to only delete dev packages

Ryan Walder requested to merge ryanw/fix-cleanup-script into master

Intent

The previous package cleanup script somehow deleted live system Packages.

This MR updates the script to add double protection when querying the Packages API to ensure it only ever returns dev packages.


  • This is meant for a hotfix
  • This is meant for the next release (see milestone)
  • This needs more reviewers than normal; there may be controversy or high complexity
  • This intentionally introduces regressions that will be addressed later
  • There is/will be documentation changes on the wiki
  • Please do not send commits here without coordinating closely with the owner
  • This is a Build System change

Steps to Test

  1. Read Diff
  2. Trust me Bro™

or

As we can't easily test the script itself (it could be edited manually on a local machine I guess by adding all the needed envars and replacing "JOB-TOKEN: ${CI_JOB_TOKEN}" with "PRIVATE-TOKEN: ${PRIVATE_TOKEN}", editing out the actual deletion part and having it echo the IDs to delete but, that's a lot of effort). This emulates a "get everything" API call to see the current packages then the new filtered API call used in the script.

Linux/MacOS:

  1. You will need bash, curl, jq
    • Other shells will work but you're on your own for setting/referencing variables
  2. Create yourself a GitLab API key with the api scope
    • Make it valid for like 1 day for safety
  3. In your local (bash) shell do the following:
    1. export SYSTEM_NAME="cyberpunk-red-core"
    2. export PRIVATE_TOKEN="YOUR_API_KEY"
    3. export PROJECT_URL="https://gitlab.com/api/v4/projects/cyberpunk-red-team/fvtt-cyberpunk-red-core
  4. curl --silent --location --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "${PROJECT_URL}/packages | jq
    • This is mostly a test to see if you have configured your environment correctly
    • This will return a list of all releases
  5. curl --silent --location --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "${PROJECT_URL}/packages?package_name=fvtt-cyberpunk-red-core-dev" | jq -r 'map(select(.version != "latest" and .name == "fvtt-cyberpunk-red-core-dev")) | sort_by(.version) | .[].id'
    • This should return nothing as we currently have no dev releases (as of writing)
      • If we have > 3 dev releases it should return N-3 package IDs
      • If this is the case compare the IDs returned with your first API call and check it is the correct one to delete

Windows:

These instructions are from ChatGPT. I have no idea if they are valid but they shouldn't be destructive if they are not

  1. Ensure you have PowerShell installed
  2. Create a GitLab API key with the api scope for security. Set it to expire in 1 day
  3. In your local (PowerShell) shell do the following:
    1. $env:SYSTEM_NAME = "cyberpunk-red-core"
    2. $env:PRIVATE_TOKEN = "YOUR_API_KEY"
    3. $env:PROJECT_URL = "https://gitlab.com/api/v4/projects/cyberpunk-red-team/fvtt-cyberpunk-red-core"
    4. $env:PACKAGE_NAME = "fvtt-cyberpunk-red-core-dev"
  4. Invoke-RestMethod -Uri $env:PROJECT_URL/packages -Headers @{"PRIVATE-TOKEN" = $env:PRIVATE_TOKEN}
    • This is mostly a test to see if you have configured your environment correctly
    • This will return a list of all releases
  5. Invoke-RestMethod -Uri "$env:PROJECT_URL/packages?package_name=$env:PACKAGE_NAME" -Headers @{"PRIVATE-TOKEN" = $env:PRIVATE_TOKEN} | Where-Object { $_.version -ne "latest" -and $_.name -eq $env:PACKAGE_NAME } | Sort-Object version | Select-Object -ExpandProperty id
    • This should return nothing as we currently have no dev releases (as of writing)
      • If we have > 3 dev releases it should return N-3 package IDs
      • If this is the case compare the IDs returned with your first API call and check it is the correct one to delete

Future Work

Once merged the Cleanup Dev Packages job in Pipeline Schedules needs to be updated to remove the [DO NOT ENABLE: Ryan] text from the name and re-enable it.

Feel free to leave this to me, it will not do anything if this merged.

Edited by Ryan Walder

Merge request reports