add tools/deps-project-tool.py to manage branches, tags in sylva-elements repo
The key goal of this MR is to share the tool I used to create tags and branches in sylva-elements repos that sylva-core.
I'm mostly sharing it "as-is", knowing that some of what it does overlaps stuff that others have been possibly doing to managee gitlab repo settings (for instance protected tags and branches) "as code", or to detect untagged work.
$ tools/deps-project-tool.py -h
usage: deps-project-tool.py [-h] [--tags-new-release] [--new-release-branch] [--check-wrong-tags] [--base-protected-branches] [--commit] [--include-only INCLUDE_ONLY] [--exclude EXCLUDE] [--verbose]
Dependency project tool
options:
-h, --help show this help message and exit
--tags-new-release Do actions related to sylva-core current branch, for sylva-elements projects tags (need --commit to actually do the actions)
--new-release-branch Do actions related to sylva-core current branch, for sylva-elements projects branch (need --commit to actually do the actions)
--check-wrong-tags On sylva-elements projects, check tags matching the x.y branch (from current sylva-core branch), but that would not be on the release-x.y branch (need --commit to actually do the
actions)
--base-protected-branches
Used to setup base protected branches ("main","release-*.*") (need --commit to actually do the actions)
--commit Used to actually realize the actions (NO ACTION is )
--include-only INCLUDE_ONLY
Only look at projects matching this regexp
--exclude EXCLUDE Exclude projects matching this regexp
--verbose, -v More verbose informationExample run (for kube-job the tool had already ran with --commit option to create the tags and branch, and on sylva-toolbox for the tag only)
$ tools/deps-project-tool.py --tags-new-release --new-release-branch
processing 21 projects...
sylva-projects/sylva-elements/container-images/kube-job ...
used: v1.2.1
next: v1.3.0
v* wildcard protected tag already exists
tag v1.3.0 already exists
release branch release-1.2 already exists
release branch release-1.2 is already protected
sylva-projects/sylva-elements/container-images/libvirt-metal ...
used: 0.1.22
next: 0.2.0
protected tag 0.2.* already exists
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.22'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.22'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/container-images/oci-tools ...
used: 0.1.3
next: 0.2.0
protected tag 0.2.* already exists
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.3'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.3'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/container-images/sylva-toolbox ...
used: v0.10.4
next: v0.11.0
v* wildcard protected tag already exists
tag v0.11.0 already exists
would have created branch {'branch': 'release-0.10', 'ref': 'v0.10.4'} (use --commit to realize)
would have protected branch release-0.10 (use --commit to realize)
sylva-projects/sylva-elements/diskimage-builder ...
used: 0.4.12
next: 0.5.0
would have created protected tag rule '0.5.*' (use --commit to realize)
would have created tag {'tag_name': '0.5.0', 'ref': '0.4.12'} (use --commit to realize)
would have created branch {'branch': 'release-0.4', 'ref': '0.4.12'} (use --commit to realize)
would have protected branch release-0.4 (use --commit to realize)
sylva-projects/sylva-elements/heat-operator ...
used: 0.1.1
next: 0.2.0
would have created protected tag rule '0.2.*' (use --commit to realize)
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.1'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.1'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas ...
used: 1.1.0
next: 1.2.0
would have created protected tag rule '1.2.*' (use --commit to realize)
would have created tag {'tag_name': '1.2.0', 'ref': '1.1.0'} (use --commit to realize)
would have created branch {'branch': 'release-1.1', 'ref': '1.1.0'} (use --commit to realize)
would have protected branch release-1.1 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/os-image-server ...
used: 2.3.0
next: 2.4.0
would have created protected tag rule '2.4.*' (use --commit to realize)
would have created tag {'tag_name': '2.4.0', 'ref': '2.3.0'} (use --commit to realize)
would have created branch {'branch': 'release-2.3', 'ref': '2.3.0'} (use --commit to realize)
would have protected branch release-2.3 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-alertmanager-resources ...
used: 0.0.4
next: 0.1.0
protected tag 0.1.* already exists
would have created tag {'tag_name': '0.1.0', 'ref': '0.0.4'} (use --commit to realize)
would have created branch {'branch': 'release-0.0', 'ref': '0.0.4'} (use --commit to realize)
would have protected branch release-0.0 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-capi-cluster ...
used: 0.9.13
next: 0.10.0
would have created protected tag rule '0.10.*' (use --commit to realize)
would have created tag {'tag_name': '0.10.0', 'ref': '0.9.13'} (use --commit to realize)
would have created branch {'branch': 'release-0.9', 'ref': '0.9.13'} (use --commit to realize)
would have protected branch release-0.9 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-dashboards ...
used: 0.0.18
next: 0.1.0
protected tag 0.1.* already exists
would have created tag {'tag_name': '0.1.0', 'ref': '0.0.18'} (use --commit to realize)
would have created branch {'branch': 'release-0.0', 'ref': '0.0.18'} (use --commit to realize)
would have protected branch release-0.0 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-logging-flows ...
used: 0.0.1
next: 0.1.0
protected tag 0.1.* already exists
would have created tag {'tag_name': '0.1.0', 'ref': '0.0.1'} (use --commit to realize)
would have created branch {'branch': 'release-0.0', 'ref': '0.0.1'} (use --commit to realize)
would have protected branch release-0.0 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-prometheus-rules ...
used: 0.0.17
next: 0.1.0
protected tag 0.1.* already exists
would have created tag {'tag_name': '0.1.0', 'ref': '0.0.17'} (use --commit to realize)
would have created branch {'branch': 'release-0.0', 'ref': '0.0.17'} (use --commit to realize)
would have protected branch release-0.0 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-snmp-resources ...
used: 0.0.9
next: 0.1.0
protected tag 0.1.* already exists
would have created tag {'tag_name': '0.1.0', 'ref': '0.0.9'} (use --commit to realize)
would have created branch {'branch': 'release-0.0', 'ref': '0.0.9'} (use --commit to realize)
would have protected branch release-0.0 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sylva-thanos-rules ...
used: 0.1.4
next: 0.2.0
would have created protected tag rule '0.2.*' (use --commit to realize)
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.4'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.4'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/sync-openstack-images ...
used: 0.4.5
next: 0.5.0
protected tag 0.5.* already exists
would have created tag {'tag_name': '0.5.0', 'ref': '0.4.5'} (use --commit to realize)
would have created branch {'branch': 'release-0.4', 'ref': '0.4.5'} (use --commit to realize)
would have protected branch release-0.4 (use --commit to realize)
sylva-projects/sylva-elements/helm-charts/workload-team-defs ...
used: 0.1.2
next: 0.2.0
would have created protected tag rule '0.2.*' (use --commit to realize)
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.2'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.2'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/kiwi-imagebuilder ...
used: 0.1.2
next: 0.2.0
would have created protected tag rule '0.2.*' (use --commit to realize)
would have created tag {'tag_name': '0.2.0', 'ref': '0.1.2'} (use --commit to realize)
would have created branch {'branch': 'release-0.1', 'ref': '0.1.2'} (use --commit to realize)
would have protected branch release-0.1 (use --commit to realize)
sylva-projects/sylva-elements/sylva-units-operator ...
used: v0.2.2
next: v0.3.0
v* wildcard protected tag already exists
would have created tag {'tag_name': 'v0.3.0', 'ref': 'v0.2.2'} (use --commit to realize)
would have created branch {'branch': 'release-0.2', 'ref': 'v0.2.2'} (use --commit to realize)
would have protected branch release-0.2 (use --commit to realize)
sylva-projects/sylva-elements/sylvactl ...
used: v0.7.6
next: v0.8.0
v* wildcard protected tag already exists
would have created tag {'tag_name': 'v0.8.0', 'ref': 'v0.7.6'} (use --commit to realize)
would have created branch {'branch': 'release-0.7', 'ref': 'v0.7.6'} (use --commit to realize)
would have protected branch release-0.7 (use --commit to realize)
sylva-projects/sylva-elements/workload-cluster-operator ...
used: v0.3.2
next: v0.4.0
v* wildcard protected tag already exists
would have created tag {'tag_name': 'v0.4.0', 'ref': 'v0.3.2'} (use --commit to realize)
would have created branch {'branch': 'release-0.3', 'ref': 'v0.3.2'} (use --commit to realize)
would have protected branch release-0.3 (use --commit to realize) The tool also has a mode to check that in sylva-elements repo there is no x.y.z tag that would not be on release-x.y (and x.y.z > the version currently used in sylva-core). This is something that can happen if an x.y.z was added in main in a sylva-elements repo and "x.y.(z-1)" was still the version used by sylva-core when Sylva 1.4 release was made. In such a case, we need to know it, and then we manually create an "x.y.(z+2)" tag in release-x.y and switch to it in sylva-core release-1.4.
Example:
$ tools/deps-project-tool.py --check-wrong-tags --verbose --exclude 'diskimage-builder'
processing 21 projects...
sylva-projects/sylva-elements/container-images/ci-image ...
used: v1.1.6
next: v1.2.0
checking v1.1.* tags
ok, tag v1.1.6 is on release-1.1
ignored v1.1.5 tag
ignored v1.1.4 tag
ignored v1.1.3 tag
ignored v1.1.2 tag
ignored v1.1.1 tag
ignored v1.1.0 tag
...
sylva-projects/sylva-elements/helm-charts/sylva-snmp-resources ...
used: 0.0.9
next: 0.1.0
checking 0.0.* tags
ok, tag 0.0.9 is on release-0.0
ignored 0.0.0-pre1 tag
ignored 0.0.8 tag
ignored 0.0.0-lenovo3 tag
ignored 0.0.0-lenovo2 tag
ignored 0.0.0-lenovo1 tag
ignored 0.0.7 tag
ignored 0.0.6 tag
ignored 0.0.6-pre2 tag
ignored 0.0.6-pre1 tag
ignored 0.0.5 tag
ignored 0.0.5-pre tag
ignored 0.0.4-pre tag
ignored 0.0.3 tag
ignored 0.0.2 tag
ignored 0.0.1 tag
sylva-projects/sylva-elements/helm-charts/sylva-thanos-rules ...
used: 0.1.4
next: 0.2.0
checking 0.1.* tags
67%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 14/21 [00:25<00:11, 1.70s/it]Traceback (most recent call last):
File "/home/teom7365/projets/sylva/sylva-core/tools/deps-project-tool.py", line 460, in <module>
main()
File "/home/teom7365/projets/sylva/sylva-core/tools/deps-project-tool.py", line 456, in main
process_repos(gl, options, gitlab_project)
File "/home/teom7365/projets/sylva/sylva-core/tools/deps-project-tool.py", line 342, in process_repos
raise Exception(f"tag {tag.name} is not on branch '{branch_name}'")
Exception: tag 0.1.5 is not on branch 'release-0.1'Since I noticed discrepancies in branch protection rules I added the ability to align them:
$ tools/deps-project-tool.py --base-protected-branches
processing 22 projects...
sylva-projects/sylva-elements/container-images/ci-image ...
used: v1.2.0
next: v1.3.0
sylva-projects/sylva-elements/container-images/kube-job ...
used: v1.3.0
next: v1.4.0
sylva-projects/sylva-elements/container-images/libvirt-metal ...
used: 0.2.0
next: 0.3.0
sylva-projects/sylva-elements/container-images/oci-tools ...
used: 0.1.3
next: 0.2.0
sylva-projects/sylva-elements/container-images/sylva-toolbox ...
used: v0.11.0
would have created branch protection rule for 'release-*.*' (use --commit to realize)
next: v0.12.0
sylva-projects/sylva-elements/diskimage-builder ...
used: 0.4.12
would have created branch protection rule for 'release-*.*' (use --commit to realize)
next: 0.5.0
sylva-projects/sylva-elements/heat-operator ...
used: 0.2.0
would have recreated branch protection rule for 'main' (use --commit to realize)
would have created branch protection rule for 'release-*.*' (use --commit to realize)
next: 0.3.0
sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas ...
used: 1.2.0
would have recreated branch protection rule for 'main' (use --commit to realize)
would have created branch protection rule for 'release-*.*' (use --commit to realize)
next: 1.3.0
...with --commit:
...
sylva-projects/sylva-elements/heat-operator ...
used: 0.2.0
recreated branch protection rule for 'main'
created branch protection rule for 'release-*.*'
next: 0.3.0
sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas ...
used: 1.2.0
recreated branch protection rule for 'main'
created branch protection rule for 'release-*.*'
next: 1.3.0
sylva-projects/sylva-elements/helm-charts/os-image-server ...
used: 2.4.0
created branch protection rule for 'release-*.*'
next: 2.5.0
sylva-projects/sylva-elements/helm-charts/sylva-alertmanager-resources ...
used: 0.1.0
recreated branch protection rule for 'main'
created branch protection rule for 'release-*.*'
next: 0.2.0
...CI configuration
Below you can choose test deployment variants to run in this MR's CI.
Click to open to CI configuration
Legend:
| Icon | Meaning | Available values |
|---|---|---|
| Infra Provider |
capd, capo, capm3
|
|
| Bootstrap Provider |
kubeadm (alias kadm), rke2
|
|
| Node OS |
ubuntu, suse
|
|
| Deployment Options |
light-deploy, dev-sources, ha, misc, maxsurge-0, logging, no-logging
|
|
| Pipeline Scenarios | Available scenario list and description |
-
🎬 preview☁️ capd🚀 kadm🐧 ubuntu -
🎬 preview☁️ capo🚀 rke2🐧 suse -
🎬 preview☁️ capm3🚀 rke2🐧 ubuntu -
☁️ capd🚀 kadm🛠️ light-deploy🐧 ubuntu -
☁️ capd🚀 rke2🛠️ light-deploy🐧 suse -
☁️ capo🚀 rke2🐧 suse -
☁️ capo🚀 kadm🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 wkld-k8s-upgrade🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update-no-wkld🛠️ ha🐧 suse -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.3.x🛠️ ha🐧 ubuntu -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.3.x🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🐧 suse -
☁️ capm3🚀 kadm🐧 ubuntu -
☁️ capm3🚀 kadm🎬 rolling-update-no-wkld🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🎬 wkld-k8s-upgrade🛠️ ha🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2🎬 sylva-upgrade-from-1.3.x🛠️ ha🐧 suse -
☁️ capm3🚀 rke2🛠️ misc,ha🐧 suse -
☁️ capm3🚀 rke2🎬 sylva-upgrade-from-1.3.x🛠️ ha,misc🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 suse -
☁️ capm3🚀 ck8s🎬 no-wkld🛠️ light-deploy🐧 ubuntu
Global config for deployment pipelines
-
autorun pipelines -
allow failure on pipelines -
record sylvactl events
Notes:
- Enabling
autorunwill make deployment pipelines to be run automatically without human interaction - Disabling
allow failurewill make deployment pipelines mandatory for pipeline success. - if both
autorunandallow failureare disabled, deployment pipelines will need manual triggering but will be blocking the pipeline
Be aware: after configuration change, pipeline is not triggered automatically.
Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.
Edited by Thomas Morin