capo-contrail-bgpaas failing to remove OpenStackMachine
This is a follow-up issue to #2941 (closed) which is a backport issue for #2736 (closed)
The reason for creating it:
- the cause of #2941 (closed) is a failure by capo-contrail-bgpaas preventing the deletion of OpenStackMachines
- #2736 (closed) describes a symptom not a cause, and no logs have been preserved to know whether or not, so it's hard to know whether or not the cause is the same
- a tentative fix for this issue was worked on in sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas#8 (closed), and the sylva-core integration MRs had been marked as "Closing" #2941 (closed), which is messy since #2941 (closed) is specific to release-1.5
The point of this issue is to have a clear issue for the capo-contrail-bgpaas problem.
Status as of 2025-10-03
The tentative fix at sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas#8 (closed) appears to not be sufficient, because the problem still occurs it's application to release-1.4 in !5746 (closed) (see job https://gitlab.com/sylva-projects/sylva-core/-/jobs/11579019137 which has OpenStackMachines pending deletion blocked by the finalizer).
Status as of 2025-10-20
While trying to troubleshoot and fix the problem, beyond the initial observations were the controller was failing to remove its finalizer on the OpenStackMachines, I also noticed potential cases where the OpenStackMachines was prevented because the HeatStack resources would either not have been removed or would have wrongly be recreated and would have prevented the deletion of the corresponding OpenStackMachine beucase of the presence of the blockOwnerDeletion ownerReference.
sylva-projects/sylva-elements/helm-charts/capo-contrail-bgpaas!106 (merged) proposes controller improvements to avoid all those potential issues
Status as of 2025-10-21
Even with the above improvement, the issue still reproduces: https://gitlab.com/sylva-projects/sylva-core/-/jobs/11781094274