Improve ready conditions of MD and CP in cluster-machines-ready script
close : #1289 (closed) #1412 (closed) \
(Tested on Capo with maxSurge set to 0)
MD rolling-upgrade related to #1289 (closed)
ControlPlane rolling-upgrade related to #1412 (closed)
A MD rolling-upgrade can be seen below. The conclusions we can take from this are :
-
updatedReplicas
is updated as soon as a new machine is created (no matter what state it is in (failed, provisionning...) -
phase
: between eachscalingUp
, the machineDeployment returns to arunning
state -
readyReplicas
: takes into account replicas that are Deleting and Running -
replicas
: take into account machine no matter their states -
availableReplicas
: seems represent running replicas (and running replicas which are in deleting state) -
unavailableReplicas
: duringscalingUp
phase this represent the replicas that are currently not updated
Based on that we can wait that readyReplicas
,replicas
,updatedReplicas
are equals to spec.replicas
and unavailableReplicas
=0 and phase
is Running
.
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-9xxq7 management-cluster management-cluster-md-md0-2a55c05a1b-pnml9 openstack:///27ef1b52-debc-4699-9bc6-904cf68fd25a Running 49m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-gvxlg management-cluster management-cluster-md-md0-2a55c05a1b-mvwhb openstack:///c81e1c91-749c-4594-871e-37f9d2d12e72 Running 53m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 53m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 50m v1.28.9+rke2r1
"availableReplicas": 4,
"observedGeneration": 2,
"phase": "Running",
"readyReplicas": 4,
"replicas": 4,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 4
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-9xxq7 management-cluster management-cluster-md-md0-2a55c05a1b-pnml9 openstack:///27ef1b52-debc-4699-9bc6-904cf68fd25a Deleting 50m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-gvxlg management-cluster management-cluster-md-md0-2a55c05a1b-mvwhb openstack:///c81e1c91-749c-4594-871e-37f9d2d12e72 Running 55m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 54m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 51m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster Provisioning 1s v1.28.9+rke2r1
"availableReplicas": 4,
"observedGeneration": 3,
"phase": "Running",
"readyReplicas": 4,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 1
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-gvxlg management-cluster management-cluster-md-md0-2a55c05a1b-mvwhb openstack:///c81e1c91-749c-4594-871e-37f9d2d12e72 Running 55m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 55m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 51m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Provisioning 2s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster Provisioning 14s v1.28.9+rke2r1
"availableReplicas": 3,
"conditions": [
{
"lastTransitionTime": "2024-06-20T15:48:14Z",
"status": "True",
"type": "Ready"
},
{
"lastTransitionTime": "2024-06-20T15:48:14Z",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 2
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-gvxlg management-cluster management-cluster-md-md0-2a55c05a1b-mvwhb openstack:///c81e1c91-749c-4594-871e-37f9d2d12e72 Running 57m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 57m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 54m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 2m25s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 2m37s v1.28.9+rke2r1
"availableReplicas": 4,
"observedGeneration": 3,
"phase": "Running",
"readyReplicas": 4,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 2
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 58m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 54m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 3m7s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster Provisioning 3s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 3m19s v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 3
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 60m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 57m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 5m34s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Provisioned 2m30s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 5m46s v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 3
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 60m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 57m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 5m36s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 2m32s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 5m48s v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 3
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-hrmf4 management-cluster management-cluster-md-md0-2a55c05a1b-7g672 openstack:///904d27f5-0900-4d61-bd7e-5b0f063d87ed Running 61m v1.28.9+rke2r1
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 57m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 6m12s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 3m8s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 6m24s v1.28.9+rke2r1
{
"availableReplicas": 4,
"observedGeneration": 3,
"phase": "Running",
"readyReplicas": 4,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 3
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 58m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 6m40s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster Provisioning 4s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 3m36s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 6m52s v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Running 61m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 9m33s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 2m57s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 6m29s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 9m45s v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 2,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-7x4x2-zd4w7 management-cluster management-cluster-md-md0-2a55c05a1b-t7jd2 openstack:///72aec881-77c2-48e7-9d60-8ba021c3dca4 Deleting 62m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 10m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 3m45s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 7m17s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 10m v1.28.9+rke2r1
{
"availableReplicas": 4,
"observedGeneration": 3,
"phase": "Running",
"readyReplicas": 4,
"replicas": 5,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Failed 10m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 3m54s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 7m26s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 10m v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 4,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 1,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-nfwbp-6wqb5 management-cluster Deleting 12m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 5m49s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 9m21s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 12m v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 4,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 1,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-nfwbp-5x69r management-cluster Provisioning 5s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 5m54s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 9m26s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 12m v1.28.9+rke2r1
{
"availableReplicas": 3,
"observedGeneration": 3,
"phase": "ScalingUp",
"readyReplicas": 3,
"replicas": 4,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 1,
"updatedReplicas": 4
}
NAMESPACE NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
sylva-system management-cluster-md0-nfwbp-5x69r management-cluster management-cluster-md-md0-2a55c05a1b-jjwhg openstack:///1120c725-1b19-4d33-9c55-01b58bc277e4 Running 3m22s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-l726l management-cluster management-cluster-md-md0-2a55c05a1b-zbbb5 openstack:///1f7abab3-0a5f-40ce-8dad-e60fefe21a5c Running 9m11s v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-rpqkk management-cluster management-cluster-md-md0-2a55c05a1b-xd94c openstack:///4eeb3cbb-98fc-424c-9dba-f419fe936ffd Running 12m v1.28.9+rke2r1
sylva-system management-cluster-md0-nfwbp-tzvpf management-cluster management-cluster-md-md0-2a55c05a1b-5b7qb openstack:///719d847f-f52e-4fe7-a167-ff357619b1b9 Running 15m v1.28.9+rke2r1
{
"availableReplicas": 4,
"observedGeneration": 3,
"phase": "Running",
"readyReplicas": 4,
"replicas": 4,
"selector": "cluster.x-k8s.io/cluster-name=management-cluster",
"unavailableReplicas": 0,
"updatedReplicas": 4
}
Rolling upgrade on controlplane. \
As explained in #1412 (closed), with capm3 (and maxSurge set to 0), when the last node is being deleted (DELETING
state), it causes the rke2controlplane updatedReplicas counter to be incremented. As a consequence rke2ControlPlane move to the "Ready" state (It thinks everything is OK as spec.replicas == status.updatedReplicas).
The solution provided is to wait/confirm that updatedReplicas
is equal to spec.replicas
and then check that a machine is currently deleting.
If yes, we wait for it to be deleted and then wait one last time until .spec.replicas == updatedReplicas. \
kubectl logs cluster-machines-ready-sylva-system-s785l -n kube-job
--- waiting for control plane to be Ready...
rke2controlplane.controlplane.cluster.x-k8s.io/management-cluster-control-plane condition met
--- waiting for controlplane replicas ...
rke2controlplane.controlplane.cluster.x-k8s.io/management-cluster-control-plane condition met
--- waiting for controlplane updatedreplicas ...
rke2controlplane.controlplane.cluster.x-k8s.io/management-cluster-control-plane condition met
--- waiting for controlplane machines with deletionTimestamp to be deleted... <<<<<<<<<<<<<<<<<<<<
machine.cluster.x-k8s.io/management-cluster-control-plane-6l42w condition met
--- waiting for controlplane updatedreplicas ...
rke2controlplane.controlplane.cluster.x-k8s.io/management-cluster-control-plane condition met
--- waiting for all machines to be Ready...
machine.cluster.x-k8s.io/management-cluster-control-plane-f5jkj condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-grdbv condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-kxnhn condition met
--- waiting for all machines to have a provider id (unset if the Node has not joined the cluster)...
machine.cluster.x-k8s.io/management-cluster-control-plane-f5jkj condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-grdbv condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-kxnhn condition met
--- waiting for all machines to have a nodeRef (reflects the fact that the Node has joined the cluster)...
machine.cluster.x-k8s.io/management-cluster-control-plane-f5jkj condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-grdbv condition met
machine.cluster.x-k8s.io/management-cluster-control-plane-kxnhn condition met
--- summary of resources
NAME AGE
management-cluster-control-plane 109m
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
management-cluster-control-plane-f5jkj management-cluster management-cluster-cp-de23bf3763-b77hr openstack:///6a1131ea-118c-4b58-ab2b-d33be121064a Running 4m52s v1.28.9
management-cluster-control-plane-grdbv management-cluster management-cluster-cp-de23bf3763-2wq7w openstack:///3e320ae0-ce7e-47fe-a36c-0a2bcc453f83 Running 18m v1.28.9
management-cluster-control-plane-kxnhn management-cluster management-cluster-cp-de23bf3763-lrsrf openstack:///f2534d12-6802-4916-a65d-805559d990db Running 11m v1.28.9