[CAPO] Failed to create management-cluster-control-plane with plain openstack image ubuntu 20.04
Summary
When using the plain Ubuntu 20.04 to create the management-cluster-control-plane, it failed with reasons 'WaitingForKubeadmInit' and 'ScalingUp'. The OpenStack server is running but with some errors in the log.
related references
Details
The image used to create the management-cluster-control-plane is downloaded from https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img.
The management-cluster-control-plane is pending.
Status of management-cluster-control-plane
# kubectl describe KubeadmControlPlane management-cluster-control-plane
<...>
Status:
Conditions:
Last Transition Time: 2023-09-12T03:29:10Z
Message: Scaling up control plane to 3 replicas (actual 1)
Reason: ScalingUp
Severity: Warning
Status: False
Type: Ready
Last Transition Time: 2023-09-12T03:10:44Z
Reason: WaitingForKubeadmInit
Severity: Info
Status: False
Type: Available
Last Transition Time: 2023-09-12T03:10:43Z
Status: True
Type: CertificatesAvailable
Last Transition Time: 2023-09-12T03:29:10Z
Status: True
Type: MachinesReady
Last Transition Time: 2023-09-12T03:10:47Z
Message: Scaling up control plane to 3 replicas (actual 1)
Reason: ScalingUp
Severity: Warning
Status: False
Type: Resized
Observed Generation: 2
Replicas: 1
Selector: cluster.x-k8s.io/cluster-name=management-cluster,cluster.x-k8s.io/control-plane
Unavailable Replicas: 1
Updated Replicas: 1
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ControlPlaneUnhealthy 2m29s (x91 over 5h35m) kubeadm-control-plane-controller Waiting for control plane to pass preflight checks to continue reconciliation: [machine management-cluster-control-plane-gdp47 does not have APIServerPodHealthy condition, machine management-cluster-control-plane-gdp47 does not have ControllerManagerPodHealthy condition, machine management-cluster-control-plane-gdp47 does not have SchedulerPodHealthy condition, machine management-cluster-control-plane-gdp47 does not have EtcdPodHealthy condition, machine management-cluster-control-plane-gdp47 does not have EtcdMemberHealthy condition]The openstack server is running but there are some Errors found in the logs.
management-cluster server info
# openstack server list
+--------------------------------------+----------------------------------------+--------+-----------------------+------------------+----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+----------------------------------------+--------+-----------------------+------------------+----------+
| 88f35cb2-f4e4-4ec2-b7ca-18718cf9b8d8 | management-cluster-cp-81b06af6d1-5xrpk | ACTIVE | public=172.29.249.161 | capo-ubuntu-2004 | m1.large |
+--------------------------------------+----------------------------------------+--------+-----------------------+------------------+----------+
# openstack server show management-cluster-cp-81b06af6d1-5xrpk
+-------------------------------------+----------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+----------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | aio1 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | aio1.openstack.local |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2023-09-12T03:29:01.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | public=172.29.249.161 |
| config_drive | |
| created | 2023-09-12T03:28:37Z |
| flavor | m1.large (8862aa73-aed8-4e21-9302-22ceeacf9e02) |
| hostId | 3e348e22e5390435041df791d447763fd026856b1c6292bc5a010aa8 |
| id | 88f35cb2-f4e4-4ec2-b7ca-18718cf9b8d8 |
| image | capo-ubuntu-2004 (4f34fc59-4212-42de-b52b-9b58bfe1861f) |
| key_name | sylva-keypair |
| name | management-cluster-cp-81b06af6d1-5xrpk |
| progress | 0 |
| project_id | 82d3218f79ab4f86b4f2081521624332 |
| properties | |
| security_groups | name='default' |
| | name='capo-management-cluster-security-group-ctrl-plane-sylva-admin' |
| | name='k8s-cluster-default-management-cluster-secgroup-controlplane' |
| status | ACTIVE |
| updated | 2023-09-12T03:29:01Z |
| user_id | b066e160a0e34c608d544179d6b3201f |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------------------------+Server Error Msg
[ 244.008910] cloud-init[1381]: * Applying /etc/sysctl.conf ...
[ 244.012345] cloud-init[1381]: fs.inotify.max_user_watches = 524288
[ 244.015210] cloud-init[1381]: fs.inotify.max_user_instances = 512
[ 244.018638] cloud-init[1381]: sed: can't read /etc/containerd/config.toml: No such file or directory
[ 244.082979] cloud-init[1381]: grep: /etc/containerd/config.toml: No such file or directory
[ 244.089832] cloud-init[1381]: /var/lib/cloud/instance/scripts/runcmd: 6: cannot create /etc/containerd/config.toml: Directory nonexistent
[ 244.232262] cloud-init[1381]: Failed to restart containerd.service: Unit containerd.service not found.
[ 250.222054] cloud-init[1381]: Failed to restart containerd.service: Unit containerd.service not found.
[ 250.233483] cloud-init[1381]: /var/lib/cloud/instance/scripts/runcmd: 10: kubeadm: not found