Skip to content

Unable to create new cluster with fresh virtual environment: AttributeError: 'builtins.Certificate' object has no attribute '_backend'

For a while now, I am unable to create a fresh cluster. Here is all the information. Any clues appreciated.

Error description

When building the k8s control plane, the ansible run crashes with a AttributeError: 'builtins.Certificate' object has no attribute '_backend' in the k8s-master : Get certificate information task.

Full ansible error 
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'builtins.Certificate' object has no attribute '_backend'
fatal: [managed-k8s-master-2 -> localhost]: FAILED! => changed=false 
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.6903296-71536-6374088286146/AnsiballZ_x509_certificate_info.py", line 107, in <module>
        _ansiballz_main()
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.6903296-71536-6374088286146/AnsiballZ_x509_certificate_info.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.6903296-71536-6374088286146/AnsiballZ_x509_certificate_info.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.x509_certificate_info', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins.modules.x509_certificate_info', _modlib_path=modlib_path),
      File "/usr/lib/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_iucsjcu9/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 452, in <module>
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_iucsjcu9/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 436, in main
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_iucsjcu9/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/certificate_info.py", line 228, in get_info
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_iucsjcu9/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/certificate_info.py", line 381, in _get_all_extensions
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_iucsjcu9/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/cryptography_support.py", line 71, in cryptography_get_extensions_from_cert
    AttributeError: 'builtins.Certificate' object has no attribute '_backend'
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'builtins.Certificate' object has no attribute '_backend'
fatal: [managed-k8s-master-1 -> localhost]: FAILED! => changed=false
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.5657644-71523-190223792788927/AnsiballZ_x509_certificate_info.py", line 107, in <module>
        _ansiballz_main()
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.5657644-71523-190223792788927/AnsiballZ_x509_certificate_info.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/jssfr/.ansible/tmp/ansible-tmp-1647849077.5657644-71523-190223792788927/AnsiballZ_x509_certificate_info.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.x509_certificate_info', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins.modules.x509_certificate_info', _modlib_path=modlib_path),
      File "/usr/lib/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_cfznqq2g/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 452, in <module>
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_cfznqq2g/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 436, in main
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_cfznqq2g/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/certificate_info.py", line 228, in get_info
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_cfznqq2g/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/certificate_info.py", line 381, in _get_all_extensions
      File "/tmp/ansible_community.crypto.x509_certificate_info_payload_cfznqq2g/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/cryptography_support.py", line 71, in cryptography_get_extensions_from_cert
    AttributeError: 'builtins.Certificate' object has no attribute '_backend'
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

How to reproduce

config.toml 
[terraform]
subnet_cidr = "172.30.80.0/24"
subnet_v6_cidr = "fd53:851c:6b8d::/120"
keypair = "jonasschaefer-antares-20190912"
workers = 3
masters = 3
gateways = 2
gateway_image_name = "my-fancy-debian-10"
worker_flavors = [ "L", "L", "L",]
enable_az_management = false
dualstack_support = false

[load-balancing]
openstack_lbaas = false
lb_ports = [ 30060,]

[ch-k8s-lbaas]
enabled = true
shared_secret = <redacted>
version = "0.3.1"
agent_port = 15203

[kubernetes]
version = "1.21.4"
use_podsecuritypolicies = false
is_gpu_cluster = false

[kubernetes.apiserver]
frontend_port = 8888

[node-scheduling]
scheduling_key_prefix = "scheduling.mk8s.cloudandheat.com"

[wireguard]
rollout_company_users = true
ip_cidr = "172.30.81.0/28"
ip_gw = "172.30.81.1/28"
ipv6_cidr = "fd53:851c:6b8d::100/120"
ipv6_gw = "fd53:851c:6b8d::101/120"
peers = []
s2s_enabled = true
s2s_ip = "169.254.0.1"
s2s_peer_ip = "169.254.0.2"
s2s_transfer_subnet = "169.254.0.0/29"
s2s_port = 2342
s2s_peer_pub_key = "yD61gKzhXcPEK0pBEcpnG8ZmVnmGB9Iv85F4g67f11E="
s2s_peer_public_endpoint = "0.0.0.0:2342"
s2s_bgp_as = 65010
s2s_peer_bgp_as = 65009
port = 48793

[ipsec]
enabled = true
proposals = [ "aes256-sha256-modp2048",]
esp_proposals = "{{ ipsec_proposals }}"
peer_networks = [ "192.168.255.0/24",]
local_networks = [ "{{ subnet_cidr }}",]
virtual_subnet_pool = false
remote_addrs = [ "0.0.0.0",]
remote_name = "dummy"

[passwordstore]
rollout_company_users = true

[cah-users]
rollout = true

[miscellaneous]
wireguard_on_workers = false

[load-balancing.priorities]
managed-k8s-gw-az1 = 150
managed-k8s-gw-az2 = 100
managed-k8s-gw-az3 = 50

[kubernetes.storage]
rook_enabled = true
nodeplugin_toleration = true

[kubernetes.local_storage.static]
enabled = true
storageclass_name = "local-storage"

[kubernetes.local_storage.dynamic]
enabled = true
storageclass_name = "local-storage-dynamic"

[kubernetes.monitoring]
enabled = true

[kubernetes.global_monitoring]
enabled = true
nodeport = 31911
nodeport_name = "ch-k8s-global-monitoring"

[kubernetes.continuous_join_key]
enabled = false

[kubernetes.network]
pod_subnet = "10.244.0.0/16"
service_subnet = "10.96.0.0/12"
plugin = "calico"
bgp_worker_as = 64520

[k8s-service-layer.rook]
enabled = true
namespace = "rook-ceph"
cluster_name = "rook-ceph"
version = "v1.6.7"
nosds = 3
osd_volume_size = "90Gi"
toolbox = true
ceph_fs = true
nodeplugin_toleration = "{{ k8s_storage_nodeplugin_toleration }}"
scheduling_key = false
operator_cpu_limit = "1"
operator_cpu_request = "1"
mds_memory_limit = "4Gi"
mds_memory_request = "1Gi"
mds_cpu_limit = "1"
mds_cpu_request = "500m"
mon_cpu_limit = "500m"
mon_cpu_request = "100m"

[[k8s-service-layer.rook.pools]]
name = "data"
create_storage_class = "block"
replicated = 3

[k8s-service-layer.prometheus]
use_thanos = true

[k8s-service-layer.cert-manager]
enabled = true

[k8s-service-layer.ingress]
enabled = true

[node-scheduling.labels]

[node-scheduling.taints]
managed-k8s-master-0 = []
managed-k8s-master-1 = []

[testing.test-nodes]
managed-k8s-worker-0 = "worker0"
managed-k8s-worker-1 = "worker1"
managed-k8s-worker-2 = "worker2"

[passwordstore.additional_users]
  1. Create a fresh virtual environment
  2. Check out most recent devel in managed-k8s
  3. pip install -r managed-k8s/requirements.txt
  4. Run managed-k8s/actions/apply.sh

This reproduces in 100% of the cases for me.

Additional information

pip freeze 
ansible==5.3.0
ansible-core==2.12.3
appdirs==1.4.4
attrs==21.4.0
autopage==0.5.0
cachetools==5.0.0
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.12
cliff==3.10.1
cmd2==2.4.0
cryptography==36.0.2
debtcollector==2.5.0
decorator==5.1.1
dogpile.cache==1.1.5
google-auth==2.6.2
idna==3.3
iso8601==1.0.2
Jinja2==3.0.3
jmespath==1.0.0
jsonpatch==1.32
jsonpointer==2.2
jsonschema==4.4.0
keystoneauth1==4.5.0
kubernetes==23.3.0
kubernetes-validate==1.23.1
loguru==0.6.0
MarkupSafe==2.1.1
mergedeep==1.3.4
msgpack==1.0.3
munch==2.5.0
netaddr==0.8.0
netifaces==0.11.0
oauthlib==3.2.0
openshift==0.13.1
openstacksdk==0.61.0
os-service-types==1.7.0
osc-lib==2.5.0
oslo.config==8.8.0
oslo.i18n==5.1.0
oslo.serialization==4.3.0
oslo.utils==4.12.2
packaging==21.3
pbr==5.8.1
prettytable==3.2.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pyparsing==3.0.7
pyperclip==1.8.2
pyrsistent==0.18.1
python-cinderclient==8.3.0
python-dateutil==2.8.2
python-keystoneclient==4.4.0
python-novaclient==17.7.0
python-openstackclient==5.8.0
python-string-utils==1.0.0
pytz==2022.1
PyYAML==6.0
requests==2.27.1
requests-oauthlib==1.3.1
requestsexceptions==1.4.0
resolvelib==0.5.4
rfc3986==2.0.0
rsa==4.8
simplejson==3.17.6
six==1.16.0
stevedore==3.5.0
toml==0.10.2
urllib3==1.26.9
wcwidth==0.2.5
websocket-client==1.3.1
wrapt==1.14.0
Python and ansible versions 
$ python --version                                                                                                                                                 
Python 3.9.10
$ ansible --version                                                                                                                                                
ansible [core 2.12.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/jssfr/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jssfr/Work/ch/managed-k8s-clusters/cah-k8s-dev-flyingdutchman-cluster/.direnv/python-3.9.10/lib/python3.9/site-packages/ansible
  ansible collection location = /home/jssfr/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jssfr/Work/ch/managed-k8s-clusters/cah-k8s-dev-flyingdutchman-cluster/.direnv/python-3.9.10/bin/ansible
  python version = 3.9.10 (main, Jan 16 2022, 17:12:18) [GCC 11.2.0]
  jinja version = 3.0.3
  libyaml = True
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information