Extend the CI coverage to test MetalLB BGP

We are hitting issues related to workload cluster update that have not been detected in CI since the scenarios currently tested in CI are too "simple" and we should add some more complex (but common) networking scenarios using secondary interfaces on nodes and MetalLB/BGP (on both CAPO and CAPM3).

Note: testing completely MetalLB/BGP in CI won't be easy (need to get a BGP peer establish sessions) but the starting point would be to verify if a configuration won't lead to issue (during upgrade or fresh install of a workload cluster) without requiring a full BGP test.

example of values that could be included (CAPO):

cluster:
  control_plane:
    network_interfaces:
      secondary:
        network:
          id:  # secondary network.id for control nodes 
        vnicType: normal
        allowedAddressPairs:
        - ipAddress: 192.168.1.1

  machine_deployments:
    md0:
      network_interfaces:
        secondary:
          network:
            id:  # secondary network.id for md nodes
          vnicType: normal

  metallb:
    bgp_lbs:
      l3_options:
        bgp_peers:
          my_peer:
            local_asn: 64511
            peer_asn: 64510
            peer_address: 10.20.30.1 # arbitrary address if peer does not really exist
            advertised_pools:
              - pool1         
            receive_routes:
              mode: all
      address_pools:
        pool1:
          addresses:
            - 192.168.1.1-192.168.1.2
Assignee Loading
Time tracking Loading