Refactor metallb-resources helm chart to enhance BGP features
Summary
Current metallb-resources Helm values schema is limiting advanced BGP MetalLB functionalities for BGPAdvertisement, because a single BGPAdvertisement object is created per BGP peer. Although this is a working setup in many topologies, being able to support more MetalLB BGP features in cluster definition requires a redesign of the metallb-resources helm chart.
Also, take the opportunity to introduce the ability to specify vrf and ebgp_multihop for MetalLB BGP peers, for experimental use.
Proposed changes
Targeted new features for BGPAdvertisement
root@vlado-jump:~/git_folder/sylva-core# k explain bgpadvertisements.metallb.io.spec
GROUP: metallb.io
KIND: BGPAdvertisement
VERSION: v1beta1
FIELD: spec <Object>
DESCRIPTION:
BGPAdvertisementSpec defines the desired state of BGPAdvertisement.
FIELDS:
aggregationLength <integer>
The aggregation-length advertisement option lets you “roll up” the /32s into
a larger prefix. Defaults to 32. Works for IPv4 addresses.
communities <[]string>
The BGP communities to be associated with the announcement. Each item can be
a standard community of the
form 1234:1234, a large community of the form large:1234:1234:1234 or the
name of an alias defined in the
Community CRD.
localPref <integer>
The BGP LOCAL_PREF attribute which is used by BGP best path algorithm,
Path with higher localpref is preferred over one with lower localpref.
nodeSelectors <[]Object>
NodeSelectors allows to limit the nodes to announce as next hops for the
LoadBalancer IP. When empty, all the nodes having are announced as next
hops.
peers <[]string>
Peers limits the bgppeer to advertise the ips of the selected pools to.
When empty, the loadbalancer IP is announced to all the BGPPeers configured.Furthermore, named communities should be supported:
apiVersion: metallb.io/v1beta1
kind: Community
metadata:
name: communities
namespace: metallb-system
spec:
communities:
- name: vpn-only
value: 1234:1
- name: NO_ADVERTISE
value: 65535:65282And named communities could be used inside BGPAdvertisements:
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
name: local
namespace: metallb-system
spec:
ipAddressPools:
- first-pool
communities:
- vpn-onlyProposed metallb-resources values schema change under metallb.bgp_lbs.l3_options:
| Item | Change | Observation |
|---|---|---|
update bgp_peers |
remove advertised_pools under <bgp_peer_name> |
bgp peers will be mentioned under bgp_advertisements.<bgp_adv_name>.peers |
add bgp_advertisements |
define named BGPAdvertisements | see next table |
add communities |
collection of aliases for communities | see example above |
Proposed metallb-resources values schema change under metallb.bgp_lbs.l3_options.bgp_advertisement.<bgp_advertisement_name>:
| Item | Change | Observation |
|---|---|---|
| aggregationLength | optional | The aggregation-length advertisement option lets you “roll up” the /32s into a larger prefix. Defaults to 32 |
| localPref | optional | The BGP LOCAL_PREF attribute which is used by BGP best path algorithm, path with higher localpref is preferred over one with lower localpref. |
| nodeSelectors | optional | NodeSelectors allows to limit the nodes to announce as next hops for the LoadBalancer IP. When empty, all the nodes having are announced as next hops. |
| peers | mandatory | Peers limits the bgppeer to advertise the ips of the selected pools to. When empty, the loadbalancer IP is announced to all the BGPPeers configured. |
| communities | optional | The BGP communities to be associated with the announcement. Each item can be a standard community of the form 1234:1234, a large community of the form large:4534:6233:1234 or the name of an alias defined in the Community CRD. |
| advertised_pools | mandatory | The list of IPAddressPools to advertise via this advertisement, selected by name |
Example of new Metallb BGP definition, comments are showing the proposed evolution:
metallb:
bgp_lbs:
address_pools:
pool1:
addresses:
- 192.168.1.1-192.168.1.2
pool2:
addresses:
- 192.168.2.1-192.168.2.2
l3_options:
bfd_profiles:
bfd-profile-bgp:
receiveInterval: 1000
transmitInterval: 1000
detectMultiplier: 3
bgp_peers:
mx1:
local_asn: 64513
peer_asn: 64512
peer_address: 172.30.140.2
password: bgp-peer-password
bfd_profile: bfd-profile-bgp
node_selectors: # should be adapted to changed for consistency with bgp_advertisements
- matchLabels:
kubernetes.io/os: linux
# advertised_pools: # removed in this proposed change
# - pool1 # removed in this proposed change
# - pool2 # removed in this proposed change
receive_routes:
mode: all
mx2:
local_asn: 64513
peer_asn: 64512
peer_address: 172.30.140.3
password: bgp-peer-password
bfd_profile: bfd-profile-bgp
node_selectors: # should be adapted to changed for consistency with bgp_advertisements
- matchLabels:
kubernetes.io/os: linux
# advertised_pools: # removed in this proposed change
# - pool1 # removed in this proposed change
# - pool2 # removed in this proposed change
receive_routes:
prefixes:
- prefix: 10.200.200.1/32
- prefix: 2.2.2.0/24
ge: 30
le: 30
communities: # new - optional
- name: vpn1-comm
value: 1234:1
- name: vpn2-comm
value: 1234:2
bgp_advertisements: # new - mandatory
bgp-adv1: # new - mandatory
advertised_pools: # new - mandatory
- pool1
peers: # new - mandatory
- mx1
- mx2
aggregationLength: 32 # new - optional
communities: # new - optional
- 64512:100
- vpn1-comm
localPref: 200 # new - optional
nodeSelectors: # new - optional
- matchLabels:
kubernetes.io/os: linux
bgp-adv2: # new - mandatory
advertised_pools: # new - mandatory
- pool2
peers: # new - mandatory
- mx1
- mx2
aggregationLength: 32 # new - optional
communities: # new - optional
- 64512:200
- vpn2-comm
localPref: 400 # new - optional
nodeSelectors: # new - optional
- matchLabels:
kubernetes.io/os: linuxTargeted new features for BGPPeer
- Introduce the ability to specify
vrffor MetalLB BGPPeers, for experimental use:
$ k explain bgppeers.metallb.io.spec.vrf
GROUP: metallb.io
KIND: BGPPeer
VERSION: v1beta2
FIELD: vrf <string>
DESCRIPTION:
To set if we want to peer with the BGPPeer using an interface belonging to
a host vrfNote: could be used in the future for symmetric routing when exposing LB services on secondary networks similar to: https://docs.okd.io/4.19/networking/ingress_load_balancing/metallb/metallb-configure-return-traffic.html#nw-metallb-configure-return-traffic-proc_metallb-configure-return-traffic
- Introduce the ability to define BGP peerings with multi-hop enabled for external BGP.
$ k explain bgppeers.metallb.io.spec.ebgpMultiHop
GROUP: metallb.io
KIND: BGPPeer
VERSION: v1beta2
FIELD: ebgpMultiHop <boolean>
DESCRIPTION:
To set if the BGPPeer is multi-hops away. Needed for FRR mode only.Prepare and test migration for production
We need to implement the change in a manner which will avoid or reduce to minimum the LB services impact on running clusters. A solution could be to not delete the legacy bgp_advertisements and clean them manually after cluster is upgraded to the new metallb-resources Helm chart version.