Chapter 31. Load balancing with MetalLB

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators OperatorHub.

2. Type a keyword into the Filter by keyword box or scroll to find the Operator you want. For example, type

   metallb

   to find the MetalLB Operator.

   You can also filter options by Infrastructure Features. For example, select Disconnected if you want to see Operators that work in disconnected environments, also known as restricted network environments.

3. On the Install Operator page, accept the defaults and click Install.

Verification

1. To confirm that the installation is successful:

   1. Navigate to the Operators Installed Operators page.
   2. Check that the Operator is installed in the

      openshift-operators

      namespace and that its status is

      Succeeded

      .

2. If the Operator is not installed successfully, check the status of the Operator and review the logs:

   1. Navigate to the Operators Installed Operators page and inspect the

      Status

      column for any errors or failures.
   2. Navigate to the Workloads Pods page and check the logs in any pods in the

      openshift-operators

      project that are reporting issues.

Procedure

1. Create a namespace for the MetalLB Operator by entering the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Namespace
   metadata:
     name: metallb-system
   EOF

2. Create an Operator group custom resource (CR) in the namespace:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: metallb-operator
     namespace: metallb-system
   EOF

3. Confirm the Operator group is installed in the namespace:

   $ oc get operatorgroup -n metallb-system

   Example output

   NAME               AGE
   metallb-operator   14m

4. Create a

   Subscription

   CR:

   1. Define the

      Subscription

      CR and save the YAML file, for example,

      metallb-sub.yaml

      :

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: metallb-operator-sub
        namespace: metallb-system
      spec:
        channel: stable
        name: metallb-operator
        source: redhat-operators 

      1

      
        sourceNamespace: openshift-marketplace

      1

      You must specify the redhat-operators value.

   2. To create the

      Subscription

      CR, run the following command:

      $ oc create -f metallb-sub.yaml

5. Optional: To ensure BGP and BFD metrics appear in Prometheus, you can label the namespace as in the following command:

   $ oc label ns metallb-system "openshift.io/cluster-monitoring=true"

1. Confirm the install plan is in the namespace:

   $ oc get installplan -n metallb-system

   Example output

   NAME            CSV                                   APPROVAL    APPROVED
   install-wzg94   metallb-operator.4.12.0-nnnnnnnnnnnn   Automatic   true

   Note

   Installation of the Operator might take a few seconds.

2. To verify that the Operator is installed, enter the following command:

   $ oc get clusterserviceversion -n metallb-system \
     -o custom-columns=Name:.metadata.name,Phase:.status.phase

   Example output

   Name                                  Phase
   metallb-operator.4.12.0-nnnnnnnnnnnn   Succeeded

1. Create a single instance of a MetalLB custom resource:

   $ cat << EOF | oc apply -f -
   apiVersion: metallb.io/v1beta1
   kind: MetalLB
   metadata:
     name: metallb
     namespace: metallb-system
   EOF

1. Verify that the deployment for the controller is running:

   $ oc get deployment -n metallb-system controller

   Example output

   NAME         READY   UP-TO-DATE   AVAILABLE   AGE
   controller   1/1     1            1           11m

2. Verify that the daemon set for the speaker is running:

   $ oc get daemonset -n metallb-system speaker

   Example output

   NAME      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
   speaker   6         6         6       6            6           kubernetes.io/os=linux   18m

   The example output indicates 6 speaker pods. The number of speaker pods in your cluster might differ from the example output. Make sure the output indicates one pod for each node in your cluster.

Procedure

1. Get the YAML definition of the

   metallb-operator

   subscription in the

   metallb-system

   namespace by entering the following command:

   $ oc -n metallb-system get subscription metallb-operator -o yaml

2. Edit the

   Subscription

   CR by setting the

   installPlanApproval

   parameter to

   Manual

   :

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: metallb-operator
     namespace: metallb-system
   # ...
   spec:
      channel: stable
      installPlanApproval: Manual
      name: metallb-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
   # ...

3. Find the latest OpenShift Container Platform 4.12 version of the MetalLB Operator by entering the following command:

   $ oc -n metallb-system get csv

   Example output

   NAME                       DISPLAY            VERSION    REPLACES    PHASE
   metallb-operator.v4.12.0   MetalLB Operator   4.12.0                 Succeeded

4. Check the install plan that exists in the namespace by entering the following command.

   $ oc -n metallb-system get installplan

   Example output that shows install-tsz2g as a manual install plan

   NAME            CSV                                     APPROVAL    APPROVED
   install-shpmd   metallb-operator.v4.12.0-202502261233   Automatic   true
   install-tsz2g   metallb-operator.v4.12.0-202503102139   Manual      false

5. Edit the install plan that exists in the namespace by entering the following command. Ensure that you replace

   <name_of_installplan>

   with the name of the install plan, such as

   install-tsz2g

   .

   $ oc edit installplan <name_of_installplan> -n metallb-system

   1. With the install plan open in your editor, set the

      spec.approval

      parameter to

      Manual

      and set the

      spec.approved

      parameter to

      true

      .
      Note

      After you edit the install plan, the upgrade operation starts. If you enter the

      oc -n metallb-system get csv

      command during the upgrade operation, the output might show the

      Replacing

      or the

      Pending

      status.

Verification

1. Verify the upgrade was successful by entering the following command:

   $ oc -n metallb-system get csv

   Example output

   NAME                                        DISPLAY             VERSION              REPLACE                                 PHASE
   metallb-operator.v<latest>.0-202503102139   MetalLB Operator    {product-version}.0-202503102139  metallb-operator.v{product-version}.0-202502261233   Succeeded

Procedure

1. Create a file, such as

   ipaddresspool.yaml

   , with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     namespace: metallb-system
     name: doc-example
     labels: 

   1

   
       zone: east
   spec:
     addresses:
     - 203.0.113.1-203.0.113.10
     - 203.0.113.65-203.0.113.75
   # ...

   1

   This label assigned to the IPAddressPool can be referenced by the ipAddressPoolSelectors in the BGPAdvertisement CRD to associate the IPAddressPool with the advertisement.

2. Apply the configuration for the IP address pool:

   $ oc apply -f ipaddresspool.yaml

Verification

1. View the address pool by entering the following command:

   $ oc describe -n metallb-system IPAddressPool doc-example

   Example output

   Name:         doc-example
   Namespace:    metallb-system
   Labels:       zone=east
   Annotations:  <none>
   API Version:  metallb.io/v1beta1
   Kind:         IPAddressPool
   Metadata:
     ...
   Spec:
     Addresses:
       203.0.113.1-203.0.113.10
       203.0.113.65-203.0.113.75
     Auto Assign:  true
   Events:         <none>

2. Confirm that the address pool name, such as

   doc-example

   , and the IP address ranges exist in the output.

Procedure

1. Create an IP address pool by using a custom resource:

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     namespace: metallb-system
     name: pool1
   spec:
     addresses:
       - 4.4.4.100-4.4.4.200
       - 2001:100:4::200-2001:100:4::400

2. Control which nodes in the cluster the IP address from

   pool1

   advertises from by defining the

   .spec.nodeSelector

   value in the BGPAdvertisement custom resource:

   apiVersion: metallb.io/v1beta1
   kind: BGPAdvertisement
   metadata:
     name: example
   spec:
     ipAddressPools:
     - pool1
     nodeSelector:
     - matchLabels:
         kubernetes.io/hostname: NodeA
     - matchLabels:
         kubernetes.io/hostname: NodeB

Procedure

1. Create an IP address pool.

   1. Create a file, such as

      ipaddresspool.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2
      spec:
        addresses:
          - 4.4.4.0/24
        autoAssign: false

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement.

   1. Create a file, such as

      l2advertisement.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement
        namespace: metallb-system
      spec:
        ipAddressPools:
         - doc-example-l2

   2. Apply the configuration:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create an IP address pool.

   1. Create a file, such as

      ipaddresspool.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2-label
        labels:
          zone: east
      spec:
        addresses:
          - 172.31.249.87/32

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement advertising the IP using

   ipAddressPoolSelectors

   .

   1. Create a file, such as

      l2advertisement.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement-label
        namespace: metallb-system
      spec:
        ipAddressPoolSelectors:
          - matchExpressions:
              - key: zone
                operator: In
                values:
                  - east

   2. Apply the configuration:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create an IP address pool:

   1. Create a file, such as

      ipaddresspool.yaml

      , and enter the configuration details like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2
      spec:
        addresses:
          - 4.4.4.0/24
        autoAssign: false

   2. Apply the configuration for the IP address pool like the following example:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement advertising the IP with

   interfaces

   selector.

   1. Create a YAML file, such as

      l2advertisement.yaml

      , and enter the configuration details like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement
        namespace: metallb-system
      spec:
        ipAddressPools:
         - doc-example-l2
         interfaces:
         - interfaceA
         - interfaceB

   2. Apply the configuration for the advertisement like the following example:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create a file, such as

   bgppeer.yaml

   , with content like the following example:

   apiVersion: metallb.io/v1beta2
   kind: BGPPeer
   metadata:
     namespace: metallb-system
     name: doc-example-peer
   spec:
     peerAddress: 10.0.0.1
     peerASN: 64501
     myASN: 64500
     routerID: 10.10.10.10

2. Apply the configuration for the BGP peer:

   $ oc apply -f bgppeer.yaml

Procedure

1. Create address pool

   pool1

   .

   1. Create a file, such as

      ipaddresspool1.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: pool1
      spec:
        addresses:
          - 4.4.4.100-4.4.4.200
          - 2001:100:4::200-2001:100:4::400

   2. Apply the configuration for the IP address pool

      pool1

      :

      $ oc apply -f ipaddresspool1.yaml

2. Create address pool

   pool2

   .

   1. Create a file, such as

      ipaddresspool2.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: pool2
      spec:
        addresses:
          - 5.5.5.100-5.5.5.200
          - 2001:100:5::200-2001:100:5::400

   2. Apply the configuration for the IP address pool

      pool2

      :

      $ oc apply -f ipaddresspool2.yaml

3. Create BGP

   peer1

   .

   1. Create a file, such as

      bgppeer1.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: peer1
      spec:
        peerAddress: 10.0.0.1
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer:

      $ oc apply -f bgppeer1.yaml

4. Create BGP

   peer2

   .

   1. Create a file, such as

      bgppeer2.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: peer2
      spec:
        peerAddress: 10.0.0.2
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer2:

      $ oc apply -f bgppeer2.yaml

5. Create BGP advertisement 1.

   1. Create a file, such as

      bgpadvertisement1.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgpadvertisement-1
        namespace: metallb-system
      spec:
        ipAddressPools:
          - pool1
        peers:
          - peer1
        communities:
          - 65535:65282
        aggregationLength: 32
        aggregationLengthV6: 128
        localPref: 100

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement1.yaml

6. Create BGP advertisement 2.

   1. Create a file, such as

      bgpadvertisement2.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgpadvertisement-2
        namespace: metallb-system
      spec:
        ipAddressPools:
          - pool2
        peers:
          - peer2
        communities:
          - 65535:65282
        aggregationLength: 32
        aggregationLengthV6: 128
        localPref: 100

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement2.yaml

Procedure

1. Create an IP address pool.

   1. Create a file, such as

      ipaddresspool.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-bgp-community
      spec:
        addresses:
          - 203.0.113.200/30
          - fc00:f853:ccd:e799::/124

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a community alias named

   community1

   .

   apiVersion: metallb.io/v1beta1
   kind: Community
   metadata:
     name: community1
     namespace: metallb-system
   spec:
     communities:
       - name: NO_ADVERTISE
         value: '65535:65282'

3. Create a BGP peer named

   doc-example-bgp-peer

   .

   1. Create a file, such as

      bgppeer.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: doc-example-bgp-peer
      spec:
        peerAddress: 10.0.0.1
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer:

      $ oc apply -f bgppeer.yaml

4. Create a BGP advertisement with the community alias.

   1. Create a file, such as

      bgpadvertisement.yaml

      , with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgp-community-sample
        namespace: metallb-system
      spec:
        aggregationLength: 32
        aggregationLengthV6: 128
        communities:
          - NO_ADVERTISE 

      1

      
        ipAddressPools:
          - doc-example-bgp-community
        peers:
          - doc-example-peer

      1 1

      Specify the CommunityAlias.name here and not the community custom resource (CR) name.

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement.yaml

Procedure

1. Create a file, such as

   bfdprofile.yaml

   , with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: BFDProfile
   metadata:
     name: doc-example-bfd-profile-full
     namespace: metallb-system
   spec:
     receiveInterval: 300
     transmitInterval: 300
     detectMultiplier: 3
     echoMode: false
     passiveMode: true
     minimumTtl: 254

2. Apply the configuration for the BFD profile:

   $ oc apply -f bfdprofile.yaml

Procedure

1. Create a

   <service_name>.yaml

   file. In the file, ensure that the

   spec.type

   field is set to

   LoadBalancer

   .

   Refer to the examples for information about how to request the external IP address that MetalLB assigns to the service.

2. Create the service:

   $ oc apply -f <service_name>.yaml

   Example output

   service/<service_name> created

Procedure

1. Create a file, such as

   setdebugloglevel.yaml

   , with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: MetalLB
   metadata:
     name: metallb
     namespace: metallb-system
   spec:
     logLevel: debug
     nodeSelector:
       node-role.kubernetes.io/worker: ""

2. Apply the configuration:

   $ oc replace -f setdebugloglevel.yaml

   Note

   Use

   oc replace

   as the understanding is the

   metallb

   CR is already created and here you are changing the log level.

3. Display the names of the

   speaker

   pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME                    READY   STATUS    RESTARTS   AGE
   speaker-2m9pm           4/4     Running   0          9m19s
   speaker-7m4qw           3/4     Running   0          19s
   speaker-szlmx           4/4     Running   0          9m19s

   Note

   Speaker and controller pods are recreated to ensure the updated logging level is applied. The logging level is modified for all the components of MetalLB.

4. View the

   speaker

   logs:

   $ oc logs -n metallb-system speaker-7m4qw -c speaker

   Example output

   {"branch":"main","caller":"main.go:92","commit":"3d052535","goversion":"gc / go1.17.1 / amd64","level":"info","msg":"MetalLB speaker starting (commit 3d052535, branch main)","ts":"2022-05-17T09:55:05Z","version":""}
   {"caller":"announcer.go:110","event":"createARPResponder","interface":"ens4","level":"info","msg":"created ARP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:119","event":"createNDPResponder","interface":"ens4","level":"info","msg":"created NDP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:110","event":"createARPResponder","interface":"tun0","level":"info","msg":"created ARP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:119","event":"createNDPResponder","interface":"tun0","level":"info","msg":"created NDP responder for interface","ts":"2022-05-17T09:55:05Z"}
   I0517 09:55:06.515686      95 request.go:665] Waited for 1.026500832s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operators.coreos.com/v1alpha1?timeout=32s
   {"Starting Manager":"(MISSING)","caller":"k8s.go:389","level":"info","ts":"2022-05-17T09:55:08Z"}
   {"caller":"speakerlist.go:310","level":"info","msg":"node event - forcing sync","node addr":"10.0.128.4","node event":"NodeJoin","node name":"ci-ln-qb8t3mb-72292-7s7rh-worker-a-vvznj","ts":"2022-05-17T09:55:08Z"}
   {"caller":"service_controller.go:113","controller":"ServiceReconciler","enqueueing":"openshift-kube-controller-manager-operator/metrics","epslice":"{\"metadata\":{\"name\":\"metrics-xtsxr\",\"generateName\":\"metrics-\",\"namespace\":\"openshift-kube-controller-manager-operator\",\"uid\":\"ac6766d7-8504-492c-9d1e-4ae8897990ad\",\"resourceVersion\":\"9041\",\"generation\":4,\"creationTimestamp\":\"2022-05-17T07:16:53Z\",\"labels\":{\"app\":\"kube-controller-manager-operator\",\"endpointslice.kubernetes.io/managed-by\":\"endpointslice-controller.k8s.io\",\"kubernetes.io/service-name\":\"metrics\"},\"annotations\":{\"endpoints.kubernetes.io/last-change-trigger-time\":\"2022-05-17T07:21:34Z\"},\"ownerReferences\":[{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"name\":\"metrics\",\"uid\":\"0518eed3-6152-42be-b566-0bd00a60faf8\",\"controller\":true,\"blockOwnerDeletion\":true}],\"managedFields\":[{\"manager\":\"kube-controller-manager\",\"operation\":\"Update\",\"apiVersion\":\"discovery.k8s.io/v1\",\"time\":\"2022-05-17T07:20:02Z\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:addressType\":{},\"f:endpoints\":{},\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:endpoints.kubernetes.io/last-change-trigger-time\":{}},\"f:generateName\":{},\"f:labels\":{\".\":{},\"f:app\":{},\"f:endpointslice.kubernetes.io/managed-by\":{},\"f:kubernetes.io/service-name\":{}},\"f:ownerReferences\":{\".\":{},\"k:{\\\"uid\\\":\\\"0518eed3-6152-42be-b566-0bd00a60faf8\\\"}\":{}}},\"f:ports\":{}}}]},\"addressType\":\"IPv4\",\"endpoints\":[{\"addresses\":[\"10.129.0.7\"],\"conditions\":{\"ready\":true,\"serving\":true,\"terminating\":false},\"targetRef\":{\"kind\":\"Pod\",\"namespace\":\"openshift-kube-controller-manager-operator\",\"name\":\"kube-controller-manager-operator-6b98b89ddd-8d4nf\",\"uid\":\"dd5139b8-e41c-4946-a31b-1a629314e844\",\"resourceVersion\":\"9038\"},\"nodeName\":\"ci-ln-qb8t3mb-72292-7s7rh-master-0\",\"zone\":\"us-central1-a\"}],\"ports\":[{\"name\":\"https\",\"protocol\":\"TCP\",\"port\":8443}]}","level":"debug","ts":"2022-05-17T09:55:08Z"}

5. View the FRR logs:

   $ oc logs -n metallb-system speaker-7m4qw -c frr

   Example output

   Started watchfrr
   2022/05/17 09:55:05 ZEBRA: client 16 says hello and bids fair to announce only bgp routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 31 says hello and bids fair to announce only vnc routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 38 says hello and bids fair to announce only static routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 43 says hello and bids fair to announce only bfd routes vrf=0
   2022/05/17 09:57:25.089 BGP: Creating Default VRF, AS 64500
   2022/05/17 09:57:25.090 BGP: dup addr detect enable max_moves 5 time 180 freeze disable freeze_time 0
   2022/05/17 09:57:25.090 BGP: bgp_get: Registering BGP instance (null) to zebra
   2022/05/17 09:57:25.090 BGP: Registering VRF 0
   2022/05/17 09:57:25.091 BGP: Rx Router Id update VRF 0 Id 10.131.0.1/32
   2022/05/17 09:57:25.091 BGP: RID change : vrf VRF default(0), RTR ID 10.131.0.1
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF br0
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF ens4
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF ens4 addr 10.0.128.4/32
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF ens4 addr fe80::c9d:84da:4d86:5618/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF lo
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF ovs-system
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF tun0
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF tun0 addr 10.131.0.1/23
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF tun0 addr fe80::40f1:d1ff:feb6:5322/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth2da49fed
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth2da49fed addr fe80::24bd:d1ff:fec1:d88/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth2fa08c8c
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth2fa08c8c addr fe80::6870:ff:fe96:efc8/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth41e356b7
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth41e356b7 addr fe80::48ff:37ff:fede:eb4b/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth1295c6e2
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth1295c6e2 addr fe80::b827:a2ff:feed:637/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth9733c6dc
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth9733c6dc addr fe80::3cf4:15ff:fe11:e541/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth336680ea
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth336680ea addr fe80::94b1:8bff:fe7e:488c/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vetha0a907b7
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vetha0a907b7 addr fe80::3855:a6ff:fe73:46c3/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vethf35a4398
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vethf35a4398 addr fe80::40ef:2fff:fe57:4c4d/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vethf831b7f4
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vethf831b7f4 addr fe80::f0d9:89ff:fe7c:1d32/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vxlan_sys_4789
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vxlan_sys_4789 addr fe80::80c1:82ff:fe4b:f078/64
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] Timer (start timer expire).
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] BGP_Start (Idle->Connect), fd -1
   2022/05/17 09:57:26.094 BGP: Allocated bnc 10.0.0.1/32(0)(VRF default) peer 0x7f807f7631a0
   2022/05/17 09:57:26.094 BGP: sendmsg_zebra_rnh: sending cmd ZEBRA_NEXTHOP_REGISTER for 10.0.0.1/32 (vrf VRF default)
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] Waiting for NHT
   2022/05/17 09:57:26.094 BGP: bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 went from Idle to Connect
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] TCP_connection_open_failed (Connect->Active), fd -1
   2022/05/17 09:57:26.094 BGP: bgp_fsm_change_status : vrf default(0), Status: Active established_peers 0
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 went from Connect to Active
   2022/05/17 09:57:26.094 ZEBRA: rnh_register msg from client bgp: hdr->length=8, type=nexthop vrf=0
   2022/05/17 09:57:26.094 ZEBRA: 0: Add RNH 10.0.0.1/32 type Nexthop
   2022/05/17 09:57:26.094 ZEBRA: 0:10.0.0.1/32: Evaluate RNH, type Nexthop (force)
   2022/05/17 09:57:26.094 ZEBRA: 0:10.0.0.1/32: NH has become unresolved
   2022/05/17 09:57:26.094 ZEBRA: 0: Client bgp registers for RNH 10.0.0.1/32 type Nexthop
   2022/05/17 09:57:26.094 BGP: VRF default(0): Rcvd NH update 10.0.0.1/32(0) - metric 0/0 #nhops 0/0 flags 0x6
   2022/05/17 09:57:26.094 BGP: NH update for 10.0.0.1/32(0)(VRF default) - flags 0x6 chgflags 0x0 - evaluate paths
   2022/05/17 09:57:26.094 BGP: evaluate_paths: Updating peer (10.0.0.1(VRF default)) status with NHT
   2022/05/17 09:57:30.081 ZEBRA: Event driven route-map update triggered
   2022/05/17 09:57:30.081 ZEBRA: Event handler for route-map: 10.0.0.1-out
   2022/05/17 09:57:30.081 ZEBRA: Event handler for route-map: 10.0.0.1-in
   2022/05/17 09:57:31.104 ZEBRA: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWNEIGH(28), len=76, seq=0, pid=0
   2022/05/17 09:57:31.104 ZEBRA: 	Neighbor Entry received is not on a VLAN or a BRIDGE, ignoring
   2022/05/17 09:57:31.105 ZEBRA: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWNEIGH(28), len=76, seq=0, pid=0
   2022/05/17 09:57:31.105 ZEBRA: 	Neighbor Entry received is not on a VLAN or a BRIDGE, ignoring

Procedure

1. Display the names of the

   speaker

   pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME            READY   STATUS    RESTARTS   AGE
   speaker-66bth   4/4     Running   0          56m
   speaker-gvfnf   4/4     Running   0          56m
   ...

2. Display the running configuration for FRR:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show running-config"

   Example output

   Building configuration...
   
   Current configuration:
   !
   frr version 7.5.1_git
   frr defaults traditional
   hostname some-hostname
   log file /etc/frr/frr.log informational
   log timestamp precision 3
   service integrated-vtysh-config
   !
   router bgp 64500  

   1

   
    bgp router-id 10.0.1.2
    no bgp ebgp-requires-policy
    no bgp default ipv4-unicast
    no bgp network import-check
    neighbor 10.0.2.3 remote-as 64500  

   2

   
    neighbor 10.0.2.3 bfd profile doc-example-bfd-profile-full  

   3

   
    neighbor 10.0.2.3 timers 5 15
    neighbor 10.0.2.4 remote-as 64500  

   4

   
    neighbor 10.0.2.4 bfd profile doc-example-bfd-profile-full  

   5

   
    neighbor 10.0.2.4 timers 5 15
    !
    address-family ipv4 unicast
     network 203.0.113.200/30   

   6

   
     neighbor 10.0.2.3 activate
     neighbor 10.0.2.3 route-map 10.0.2.3-in in
     neighbor 10.0.2.4 activate
     neighbor 10.0.2.4 route-map 10.0.2.4-in in
    exit-address-family
    !
    address-family ipv6 unicast
     network fc00:f853:ccd:e799::/124  

   7

   
     neighbor 10.0.2.3 activate
     neighbor 10.0.2.3 route-map 10.0.2.3-in in
     neighbor 10.0.2.4 activate
     neighbor 10.0.2.4 route-map 10.0.2.4-in in
    exit-address-family
   !
   route-map 10.0.2.3-in deny 20
   !
   route-map 10.0.2.4-in deny 20
   !
   ip nht resolve-via-default
   !
   ipv6 nht resolve-via-default
   !
   line vty
   !
   bfd
    profile doc-example-bfd-profile-full  

   8

   
     transmit-interval 35
     receive-interval 35
     passive-mode
     echo-mode
     echo-interval 35
     minimum-ttl 10
    !
   !
   end

   <.> The

   router bgp

   section indicates the ASN for MetalLB. <.> Confirm that a

   neighbor <ip-address> remote-as <peer-ASN>

   line exists for each BGP peer custom resource that you added. <.> If you configured BFD, confirm that the BFD profile is associated with the correct BGP peer and that the BFD profile appears in the command output. <.> Confirm that the

   network <ip-address-range>

   lines match the IP address ranges that you specified in address pool custom resources that you added.

3. Display the BGP summary:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bgp summary"

   Example output

   IPv4 Unicast Summary:
   BGP router identifier 10.0.1.2, local AS number 64500 vrf-id 0
   BGP table version 1
   RIB entries 1, using 192 bytes of memory
   Peers 2, using 29 KiB of memory
   
   Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
   10.0.2.3        4      64500       387       389        0    0    0 00:32:02            0        1  

   1

   
   10.0.2.4        4      64500         0         0        0    0    0    never       Active        0  

   2

   
   
   Total number of neighbors 2
   
   IPv6 Unicast Summary:
   BGP router identifier 10.0.1.2, local AS number 64500 vrf-id 0
   BGP table version 1
   RIB entries 1, using 192 bytes of memory
   Peers 2, using 29 KiB of memory
   
   Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
   10.0.2.3        4      64500       387       389        0    0    0 00:32:02 NoNeg  

   3

   
   10.0.2.4        4      64500         0         0        0    0    0    never       Active        0  

   4

   
   
   Total number of neighbors 2

   1 1 3

   Confirm that the output includes a line for each BGP peer custom resource that you added.

   2 4 2 4

   Output that shows 0 messages received and messages sent indicates a BGP peer that does not have a BGP session. Check network connectivity and the BGP configuration of the BGP peer.

4. Display the BGP peers that received an address pool:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bgp ipv4 unicast 203.0.113.200/30"

   Replace

   ipv4

   with

   ipv6

   to display the BGP peers that received an IPv6 address pool. Replace

   203.0.113.200/30

   with an IPv4 or IPv6 IP address range from an address pool.

   Example output

   BGP routing table entry for 203.0.113.200/30
   Paths: (1 available, best #1, table default)
     Advertised to non peer-group peers:
     10.0.2.3  <.>
     Local
       0.0.0.0 from 0.0.0.0 (10.0.1.2)
         Origin IGP, metric 0, weight 32768, valid, sourced, local, best (First path received)
         Last update: Mon Jan 10 19:49:07 2022

   <.> Confirm that the output includes an IP address for a BGP peer.

Procedure

1. Display the names of the

   speaker

   pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME            READY   STATUS    RESTARTS   AGE
   speaker-66bth   4/4     Running   0          26m
   speaker-gvfnf   4/4     Running   0          26m
   ...

2. Display the BFD peers:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bfd peers brief"

   Example output

   Session count: 2
   SessionId  LocalAddress              PeerAddress              Status
   =========  ============              ===========              ======
   3909139637 10.0.1.2                  10.0.2.3                 up  <.>

   <.> Confirm that the

   PeerAddress

   column includes each BFD peer. If the output does not list a BFD peer IP address that you expected the output to include, troubleshoot BGP connectivity with the peer. If the status field indicates

   down

   , check for connectivity on the links and equipment between the node and the peer. You can determine the node name for the speaker pod with a command like

   oc get pods -n metallb-system speaker-66bth -o jsonpath='{.spec.nodeName}'

   .