Chapter 33. Load balancing with MetalLB

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators OperatorHub.

2. Type a keyword into the Filter by keyword box or scroll to find the Operator you want. For example, type metallb to find the MetalLB Operator.

   You can also filter options by Infrastructure Features. For example, select Disconnected if you want to see Operators that work in disconnected environments, also known as restricted network environments.

3. On the Install Operator page, accept the defaults and click Install.

Verification

1. To confirm that the installation is successful:

   1. Navigate to the Operators Installed Operators page.
   2. Check that the Operator is installed in the openshift-operators namespace and that its status is Succeeded.

2. If the Operator is not installed successfully, check the status of the Operator and review the logs:

   1. Navigate to the Operators Installed Operators page and inspect the Status column for any errors or failures.
   2. Navigate to the Workloads Pods page and check the logs in any pods in the openshift-operators project that are reporting issues.

Procedure

1. Create a namespace for the MetalLB Operator by entering the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Namespace
   metadata:
     name: metallb-system
   EOF

2. Create an Operator group custom resource (CR) in the namespace:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: metallb-operator
     namespace: metallb-system
   EOF

3. Confirm the Operator group is installed in the namespace:

   $ oc get operatorgroup -n metallb-system

   Example output

   NAME               AGE
   metallb-operator   14m

4. Create a Subscription CR:

   1. Define the Subscription CR and save the YAML file, for example, metallb-sub.yaml:

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: metallb-operator-sub
        namespace: metallb-system
      spec:
        channel: stable
        name: metallb-operator
        source: redhat-operators 1
        sourceNamespace: openshift-marketplace

      1

      You must specify the redhat-operators value.

   2. To create the Subscription CR, run the following command:

      $ oc create -f metallb-sub.yaml

5. Optional: To ensure BGP and BFD metrics appear in Prometheus, you can label the namespace as in the following command:

   $ oc label ns metallb-system "openshift.io/cluster-monitoring=true"

1. Confirm the install plan is in the namespace:

   $ oc get installplan -n metallb-system

   Example output

   NAME            CSV                                   APPROVAL    APPROVED
   install-wzg94   metallb-operator.4.13.0-nnnnnnnnnnnn   Automatic   true

   Note

   Installation of the Operator might take a few seconds.

2. To verify that the Operator is installed, enter the following command:

   $ oc get clusterserviceversion -n metallb-system \
     -o custom-columns=Name:.metadata.name,Phase:.status.phase

   Example output

   Name                                  Phase
   metallb-operator.4.13.0-nnnnnnnnnnnn   Succeeded

1. Create a single instance of a MetalLB custom resource:

   $ cat << EOF | oc apply -f -
   apiVersion: metallb.io/v1beta1
   kind: MetalLB
   metadata:
     name: metallb
     namespace: metallb-system
   EOF

1. Verify that the deployment for the controller is running:

   $ oc get deployment -n metallb-system controller

   Example output

   NAME         READY   UP-TO-DATE   AVAILABLE   AGE
   controller   1/1     1            1           11m

2. Verify that the daemon set for the speaker is running:

   $ oc get daemonset -n metallb-system speaker

   Example output

   NAME      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
   speaker   6         6         6       6            6           kubernetes.io/os=linux   18m

   The example output indicates 6 speaker pods. The number of speaker pods in your cluster might differ from the example output. Make sure the output indicates one pod for each node in your cluster.

Procedure

1. Navigate to the Operators Installed Operators page.
2. Search for the MetalLB Operator. Then, click on it.

3. On the right side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.

   An Uninstall Operator? dialog box is displayed.

4. Select Uninstall to remove the Operator, Operator deployments, and pods. Following this action, the Operator stops running and no longer receives updates.

   Note

   This action does not remove resources managed by the Operator, including custom resource definitions (CRDs) and custom resources (CRs). Dashboards and navigation items enabled by the web console and off-cluster resources that continue to run might need manual clean up. To remove these after uninstalling the Operator, you might need to manually delete the Operator CRDs.

Procedure

1. Check the current version of the subscribed MetalLB Operator in the currentCSV field:

   $ oc get subscription metallb-operator -n metallb-system -o yaml | grep currentCSV

   Example output

     currentCSV: metallb-operator.4.10.0-202207051316

2. Delete the subscription:

   $ oc delete subscription metallb-operator -n metallb-system

   Example output

   subscription.operators.coreos.com "metallb-operator" deleted

3. Delete the CSV for the Operator in the target namespace using the currentCSV value from the previous step:

   $ oc delete clusterserviceversion metallb-operator.4.10.0-202207051316 -n metallb-system

   Example output

   clusterserviceversion.operators.coreos.com "metallb-operator.4.10.0-202207051316" deleted

Procedure

1. List the Operator groups in the metallb-system namespace by running the following command:

   $ oc get operatorgroup -n metallb-system

   Example output

   NAME                   AGE
   metallb-system-7jc66   85m

2. Verify that the spec.targetNamespaces is present in the Operator group CR associated with the metallb-system namespace by running the following command:

   $ oc get operatorgroup metallb-system-7jc66 -n metallb-system -o yaml

   Example output

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     annotations:
       olm.providedAPIs: ""
     creationTimestamp: "2023-10-25T09:42:49Z"
     generateName: metallb-system-
     generation: 1
     name: metallb-system-7jc66
     namespace: metallb-system
     resourceVersion: "25027"
     uid: f5f644a0-eef8-4e31-a306-e2bbcfaffab3
   spec:
     targetNamespaces:
     - metallb-system
     upgradeStrategy: Default
   status:
     lastUpdated: "2023-10-25T09:42:49Z"
     namespaces:
     - metallb-system

3. Edit the Operator group and remove the targetNamespaces and metallb-system present under the spec section by running the following command:

   $ oc edit n metallb-system

   Example output

   operatorgroup.operators.coreos.com/metallb-system-7jc66 edited

4. Verify the spec.targetNamespaces is removed from the Operator group custom resource associated with the metallb-system namespace by running the following command:

   $ oc get operatorgroup metallb-system-7jc66 -n metallb-system -o yaml

   Example output

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     annotations:
       olm.providedAPIs: ""
     creationTimestamp: "2023-10-25T09:42:49Z"
     generateName: metallb-system-
     generation: 2
     name: metallb-system-7jc66
     namespace: metallb-system
     resourceVersion: "61658"
     uid: f5f644a0-eef8-4e31-a306-e2bbcfaffab3
   spec:
     upgradeStrategy: Default
   status:
     lastUpdated: "2023-10-25T14:31:30Z"
     namespaces:
     - ""

Procedure

1. Verify that the metallb-system namespace still exists:

   $ oc get namespaces | grep metallb-system

   Example output

   metallb-system                                     Active   31m

2. Verify the metallb custom resource still exists:

   $ oc get metallb -n metallb-system

   Example output

   NAME      AGE
   metallb   33m

3. Follow the guidance in "Installing from OperatorHub using the CLI" to install the latest 4.13 version of the MetalLB Operator.

   Note

   When installing the latest 4.13 version of the MetalLB Operator, you must install the Operator to the same namespace it was previously installed to.

4. Verify the upgraded version of the Operator is now the 4.13 version.

   $ oc get csv -n metallb-system

   Example output

   NAME                                   DISPLAY            VERSION               REPLACES   PHASE
   metallb-operator.4.13.0-202207051316   MetalLB Operator   4.13.0-202207051316              Succeeded

Procedure

1. Create a file, such as ipaddresspool.yaml, with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     namespace: metallb-system
     name: doc-example
     labels: 1
       zone: east
   spec:
     addresses:
     - 203.0.113.1-203.0.113.10
     - 203.0.113.65-203.0.113.75

   1

   This label assigned to the IPAddressPool can be referenced by the ipAddressPoolSelectors in the BGPAdvertisement CRD to associate the IPAddressPool with the advertisement.

2. Apply the configuration for the IP address pool:

   $ oc apply -f ipaddresspool.yaml

Procedure

1. Create an IP address pool by using a custom resource:

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     namespace: metallb-system
     name: pool1
   spec:
     addresses:
       - 4.4.4.100-4.4.4.200
       - 2001:100:4::200-2001:100:4::400

2. Control which nodes in the cluster the IP address from pool1 advertises from by defining the .spec.nodeSelector value in the BGPAdvertisement custom resource:

   apiVersion: metallb.io/v1beta1
   kind: BGPAdvertisement
   metadata:
     name: example
   spec:
     ipAddressPools:
     - pool1
     nodeSelector:
     - matchLabels:
         kubernetes.io/hostname: NodeA
     - matchLabels:
         kubernetes.io/hostname: NodeB

Procedure

1. Create an IP address pool.

   1. Create a file, such as ipaddresspool.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2
      spec:
        addresses:
          - 4.4.4.0/24
        autoAssign: false

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement.

   1. Create a file, such as l2advertisement.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement
        namespace: metallb-system
      spec:
        ipAddressPools:
         - doc-example-l2

   2. Apply the configuration:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create an IP address pool.

   1. Create a file, such as ipaddresspool.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2-label
        labels:
          zone: east
      spec:
        addresses:
          - 172.31.249.87/32

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement advertising the IP using ipAddressPoolSelectors.

   1. Create a file, such as l2advertisement.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement-label
        namespace: metallb-system
      spec:
        ipAddressPoolSelectors:
          - matchExpressions:
              - key: zone
                operator: In
                values:
                  - east

   2. Apply the configuration:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create an IP address pool.

   1. Create a file, such as ipaddresspool.yaml, and enter the configuration details like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-l2
      spec:
        addresses:
          - 4.4.4.0/24
        autoAssign: false

   2. Apply the configuration for the IP address pool like the following example:

      $ oc apply -f ipaddresspool.yaml

2. Create a L2 advertisement advertising the IP with interfaces selector.

   1. Create a YAML file, such as l2advertisement.yaml, and enter the configuration details like the following example:

      apiVersion: metallb.io/v1beta1
      kind: L2Advertisement
      metadata:
        name: l2advertisement
        namespace: metallb-system
      spec:
        ipAddressPools:
         - doc-example-l2
         interfaces:
         - interfaceA
         - interfaceB

   2. Apply the configuration for the advertisement like the following example:

      $ oc apply -f l2advertisement.yaml

Procedure

1. Create a file, such as bgppeer.yaml, with content like the following example:

   apiVersion: metallb.io/v1beta2
   kind: BGPPeer
   metadata:
     namespace: metallb-system
     name: doc-example-peer
   spec:
     peerAddress: 10.0.0.1
     peerASN: 64501
     myASN: 64500
     routerID: 10.10.10.10

2. Apply the configuration for the BGP peer:

   $ oc apply -f bgppeer.yaml

Procedure

1. Create address pool pool1.

   1. Create a file, such as ipaddresspool1.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: pool1
      spec:
        addresses:
          - 4.4.4.100-4.4.4.200
          - 2001:100:4::200-2001:100:4::400

   2. Apply the configuration for the IP address pool pool1:

      $ oc apply -f ipaddresspool1.yaml

2. Create address pool pool2.

   1. Create a file, such as ipaddresspool2.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: pool2
      spec:
        addresses:
          - 5.5.5.100-5.5.5.200
          - 2001:100:5::200-2001:100:5::400

   2. Apply the configuration for the IP address pool pool2:

      $ oc apply -f ipaddresspool2.yaml

3. Create BGP peer1.

   1. Create a file, such as bgppeer1.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: peer1
      spec:
        peerAddress: 10.0.0.1
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer:

      $ oc apply -f bgppeer1.yaml

4. Create BGP peer2.

   1. Create a file, such as bgppeer2.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: peer2
      spec:
        peerAddress: 10.0.0.2
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer2:

      $ oc apply -f bgppeer2.yaml

5. Create BGP advertisement 1.

   1. Create a file, such as bgpadvertisement1.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgpadvertisement-1
        namespace: metallb-system
      spec:
        ipAddressPools:
          - pool1
        peers:
          - peer1
        communities:
          - 65535:65282
        aggregationLength: 32
        aggregationLengthV6: 128
        localPref: 100

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement1.yaml

6. Create BGP advertisement 2.

   1. Create a file, such as bgpadvertisement2.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgpadvertisement-2
        namespace: metallb-system
      spec:
        ipAddressPools:
          - pool2
        peers:
          - peer2
        communities:
          - 65535:65282
        aggregationLength: 32
        aggregationLengthV6: 128
        localPref: 100

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement2.yaml

Procedure

1. Create an IP address pool.

   1. Create a file, such as ipaddresspool.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        namespace: metallb-system
        name: doc-example-bgp-community
      spec:
        addresses:
          - 203.0.113.200/30
          - fc00:f853:ccd:e799::/124

   2. Apply the configuration for the IP address pool:

      $ oc apply -f ipaddresspool.yaml

2. Create a community alias named community1.

   apiVersion: metallb.io/v1beta1
   kind: Community
   metadata:
     name: community1
     namespace: metallb-system
   spec:
     communities:
       - name: NO_ADVERTISE
         value: '65535:65282'

3. Create a BGP peer named doc-example-bgp-peer.

   1. Create a file, such as bgppeer.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        namespace: metallb-system
        name: doc-example-bgp-peer
      spec:
        peerAddress: 10.0.0.1
        peerASN: 64501
        myASN: 64500
        routerID: 10.10.10.10

   2. Apply the configuration for the BGP peer:

      $ oc apply -f bgppeer.yaml

4. Create a BGP advertisement with the community alias.

   1. Create a file, such as bgpadvertisement.yaml, with content like the following example:

      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: bgp-community-sample
        namespace: metallb-system
      spec:
        aggregationLength: 32
        aggregationLengthV6: 128
        communities:
          - NO_ADVERTISE 1
        ipAddressPools:
          - doc-example-bgp-community
        peers:
          - doc-example-peer

      1

      Specify the CommunityAlias.name here and not the community custom resource (CR) name.

   2. Apply the configuration:

      $ oc apply -f bgpadvertisement.yaml

Procedure

1. Create a file, such as bfdprofile.yaml, with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: BFDProfile
   metadata:
     name: doc-example-bfd-profile-full
     namespace: metallb-system
   spec:
     receiveInterval: 300
     transmitInterval: 300
     detectMultiplier: 3
     echoMode: false
     passiveMode: true
     minimumTtl: 254

2. Apply the configuration for the BFD profile:

   $ oc apply -f bfdprofile.yaml

Procedure

1. Create a <service_name>.yaml file. In the file, ensure that the spec.type field is set to LoadBalancer.

   Refer to the examples for information about how to request the external IP address that MetalLB assigns to the service.

2. Create the service:

   $ oc apply -f <service_name>.yaml

   Example output

   service/<service_name> created

Procedure

1. Create a file, such as setdebugloglevel.yaml, with content like the following example:

   apiVersion: metallb.io/v1beta1
   kind: MetalLB
   metadata:
     name: metallb
     namespace: metallb-system
   spec:
     logLevel: debug
     nodeSelector:
       node-role.kubernetes.io/worker: ""

2. Apply the configuration:

   $ oc replace -f setdebugloglevel.yaml

   Note

   Use oc replace as the understanding is the metallb CR is already created and here you are changing the log level.

3. Display the names of the speaker pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME                    READY   STATUS    RESTARTS   AGE
   speaker-2m9pm           4/4     Running   0          9m19s
   speaker-7m4qw           3/4     Running   0          19s
   speaker-szlmx           4/4     Running   0          9m19s

   Note

   Speaker and controller pods are recreated to ensure the updated logging level is applied. The logging level is modified for all the components of MetalLB.

4. View the speaker logs:

   $ oc logs -n metallb-system speaker-7m4qw -c speaker

   Example output

   {"branch":"main","caller":"main.go:92","commit":"3d052535","goversion":"gc / go1.17.1 / amd64","level":"info","msg":"MetalLB speaker starting (commit 3d052535, branch main)","ts":"2022-05-17T09:55:05Z","version":""}
   {"caller":"announcer.go:110","event":"createARPResponder","interface":"ens4","level":"info","msg":"created ARP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:119","event":"createNDPResponder","interface":"ens4","level":"info","msg":"created NDP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:110","event":"createARPResponder","interface":"tun0","level":"info","msg":"created ARP responder for interface","ts":"2022-05-17T09:55:05Z"}
   {"caller":"announcer.go:119","event":"createNDPResponder","interface":"tun0","level":"info","msg":"created NDP responder for interface","ts":"2022-05-17T09:55:05Z"}
   I0517 09:55:06.515686      95 request.go:665] Waited for 1.026500832s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operators.coreos.com/v1alpha1?timeout=32s
   {"Starting Manager":"(MISSING)","caller":"k8s.go:389","level":"info","ts":"2022-05-17T09:55:08Z"}
   {"caller":"speakerlist.go:310","level":"info","msg":"node event - forcing sync","node addr":"10.0.128.4","node event":"NodeJoin","node name":"ci-ln-qb8t3mb-72292-7s7rh-worker-a-vvznj","ts":"2022-05-17T09:55:08Z"}
   {"caller":"service_controller.go:113","controller":"ServiceReconciler","enqueueing":"openshift-kube-controller-manager-operator/metrics","epslice":"{\"metadata\":{\"name\":\"metrics-xtsxr\",\"generateName\":\"metrics-\",\"namespace\":\"openshift-kube-controller-manager-operator\",\"uid\":\"ac6766d7-8504-492c-9d1e-4ae8897990ad\",\"resourceVersion\":\"9041\",\"generation\":4,\"creationTimestamp\":\"2022-05-17T07:16:53Z\",\"labels\":{\"app\":\"kube-controller-manager-operator\",\"endpointslice.kubernetes.io/managed-by\":\"endpointslice-controller.k8s.io\",\"kubernetes.io/service-name\":\"metrics\"},\"annotations\":{\"endpoints.kubernetes.io/last-change-trigger-time\":\"2022-05-17T07:21:34Z\"},\"ownerReferences\":[{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"name\":\"metrics\",\"uid\":\"0518eed3-6152-42be-b566-0bd00a60faf8\",\"controller\":true,\"blockOwnerDeletion\":true}],\"managedFields\":[{\"manager\":\"kube-controller-manager\",\"operation\":\"Update\",\"apiVersion\":\"discovery.k8s.io/v1\",\"time\":\"2022-05-17T07:20:02Z\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:addressType\":{},\"f:endpoints\":{},\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:endpoints.kubernetes.io/last-change-trigger-time\":{}},\"f:generateName\":{},\"f:labels\":{\".\":{},\"f:app\":{},\"f:endpointslice.kubernetes.io/managed-by\":{},\"f:kubernetes.io/service-name\":{}},\"f:ownerReferences\":{\".\":{},\"k:{\\\"uid\\\":\\\"0518eed3-6152-42be-b566-0bd00a60faf8\\\"}\":{}}},\"f:ports\":{}}}]},\"addressType\":\"IPv4\",\"endpoints\":[{\"addresses\":[\"10.129.0.7\"],\"conditions\":{\"ready\":true,\"serving\":true,\"terminating\":false},\"targetRef\":{\"kind\":\"Pod\",\"namespace\":\"openshift-kube-controller-manager-operator\",\"name\":\"kube-controller-manager-operator-6b98b89ddd-8d4nf\",\"uid\":\"dd5139b8-e41c-4946-a31b-1a629314e844\",\"resourceVersion\":\"9038\"},\"nodeName\":\"ci-ln-qb8t3mb-72292-7s7rh-master-0\",\"zone\":\"us-central1-a\"}],\"ports\":[{\"name\":\"https\",\"protocol\":\"TCP\",\"port\":8443}]}","level":"debug","ts":"2022-05-17T09:55:08Z"}

5. View the FRR logs:

   $ oc logs -n metallb-system speaker-7m4qw -c frr

   Example output

   Started watchfrr
   2022/05/17 09:55:05 ZEBRA: client 16 says hello and bids fair to announce only bgp routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 31 says hello and bids fair to announce only vnc routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 38 says hello and bids fair to announce only static routes vrf=0
   2022/05/17 09:55:05 ZEBRA: client 43 says hello and bids fair to announce only bfd routes vrf=0
   2022/05/17 09:57:25.089 BGP: Creating Default VRF, AS 64500
   2022/05/17 09:57:25.090 BGP: dup addr detect enable max_moves 5 time 180 freeze disable freeze_time 0
   2022/05/17 09:57:25.090 BGP: bgp_get: Registering BGP instance (null) to zebra
   2022/05/17 09:57:25.090 BGP: Registering VRF 0
   2022/05/17 09:57:25.091 BGP: Rx Router Id update VRF 0 Id 10.131.0.1/32
   2022/05/17 09:57:25.091 BGP: RID change : vrf VRF default(0), RTR ID 10.131.0.1
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF br0
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF ens4
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF ens4 addr 10.0.128.4/32
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF ens4 addr fe80::c9d:84da:4d86:5618/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF lo
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF ovs-system
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF tun0
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF tun0 addr 10.131.0.1/23
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF tun0 addr fe80::40f1:d1ff:feb6:5322/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth2da49fed
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth2da49fed addr fe80::24bd:d1ff:fec1:d88/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth2fa08c8c
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth2fa08c8c addr fe80::6870:ff:fe96:efc8/64
   2022/05/17 09:57:25.091 BGP: Rx Intf add VRF 0 IF veth41e356b7
   2022/05/17 09:57:25.091 BGP: Rx Intf address add VRF 0 IF veth41e356b7 addr fe80::48ff:37ff:fede:eb4b/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth1295c6e2
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth1295c6e2 addr fe80::b827:a2ff:feed:637/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth9733c6dc
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth9733c6dc addr fe80::3cf4:15ff:fe11:e541/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF veth336680ea
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF veth336680ea addr fe80::94b1:8bff:fe7e:488c/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vetha0a907b7
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vetha0a907b7 addr fe80::3855:a6ff:fe73:46c3/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vethf35a4398
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vethf35a4398 addr fe80::40ef:2fff:fe57:4c4d/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vethf831b7f4
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vethf831b7f4 addr fe80::f0d9:89ff:fe7c:1d32/64
   2022/05/17 09:57:25.092 BGP: Rx Intf add VRF 0 IF vxlan_sys_4789
   2022/05/17 09:57:25.092 BGP: Rx Intf address add VRF 0 IF vxlan_sys_4789 addr fe80::80c1:82ff:fe4b:f078/64
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] Timer (start timer expire).
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] BGP_Start (Idle->Connect), fd -1
   2022/05/17 09:57:26.094 BGP: Allocated bnc 10.0.0.1/32(0)(VRF default) peer 0x7f807f7631a0
   2022/05/17 09:57:26.094 BGP: sendmsg_zebra_rnh: sending cmd ZEBRA_NEXTHOP_REGISTER for 10.0.0.1/32 (vrf VRF default)
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] Waiting for NHT
   2022/05/17 09:57:26.094 BGP: bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 went from Idle to Connect
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 [FSM] TCP_connection_open_failed (Connect->Active), fd -1
   2022/05/17 09:57:26.094 BGP: bgp_fsm_change_status : vrf default(0), Status: Active established_peers 0
   2022/05/17 09:57:26.094 BGP: 10.0.0.1 went from Connect to Active
   2022/05/17 09:57:26.094 ZEBRA: rnh_register msg from client bgp: hdr->length=8, type=nexthop vrf=0
   2022/05/17 09:57:26.094 ZEBRA: 0: Add RNH 10.0.0.1/32 type Nexthop
   2022/05/17 09:57:26.094 ZEBRA: 0:10.0.0.1/32: Evaluate RNH, type Nexthop (force)
   2022/05/17 09:57:26.094 ZEBRA: 0:10.0.0.1/32: NH has become unresolved
   2022/05/17 09:57:26.094 ZEBRA: 0: Client bgp registers for RNH 10.0.0.1/32 type Nexthop
   2022/05/17 09:57:26.094 BGP: VRF default(0): Rcvd NH update 10.0.0.1/32(0) - metric 0/0 #nhops 0/0 flags 0x6
   2022/05/17 09:57:26.094 BGP: NH update for 10.0.0.1/32(0)(VRF default) - flags 0x6 chgflags 0x0 - evaluate paths
   2022/05/17 09:57:26.094 BGP: evaluate_paths: Updating peer (10.0.0.1(VRF default)) status with NHT
   2022/05/17 09:57:30.081 ZEBRA: Event driven route-map update triggered
   2022/05/17 09:57:30.081 ZEBRA: Event handler for route-map: 10.0.0.1-out
   2022/05/17 09:57:30.081 ZEBRA: Event handler for route-map: 10.0.0.1-in
   2022/05/17 09:57:31.104 ZEBRA: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWNEIGH(28), len=76, seq=0, pid=0
   2022/05/17 09:57:31.104 ZEBRA: 	Neighbor Entry received is not on a VLAN or a BRIDGE, ignoring
   2022/05/17 09:57:31.105 ZEBRA: netlink_parse_info: netlink-listen (NS 0) type RTM_NEWNEIGH(28), len=76, seq=0, pid=0
   2022/05/17 09:57:31.105 ZEBRA: 	Neighbor Entry received is not on a VLAN or a BRIDGE, ignoring

Procedure

1. Display the names of the speaker pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME            READY   STATUS    RESTARTS   AGE
   speaker-66bth   4/4     Running   0          56m
   speaker-gvfnf   4/4     Running   0          56m
   ...

2. Display the running configuration for FRR:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show running-config"

   Example output

   Building configuration...
   
   Current configuration:
   !
   frr version 7.5.1_git
   frr defaults traditional
   hostname some-hostname
   log file /etc/frr/frr.log informational
   log timestamp precision 3
   service integrated-vtysh-config
   !
   router bgp 64500  1
    bgp router-id 10.0.1.2
    no bgp ebgp-requires-policy
    no bgp default ipv4-unicast
    no bgp network import-check
    neighbor 10.0.2.3 remote-as 64500  2
    neighbor 10.0.2.3 bfd profile doc-example-bfd-profile-full  3
    neighbor 10.0.2.3 timers 5 15
    neighbor 10.0.2.4 remote-as 64500  4
    neighbor 10.0.2.4 bfd profile doc-example-bfd-profile-full  5
    neighbor 10.0.2.4 timers 5 15
    !
    address-family ipv4 unicast
     network 203.0.113.200/30   6
     neighbor 10.0.2.3 activate
     neighbor 10.0.2.3 route-map 10.0.2.3-in in
     neighbor 10.0.2.4 activate
     neighbor 10.0.2.4 route-map 10.0.2.4-in in
    exit-address-family
    !
    address-family ipv6 unicast
     network fc00:f853:ccd:e799::/124  7
     neighbor 10.0.2.3 activate
     neighbor 10.0.2.3 route-map 10.0.2.3-in in
     neighbor 10.0.2.4 activate
     neighbor 10.0.2.4 route-map 10.0.2.4-in in
    exit-address-family
   !
   route-map 10.0.2.3-in deny 20
   !
   route-map 10.0.2.4-in deny 20
   !
   ip nht resolve-via-default
   !
   ipv6 nht resolve-via-default
   !
   line vty
   !
   bfd
    profile doc-example-bfd-profile-full  8
     transmit-interval 35
     receive-interval 35
     passive-mode
     echo-mode
     echo-interval 35
     minimum-ttl 10
    !
   !
   end

   <.> The router bgp section indicates the ASN for MetalLB. <.> Confirm that a neighbor <ip-address> remote-as <peer-ASN> line exists for each BGP peer custom resource that you added. <.> If you configured BFD, confirm that the BFD profile is associated with the correct BGP peer and that the BFD profile appears in the command output. <.> Confirm that the network <ip-address-range> lines match the IP address ranges that you specified in address pool custom resources that you added.

3. Display the BGP summary:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bgp summary"

   Example output

   IPv4 Unicast Summary:
   BGP router identifier 10.0.1.2, local AS number 64500 vrf-id 0
   BGP table version 1
   RIB entries 1, using 192 bytes of memory
   Peers 2, using 29 KiB of memory
   
   Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
   10.0.2.3        4      64500       387       389        0    0    0 00:32:02            0        1  1
   10.0.2.4        4      64500         0         0        0    0    0    never       Active        0  2
   
   Total number of neighbors 2
   
   IPv6 Unicast Summary:
   BGP router identifier 10.0.1.2, local AS number 64500 vrf-id 0
   BGP table version 1
   RIB entries 1, using 192 bytes of memory
   Peers 2, using 29 KiB of memory
   
   Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
   10.0.2.3        4      64500       387       389        0    0    0 00:32:02 NoNeg  3
   10.0.2.4        4      64500         0         0        0    0    0    never       Active        0  4
   
   Total number of neighbors 2

   1 1 3

   Confirm that the output includes a line for each BGP peer custom resource that you added.

   2 4 2 4

   Output that shows 0 messages received and messages sent indicates a BGP peer that does not have a BGP session. Check network connectivity and the BGP configuration of the BGP peer.

4. Display the BGP peers that received an address pool:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bgp ipv4 unicast 203.0.113.200/30"

   Replace ipv4 with ipv6 to display the BGP peers that received an IPv6 address pool. Replace 203.0.113.200/30 with an IPv4 or IPv6 IP address range from an address pool.

   Example output

   BGP routing table entry for 203.0.113.200/30
   Paths: (1 available, best #1, table default)
     Advertised to non peer-group peers:
     10.0.2.3  <.>
     Local
       0.0.0.0 from 0.0.0.0 (10.0.1.2)
         Origin IGP, metric 0, weight 32768, valid, sourced, local, best (First path received)
         Last update: Mon Jan 10 19:49:07 2022

   <.> Confirm that the output includes an IP address for a BGP peer.

Procedure

1. Display the names of the speaker pods:

   $ oc get -n metallb-system pods -l component=speaker

   Example output

   NAME            READY   STATUS    RESTARTS   AGE
   speaker-66bth   4/4     Running   0          26m
   speaker-gvfnf   4/4     Running   0          26m
   ...

2. Display the BFD peers:

   $ oc exec -n metallb-system speaker-66bth -c frr -- vtysh -c "show bfd peers brief"

   Example output

   Session count: 2
   SessionId  LocalAddress              PeerAddress              Status
   =========  ============              ===========              ======
   3909139637 10.0.1.2                  10.0.2.3                 up  <.>

   <.> Confirm that the PeerAddress column includes each BFD peer. If the output does not list a BFD peer IP address that you expected the output to include, troubleshoot BGP connectivity with the peer. If the status field indicates down, check for connectivity on the links and equipment between the node and the peer. You can determine the node name for the speaker pod with a command like oc get pods -n metallb-system speaker-66bth -o jsonpath='{.spec.nodeName}'.