Chapter 12. Switching Bluefield-2 from DPU to NIC

Procedure

1. Add the following labels to each of your worker nodes by entering the following commands:

   $ oc label node <example_node_name_one> node-role.kubernetes.io/sriov=

   Copy to Clipboard

   $ oc label node <example_node_name_two> node-role.kubernetes.io/sriov=

   Copy to Clipboard

2. Create a machine config pool for the SR-IOV Network Operator, for example:

   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfigPool
   metadata:
     name: sriov
   spec:
     machineConfigSelector:
       matchExpressions:
       - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,sriov]}
     nodeSelector:
       matchLabels:
         node-role.kubernetes.io/sriov: ""

   Copy to Clipboard

3. Apply the following machineconfig.yaml file to the worker nodes:

   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     labels:
       machineconfiguration.openshift.io/role: sriov
     name: 99-bf2-dpu
   spec:
     config:
       ignition:
         version: 3.2.0
       storage:
         files:
         - contents:
             source: data:text/plain;charset=utf-8;base64,ZmluZF9jb250YWluZXIoKSB7CiAgY3JpY3RsIHBzIC1vIGpzb24gfCBqcSAtciAnLmNvbnRhaW5lcnNbXSB8IHNlbGVjdCgubWV0YWRhdGEubmFtZT09InNyaW92LW5ldHdvcmstY29uZmlnLWRhZW1vbiIpIHwgLmlkJwp9CnVudGlsIG91dHB1dD0kKGZpbmRfY29udGFpbmVyKTsgW1sgLW4gIiRvdXRwdXQiIF1dOyBkbwogIGVjaG8gIndhaXRpbmcgZm9yIGNvbnRhaW5lciB0byBjb21lIHVwIgogIHNsZWVwIDE7CmRvbmUKISBzdWRvIGNyaWN0bCBleGVjICRvdXRwdXQgL2JpbmRhdGEvc2NyaXB0cy9iZjItc3dpdGNoLW1vZGUuc2ggIiRAIgo=
           mode: 0755
           overwrite: true
           path: /etc/default/switch_in_sriov_config_daemon.sh
       systemd:
         units:
         - name: dpu-switch.service
           enabled: true
           contents: |
             [Unit]
             Description=Switch BlueField2 card to NIC/DPU mode
             RequiresMountsFor=%t/containers
             Wants=network.target
             After=network-online.target kubelet.service
             [Service]
             SuccessExitStatus=0 120
             RemainAfterExit=True
             ExecStart=/bin/bash -c '/etc/default/switch_in_sriov_config_daemon.sh nic || shutdown -r now' 

   1

   
             Type=oneshot
             [Install]
             WantedBy=multi-user.target

   Copy to Clipboard

   1

   Optional: The PCI address of a specific card can optionally be specified, for example ExecStart=/bin/bash -c '/etc/default/switch_in_sriov_config_daemon.sh nic 0000:5e:00.0 || echo done'. By default, the first device is selected. If there is more than one device, you must specify which PCI address to be used. The PCI address must be the same on all nodes that are switching Bluefield-2 from DPU mode to NIC mode.

4. Wait for the worker nodes to restart. After restarting, the Bluefield-2 network device on the worker nodes is switched into NIC mode.
5. Optional: You might need to restart the host hardware because most recent Bluefield-2 firmware releases require a hardware restart to switch into NIC mode.