6.2. 为启用 SR-IOV 的工作负载配置 QinQ 支持
先决条件
-
已安装 OpenShift CLI(
oc)。 -
您可以使用具有
cluster-admin角色的用户访问集群。 - 已安装 SR-IOV Network Operator。
流程
使用以下内容创建一个名为
sriovnetpolicy-810-sriov-node-network.yaml的文件:apiVersion: sriovnetwork.openshift.io/v1 kind: SriovNetworkNodePolicy metadata: name: sriovnetpolicy-810 namespace: openshift-sriov-network-operator spec: deviceType: netdevice nicSelector: pfNames: - ens5f0#0-9 nodeSelector: node-role.kubernetes.io/worker-cnf: "" numVfs: 10 priority: 99 resourceName: resource810运行以下命令来创建
SriovNetworkNodePolicy对象:$ oc create -f sriovnetpolicy-810-sriov-node-network.yaml打开一个单独的终端窗口,运行以下命令来监控
openshift-sriov-network-operator命名空间中指定节点的 SR-IOV 网络节点状态的同步状态:$ watch -n 1 'oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath="{.status.syncStatus}"'同步状态表示从
InProgress更改为Succeeded。创建一个
SriovNetwork对象,并设置名为 S-tag 或Service Tag的外部 VLAN,因为它属于基础架构。重要您必须在交换机的中继接口上配置 VLAN。另外,您可能需要进一步配置一些交换机来支持 QinQ 标记。
使用以下内容创建一个名为
nad-sriovnetwork-1ad-810.yaml的文件:apiVersion: sriovnetwork.openshift.io/v1 kind: SriovNetwork metadata: name: sriovnetwork-1ad-810 namespace: openshift-sriov-network-operator spec: ipam: '{}' vlan: 1711 vlanProto: "802.1ad"2 networkNamespace: default resourceName: resource810运行以下命令来创建对象:
$ oc create -f nad-sriovnetwork-1ad-810.yaml
使用内部 VLAN 创建
NetworkAttachmentDefinition对象。内部 VLAN 通常被称为 C-tag 或Customer Tag,它属于 Network Function:使用以下内容,创建一个名为
nad-cvlan100.yaml的文件:apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: nad-cvlan100 namespace: default spec: config: '{ "name": "vlan-100", "cniVersion": "0.3.1", "type": "vlan", "linkInContainer": true, "master": "net1",1 "vlanId": 100, "ipam": {"type": "static"} }'- 1
- 指定 pod 中的 VF 接口。默认名称为
net1,因为 pod 注解中没有设置名称。
运行以下命令来应用 YAML 文件:
$ oc apply -f nad-cvlan100.yaml
验证
按照以下步骤,在节点上验证 QinQ 是否活跃:
使用以下内容创建一个名为
test-qinq-pod.yaml的文件:apiVersion: v1 kind: Pod metadata: name: test-pod annotations: k8s.v1.cni.cncf.io/networks: sriovnetwork-1ad-810, nad-cvlan100 spec: containers: - name: test-container image: quay.io/ocp-edge-qe/cnf-gotests-client:v4.10 imagePullPolicy: Always securityContext: privileged: true运行以下命令来创建测试 pod:
$ oc create -f test-qinq-pod.yaml在存在 pod 的目标节点上进入 debug 会话,运行以下命令显示网络接口
ens5f0的信息:$ oc debug node/my-cluster-node -- bash -c "ip link show ens5f0"输出示例
6: ens5f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether b4:96:91:a5:22:10 brd ff:ff:ff:ff:ff:ff vf 0 link/ether a2:81:ba:d0:6f:f3 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 1 link/ether 8a:bb:0a:36:f2:ed brd ff:ff:ff:ff:ff:ff, vlan 171, vlan protocol 802.1ad, spoof checking on, link-state auto, trust off vf 2 link/ether ca:0e:e1:5b:0c:d2 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 3 link/ether ee:6c:e2:f5:2c:70 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 4 link/ether 0a:d6:b7:66:5e:e8 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 5 link/ether da:d5:e7:14:4f:aa brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 6 link/ether d6:8e:85:75:12:5c brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 7 link/ether d6:eb:ce:9c:ea:78 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 8 link/ether 5e:c5:cc:05:93:3c brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust on vf 9 link/ether a6:5a:7c:1c:2a:16 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off输出中的
vlan protocol 802.1adID 表示接口支持带有协议 802.1ad (QinQ)的 VLAN 标记。VLAN ID 为 171。