Chapter 6. Managing multiple networks

Create a project to store CNI configurations as CRs and the Pods that will use the CRs.

$ oc new-project multinetwork-example

Create the CR that will define an additional network interface. Create a YAML file called macvlan-conf.yaml with the following contents:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition 1
metadata:
  name: macvlan-conf 2
spec:
  config: '{ 3
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.1.0/24",
        "rangeStart": "192.168.1.200",
        "rangeEnd": "192.168.1.216",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "192.168.1.1"
      }
    }'

The configuration is specific to a plug-in, which enables macvlan. Note the type line in the CNI configuration portion. Aside from the IPAM (IP address management) parameters for networking, in this example the master field must reference a network interface that resides on the node(s) hosting the Pod(s).

Run the following command to create the CR:

$ oc create -f macvlan-conf.yaml

Create a YAML file for the openshift-sriov namespace with the following contents:

apiVersion: v1
kind: Namespace
metadata:
  name: openshift-sriov
  labels:
    name: openshift-sriov
    openshift.io/run-level: "0"
  annotations:
    openshift.io/node-selector: ""
    openshift.io/description: "Openshift SR-IOV network components"

Run the following command to create the openshift-sriov namespace:

$ oc create -f openshift-sriov.yaml

Create a YAML file for the sriov-device-plugin service account with the following contents:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sriov-device-plugin
  namespace: openshift-sriov

Run the following command to create the sriov-device-plugin service account:

$ oc create -f sriov-device-plugin.yaml

Create a YAML file for the sriov-cni service account with the following contents:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sriov-cni
  namespace: openshift-sriov

Run the following command to create the sriov-cni service account:

$ oc create -f sriov-cni.yaml

Create a YAML file for the sriov-device-plugin DaemonSet with the following contents:

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: sriov-device-plugin
  namespace: openshift-sriov
  annotations:
    kubernetes.io/description: |
      This daemon set launches the SR-IOV network device plugin on each node.
spec:
  selector:
    matchLabels:
      app: sriov-device-plugin
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: sriov-device-plugin
        component: network
        type: infra
        openshift.io/component: network
    spec:
      hostNetwork: true
      nodeSelector:
        beta.kubernetes.io/os: linux
      tolerations:
      - operator: Exists
      serviceAccountName: sriov-device-plugin
      containers:
      - name: sriov-device-plugin
        image: quay.io/openshift/ose-sriov-network-device-plugin:v4.0.0
        args:
        - --log-level=10
        securityContext:
          privileged: true
        volumeMounts:
        - name: devicesock
          mountPath: /var/lib/kubelet/
          readOnly: false
        - name: net
          mountPath: /sys/class/net
          readOnly: true
      volumes:
        - name: devicesock
          hostPath:
            path: /var/lib/kubelet/
        - name: net
          hostPath:
            path: /sys/class/net

Run the following command to create the sriov-device-plugin DaemonSet:

oc create -f sriov-device-plugin.yaml

Create a YAML file for the sriov-cni DaemonSet with the following contents:

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: sriov-cni
  namespace: openshift-sriov
  annotations:
    kubernetes.io/description: |
      This daemon set launches the SR-IOV CNI plugin on SR-IOV capable worker nodes.
spec:
  selector:
    matchLabels:
      app: sriov-cni
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: sriov-cni
        component: network
        type: infra
        openshift.io/component: network
    spec:
      nodeSelector:
        beta.kubernetes.io/os: linux
      tolerations:
      - operator: Exists
      serviceAccountName: sriov-cni
      containers:
      - name: sriov-cni
        image: quay.io/openshift/ose-sriov-cni:v4.0.0
        securityContext:
          privileged: true
        volumeMounts:
        - name: cnibin
          mountPath: /host/opt/cni/bin
      volumes:
        - name: cnibin
          hostPath:
            path: /var/lib/cni/bin

Run the following command to create the sriov-cni DaemonSet:

$ oc create -f sriov-cni.yaml

Create a YAML file for the Custom Resource (CR) with SR-IOV configuration. The name field in the following CR has the value sriov-conf.

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov-conf
  annotations:
    k8s.v1.cni.cncf.io/resourceName: openshift.io/sriov 1
spec:
  config: '{
      "type": "sriov", 2
      "name": "sriov-conf",
      "ipam": {
        "type": "host-local",
        "subnet": "10.56.217.0/24",
        "routes": [{
          "dst": "0.0.0.0/0"
        }],
        "gateway": "10.56.217.1"
      }
    }'

Run the following command to create the sriov-conf CR:

$ oc create -f sriov-conf.yaml

Create a YAML file for a Pod which references the name of the NetworkAttachmentDefinition and requests one openshift.io/sriov resource:

apiVersion: v1
kind: Pod
metadata:
  name: sriovsamplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: sriov-conf
spec:
  containers:
  - name: sriovsamplepod
    command: ["/bin/bash", "-c", "sleep 2000000000000"]
    image: centos/tools
    resources:
      requests:
        openshift.io/sriov: '1'
      limits:
        openshift.io/sriov: '1'

Run the following command to create the sriovsamplepod Pod:

$ oc create -f sriovsamplepod.yaml

View the additional interface by executing the ip command:

$ oc exec sriovsamplepod -- ip a