Chapter 2. Configuring IPv6 single or dual-stack networking

Procedure

1. If you have not done so, make a copy of the provided config.yaml.default file in the /etc/microshift/ directory, renaming it config.yaml.

2. Keep the new MicroShift config.yaml in the /etc/microshift/ directory. Your config.yaml file is read every time the MicroShift service starts.

   Note

   After you create it, the config.yaml file takes precedence over built-in settings.

3. Replace the default values in the network section of the MicroShift YAML with your valid values.

   Example single-stack IPv6 networking configuration

   apiServer:
   # ...
   network:
     clusterNetwork:
     - fd01::/48 1
     serviceNetwork:
     - fd02::/112 2
   node:
     nodeIP: 2600:1f14:1c48:ee00:2d76:3190:5bc2:5aef 3
   # ...

   1

   Specify a clusterNetwork with a CIDR value that is less than 64.

   2

   Specify an IPv6 CIDR with a prefix of 112. Kubernetes uses only the lowest 16 bits. For a prefix of 112, IP addresses are assigned from 112 to 128 bits.

   3

   Example node IP address. Valid values are IP addresses in the IPv6 address family. You must only specify an IPv6 address when an IPv4 network is also present. If an IPv4 network is not present, the MicroShift service automatically fills in this value upon restart.

4. Complete any other configurations you require, then start MicroShift by running the following command:

   $ sudo systemctl start microshift

Verification

1. Retrieve the networks defined in the node resource by running the following command:

   $ oc get node -o jsonpath='{.items[].spec.podCIDRs[]}'

   Example output

   fd01::/48

2. Retrieve the status of the pods by running the following command:

   $ oc get pod -A -o wide

   Example output

   NAMESPACE                  NAME                                      READY   STATUS    RESTARTS   AGE   IP                      NODE           NOMINATED NODE   READINESS GATES
   kube-system                csi-snapshot-controller-bb7cb654b-rqrt6   1/1     Running   0          65s   fd01:0:0:1::5           microshift-9   <none>           <none>
   kube-system                csi-snapshot-webhook-95f475949-nbz8x      1/1     Running   0          61s   fd01:0:0:1::6           microshift-9   <none>           <none>
   openshift-dns              dns-default-cjn66                         2/2     Running   0          62s   fd01:0:0:1::9           microshift-9   <none>           <none>
   openshift-dns              node-resolver-ppnjb                       1/1     Running   0          63s   2001:db9:ca7:ff::1db8   microshift-9   <none>           <none>
   openshift-ingress          router-default-6d97d7b8b6-wdtmg           1/1     Running   0          61s   fd01:0:0:1::8           microshift-9   <none>           <none>
   openshift-ovn-kubernetes   ovnkube-master-gfvp5                      4/4     Running   0          63s   2001:db9:ca7:ff::1db8   microshift-9   <none>           <none>
   openshift-ovn-kubernetes   ovnkube-node-bnpjh                        1/1     Running   0          63s   2001:db9:ca7:ff::1db8   microshift-9   <none>           <none>
   openshift-service-ca       service-ca-5d7bd9db6-j25bd                1/1     Running   0          60s   fd01:0:0:1::4           microshift-9   <none>           <none>
   openshift-storage          lvms-operator-656cd9b59b-bwr47            1/1     Running   0          63s   fd01:0:0:1::7           microshift-9   <none>           <none>
   openshift-storage          vg-manager-f7dmk                          1/1     Running   0          27s   fd01:0:0:1::a           microshift-9   <none>           <none>

3. Retrieve the status of services by running the following command:

   $ oc get svc -A

   Example output

   NAMESPACE           NAME                            TYPE           CLUSTER-IP   EXTERNAL-IP                                             PORT(S)                      AGE
   default             kubernetes                      ClusterIP      fd02::1      <none>                                                  443/TCP                      3m42s
   kube-system         csi-snapshot-webhook            ClusterIP      fd02::4c4f   <none>                                                  443/TCP                      3m20s
   openshift-dns       dns-default                     ClusterIP      fd02::a      <none>                                                  53/UDP,53/TCP,9154/TCP       2m58s
   openshift-ingress   router-default                  LoadBalancer   fd02::f2e6   2001:db9:ca7:ff::1db8,fd01:0:0:1::2,fd02::1:0,fd69::2   80:31133/TCP,443:31996/TCP   2m58s
   openshift-ingress   router-internal-default         ClusterIP      fd02::c55e   <none>                                                  80/TCP,443/TCP,1936/TCP      2m58s
   openshift-storage   lvms-operator-metrics-service   ClusterIP      fd02::7afb   <none>                                                  443/TCP                      2m58s
   openshift-storage   lvms-webhook-service            ClusterIP      fd02::d8dd   <none>                                                  443/TCP                      2m58s
   openshift-storage   vg-manager-metrics-service      ClusterIP      fd02::fc1    <none>                                                  443/TCP                      2m58s

Procedure

1. If you have not done so, make a copy of the provided config.yaml.default file in the /etc/microshift/ directory, renaming it config.yaml.

2. Keep the new MicroShift config.yaml in the /etc/microshift/ directory. Your config.yaml file is read every time the MicroShift service starts.

   Note

   After you create it, the config.yaml file takes precedence over built-in settings.

3. If you have not started MicroShift, replace the default values in the network section of the MicroShift YAML with your valid values.

   Example dual-stack IPv6 networking configuration with network assignments

   apiServer:
   # ...
   apiServer:
     subjectAltNames:
     - 192.168.113.117
     - 2001:db9:ca7:ff::1db8
   network:
     clusterNetwork:
     - 10.42.0.0/16
     - fd01::/48 1
     serviceNetwork:
     - 10.43.0.0/16
     - fd02::/112 2
   node:
     nodeIP: 192.168.113.117 3
     nodeIPv6: 2001:db9:ca7:ff::1db8 4
   # ...

   1

   Specify an IPv6 clusterNetwork with a CIDR value that is less than 64.

   2

   Specify an IPv6 CIDR with a prefix of 112. Kubernetes uses only the lowest 16 bits. For a prefix of 112, IP addresses are assigned from 112 to 128 bits.

   3

   Example node IP address. Must be an IPv4 address family.

   4

   Example node IP address for dual-stack configuration. Must be an IPv6 address family. Configurable only with dual-stack networking.

4. Complete any other MicroShift configurations you require, then start MicroShift by running the following command:

   $ sudo systemctl start microshift

5. Reset the IP family policy for application pods and services as needed, then restart those application pods and services to enable dual-stack networking. See "Resetting the IP family policy for application pods and services" for a simple example.

Verification

1. You can verify that all of the system services and pods to have two IP addresses, one for each family, by using the following steps:

   1. Retrieve the networks defined in the node resource by running the following command:

      $ oc get pod -n openshift-ingress router-default-5b75594b4-w7w6s -o jsonpath='{.status.podIPs}'

      Example output

      [{"ip":"10.42.0.4"},{"ip":"fd01:0:0:1::4"}]

   2. Retrieve the networks defined by the host network pods by running the following command:

      $ oc get pod -n openshift-ovn-kubernetes ovnkube-master-2fm2k -o jsonpath='{.status.podIPs}'

      Example output

      [{"ip":"192.168.113.117"},{"ip":"2001:db9:ca7:ff::1db8"}]

Procedure

1. If you have not done so, make a copy of the provided config.yaml.default file in the /etc/microshift/ directory, renaming it config.yaml.

2. Keep the new MicroShift config.yaml in the /etc/microshift/ directory. Your config.yaml file is read every time the MicroShift service starts.

   Note

   After you create it, the config.yaml file takes precedence over built-in settings.

3. Add IPv6 configurations to the network section of the MicroShift YAML with your valid values:

   Warning

   You must keep the same first entry across restarts and migrations. This is true for any migration: single-to-dual stack, or dual-to-single stack. A complete wipe of the etcd database is required if a change to the first entry is needed. This might result in application data loss and is not supported.

   1. Add an IPv6 configuration for a second network in the network section of the MicroShift YAML with your valid values.

   2. Add network assignments to the network section of the MicroShift config.yaml to enable dual stack with IPv6 as secondary network.

      Example dual-stack IPv6 configuration with network assignments

      # ...
      apiServer:
        subjectAltNames:
        - 192.168.113.117
        - 2001:db9:ca7:ff::1db8 1
      network:
        clusterNetwork:
        - 10.42.0.0/16 2
        - fd01::/48 3
        serviceNetwork:
        - 10.43.0.0/16
        - fd02::/112 4
      node:
        nodeIP: 192.168.113.117 5
        nodeIPv6: 2001:db9:ca7:ff::1db8 6
      # ...

      1

      The IPv6 node address.

      2

      IPv4 network. Specify a clusterNetwork with a CIDR value that is less than 24.

      3

      IPv6 network. Specify a clusterNetwork with a CIDR value that is less than 64.

      4

      Specify an IPv6 CIDR with a prefix of 112. Kubernetes uses only the lowest 16 bits. For a prefix of 112, IP addresses are assigned from 112 to 128 bits.

      5

      Example node IP address. Maintain the previous IPv4 IP address.

      6

      Example node IP address. Must be an IPv6 address family.

4. Complete any other configurations you require, then restart MicroShift by running the following command:

   $ sudo systemctl restart microshift

5. Reset the IP family policy for application pods and services as needed, then restart those application pods and services to enable dual-stack networking. See "Resetting the IP family policy for application pods and services" for a simple example.

1. Retrieve the status of the pods by running the following command:

   $ oc get pod -A -o wide

   Example output

   NAMESPACE                  NAME                                      READY   STATUS    RESTARTS        AGE     IP                NODE           NOMINATED NODE   READINESS GATES
   kube-system                csi-snapshot-controller-bb7cb654b-7s5ql   1/1     Running   0               46m     10.42.0.6         microshift-9   <none>           <none>
   kube-system                csi-snapshot-webhook-95f475949-jrqv8      1/1     Running   0               46m     10.42.0.4         microshift-9   <none>           <none>
   openshift-dns              dns-default-zxkqn                         2/2     Running   0               46m     10.42.0.5         microshift-9   <none>           <none>
   openshift-dns              node-resolver-r2h5z                       1/1     Running   0               46m     192.168.113.117   microshift-9   <none>           <none>
   openshift-ingress          router-default-5b75594b4-228z7            1/1     Running   0               2m5s    10.42.0.3         microshift-9   <none>           <none>
   openshift-ovn-kubernetes   ovnkube-master-bltk7                      4/4     Running   2 (2m32s ago)   2m36s   192.168.113.117   microshift-9   <none>           <none>
   openshift-ovn-kubernetes   ovnkube-node-9ghgs                        1/1     Running   2 (2m32s ago)   46m     192.168.113.117   microshift-9   <none>           <none>
   openshift-service-ca       service-ca-5d7bd9db6-qgwgw                1/1     Running   0               46m     10.42.0.7         microshift-9   <none>           <none>
   openshift-storage          lvms-operator-656cd9b59b-8rpf4            1/1     Running   0               46m     10.42.0.8         microshift-9   <none>           <none>
   openshift-storage          vg-manager-wqmh4                          1/1     Running   2 (2m39s ago)   46m     10.42.0.10        microshift-9   <none>           <none>

2. Retrieve the networks defined by the OVN-K network plugin by running the following command:

   $ oc get pod -n openshift-ovn-kubernetes ovnkube-master-bltk7 -o jsonpath='{.status.podIPs}'

   Example output

   [{"ip":"192.168.113.117"},{"ip":"2001:db9:ca7:ff::1db8"}]

3. Retrieve the networks defined in the node resource by running the following command:

   $ oc get pod -n openshift-ingress router-default-5b75594b4-228z7 -o jsonpath='{.status.podIPs}'

   Example output

   [{"ip":"10.42.0.3"},{"ip":"fd01:0:0:1::3"}]

Procedure

1. Set the spec.ipFamilyPolicy field to a valid value for dual-stack networking in your service or pod by using the following example:

   Example dual-stack network configuration for a service

   kind: Service
   apiVersion: v1
   metadata:
     name: microshift-new-service
     labels: app: microshift-application
   spec:
     type: NodePort
     ipFamilyPolicy: `PreferDualStack` 1
   # ...

   1

   Required. Valid values for dual-stack networking are PreferDualStack and RequireDualStack. The value you set depends on the requirements of your application. PreferSingleStack is the default value for the ipFamilyPolicy field.

2. Restart any application pods that do not have a hostNetwork defined. Pods that do have a hostNetwork defined do not need to be restarted to update the ipFamilyPolicy value.