Chapter 6. Configuring IPsec encryption

Procedure

1. To enable IPsec encryption, enter the following command:

   $ oc patch networks.operator.openshift.io cluster --type=merge -p \
     '{
     "spec":{
       "defaultNetwork":{
         "ovnKubernetesConfig":{
           "ipsecConfig":{
             "mode":"<mode"> 

   1

   
           }}}}}'

   Copy to Clipboard Toggle word wrap

   1

   Specify External to encrypt traffic to external hosts or specify Full to encrypt pod-to-pod traffic and, optionally, traffic to external hosts. By default, IPsec is disabled.

   Example configuration that has IPsec enabled in Full mode and encapsulation set to Always

   $ oc patch networks.operator.openshift.io cluster --type=merge -p \
     '{
     "spec":{
       "defaultNetwork":{
         "ovnKubernetesConfig":{
           "ipsecConfig":{
             "mode":"Full",
             "full":{
               "encapsulation": "Always"
             }}}}}}'

   Copy to Clipboard Toggle word wrap

2. Encrypt external traffic with IPsec by completing the "Configuring IPsec encryption for external traffic" procedure.

Verification

1. To find the names of the OVN-Kubernetes data plane pods, enter the following command:

   $ oc get pods -n openshift-ovn-kubernetes -l=app=ovnkube-node

   Copy to Clipboard Toggle word wrap

   Example output

   ovnkube-node-5xqbf                       8/8     Running   0              28m
   ovnkube-node-6mwcx                       8/8     Running   0              29m
   ovnkube-node-ck5fr                       8/8     Running   0              31m
   ovnkube-node-fr4ld                       8/8     Running   0              26m
   ovnkube-node-wgs4l                       8/8     Running   0              33m
   ovnkube-node-zfvcl                       8/8     Running   0              34m
   ...

   Copy to Clipboard Toggle word wrap

2. Verify that you enabled IPsec on your cluster by running the following command:

   Note

   As a cluster administrator, you can verify that you enabled IPsec between pods on your cluster when you configured IPsec in Full mode. This step does not verify whether IPsec is working between your cluster and external hosts.

   $ oc -n openshift-ovn-kubernetes rsh ovnkube-node-<XXXXX> ovn-nbctl --no-leader-only get nb_global . ipsec 

   1

   Copy to Clipboard Toggle word wrap

   where: <XXXXX> specifies the random sequence of letters for a pod from an earlier step.

   Successful output from the command shows the status as true.

Procedure

1. Create an IPsec configuration with an NMState Operator node network configuration policy. For more information, see Libreswan as an IPsec VPN implementation.

   1. To identify the IP address of the cluster node that is the IPsec endpoint, enter the following command:

      $ oc get nodes

      Copy to Clipboard Toggle word wrap

   2. Create a file named ipsec-config.yaml that contains a node network configuration policy for the NMState Operator, such as in the following examples. For an overview about NodeNetworkConfigurationPolicy objects, see The Kubernetes NMState project.

      Example NMState IPsec transport configuration

      apiVersion: nmstate.io/v1
      kind: NodeNetworkConfigurationPolicy
      metadata:
        name: ipsec-config
      spec:
        nodeSelector:
          kubernetes.io/hostname: "<hostname>" 

      1

      
        desiredState:
          interfaces:
          - name: <interface_name> 

      2

      
            type: ipsec
            libreswan:
              left: <cluster_node> 

      3

      
              leftid: '%fromcert'
              leftrsasigkey: '%cert'
              leftcert: left_server
              leftmodecfgclient: false
              right: <external_host> 

      4

      
              rightid: '%fromcert'
              rightrsasigkey: '%cert'
              rightsubnet: <external_address>/32 

      5

      
              ikev2: insist
              type: transport

      Copy to Clipboard Toggle word wrap

      1 1

      Specifies the host name to apply the policy to. This host serves as the left side host in the IPsec configuration.

      2

      Specifies the name of the interface to create on the host.

      3

      Specifies the host name of the cluster node that terminates the IPsec tunnel on the cluster side. The name should match SAN [Subject Alternate Name] from your supplied PKCS#12 certificates.

      4

      Specifies the external host name, such as host.example.com. The name should match the SAN [Subject Alternate Name] from your supplied PKCS#12 certificates.

      5

      Specifies the IP address of the external host, such as 10.1.2.3/32.

      Example NMState IPsec tunnel configuration

      apiVersion: nmstate.io/v1
      kind: NodeNetworkConfigurationPolicy
      metadata:
        name: ipsec-config
      spec:
        nodeSelector:
          kubernetes.io/hostname: "<hostname>" 

      1

      
        desiredState:
          interfaces:
          - name: <interface_name> 

      2

      
            type: ipsec
            libreswan:
              left: <cluster_node> 

      3

      
              leftid: '%fromcert'
              leftmodecfgclient: false
              leftrsasigkey: '%cert'
              leftcert: left_server
              right: <external_host> 

      4

      
              rightid: '%fromcert'
              rightrsasigkey: '%cert'
              rightsubnet: <external_address>/32 

      5

      
              ikev2: insist
              type: tunnel

      Copy to Clipboard Toggle word wrap

      1

      Specifies the host name to apply the policy to. This host serves as the left side host in the IPsec configuration.

      2

      Specifies the name of the interface to create on the host.

      3

      Specifies the host name of the cluster node that terminates the IPsec tunnel on the cluster side. The name should match SAN [Subject Alternate Name] from your supplied PKCS#12 certificates.

      4

      Specifies the external host name, such as host.example.com. The name should match the SAN [Subject Alternate Name] from your supplied PKCS#12 certificates.

      5

      Specifies the IP address of the external host, such as 10.1.2.3/32.

   3. To configure the IPsec interface, enter the following command:

      $ oc create -f ipsec-config.yaml

      Copy to Clipboard Toggle word wrap

2. Provide the following certificate files to add to the Network Security Services (NSS) database on each host. These files are imported as part of the Butane configuration in later steps.

   • left_server.p12: The certificate bundle for the IPsec endpoints
   • ca.pem: The certificate authority that you signed your certificates with

3. Create a machine config to add your certificates to the cluster:

   1. To create Butane config files for the control plane and worker nodes, enter the following command:

      Note

      The Butane version you specify in the config file should match the OpenShift Container Platform version and always ends in 0. For example, 4.19.0. See "Creating machine configs with Butane" for information about Butane.

      $ for role in master worker; do
        cat >> "99-ipsec-${role}-endpoint-config.bu" <<-EOF
        variant: openshift
        version: 4.19.0
        metadata:
          name: 99-${role}-import-certs
          labels:
            machineconfiguration.openshift.io/role: $role
        systemd:
          units:
          - name: ipsec-import.service
            enabled: true
            contents: |
              [Unit]
              Description=Import external certs into ipsec NSS
              Before=ipsec.service
      
              [Service]
              Type=oneshot
              ExecStart=/usr/local/bin/ipsec-addcert.sh
              RemainAfterExit=false
              StandardOutput=journal
      
              [Install]
              WantedBy=multi-user.target
        storage:
          files:
          - path: /etc/pki/certs/ca.pem
            mode: 0400
            overwrite: true
            contents:
              local: ca.pem
          - path: /etc/pki/certs/left_server.p12
            mode: 0400
            overwrite: true
            contents:
              local: left_server.p12
          - path: /usr/local/bin/ipsec-addcert.sh
            mode: 0740
            overwrite: true
            contents:
              inline: |
                #!/bin/bash -e
                echo "importing cert to NSS"
                certutil -A -n "CA" -t "CT,C,C" -d /var/lib/ipsec/nss/ -i /etc/pki/certs/ca.pem
                pk12util -W "" -i /etc/pki/certs/left_server.p12 -d /var/lib/ipsec/nss/
                certutil -M -n "left_server" -t "u,u,u" -d /var/lib/ipsec/nss/
      EOF
      done

      Copy to Clipboard Toggle word wrap

   2. To transform the Butane files that you created in an earlier step into machine configs, enter the following command:

      $ for role in master worker; do
        butane -d . 99-ipsec-${role}-endpoint-config.bu -o ./99-ipsec-$role-endpoint-config.yaml
      done

      Copy to Clipboard Toggle word wrap

4. To apply the machine configs to your cluster, enter the following command:

   $ for role in master worker; do
     oc apply -f 99-ipsec-${role}-endpoint-config.yaml
   done

   Copy to Clipboard Toggle word wrap
   Important

   As the Machine Config Operator (MCO) updates machines in each machine config pool, it reboots each node one by one. You must wait until all the nodes to update before external IPsec connectivity is available.

5. Check the machine config pool status by entering the following command:

   $ oc get mcp

   Copy to Clipboard Toggle word wrap

   A successfully updated node has the following status: UPDATED=true, UPDATING=false, DEGRADED=false.

   Note

   By default, the MCO updates one machine per pool at a time, causing the total time the migration takes to increase with the size of the cluster.

6. To confirm that IPsec machine configs rolled out successfully, enter the following commands:

   1. Confirm the creation of the IPsec machine configs:

      $ oc get mc | grep ipsec

      Copy to Clipboard Toggle word wrap

      Example output

      80-ipsec-master-extensions        3.2.0        6d15h
      80-ipsec-worker-extensions        3.2.0        6d15h

      Copy to Clipboard Toggle word wrap

   2. Confirm the application of the IPsec extension to control plane nodes. Example output would show 2.

      $ oc get mcp master -o yaml | grep 80-ipsec-master-extensions -c

      Copy to Clipboard Toggle word wrap

   3. Confirm the application of the IPsec extension to compute nodes. Example output would show 2.

      $ oc get mcp worker -o yaml | grep 80-ipsec-worker-extensions -c

      Copy to Clipboard Toggle word wrap

Procedure

1. Create a file named remove-ipsec-tunnel.yaml with the following YAML:

   kind: NodeNetworkConfigurationPolicy
   apiVersion: nmstate.io/v1
   metadata:
     name: <name>
   spec:
     nodeSelector:
       kubernetes.io/hostname: <node_name>
     desiredState:
       interfaces:
       - name: <tunnel_name>
         type: ipsec
         state: absent

   Copy to Clipboard Toggle word wrap

   where:

   name

   Specifies a name for the node network configuration policy.

   node_name

   Specifies the name of the node where the IPsec tunnel that you want to remove exists.

   tunnel_name

   Specifies the interface name for the existing IPsec tunnel.

2. To remove the IPsec tunnel, enter the following command:

   $ oc apply -f remove-ipsec-tunnel.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Choose one of the following options to disable IPsec encryption:

   1. Where the ipsecConfig.mode parameter is set to either External or Full and the ipsecConfig.full schema is not added to networks.operator.openshift.io, enter the following command:

      $ oc patch networks.operator.openshift.io cluster --type=merge -p \
        '{
        "spec":{
          "defaultNetwork":{
            "ovnKubernetesConfig":{
              "ipsecConfig":{
                "mode":"Disabled"
              }}}}}'

      Copy to Clipboard Toggle word wrap

   2. Where the ipsecConfig.mode parameter is set to Full and the ipsecConfig.full configuration is added to networks.operator.openshift.io, enter the following command:

      $ oc patch networks.operator.openshift.io cluster --type='json' -p \
            '[{"op": "remove", "path": "/spec/defaultNetwork/ovnKubernetesConfig/ipsecConfig/full"},
            {"op": "replace", "path": "/spec/defaultNetwork/ovnKubernetesConfig/ipsecConfig/mode", "value": "Disabled"}]'

      Copy to Clipboard Toggle word wrap
2. Optional: You can increase the size of your cluster MTU by 46 bytes because there is no longer any overhead from the IPsec Encapsulating Security Payload (ESP) header in IP packets.