Chapter 16. Configuring Kuryr SDN

Kuryr (or more specifically Kuryr-Kubernetes) is an SDN solution built using CNI and OpenStack Neutron. Its advantages include being able to use a wide range of Neutron SDN backends and providing inter-connectivity between Kubernetes pods and OpenStack virtual machines (VMs).

Kuryr-Kubernetes and OpenShift Container Platform integration is primarily designed for OpenShift Container Platform clusters running on OpenStack VMs. Kuryr-Kubernetes components are installed as pods on OpenShift Container Platform in the openshift-infra namespace:

The system running openshift-ansible must be subscribed to the OSP as well as OCP repositories. The OpenStack integration requires a few extra packages. To install the dependencies, run:

$ sudo yum install -y ansible openshift-ansible python2-shade python-dns \
    python2-heatclient python2-octaviaclient python-openstackclient bind-utils

In the Ansible nodes file, specify the following parameters in order to set up Kuryr-Kubernetes as the network plug-in:

 # Enable Kuryr.
 openshift_use_openshift_sdn=False
 openshift_use_kuryr=True
 os_sdn_network_plugin_name=cni

 # Set userspace so that there are no iptables remains.
 openshift_node_proxy_mode='userspace'

 # Keystone URL.
 kuryr_openstack_auth_url=http://127.0.0.1/identity

 # OpenStack domain name of user owning Kuryr resources.
 kuryr_openstack_user_domain_name=default

 # OpenStack project name of user owning Kuryr resources.
 kuryr_openstack_user_project_name=admin

 # OpenStack project id for Kuryr resources.
 kuryr_openstack_project_id=ec0b31802fd043c08bc15b74d2f9a3d3

 # OpenStack username that will own kuryr resources.
 kuryr_openstack_username=admin

 # Password for that user.
 kuryr_openstack_password=password

 # Default Neutron security groups' IDs for Kubernetes pods
 kuryr_openstack_pod_sg_id=f74c83a8-a520-421a-930e-21b6cd098c6a,01f85594-9950-4ded-a92c-5ad546a41188

 # Default Neutron subnet ID for Kubernetes pods.
 kuryr_openstack_pod_subnet_id=c85cdee6-0ed1-4d8f-ae61-7afa4674b311

 # Default OpenStack project ID for Kubernetes resources.
 kuryr_openstack_pod_project_id=ec0b31802fd043c08bc15b74d2f9a3d3

 # Neutron subnet ID for Kubernetes worker node VMs.
 kuryr_openstack_worker_nodes_subnet_id=477cfa49-e641-4d31-a7b5-5bc834743f61

 # Default Neutron subnet ID for Kubernetes services.
 kuryr_openstack_service_subnet_id=3b31a106-4084-4db9-bc0c-00b97afe186e

You must also specify an OpenStack cloud provider as described in the OpenStack configuration documentation.

Prior to the installation, you must also provide a DNS server the OpenShift Container Platform nodes will be using for internal name resolution. OpenStack does not provide a node name resolution out of the box. In the following example, 10.20.30.40 is the IP address of the DNS server:

openshift_openstack_dns_nameservers=[10.20.30.40]

If the DNS server supports remote updates via nsupdate (RFC 2136), the playbooks can populate it automatically, if you add the following configuration:

openshift_openstack_external_nsupdate_keys={private: {"key_secret": "<nsupdate key>", "key_algorithm": "<nsupdate key algorithm>", "key_name": "<nsupdate key name>", "server": 10.20.30.40}}

Finally, install OpenShift Container Platform by running the provision_install.yml playbook. You must specify the dynamic inventory file, inventory.py, and the the path to the Ansible nodes file that you created:

$ ansible-playbook --user openshift -i /usr/share/ansible/openshift-ansible/playbooks/openstack/inventory.py -i ansible-nodes.txt /usr/share/ansible/openshift-ansible/playbooks/openstack/openshift-cluster/provision_install.yml

If you want to do any custom setup on the created nodes before the OpenShift Container Platform installation, you can run the provision.yml and install.yml playbooks separately. provision.yml will create the OpenStack resources (nodes, networks, and so on) and install.yml will install OpenShift Container Platform.

Once the installation of OpenShift Container Platform is finished, you can check if Kuryr pods are deployed successfully:

$ oc -n openshift-infra get pods -o wide
NAME                                READY     STATUS    RESTARTS   AGE       IP              NODE
bootstrap-autoapprover-0            1/1       Running   0          3d        10.11.0.7       master-0.openshift.example.com
kuryr-cni-ds-66kt2                  2/2       Running   0          3d        192.168.99.14   infra-node-0.openshift.example.com
kuryr-cni-ds-ggcpz                  2/2       Running   0          3d        192.168.99.16   master-0.openshift.example.com
kuryr-cni-ds-mhzjt                  2/2       Running   0          3d        192.168.99.6    app-node-1.openshift.example.com
kuryr-cni-ds-njctb                  2/2       Running   0          3d        192.168.99.12   app-node-0.openshift.example.com
kuryr-cni-ds-v8hp8                  2/2       Running   0          3d        192.168.99.5    infra-node-1.openshift.example.com
kuryr-controller-59fc7f478b-qwk4k   1/1       Running   0          3d        192.168.99.5    infra-node-1.openshift.example.com

kuryr-cni pods should run on every OpenShift Container Platform node. Single kuryr-controller instances should run on any of the nodes.