Chapter 18. Configuring NSX-T SDN

ESXi servers that host OpenShift Container Platform node VMs must be NSX-T Transport Nodes.

Figure 18.1. NSX UI dislaying the Transport Nodes for a typical high availability environment:

The second vNIC on all VMs must be tagged in NSX-T so that the NSX Container Plug-in (NCP) knows which port needs to be used as a parent VIF for all Pods running in a particular OpenShift Container Platform node. The tags must be the following:

{'ncp/node_name': 'node_name'}
{'ncp/cluster': 'cluster_name'}

The following image shows how the tags in NSX UI for all nodes. For a large scale cluster, you can automate the tagging using API Call or by using Ansible.

Figure 18.2. NSX UI dislaying node tags

The order of the tags in the NSX UI is opposite from the API. The node name must be exactly as kubelet expects and the cluster name must be the same as the nsx_openshift_cluster_name in the Ansible hosts file, as shown below. Ensure that the proper tags are applied on the second vNIC on every node.

Run the following command to install required software packages, if any, for OpenShift Container Platform:

$ ansible-playbook -i hosts openshift-ansible/playbooks/prerequisites.yml

After the prerequisites.yml playbook has successfully executed, run the following command on all nodes, replacing the xxx with the NCP build version:

$ docker load -i nsx-ncp-rhel-xxx.tar

For example:

$ docker load -i nsx-ncp-rhel-2.4.0.12511604.tar

Get the image name and retag it:

$ docker images
$ docker image tag registry.local/xxxxx/nsx-ncp-rhel nsx-ncp 1

docker image tag registry.local/2.4.0.12511604/nsx-ncp-rhel nsx-ncp

In the OpenShift Container Platform Ansible hosts file, specify the following parameters to set up NSX-T as the network plug-in:

[OSEv3:children]
masters
nodes
etcd

[OSEv3:vars]
ansible_ssh_user=root
openshift_deployment_type=origin
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_master_htpasswd_users={"admin" : "$apr1$H0QeP6oX$HHdscz5gqMdtTcT5eoCJ20"}
openshift_master_default_subdomain=demo.example.com
openshift_use_nsx=true
os_sdn_network_plugin_name=cni
openshift_use_openshift_sdn=false
openshift_node_sdn_mtu=1500
openshift_master_cluster_method=native
openshift_master_cluster_hostname=master01.example.com
openshift_master_cluster_public_hostname=master01.example.com
openshift_hosted_manage_registry=true
openshift_hosted_manage_router=true
openshift_enable_service_catalog=true
openshift_cluster_monitoring_operator_install=true
openshift_web_console_install=true
openshift_console_install=true

# NSX-T specific configuration
#nsx_use_loadbalancer=false
nsx_openshift_cluster_name='cluster01'
nsx_api_managers='nsxmgr.example.com'
nsx_api_user='nsx_admin'
nsx_api_password='nsx_api_password_example'
nsx_tier0_router='LR-Tier-0'
nsx_overlay_transport_zone='TZ-Overlay'
nsx_container_ip_block='pod-networking'
nsx_no_snat_ip_block='pod-nonat'
nsx_external_ip_pool='pod-external'
nsx_top_fw_section='containers-top'
nsx_bottom_fw_section='containers-bottom'
nsx_ovs_uplink_port='ens224'
nsx_cni_url='http://websrv.example.com/nsx-cni-buildversion.x86_64.rpm'
nsx_ovs_url='http://websrv.example.com/openvswitch-buildversion.rhel75-1.x86_64.rpm'
nsx_kmod_ovs_url='http://websrv.example.com/kmod-openvswitch-buildversion.rhel75-1.el7.x86_64.rpm'
nsx_insecure_ssl=true
# vSphere Cloud Provider
#openshift_cloudprovider_kind=vsphere
#openshift_cloudprovider_vsphere_username='administrator@example.com'
#openshift_cloudprovider_vsphere_password='viadmin_password'
#openshift_cloudprovider_vsphere_host='vcsa.example.com'
#openshift_cloudprovider_vsphere_datacenter='Example-Datacenter'
#openshift_cloudprovider_vsphere_cluster='example-Cluster'
#openshift_cloudprovider_vsphere_resource_pool='ocp'
#openshift_cloudprovider_vsphere_datastore='example-Datastore-name'
#openshift_cloudprovider_vsphere_folder='ocp'

[masters]
master01.example.com
master02.example.com
master03.example.com

[etcd]
master01.example.com
master02.example.com
master03.example.com

[nodes]
master01.example.com ansible_ssh_host=192.168.220.2 openshift_node_group_name='node-config-master'
master02.example.com ansible_ssh_host=192.168.220.3 openshift_node_group_name='node-config-master'
master03.example.com ansible_ssh_host=192.168.220.4 openshift_node_group_name='node-config-master'
node01.example.com ansible_ssh_host=192.168.220.5 openshift_node_group_name='node-config-infra'
node02.example.com ansible_ssh_host=192.168.220.6 openshift_node_group_name='node-config-infra'
node03.example.com ansible_ssh_host=192.168.220.7 openshift_node_group_name='node-config-compute'
node04.example.com ansible_ssh_host=192.168.220.8 openshift_node_group_name='node-config-compute'

For information on the OpenShift Container Platform installation parameters, see Configuring Your Inventory File.