Chapter 10. Configuring ingress cluster traffic
10.1. Configuring ingress cluster traffic overview
OpenShift Container Platform provides the following methods for communicating from outside the cluster with services running in the cluster.
The methods are recommended, in order or preference:
- If you have HTTP/HTTPS, use an Ingress Controller.
- If you have a TLS-encrypted protocol other than HTTPS. For example, for TLS with the SNI header, use an Ingress Controller.
-
Otherwise, use a Load Balancer, an External IP, or a
NodePort
.
Method | Purpose |
---|---|
Allows access to HTTP/HTTPS traffic and TLS-encrypted protocols other than HTTPS (for example, TLS with the SNI header). | |
Automatically assign an external IP using a load balancer service | Allows traffic to non-standard ports through an IP address assigned from a pool. |
Allows traffic to non-standard ports through a specific IP address. | |
Expose a service on all nodes in the cluster. |
10.2. Configuring ingress cluster traffic using an Ingress Controller
OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses an Ingress Controller.
10.2.1. Using Ingress Controllers and routes
The Ingress Operator manages Ingress Controllers and wildcard DNS.
Using an Ingress Controller is the most common way to allow external access to an OpenShift Container Platform cluster.
An Ingress Controller is configured to accept external requests and proxy them based on the configured routes. This is limited to HTTP, HTTPS using SNI, and TLS using SNI, which is sufficient for web applications and services that work over TLS with SNI.
Work with your administrator to configure an Ingress Controller to accept external requests and proxy them based on the configured routes.
The administrator can create a wildcard DNS entry and then set up an Ingress Controller. Then, you can work with the edge Ingress Controller without having to contact the administrators.
When a set of routes is created in various projects, the overall set of routes is available to the set of Ingress Controllers. Each Ingress Controller admits routes from the set of routes. By default, all Ingress Controllers admit all routes.
The Ingress Controller:
- Has two replicas by default, which means it should be running on two worker nodes.
- Can be scaled up to have more replicas on more nodes.
The procedures in this section require prerequisites performed by the cluster administrator.
Prerequisites
Before starting the following procedures, the administrator must:
- Set up the external port to the cluster networking environment so that requests can reach the cluster.
Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command:
oc adm policy add-cluster-role-to-user cluster-admin username
- Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.
10.2.2. Creating a project and service
If the project and service that you want to expose do not exist, first create the project, then the service.
If the project and service already exist, skip to the procedure on exposing the service to create a route.
Prerequisites
-
Install the
oc
CLI and log in as a cluster administrator.
Procedure
Create a new project for your service:
$ oc new-project <project_name>
For example:
$ oc new-project myproject
Use the
oc new-app
command to create a service. For example:$ oc new-app \ -e MYSQL_USER=admin \ -e MYSQL_PASSWORD=redhat \ -e MYSQL_DATABASE=mysqldb \ registry.redhat.io/rhscl/mysql-80-rhel7
Run the following command to see that the new service is created:
$ oc get svc -n myproject NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-80-rhel7 ClusterIP 172.30.63.31 <none> 3306/TCP 4m55s
By default, the new service does not have an external IP address.
10.2.3. Exposing the service by creating a route
You can expose the service as a route by using the oc expose
command.
Procedure
To expose the service:
- Log in to OpenShift Container Platform.
Log in to the project where the service you want to expose is located:
$ oc project project1
Run the following command to expose the route:
$ oc expose service <service_name>
For example:
$ oc expose service mysql-80-rhel7 route "mysql-80-rhel7" exposed
Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service:
$ curl <pod_ip>:<port>
For example:
$ curl 172.30.131.89:3306
The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the
Got packets out of order
message, you are connected to the service.If you have a MySQL client, log in with the standard CLI command:
$ mysql -h 172.30.131.89 -u admin -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. MySQL [(none)]>
10.2.4. Configuring Ingress Controller sharding by using route labels
Ingress Controller sharding by using route labels means that the Ingress Controller serves any route in any namespace that is selected by the route selector.
Ingress Controller sharding is useful when balancing incoming traffic load among a set of Ingress Controllers and when isolating traffic to a specific Ingress Controller. For example, company A goes to one Ingress Controller and company B to another.
Procedure
Edit the
router-internal.yaml
file:# cat router-internal.yaml apiVersion: v1 items: - apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: sharded namespace: openshift-ingress-operator spec: domain: <apps-sharded.basedomain.example.net> nodePlacement: nodeSelector: matchLabels: node-role.kubernetes.io/worker: "" routeSelector: matchLabels: type: sharded status: {} kind: List metadata: resourceVersion: "" selfLink: ""
Apply the Ingress Controller
router-internal.yaml
file:# oc apply -f router-internal.yaml
The Ingress Controller selects routes in any namespace that have the label
type: sharded
.
10.2.5. Configuring Ingress Controller sharding by using namespace labels
Ingress Controller sharding by using namespace labels means that the Ingress Controller serves any route in any namespace that is selected by the namespace selector.
Ingress Controller sharding is useful when balancing incoming traffic load among a set of Ingress Controllers and when isolating traffic to a specific Ingress Controller. For example, company A goes to one Ingress Controller and company B to another.
Procedure
Edit the
router-internal.yaml
file:# cat router-internal.yaml apiVersion: v1 items: - apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: sharded namespace: openshift-ingress-operator spec: domain: <apps-sharded.basedomain.example.net> nodePlacement: nodeSelector: matchLabels: node-role.kubernetes.io/worker: "" namespaceSelector: matchLabels: type: sharded status: {} kind: List metadata: resourceVersion: "" selfLink: ""
Apply the Ingress Controller
router-internal.yaml
file:# oc apply -f router-internal.yaml
The Ingress Controller selects routes in any namespace that is selected by the namespace selector that have the label
type: sharded
.
10.2.6. Additional resources
- The Ingress Operator manages wildcard DNS. For more information, see Ingress Operator in OpenShift Container Platform, Installing a cluster on bare metal, and Installing a cluster on vSphere.
10.3. Configuring ingress cluster traffic using a load balancer
OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses a load balancer.
10.3.1. Using a load balancer to get traffic into the cluster
If you do not need a specific external IP address, you can configure a load balancer service to allow external access to an OpenShift Container Platform cluster.
A load balancer service allocates a unique IP. The load balancer has a single edge router IP, which can be a virtual IP (VIP), but is still a single machine for initial load balancing.
If a pool is configured, it is done at the infrastructure level, not by a cluster administrator.
The procedures in this section require prerequisites performed by the cluster administrator.
Prerequisites
Before starting the following procedures, the administrator must:
- Set up the external port to the cluster networking environment so that requests can reach the cluster.
Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command:
oc adm policy add-cluster-role-to-user cluster-admin username
- Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.
10.3.2. Creating a project and service
If the project and service that you want to expose do not exist, first create the project, then the service.
If the project and service already exist, skip to the procedure on exposing the service to create a route.
Prerequisites
-
Install the
oc
CLI and log in as a cluster administrator.
Procedure
Create a new project for your service:
$ oc new-project <project_name>
For example:
$ oc new-project myproject
Use the
oc new-app
command to create a service. For example:$ oc new-app \ -e MYSQL_USER=admin \ -e MYSQL_PASSWORD=redhat \ -e MYSQL_DATABASE=mysqldb \ registry.redhat.io/rhscl/mysql-80-rhel7
Run the following command to see that the new service is created:
$ oc get svc -n myproject NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-80-rhel7 ClusterIP 172.30.63.31 <none> 3306/TCP 4m55s
By default, the new service does not have an external IP address.
10.3.3. Exposing the service by creating a route
You can expose the service as a route by using the oc expose
command.
Procedure
To expose the service:
- Log in to OpenShift Container Platform.
Log in to the project where the service you want to expose is located:
$ oc project project1
Run the following command to expose the route:
$ oc expose service <service_name>
For example:
$ oc expose service mysql-80-rhel7 route "mysql-80-rhel7" exposed
Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service:
$ curl <pod_ip>:<port>
For example:
$ curl 172.30.131.89:3306
The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the
Got packets out of order
message, you are connected to the service.If you have a MySQL client, log in with the standard CLI command:
$ mysql -h 172.30.131.89 -u admin -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. MySQL [(none)]>
10.3.4. Creating a load balancer service
Use the following procedure to create a load balancer service.
Prerequisites
- Make sure that the project and service you want to expose exist.
Procedure
To create a load balancer service:
- Log in to OpenShift Container Platform.
Load the project where the service you want to expose is located.
$ oc project project1
Open a text file on the master node and paste the following text, editing the file as needed:
Sample load balancer configuration file
apiVersion: v1 kind: Service metadata: name: egress-2 1 spec: ports: - name: db port: 3306 2 loadBalancerIP: type: LoadBalancer 3 selector: name: mysql 4
- Save and exit the file.
Run the following command to create the service:
oc create -f <file-name>
For example:
oc create -f mysql-lb.yaml
Execute the following command to view the new service:
$ oc get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE egress-2 LoadBalancer 172.30.22.226 ad42f5d8b303045-487804948.example.com 3306:30357/TCP 15m
The service has an external IP address automatically assigned if there is a cloud provider enabled.
On the master, use a tool, such as cURL, to make sure you can reach the service using the public IP address:
$ curl <public-ip>:<port>
For example:
$ curl 172.29.121.74:3306
The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the
Got packets out of order
message, you are connecting with the service:If you have a MySQL client, log in with the standard CLI command:
$ mysql -h 172.30.131.89 -u admin -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. MySQL [(none)]>
10.4. Configuring ingress cluster traffic using a service external IP
OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses a service external IP.
10.4.1. Using a service external IP to get traffic into the cluster
One method to expose a service is to assign an external IP address directly to the service you want to make accessible from outside the cluster.
The external IP address that you use must be provisioned on your infrastructure platform and attached to a cluster node.
With an external IP on the service, OpenShift Container Platform sets up NAT rules to allow traffic arriving at any cluster node attached to that IP address to be sent to one of the internal pods. This is similar to the internal service IP addresses, but the external IP tells OpenShift Container Platform that this service should also be exposed externally at the given IP. The administrator must assign the IP address to a host (node) interface on one of the nodes in the cluster. Alternatively, the address can be used as a virtual IP (VIP).
These IPs are not managed by OpenShift Container Platform and administrators are responsible for ensuring that traffic arrives at a node with this IP.
The procedures in this section require prerequisites performed by the cluster administrator.
Prerequisites
Before starting the following procedures, the administrator must:
- Set up the external port to the cluster networking environment so that requests can reach the cluster.
Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command:
oc adm policy add-cluster-role-to-user cluster-admin username
- Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.
10.4.2. Creating a project and service
If the project and service that you want to expose do not exist, first create the project, then the service.
If the project and service already exist, skip to the procedure on exposing the service to create a route.
Prerequisites
-
Install the
oc
CLI and log in as a cluster administrator.
Procedure
Create a new project for your service:
$ oc new-project <project_name>
For example:
$ oc new-project myproject
Use the
oc new-app
command to create a service. For example:$ oc new-app \ -e MYSQL_USER=admin \ -e MYSQL_PASSWORD=redhat \ -e MYSQL_DATABASE=mysqldb \ registry.redhat.io/rhscl/mysql-80-rhel7
Run the following command to see that the new service is created:
$ oc get svc -n myproject NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-80-rhel7 ClusterIP 172.30.63.31 <none> 3306/TCP 4m55s
By default, the new service does not have an external IP address.
10.4.3. Exposing the service by creating a route
You can expose the service as a route by using the oc expose
command.
Procedure
To expose the service:
- Log in to OpenShift Container Platform.
Log in to the project where the service you want to expose is located:
$ oc project project1
Run the following command to expose the route:
$ oc expose service <service_name>
For example:
$ oc expose service mysql-80-rhel7 route "mysql-80-rhel7" exposed
Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service:
$ curl <pod_ip>:<port>
For example:
$ curl 172.30.131.89:3306
The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the
Got packets out of order
message, you are connected to the service.If you have a MySQL client, log in with the standard CLI command:
$ mysql -h 172.30.131.89 -u admin -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. MySQL [(none)]>
10.5. Configuring ingress cluster traffic using a NodePort
OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses a NodePort
.
10.5.1. Using a NodePort to get traffic into the cluster
Use a NodePort
-type Service
resource to expose a service on a specific port on all nodes in the cluster. The port is specified in the Service
resource’s .spec.ports[*].nodePort
field.
Using NodePort
s requires additional port resources.
A NodePort
exposes the service on a static port on the node’s IP address. NodePort
s are in the 30000
to 32767
range by default, which means a NodePort
is unlikely to match a service’s intended port. For example, port 8080
may be exposed as port 31020
on the node.
The administrator must ensure the external IP addresses are routed to the nodes.
NodePort
s and external IPs are independent and both can be used concurrently.
The procedures in this section require prerequisites performed by the cluster administrator.
Prerequisites
Before starting the following procedures, the administrator must:
- Set up the external port to the cluster networking environment so that requests can reach the cluster.
Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command:
$ oc adm policy add-cluster-role-to-user cluster-admin <user_name>
- Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.
10.5.2. Creating a project and service
If the project and service that you want to expose do not exist, first create the project, then the service.
If the project and service already exist, skip to the procedure on exposing the service to create a route.
Prerequisites
-
Install the
oc
CLI and log in as a cluster administrator.
Procedure
Create a new project for your service:
$ oc new-project <project_name>
For example:
$ oc new-project myproject
Use the
oc new-app
command to create a service. For example:$ oc new-app \ -e MYSQL_USER=admin \ -e MYSQL_PASSWORD=redhat \ -e MYSQL_DATABASE=mysqldb \ registry.redhat.io/rhscl/mysql-80-rhel7
Run the following command to see that the new service is created:
$ oc get svc -n myproject NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-80-rhel7 ClusterIP 172.30.63.31 <none> 3306/TCP 4m55s
By default, the new service does not have an external IP address.
10.5.3. Exposing the service by creating a route
You can expose the service as a route by using the oc expose
command.
Procedure
To expose the service:
- Log in to OpenShift Container Platform.
Log in to the project where the service you want to expose is located:
$ oc project project1
To expose a node port for the application, enter the following command. OpenShift Container Platform automatically selects an available port in the
30000-32767
range.$ oc expose dc mysql-80-rhel7 --type=NodePort --name=mysql-ingress
Optional: To confirm the service is available with a node port exposed, enter the following command:
$ oc get svc -n myproject NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-80-rhel7 ClusterIP 172.30.217.127 <none> 3306/TCP 9m44s mysql-ingress NodePort 172.30.107.72 <none> 3306:31345/TCP 39s
Optional: To remove the service created automatically by the
oc new-app
command, enter the following command:$ oc delete svc mysql-80-rhel7