Chapter 14. Getting Traffic into the Cluster

There are many ways to access the cluster. This section describes some commonly used approaches.

The recommendation is:

TCP or UDP offers several approaches:

The router is the most common way to access the cluster. This is limited to HTTP/HTTPS(SNI)/TLS(SNI), which covers web applications.

ExternalIP or NodePort is useful when the HTTP protocol is not being used or non-standard ports are in use. There is more manual setup and monitoring involved.

The administrator must set up the external port to the cluster networking environment so that requests can reach the cluster. For example, names can be configured into DNS to point to specific nodes or other IP addresses in the cluster. The DNS wildcard feature can be used to configure a subset of names to an IP address in the cluster. This is convenient when using routers because it allows the users to set up routes within the cluster without further administrator attention.

The administrator must ensure that the local firewall on each node permits the request to reach the IP address.

This is the most common way to access the cluster. A router is configured to accept external requests and proxy them based on the configured routes. This is limited to HTTP/HTTPS(SNI)/TLS(SNI), which covers web applications.

An administrator can create a wildcard DNS entry, and then set up a router. Afterward, the users can self-service the edge router without having to contact the administrators. The router has controls to allow the administrator to specify whether the users can self-provision host names, or if they must fit a pattern the administrator defines. The other solutions require the administrator to do the provisioning, or they require that the administrator delegates a lot of privilege.

A set of routes can be created in the various projects. The overall set of routes is available to the set of routers. Each router selects from the set of routes. All routers see all routes unless restricted by labels on the router, which is called router sharding.

Load balancers are available on AWS and GCE clouds, and non-cloud options are also available.

The non-cloud load balancer allocates a unique IP from a configured pool. This limits you to a single edge router IP, which can be a VIP, but still will be a single machine for initial load balancing. The non-cloud load balancer simplifies the administrator’s job by providing the needed IP address, but uses one IP per service.

Administrators can assign a list of externalIPs, for which nodes in the cluster will also accept traffic for the service. These IPs are not managed by OpenShift Container Platform and administrators are responsible for ensuring that traffic arrives at a node with this IP. A common example of this is configuring a highly available service.

The supplied list of IP addresses is used for load balancing incoming requests. The service port is opened on the externalIPs on all nodes running kube-proxy.

Traffic from hosts that are external to the cluster that is going to the external IP address ends up on a node in the cluster. When it arrives, the service that set up the external IP redirects it to a service endpoint that it is managing. The service load balances among its endpoints.

An externally visible IP for the service can be configured in several ways:

The administrator must ensure the external IPs are routed to the nodes and local firewall rules on all nodes allow access to the open port.

$ oc create -f - <<INGRESS
apiVersion: v1
kind: Service
metadata:
  name: postgresql-ingress
spec:
  externalIPs:
  - 10.9.54.100 1
  ports:
  - port: 5432
    protocol: TCP
  selector:
    name: postgresql
  type: LoadBalancer
INGRESS

The external IP address is not managed by the underlying Kubernetes infrastructure and must be maintained and provided by a cluster administrator. While external IPs provide a solution for accessing services on the OpenShift Container Platform cluster, there are several shortcomings:

Ingress IP Self-Service provides a solution.

networkConfig:
  ingressIPNetworkCIDR: 10.9.54.0/25

$ oc create -f - <<INGRESS
apiVersion: v1
kind: Service
metadata:
  name: postgresql-ingress
spec:
  ports:
  - name: postgresql
    port: 5432
  type: LoadBalancer 1
  selector:
    name: postgresql
INGRESS

$ oc expose dc postgresql --type=LoadBalancer --name=postgresql-ingress

$ oc get svc postgresql-ingress

NAME         CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
postgresql-ingress    172.30.74.106   10.9.54.100,10.9.54.100    5432/TCP    30s

$ oc export svc postgresql-ingress

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: postgresql-persistent
    template: postgresql-persistent-template
  name: postgresql-ingress
spec:
  ports:
  - nodePort: 32439 1
    port: 5432
    protocol: TCP
    targetPort: 5432
  selector:
    name: postgresql
  sessionAffinity: None
  type: LoadBalancer

Use NodePorts to expose the service nodePort on all nodes in the cluster. NodePorts are in the 30-60k range by default, which means a NodePort is unlikely to match a service’s intended port (for example, 8080 may be exposed as 31020). This use of ports is wasteful of scarce port resources. However, it is slightly easier to set up. Again, this requires more privileges.

The administrator must ensure the external IPs are routed to the nodes and local firewall rules on all nodes allow access to the open port.

NodePorts and externalIP are independent and both can be used concurrently.

High availability improves the chances that an IP address will remain active, by assigning a virtual IP address to the host in a configured pool of hosts. If the host goes down, the virtual IP address is automatically transferred to another host in the pool.

In a non-cloud environment, cluster administrators can assign a unique external IP address to a service (as described here). When routed correctly, external traffic can reach the service endpoints via any TCP/UDP port the service exposes. This is simpler than having to manage the port space of a limited number of shared IP addresses, when manually assigning external IPs to services.

An edge load balancer can be used to accept traffic from outside networks and proxy the traffic to pods inside the cluster.

In this configuration, the internal pod network is visible to the outside.