Chapter 23. Assigning Unique External IPs for Ingress Traffic
23.1. Overview
One approach to getting external traffic into the cluster is by using ExternalIP or IngressIP addresses.
This feature is only supported in non-cloud deployments. For cloud (GCE, AWS, and OpenStack) deployments, use the Load Balancer services for automatic deployment of a cloud load balancer to target the service’s endpoints.
OpenShift Container Platform supports two pools of IP addresses:
- IngressIP uses by the Loadbalancer when choosing an external IP address for the service.
- ExternalIP is used when the user selects a specific IP from the configured pool.
Both have to be configured to a device on an OpenShift Container Platform host to be used, whether with network interface controller (NIC) or virtual ethernet, as well as external routing. Ipfailover is recommended for this, because it selects the host and configures the NIC.
IngressIP and ExternalIP both allow external traffic access to the cluster, and, if routed correctly, external traffic can reach that service’s endpoints via any TCP/UDP port the service exposes. This can be simpler than having to manage the port space of a limited number of shared IP addresses when manually assigning external IPs to services. Also, these addresses can be used as virtual IPs (VIPs) when configuring high availability.
OpenShift Container Platform supports both the automatic and manual assignment of IP addresses, and each address is guaranteed to be assigned to a maximum of one service. This ensures that each service can expose its chosen ports regardless of the ports exposed by other services.
23.2. Restrictions
To use an ExternalIP, you can:
-
Select an IP address from the
externalIPNetworkCIDRsrange. Have an IP address assigned from the
ingressIPNetworkCIDRpool in the master configuration file. In this case, OpenShift Container Platform implements a non-cloud version of the load balancer service type and assigns IP addresses to the services.CautionYou must ensure that the IP address pool you assign terminates at one or more nodes in your cluster. You can use the existing
oc adm ipfailoverto ensure that the external IPs are highly available.
For manually-configured external IPs, potential port clashes are handled on a first-come, first-served basis. If you request a port, it is only available if it has not yet been assigned for that IP address. For example:
Port clash example for manually-configured external IPs
Two services have been manually configured with the same external IP address of 172.7.7.7.
MongoDB service A requests port 27017, and then MongoDB service B requests the same port; the first request gets the port.
However, port clashes are not an issue for external IPs assigned by the ingress controller, because the controller assigns each service a unique address.
23.3. Configuring the Cluster to Use Unique External IPs
In non-cloud clusters, ingressIPNetworkCIDR is set by default to 172.29.0.0/16. If your cluster environment is not already using this private range, you can use the default. However, if you want to use a different range, then you must set ingressIPNetworkCIDR in the /etc/origin/master/master-config.yaml file before you assign an ingress IP. Then, restart the master service.
External IPs assigned to services of type LoadBalancer will always be in the range of ingressIPNetworkCIDR. If ingressIPNetworkCIDR is changed such that the assigned external IPs are no longer in range, the affected services will be assigned new external IPs compatible with the new range.
If you are using high availibility, then this range must be less than 255 IP addresses.
Sample /etc/origin/master/master-config.yaml
networkConfig: ingressIPNetworkCIDR: 172.29.0.0/16
23.3.1. Configuring an Ingress IP for a Service
To assign an ingress IP:
Create a YAML file for a LoadBalancer service that requests a specific IP via the
loadBalancerIPsetting:Sample LoadBalancer Configuration
apiVersion: v1 kind: Service metadata: name: egress-1 spec: ports: - name: db port: 3306 loadBalancerIP: 172.29.0.1 type: LoadBalancer selector: name: my-db-selectorCreate a LoadBalancer service on your pod:
$ oc create -f loadbalancer.yaml
Check the service for an external IP. For example, for a service named
myservice:$ oc get svc myservice
When your LoadBalancer-type service has an external IP assigned, the output displays the IP:
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE myservice 172.30.74.106 172.29.0.1 3306/TCP 30s
23.4. Routing the Ingress CIDR for Development or Testing
Add a static route directing traffic for the ingress CIDR to a node in the cluster. For example:
# route add -net 172.29.0.0/16 gw 10.66.140.17 eth0
In the example above, 172.29.0.0/16 is the ingressIPNetworkCIDR, and 10.66.140.17 is the node IP.
23.4.1. Service externalIPs
In addition to the cluster’s internal IP addresses, the application developer can configure IP addresses that are external to the cluster. As the OpenShift Container Platform administrator, you are responsible for ensuring that traffic arrives at a node with this IP.
The externalIPs must be selected by the administrator from the externalIPNetworkCIDRs range configured in the master-config.yaml file. When master-config.yaml changes, the master services must be restarted.
# systemctl restart atomic-openshift-master-api atomic-openshift-master-controllers
Sample externalIPNetworkCIDR /etc/origin/master/master-config.yaml
networkConfig: externalIPNetworkCIDR: 172.47.0.0/24
Service externalIPs Definition (JSON)
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "my-service"
},
"spec": {
"selector": {
"app": "MyApp"
},
"ports": [
{
"name": "http",
"protocol": "TCP",
"port": 80,
"targetPort": 9376
}
],
"externalIPs" : [
"80.11.12.10" 1
]
}
}
- 1
- List of External IP addresses on which the port is exposed. In addition to the internal IP addresses)
