Chapter 12. Tutorial: Assigning a consistent egress IP for external traffic

Procedure

1. Obtain your pending egress IP assignments by running the following command:

   $ oc get egressips

   For example:

   NAME              EGRESSIPS       ASSIGNED NODE   ASSIGNED EGRESSIPS
   demo-egress-ns    10.10.100.253
   demo-egress-pod   10.10.100.254

   The egress IP rule that you created only applies to nodes with the k8s.ovn.org/egress-assignable label. Make sure that the label is only on a specific machine pool.

2. Assign the label to your machine pool using the following command:

   Warning

   If you rely on node labels for your machine pool, this command will replace those labels. Be sure to input your desired labels into the --labels field to ensure your node labels remain.

   $ rosa update machinepool ${ROSA_MACHINE_POOL_NAME} \
     --cluster="${ROSA_CLUSTER_NAME}" \
     --labels "k8s.ovn.org/egress-assignable="

Procedure

1. Run the echoserver command to replicate a request:

   $ oc -n default run demo-service --image=gcr.io/google_containers/echoserver:1.4

2. Expose the pod as a service and limit the ingress to the egress IP addresses you specified by running the following command:

   $ cat <<EOF | oc apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: demo-service
     namespace: default
     annotations:
       service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
       service.beta.kubernetes.io/aws-load-balancer-internal: "true"
   spec:
     selector:
       run: demo-service
     ports:
       - port: 80
         targetPort: 8080
     type: LoadBalancer
     externalTrafficPolicy: Local
     # NOTE: this limits the source IPs that are allowed to connect to our service.  It
     #       is being used as part of this demo, restricting connectivity to our egress
     #       IP addresses only.
     # NOTE: these egress IPs are within the subnet range(s) in which my worker nodes
     #       are deployed.
     loadBalancerSourceRanges:
       - 10.10.100.254/32
       - 10.10.150.254/32
       - 10.10.200.254/32
       - 10.10.100.253/32
       - 10.10.150.253/32
       - 10.10.200.253/32
   EOF

3. Retrieve the load balancer hostname and save it as an environment variable by running the following command:

   $ export LOAD_BALANCER_HOSTNAME=$(oc get svc -n default demo-service -o json | jq -r '.status.loadBalancer.ingress[].hostname')

Procedure

1. Start an interactive shell to test the namespace egress rule:

   $ oc run \
     demo-egress-ns \
     -it \
     --namespace=demo-egress-ns \
     --env=LOAD_BALANCER_HOSTNAME=$LOAD_BALANCER_HOSTNAME \
     --image=registry.access.redhat.com/ubi9/ubi -- \
     bash

2. Send a request to the load balancer and ensure that you can successfully connect:

   $ curl -s http://$LOAD_BALANCER_HOSTNAME

3. Check the output for a successful connection:

   Note

   The client_address is the internal IP address of the load balancer not your egress IP. You can verify that you have configured the client address correctly by connecting with your service limited to .spec.loadBalancerSourceRanges.

   For example:

   CLIENT VALUES:
   client_address=10.10.207.247
   command=GET
   real path=/
   query=nil
   request_version=1.1
   request_uri=http://internal-a3e61de18bfca4a53a94a208752b7263-148284314.us-east-1.elb.amazonaws.com:8080/
   
   SERVER VALUES:
   server_version=nginx: 1.10.0 - lua: 10001
   
   HEADERS RECEIVED:
   accept=*/*
   host=internal-a3e61de18bfca4a53a94a208752b7263-148284314.us-east-1.elb.amazonaws.com
   user-agent=curl/7.76.1
   BODY:
   -no body in request-

4. Exit the pod by running the following command:

   $ exit

Procedure

1. Start an interactive shell to test the pod egress rule:

   $ oc run \
     demo-egress-pod \
     -it \
     --namespace=demo-egress-pod \
     --env=LOAD_BALANCER_HOSTNAME=$LOAD_BALANCER_HOSTNAME \
     --image=registry.access.redhat.com/ubi9/ubi -- \
     bash

2. Send a request to the load balancer by running the following command:

   $ curl -s http://$LOAD_BALANCER_HOSTNAME

3. Check the output for a successful connection:

   Note

   The client_address is the internal IP address of the load balancer not your egress IP. You can verify that you have configured the client address correctly by connecting with your service limited to .spec.loadBalancerSourceRanges.

   For example:

   CLIENT VALUES:
   client_address=10.10.207.247
   command=GET
   real path=/
   query=nil
   request_version=1.1
   request_uri=http://internal-a3e61de18bfca4a53a94a208752b7263-148284314.us-east-1.elb.amazonaws.com:8080/
   
   SERVER VALUES:
   server_version=nginx: 1.10.0 - lua: 10001
   
   HEADERS RECEIVED:
   accept=*/*
   host=internal-a3e61de18bfca4a53a94a208752b7263-148284314.us-east-1.elb.amazonaws.com
   user-agent=curl/7.76.1
   BODY:
   -no body in request-

4. Exit the pod by running the following command:

   $ exit

Procedure

1. Test that the traffic is successfully blocked when the egress rules do not apply by running the following command:

   $ oc run \
     demo-egress-pod-fail \
     -it \
     --namespace=demo-egress-pod \
     --env=LOAD_BALANCER_HOSTNAME=$LOAD_BALANCER_HOSTNAME \
     --image=registry.access.redhat.com/ubi9/ubi -- \
     bash

2. Send a request to the load balancer by running the following command:

   $ curl -s http://$LOAD_BALANCER_HOSTNAME

3. If the command is unsuccessful, egress is successfully blocked.

4. Exit the pod by running the following command:

   $ exit