Chapter 13. Submariner

Create the inbound and outbound firewall rules on your Google Cloud Platform to open the IPsec IKE (by default 500/UDP) and NAT traversal ports (by default 4500/UDP) to enable Submariner communication:

$ gcloud compute firewall-rules create <name> --network=<network-name> --allow=udp:<ipsec-port> --direction=IN
$ gcloud compute firewall-rules create <rule-name> --network=<network-name> --allow=udp:<ipsec-port>  --direction=OUT

Replace rule-name with your rule name.

Replace network-name with your Google Cloud Platform cluster network name.

Replace ipsec-port with your IPsec port.

Create the inbound and outbound firewall rules on your Google Cloud Platform to open the 4800/UDP port to encapsulate Pod traffic from the worker and master nodes to the Submariner Gateway nodes:

$ gcloud compute firewall-rules create <name> --network=<network-name> --allow=udp:4800 --direction=IN
$ gcloud compute firewall-rules create <name> --network=<network-name> --allow=udp:4800 --direction=OUT

Replace name with your rule name.

Replace network-name with your Google Cloud Platform cluster network name.

Create the inbound and outbound firewall rules on your Google Cloud Platform to open the 8080/TCP port to export metrics service from the Submariner Gateway nodes:

$ gcloud compute firewall-rules create <name> --network=<network-name> --allow=tcp:8080 --direction=IN
$ gcloud compute firewall-rules create <name> --network=<network-name> --allow=tcp:8080 --direction=OUT

Replace name with your rule name.

Replace network-name with your Google Cloud Platform cluster network name.

Create the inbound and outbound firewall rules on your Microsoft Azure environment to open the IP security IKE (by default 500/UDP) and NAT traversal ports (by default 4500/UDP) to enable Submariner communication:

# create inbound nat rule
$ az network lb inbound-nat-rule create --lb-name <lb-name> \
--resource-group <res-group> \
--name <name> \
--protocol Udp --frontend-port <ipsec-port> \
--backend-port <ipsec-port> \
--frontend-ip-name <frontend-ip-name>

# add your vm network interface to the created inbound nat rule
$ az network nic ip-config inbound-nat-rule add \
--lb-name <lb-name> --resource-group <res-group> \
--inbound-nat-rule <nat-name> \
--nic-name <nic-name> --ip-config-name <pipConfig>

Replace lb-name with your load balancer name.

Replace res-group with your resource group name.

Replace nat-name with your load balancing inbound NAT rule name.

Replace ipsec-port with your IPsec port.

Replace pipConfig with your cluster frontend IP configuration name.

Replace nic-name with your network interface card (NIC) name.

Create one load balancing inbound NAT rules to forward Submariner gateway metrics service request:

# create inbound nat rule
$ az network lb inbound-nat-rule create --lb-name <lb-name> \
--resource-group <res-group> \
--name <name> \
--protocol Tcp --frontend-port 8080 --backend-port 8080 \
--frontend-ip-name <frontend-ip-name>

# add your vm network interface to the created inbound nat rule
$ az network nic ip-config inbound-nat-rule add \
--lb-name <lb-name> --resource-group <res-group> \
--inbound-nat-rule <nat-name> \
--nic-name <nic-name> --ip-config-name <pipConfig>

Replace lb-name with your load balancer name.

Replace res-group with your resource group name.

Replace nat-name with your load balancing inbound NAT rule name.

Replace pipConfig with your cluster frontend IP configuration name.

Replace nic-name with your network interface card (NIC) name.

Create network security groups {NSG) security rules on your Azure to open IPsec IKE (by default 500/UDP) and NAT traversal ports (by default 4500/UDP) for Submariner:

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Inbound --access Allow \
--protocol Udp --destination-port-ranges <ipsec-port>

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Outbound --access Allow \
--protocol Udp --destination-port-ranges <ipsec-port>

Replace res-group with your resource group name.

Replace nsg-name with your NSG name.

Replace priority with your rule priority.

Replace name with your rule name.

Replace ipsec-port with your IPsec port.

Create the NSG rules to open 4800/UDP port to encapsulate pod traffic from the worker and master nodes to the Submariner Gateway nodes:

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Inbound --access Allow \
--protocol Udp --destination-port-ranges 4800 \

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Outbound --access Allow \
--protocol Udp --destination-port-ranges 4800

Replace res-group with your resource group name.

Replace nsg-name with your NSG name.

Replace priority with your rule priority.

Replace name with your rule name.

Create the NSG rules to open 8080/TCP port to export metrics service from the Submariner Gateway nodes:

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Inbound --access Allow \
--protocol Tcp --destination-port-ranges 8080 \

$ az network nsg rule create --resource-group <res-group> \
--nsg-name <nsg-name> --priority <priority> \
--name <name> --direction Outbound --access Allow \
--protocol Udp --destination-port-ranges 8080

Replace res-group with your resource group name.

Replace nsg-name with your NSG name.

Replace priority with your rule priority.

Replace name with your rule name.

Log on to your hub cluster with cluster administrator permissions.

For AWS clusters that were created by Red Hat Advanced Cluster Management for Kubernetes, create a file called submarinerconfig.yaml with the following SubmarinerConfig content:

apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: <your-cluster-namespace>
spec:
  credentialsSecret:
    name: <your-cluster-name>-aws-creds
  subscriptionConfig:
    channel: alpha
    startingCSV: submariner.v0.8.1

For AWS clusters that were created by OpenShift Container Platform but imported into Red Hat Advanced Cluster Management, create the aws-cloud-credentials.

Apply the SubmarinerConfig resource to the managed cluster namespace on the hub cluster by entering the following command:

oc apply -f submarinerconfig.yaml

Create a ManagedClusterSet on the hub cluster.

apiVersion: cluster.open-cluster-management.io/v1alpha1
kind: ManagedClusterSet
metadata:
  name: my-clusterset

Add the managed clusters to the ManagedClusterSet by entering the following command:

oc label managedclusters <managedcluster-name> "cluster.open-cluster-management.io/clusterset=<ManagedClusterSet-name>" --overwrite

Run the following command to allow Red Hat Advanced Cluster Management to automatically deploy Submariner:app-name:

oc label managedclusters <managedcluster-name> "cluster.open-cluster-management.io/submariner-agent=true" --overwrite