Installing on OpenStack
Installing OpenShift Container Platform 4.3 OpenStack clusters
Abstract
Chapter 1. Installing on OpenStack
1.1. Installing a cluster on OpenStack with customizations
In OpenShift Container Platform version 4.3, you can install a customized cluster on Red Hat OpenStack Platform (RHOSP). To customize the installation, modify parameters in the install-config.yaml
before you install the cluster.
1.1.1. Prerequisites
Review details about the OpenShift Container Platform installation and update processes.
- Verify that OpenShift Container Platform 4.3 is compatible with your RHOSP version in the Available platforms section. You can also compare platform support across different versions by viewing the OpenShift Container Platform on RHOSP support matrix.
- Have metadata service enabled in RHOSP
1.1.2. Resource guidelines for installing OpenShift Container Platform on RHOSP
Your quota must meet the following requirements to run the OpenShift Container Platform installation program in Red Hat OpenStack Platform (RHOSP).
Resource | Value |
---|---|
Floating IP addresses | 2 |
Ports | 15 |
Routers | 1 |
Subnets | 1 |
RAM | 112 GB |
vCPUs | 28 |
Volume storage | 175 GB |
Instances | 7 |
Security groups | 3 |
Security group rules | 60 |
Swift containers | 2 |
Swift objects | 1 |
Swift available space | 10 MB or more |
Swift space requirements vary depending on the size of the bootstrap Ignition file and image registry.
A cluster might function with fewer than recommended resources, but its performance is not guaranteed.
By default, your security group and security group rule quotas might be low. If you encounter problems, run openstack quota set --secgroups 3 --secgroup-rules 60 <project>
as an administrator to increase them.
An OpenShift Container Platform deployment comprises control plane machines, compute machines, and a bootstrap machine.
1.1.2.1. Control plane and compute machines
By default, the OpenShift Container Platform installation program stands up three control plane and compute machines.
Each machine requires:
- An instance from the RHOSP quota
- A port from the RHOSP quota
- A flavor with at least 16 GB memory, 4 vCPUs, and 25 GB storage space
Compute machines host the applications that you run on OpenShift Container Platform; aim to run as many as you can.
1.1.2.2. Bootstrap machine
During installation, a bootstrap machine is temporarily provisioned to stand up the control plane. After the production control plane is ready, the bootstrap machine is deprovisioned.
The bootstrap machine requires:
- An instance from the RHOSP quota
- A port from the RHOSP quota
- A flavor with at least 16 GB memory, 4 vCPUs, and 25 GB storage space
The installation program cannot pass certificate authority bundles to Ignition on control plane machines. Therefore, the bootstrap machine cannot retrieve Ignition configurations from Swift if your endpoint uses self-signed certificates.
1.1.3. Internet and Telemetry access for OpenShift Container Platform
In OpenShift Container Platform 4.3, you require access to the internet to install your cluster. The Telemetry service, which runs by default to provide metrics about cluster health and the success of updates, also requires internet access. If your cluster is connected to the internet, Telemetry runs automatically, and your cluster is registered to the Red Hat OpenShift Cluster Manager (OCM).
Once you confirm that your Red Hat OpenShift Cluster Manager inventory is correct, either maintained automatically by Telemetry or manually using OCM, use subscription watch to track your OpenShift Container Platform subscriptions at the account or multi-cluster level.
You must have internet access to:
- Access the Red Hat OpenShift Cluster Manager page to download the installation program and perform subscription management. If the cluster has internet access and you do not disable Telemetry, that service automatically entitles your cluster.
- Access Quay.io to obtain the packages that are required to install your cluster.
- Obtain the packages that are required to perform cluster updates.
If your cluster cannot have direct internet access, you can perform a restricted network installation on some types of infrastructure that you provision. During that process, you download the content that is required and use it to populate a mirror registry with the packages that you need to install a cluster and generate the installation program. With some installation types, the environment that you install your cluster in will not require internet access. Before you update the cluster, you update the content of the mirror registry.
1.1.4. Enabling Swift on OpenStack
OpenShift Container Platform on Red Hat OpenStack Platform (RHOSP) uses RHOSP Object Storage (Swift) to store and serve user configuration files.
Swift is operated by a user account with the swiftoperator
role.
Prerequisites
- A RHOSP administrator account on the target environment
-
On Ceph RGW, the
account in url
option must be enabled
Procedure
To enable Swift on RHOSP:
As an administrator in the RHOSP CLI, add the
swiftoperator
role to the account that will access Swift:$ openstack role add --user <user> --project <project> swiftoperator
Your RHOSP deployment can now use Swift to store and serve files.
1.1.5. Verifying external network access
The OpenShift Container Platform installer requires external network access. You must provide an external network value to it, or deployment fails. Before you run the installer, verify that a network with the External router type exists in Red Hat OpenStack Platform (RHOSP).
Prerequisites
On RHOSP, the
NeutronDhcpAgentDnsmasqDnsServers
parameter must be configured to allow DHCP agents to forward instances' DNS queries. One way to set this parameter is to:- Create a new environment file in the template directory.
Provide parameter values in the file. For example:
Sample
neutron-dhcp-agent-dnsmasq-dns-servers.yaml
fileparameter_defaults: NeutronDhcpAgentDnsmasqDnsServers: ['<DNS_server_address_1>','<DNS_server_address_2']
Include the environment file in your Overcloud deploy command. For example:
$ openstack overcloud deploy --templates -e neutron-dhcp-agent-dnsmasq-dns-servers.yaml ...
Procedure
Using the RHOSP CLI, verify the name and ID of the 'External' network:
$ openstack network list --long -c ID -c Name -c "Router Type" +--------------------------------------+----------------+-------------+ | ID | Name | Router Type | +--------------------------------------+----------------+-------------+ | 148a8023-62a7-4672-b018-003462f8d7dc | public_network | External | +--------------------------------------+----------------+-------------+
A network with an External router type appears in the network list. If at least one does not, see Creating a default floating IP network and Creating a default provider network.
If the external network’s CIDR range overlaps one of the default network ranges, you must change the matching network ranges in the install-config.yaml
file before you run the installation program.
The default network ranges are:
Network | Range |
---|---|
machineCIDR | 10.0.0.0/16 |
serviceNetwork | 172.30.0.0/16 |
clusterNetwork | 10.128.0.0/14 |
If the installation program finds multiple networks with the same name, it sets one of them at random. To avoid this behavior, create unique names for resources in RHOSP.
If the Neutron trunk service plug-in is enabled, a trunk port is created by default. For more information, see Neutron trunk port.
1.1.6. Defining parameters for the installation program
The OpenShift Container Platform installation program relies on a file called clouds.yaml
. The file describes Red Hat OpenStack Platform (RHOSP) configuration parameters, including the project name, log in information, and authorization service URLs.
Procedure
Create the
clouds.yaml
file:If your RHOSP distribution includes the Horizon web UI, generate a
clouds.yaml
file in it.ImportantRemember to add a password to the
auth
field. You can also keep secrets in a separate file fromclouds.yaml
.If your RHOSP distribution does not include the Horizon web UI, or you do not want to use Horizon, create the file yourself. For detailed information about
clouds.yaml
, see Config files in the RHOSP documentation.clouds: shiftstack: auth: auth_url: http://10.10.14.42:5000/v3 project_name: shiftstack username: shiftstack_user password: XXX user_domain_name: Default project_domain_name: Default dev-env: region_name: RegionOne auth: username: 'devuser' password: XXX project_name: 'devonly' auth_url: 'https://10.10.14.22:5001/v2.0'
Place the file that you generate in one of the following locations:
-
The value of the
OS_CLIENT_CONFIG_FILE
environment variable - The current directory
-
A Unix-specific user configuration directory, for example
~/.config/openstack/clouds.yaml
A Unix-specific site configuration directory, for example
/etc/openstack/clouds.yaml
The installation program searches for
clouds.yaml
in that order.
-
The value of the
1.1.7. Obtaining the installation program
Before you install OpenShift Container Platform, download the installation file on a local computer.
Prerequisites
- You must install the cluster from a computer that uses Linux or macOS.
- You need 500 MB of local disk space to download the installation program.
Procedure
- Access the Infrastructure Provider page on the Red Hat OpenShift Cluster Manager site. If you have a Red Hat account, log in with your credentials. If you do not, create an account.
Navigate to the page for your installation type, download the installation program for your operating system, and place the file in the directory where you will store the installation configuration files.
ImportantThe installation program creates several files on the computer that you use to install your cluster. You must keep both the installation program and the files that the installation program creates after you finish installing the cluster.
ImportantDeleting the files created by the installation program does not remove your cluster, even if the cluster failed during installation. You must complete the OpenShift Container Platform uninstallation procedures outlined for your specific cloud provider to remove your cluster entirely.
Extract the installation program. For example, on a computer that uses a Linux operating system, run the following command:
$ tar xvf <installation_program>.tar.gz
-
From the Pull Secret page on the Red Hat OpenShift Cluster Manager site, download your installation pull secret as a
.txt
file. This pull secret allows you to authenticate with the services that are provided by the included authorities, including Quay.io, which serves the container images for OpenShift Container Platform components.
1.1.8. Creating the installation configuration file
You can customize your installation of OpenShift Container Platform on Red Hat OpenStack Platform (RHOSP).
Prerequisites
- Obtain the OpenShift Container Platform installation program and the pull secret for your cluster.
Procedure
Create the
install-config.yaml
file.Run the following command:
$ ./openshift-install create install-config --dir=<installation_directory> 1
- 1
- For
<installation_directory>
, specify the directory name to store the files that the installation program creates.
ImportantSpecify an empty directory. Some installation assets, like bootstrap X.509 certificates have short expiration intervals, so you must not reuse an installation directory. If you want to reuse individual files from another cluster installation, you can copy them into your directory. However, the file names for the installation assets might change between releases. Use caution when copying installation files from an earlier OpenShift Container Platform version.
At the prompts, provide the configuration details for your cloud:
Optional: Select an SSH key to use to access your cluster machines.
NoteFor production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery on, specify an SSH key that your
ssh-agent
process uses.- Select openstack as the platform to target.
- Specify the Red Hat OpenStack Platform (RHOSP) external network name to use for installing the cluster.
- Specify the Floating IP address to use for external access to the OpenShift API.
- Specify a RHOSP flavor with at least 16 GB RAM to use for control plane and compute nodes.
- Select the base domain to deploy the cluster to. All DNS records will be sub-domains of this base and will also include the cluster name.
- Enter a name for your cluster. The name must be 14 or fewer characters long.
- Paste the pull secret that you obtained from the Pull Secret page on the Red Hat OpenShift Cluster Manager site.
-
Modify the
install-config.yaml
file. You can find more information about the available parameters in the Installation configuration parameters section. Back up the
install-config.yaml
file so that you can use it to install multiple clusters.ImportantThe
install-config.yaml
file is consumed during the installation process. If you want to reuse the file, you must back it up now.
1.1.9. Installation configuration parameters
Before you deploy an OpenShift Container Platform cluster, you provide parameter values to describe your account on the cloud platform that hosts your cluster and optionally customize your cluster’s platform. When you create the install-config.yaml
installation configuration file, you provide values for the required parameters through the command line. If you customize your cluster, you can modify the install-config.yaml
file to provide more details about the platform.
You cannot modify these parameters in the install-config.yaml
file after installation.
Parameter | Description | Values |
---|---|---|
|
The base domain of your cloud provider. This value is used to create routes to your OpenShift Container Platform cluster components. The full DNS name for your cluster is a combination of the |
A fully-qualified domain or subdomain name, such as |
|
The cloud provider to host the control plane machines. This parameter value must match the |
|
|
The cloud provider to host the worker machines. This parameter value must match the |
|
| The name of your cluster. |
A string that contains uppercase or lowercase letters, such as |
| The region to deploy your cluster in. |
A valid region for your cloud, such as |
| The pull secret that you obtained from the Pull Secret page on the Red Hat OpenShift Cluster Manager site. You use this pull secret to authenticate with the services that are provided by the included authorities, including Quay.io, which serves the container images for OpenShift Container Platform components. |
{ "auths":{ "cloud.openshift.com":{ "auth":"b3Blb=", "email":"you@example.com" }, "quay.io":{ "auth":"b3Blb=", "email":"you@example.com" } } } |
Parameter | Description | Values |
---|---|---|
| The SSH key to use to access your cluster machines. Note
For production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery on, specify an SSH key that your |
A valid, local public SSH key that you added to the |
| Whether to enable or disable FIPS mode. By default, FIPS mode is not enabled. If FIPS mode is enabled, the Red Hat Enterprise Linux CoreOS (RHCOS) machines that OpenShift Container Platform runs on bypass the default Kubernetes cryptography suite and use the cryptography modules that are provided with RHCOS instead. |
|
| How to publish the user-facing endpoints of your cluster. |
|
|
Whether to enable or disable simultaneous multithreading, or Important If you disable simultaneous multithreading, ensure that your capacity planning accounts for the dramatically decreased machine performance. |
|
| The number of compute machines, which are also known as worker machines, to provision. |
A positive integer greater than or equal to |
|
Whether to enable or disable simultaneous multithreading, or Important If you disable simultaneous multithreading, ensure that your capacity planning accounts for the dramatically decreased machine performance. |
|
| The number of control plane machines to provision. |
A positive integer greater than or equal to |
Parameter | Description | Values |
---|---|---|
| For compute machines, the size in gigabytes of the root volume. If you do not set this value, machines use ephemeral storage. |
Integer, for example |
| For compute machines, the root volume’s type. |
String, for example |
| For control plane machines, the size in gigabytes of the root volume. If you do not set this value, machines use ephemeral storage. |
Integer, for example |
| For control plane machines, the root volume’s type. |
String, for example |
| The region where the RHOSP cluster is created. |
String, for example |
|
The name of the RHOSP cloud to use from the list of clouds in the |
String, for example |
| The RHOSP external network name to be used for installation. |
String, for example |
| The RHOSP flavor to use for control plane and compute machines. |
String, for example |
| An existing floating IP address to associate with the load balancer API. |
An IP address, for example |
Parameter | Description | Values |
---|---|---|
| IP addresses for external DNS servers that cluster instances use for DNS resolution. |
A list of IP addresses as strings, for example |
| The default machine pool platform configuration. |
{ "type": "ml.large", "rootVolume": { "size": 30, "type": "performance" } } |
| The location from which the installer downloads the RHCOS image. You must set this parameter to perform an installation in a restricted network. | An HTTP or HTTPS URL, optionally with an SHA-256 checksum.
For example,
The value can also be the name of an existing Glance image, for example |
1.1.9.1. Sample customized install-config.yaml
file for RHOSP
This sample install-config.yaml
demonstrates all of the possible Red Hat OpenStack Platform (RHOSP) customization options.
This sample file is provided for reference only. You must obtain your install-config.yaml
file by using the installation program.
apiVersion: v1 baseDomain: example.com clusterID: os-test controlPlane: name: master platform: {} replicas: 3 compute: - name: worker platform: openstack: type: ml.large replicas: 3 metadata: name: example networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 machineCIDR: 10.0.0.0/16 serviceNetwork: - 172.30.0.0/16 networkType: OpenShiftSDN platform: openstack: region: region1 cloud: mycloud externalNetwork: external computeFlavor: m1.xlarge lbFloatingIP: 128.0.0.1 fips: false pullSecret: '{"auths": ...}' sshKey: ssh-ed25519 AAAA...
1.1.10. Generating an SSH private key and adding it to the agent
If you want to perform installation debugging or disaster recovery on your cluster, you must provide an SSH key to both your ssh-agent
and to the installation program.
In a production environment, you require disaster recovery and debugging.
You can use this key to SSH into the master nodes as the user core
. When you deploy the cluster, the key is added to the core
user’s ~/.ssh/authorized_keys
list.
Procedure
If you do not have an SSH key that is configured for password-less authentication on your computer, create one. For example, on a computer that uses a Linux operating system, run the following command:
$ ssh-keygen -t rsa -b 4096 -N '' \ -f <path>/<file_name> 1
- 1
- Specify the path and file name, such as
~/.ssh/id_rsa
, of the SSH key. Do not specify an existing SSH key, as it will be overwritten.
Running this command generates an SSH key that does not require a password in the location that you specified.
Start the
ssh-agent
process as a background task:$ eval "$(ssh-agent -s)" Agent pid 31874
Add your SSH private key to the
ssh-agent
:$ ssh-add <path>/<file_name> 1 Identity added: /home/<you>/<path>/<file_name> (<computer_name>)
- 1
- Specify the path and file name for your SSH private key, such as
~/.ssh/id_rsa
Next steps
- When you install OpenShift Container Platform, provide the SSH public key to the installation program.
1.1.11. Enabling access to the environment
At deployment, all OpenShift Container Platform machines are created in a Red Hat OpenStack Platform (RHOSP)-tenant network. Therefore, they are not accessible directly in most RHOSP deployments.
You can configure the OpenShift Container Platform API to be accessible either with or without floating IP addresses.
1.1.11.1. Enabling access with floating IP addresses
Make OpenShift Container Platform API endpoints accessible by attaching two floating IP (FIP) addresses to them: one for the API load balancer (lb FIP
), and one for OpenShift Container Platform applications (apps FIP
).
The load balancer FIP is also used in the install-config.yaml
file.
Procedure
Using the Red Hat OpenStack Platform (RHOSP) CLI, create a new external network:
$ openstack floating ip create <external network>
Add a record that follows this pattern to your DNS server:
api.<cluster name>.<base domain> IN A <lb FIP>
NoteIf you do not control the DNS server you can add the record to your
/etc/hosts
file instead. This action makes the API accessible to you only, which is not suitable for production deployment but does allow installation for development and testing.
You can make OpenShift Container Platform resources available outside of the cluster by assigning a floating IP address and updating your firewall configuration.
1.1.11.2. Enabling access without floating IP addresses
If you cannot use floating IP addresses, the OpenShift Container Platform installation might still finish. However, the installation program fails after it times out waiting for API access.
After the installation program times out, the cluster might still initialize. After the bootstrapping processing begins, it must complete. You must edit the cluster’s networking configuration after it is deployed, however.
1.1.12. Deploy the cluster
You can install OpenShift Container Platform on a compatible cloud platform.
You can run the create cluster
command of the installation program only once, during initial installation.
Prerequisites
- Obtain the OpenShift Container Platform installation program and the pull secret for your cluster.
Procedure
Run the installation program:
$ ./openshift-install create cluster --dir=<installation_directory> \ 1 --log-level=info 2
NoteIf the cloud provider account that you configured on your host does not have sufficient permissions to deploy the cluster, the installation process stops, and the missing permissions are displayed.
When the cluster deployment completes, directions for accessing your cluster, including a link to its web console and credentials for the
kubeadmin
user, display in your terminal.ImportantThe Ignition config files that the installation program generates contain certificates that expire after 24 hours. You must keep the cluster running for 24 hours in a non-degraded state to ensure that the first certificate rotation has finished.
ImportantYou must not delete the installation program or the files that the installation program creates. Both are required to delete the cluster.
1.1.13. Verifying cluster status
To verify your OpenShift Container Platform cluster’s status during or after installation:
Procedure
In the cluster environment, export the administrator’s kubeconfig file:
$ export KUBECONFIG=<installation_directory>/auth/kubeconfig 1
- 1
- For
<installation_directory>
, specify the path to the directory that you stored the installation files in.
The
kubeconfig
file contains information about the cluster that is used by the CLI to connect a client to the correct cluster and API server.View the control plane and compute machines created after a deployment:
$ oc get nodes
View your cluster’s version:
$ oc get clusterversion
View your operators' status:
$ oc get clusteroperator
View all running Pods in the cluster:
$ oc get pods -A
1.1.14. Logging in to the cluster
You can log in to your cluster as a default system user by exporting the cluster kubeconfig
file. The kubeconfig
file contains information about the cluster that is used by the CLI to connect a client to the correct cluster and API server. The file is specific to a cluster and is created during OpenShift Container Platform installation.
Prerequisites
- Deploy an OpenShift Container Platform cluster.
-
Install the
oc
CLI.
Procedure
Export the
kubeadmin
credentials:$ export KUBECONFIG=<installation_directory>/auth/kubeconfig 1
- 1
- For
<installation_directory>
, specify the path to the directory that you stored the installation files in.
Verify you can run
oc
commands successfully using the exported configuration:$ oc whoami system:admin
1.1.15. Configuring application access with floating IP addresses
After you install OpenShift Container Platform, configure Red Hat OpenStack Platform (RHOSP) to allow application network traffic.
Prerequisites
- OpenShift Container Platform cluster must be installed
- Floating IP addresses are enabled as described in Enabling access to the environment.
Procedure
After you install the OpenShift Container Platform cluster, attach a floating IP address to the ingress port:
Show the port:
$ openstack port show <cluster name>-<clusterID>-ingress-port
Attach the port to the IP address:
$ openstack floating ip set --port <ingress port ID> <apps FIP>
Add a wildcard
A
record for*apps.
to your DNS file:*.apps.<cluster name>.<base domain> IN A <apps FIP>
If you do not control the DNS server but want to enable application access for non-production purposes, you can add these hostnames to /etc/hosts
:
<apps FIP> console-openshift-console.apps.<cluster name>.<base domain> <apps FIP> integrated-oauth-server-openshift-authentication.apps.<cluster name>.<base domain> <apps FIP> oauth-openshift.apps.<cluster name>.<base domain> <apps FIP> prometheus-k8s-openshift-monitoring.apps.<cluster name>.<base domain> <apps FIP> grafana-openshift-monitoring.apps.<cluster name>.<base domain> <apps FIP> <app name>.apps.<cluster name>.<base domain>
1.1.16. Next steps
- Customize your cluster.
- If necessary, you can opt out of remote health reporting.
- If you need to enable external access to node ports, configure ingress cluster traffic by using a NodePort.
1.2. Installing a cluster on OpenStack with Kuryr
In OpenShift Container Platform version 4.3, you can install a customized cluster on Red Hat OpenStack Platform (RHOSP) that uses Kuryr SDN. To customize the installation, modify parameters in the install-config.yaml
before you install the cluster.
1.2.1. Prerequisites
Review details about the OpenShift Container Platform installation and update processes.
- Verify that OpenShift Container Platform 4.3 is compatible with your RHOSP version in the Available platforms section. You can also compare platform support across different versions by viewing the OpenShift Container Platform on RHOSP support matrix.
1.2.2. About Kuryr SDN
Kuryr is a container network interface (CNI) plug-in solution that uses the Neutron and Octavia Red Hat OpenStack Platform (RHOSP) services to provide networking for Pods and Services.
Kuryr and OpenShift Container Platform integration is primarily designed for OpenShift Container Platform clusters running on RHOSP VMs. Kuryr improves the network performance by plugging OpenShift Container Platform Pods into RHOSP SDN. In addition, it provides interconnectivity between Pods and RHOSP virtual instances.
Kuryr components are installed as Pods in OpenShift Container Platform using the openshift-kuryr
namespace:
-
kuryr-controller
- a single Service instance installed on amaster
node. This is modeled in OpenShift Container Platform as aDeployment
. -
kuryr-cni
- a container installing and configuring Kuryr as a CNI driver on each OpenShift Container Platform node. This is modeled in OpenShift Container Platform as aDaemonSet
.
The Kuryr controller watches the OpenShift API server for Pod, Service, and namespace create, update, and delete events. It maps the OpenShift Container Platform API calls to corresponding objects in Neutron and Octavia. This means that every network solution that implements the Neutron trunk port functionality can be used to back OpenShift Container Platform via Kuryr. This includes open source solutions such as Open vSwitch (OVS) and Open Virtual Network (OVN) as well as Neutron-compatible commercial SDNs.
Kuryr is recommended for OpenShift Container Platform deployments on encapsulated RHOSP tenant networks to avoid double encapsulation, such as running an encapsulated OpenShift Container Platform SDN over an RHOSP network.
Conversely, Kuryr is not recommended in the following cases:
- You use provider networks or tenant VLANs.
- Your deployment uses many Services on a few hypervisors. Each OpenShift Service creates an Octavia Amphora virtual machine in OpenStack that hosts a required load balancer.
- UDP Services are needed.
1.2.3. Resource guidelines for installing OpenShift Container Platform on RHOSP with Kuryr
When using Kuryr SDN, the Pods, Services, namespaces, and network policies are using resources from the RHOSP quota; this increases the minimum requirements. Kuryr also has some additional requirements on top of what a default install requires.
Use the following quota to satisfy a default cluster’s minimum requirements:
Resource | Value |
---|---|
Floating IP addresses | 3 - plus the expected number of Services of LoadBalancer type |
Ports | 1500 - 1 needed per Pod |
Routers | 1 |
Subnets | 250 - 1 needed per Namespace/Project |
Networks | 250 - 1 needed per Namespace/Project |
RAM | 112 GB |
vCPUs | 28 |
Volume storage | 175 GB |
Instances | 7 |
Security groups | 250 - 1 needed per Service and per NetworkPolicy |
Security group rules | 1000 |
Swift containers | 2 |
Swift objects | 1 |
Swift available space | 10 MB or more |
Load balancers | 100 - 1 needed per Service |
Load balancer listeners | 500 - 1 needed per Service-exposed port |
Load balancer pools | 500 - 1 needed per Service-exposed port |
A cluster might function with fewer than recommended resources, but its performance is not guaranteed.
Take the following notes into consideration when setting resources:
- The number of ports required is actually larger than the number of Pods. Kuryr uses ports pools to have pre-created ports ready to be used by Pods and speed up the Pods booting time.
- Each NetworkPolicy is mapped into an RHOSP security group, and depending on the NetworkPolicy spec, one or more rules are added to the security group.
- Each Service is mapped into an RHOSP load balancer. Each load balancer has a security group with the user project; therefore, it must be taken into account when estimating the number of security groups required for the quota.
- Swift space requirements vary depending on the size of the bootstrap Ignition file and image registry.
- The quota does not account for load balancer resources (such as VM resources), but you must consider these resources when you decide the RHOSP deployment’s size. The default installation will have more than 50 load balancers; the clusters must be able to accommodate them.
An OpenShift Container Platform deployment comprises control plane machines, compute machines, and a bootstrap machine.
To enable Kuryr SDN, your environment must meet the following requirements:
- Run RHOSP 13+.
- Have Overcloud with Octavia.
- Use Neutron Trunk ports extension.
-
Use
openvswitch
firewall driver if ML2/OVS Neutron driver is used instead ofovs-hybrid
.
1.2.3.1. Increasing quota
When using Kuryr SDN, you must increase quotas to satisfy the Red Hat OpenStack Platform (RHOSP) resources used by Pods, Services, namespaces, and network policies.
Procedure
Increase the quotas for a project by running the following command:
$ sudo openstack quota set --secgroups 250 --secgroup-rules 1000 --ports 1500 --subnets 250 --networks 250 <project>
1.2.3.2. Configuring Neutron
Kuryr CNI leverages the Neutron Trunks extension to plug containers into the Red Hat OpenStack Platform (RHOSP) SDN, so you must use the trunks
extension for Kuryr to properly work.
In addition, if you leverage the default ML2/OVS Neutron driver, the firewall must be set to openvswitch
instead of ovs_hybrid
so that security groups are enforced on trunk subports and Kuryr can properly handle network policies.
1.2.3.3. Configuring Octavia
Kuryr SDN uses Red Hat OpenStack Platform (RHOSP)'s Octavia LBaaS to implement OpenShift Container Platform Services. Thus, you must install and configure Octavia components in RHOSP to use Kuryr SDN.
To enable Octavia, you must include the Octavia Service during the installation of the RHOSP Overcloud, or upgrade the Octavia Service if the Overcloud already exists. The following steps for enabling Octavia apply to both a clean install of the Overcloud or an Overcloud update.
The following steps only capture the key pieces required during the deployment of RHOSP when dealing with Octavia. It is also important to note that registry methods vary.
This example uses the local registry method.
Procedure
If you are using the local registry, create a template to upload the images to the registry. For example:
(undercloud) $ openstack overcloud container image prepare \ -e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/octavia.yaml \ --namespace=registry.access.redhat.com/rhosp13 \ --push-destination=<local-ip-from-undercloud.conf>:8787 \ --prefix=openstack- \ --tag-from-label {version}-{release} \ --output-env-file=/home/stack/templates/overcloud_images.yaml \ --output-images-file /home/stack/local_registry_images.yaml
Verify that the
local_registry_images.yaml
file contains the Octavia images. For example:... - imagename: registry.access.redhat.com/rhosp13/openstack-octavia-api:13.0-43 push_destination: <local-ip-from-undercloud.conf>:8787 - imagename: registry.access.redhat.com/rhosp13/openstack-octavia-health-manager:13.0-45 push_destination: <local-ip-from-undercloud.conf>:8787 - imagename: registry.access.redhat.com/rhosp13/openstack-octavia-housekeeping:13.0-45 push_destination: <local-ip-from-undercloud.conf>:8787 - imagename: registry.access.redhat.com/rhosp13/openstack-octavia-worker:13.0-44 push_destination: <local-ip-from-undercloud.conf>:8787
NoteThe Octavia container versions vary depending upon the specific RHOSP release installed.
Pull the container images from registry.redhat.io to the Undercloud node:
(undercloud) $ sudo openstack overcloud container image upload \ --config-file /home/stack/local_registry_images.yaml \ --verbose
This may take some time depending on the speed of your network and Undercloud disk.
Since an Octavia load balancer is used to access the OpenShift API, you must increase their listeners' default timeouts for the connections. The default timeout is 50 seconds. Increase the timeout to 20 minutes by passing the following file to the Overcloud deploy command:
(undercloud) $ cat octavia_timeouts.yaml parameter_defaults: OctaviaTimeoutClientData: 1200000 OctaviaTimeoutMemberData: 1200000
NoteThis is not needed for RHOSP 14+.
Install or update your Overcloud environment with Octavia:
openstack overcloud deploy --templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/octavia.yaml \ -e octavia_timeouts.yaml
NoteThis command only includes the files associated with Octavia; it varies based on your specific installation of RHOSP. See the RHOSP documentation for further information. For more information on customizing your Octavia installation, see installation of Octavia using Director.
NoteWhen leveraging Kuryr SDN, the Overcloud installation requires the Neutron
trunk
extension. This is available by default on Director deployments. Use theopenvswitch
firewall instead of the defaultovs-hybrid
when the Neutron backend is ML2/OVS. There is no need for modifications if the backend is ML2/OVN.In RHOSP versions 13 and 15, add the project ID to the
octavia.conf
configuration file after you create the project.To enforce network policies across Services, like when traffic goes through the Octavia load balancer, you must ensure Octavia creates the Amphora VM security groups on the user project.
This change ensures that required LoadBalancer security groups belong to that project, and that they can be updated to enforce Services isolation.
NoteThis task is unnecessary in RHOSP version 16 or later.
Octavia implements a new ACL API that restricts access to the Load Balancers VIP.
Get the project ID
$ openstack project show <project> +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | | | domain_id | default | | enabled | True | | id | PROJECT_ID | | is_domain | False | | name | *<project>* | | parent_id | default | | tags | [] | +-------------+----------------------------------+
Add the project ID to
octavia.conf
for the controllers.List the Overcloud controllers.
$ source stackrc # Undercloud credentials $ openstack server list +--------------------------------------+--------------+--------+-----------------------+----------------+------------+ │ | ID | Name | Status | Networks | Image | Flavor | │ +--------------------------------------+--------------+--------+-----------------------+----------------+------------+ │ | 6bef8e73-2ba5-4860-a0b1-3937f8ca7e01 | controller-0 | ACTIVE | ctlplane=192.168.24.8 | overcloud-full | controller | │ | dda3173a-ab26-47f8-a2dc-8473b4a67ab9 | compute-0 | ACTIVE | ctlplane=192.168.24.6 | overcloud-full | compute | │ +--------------------------------------+--------------+--------+-----------------------+----------------+------------+
SSH into the controller(s).
$ ssh heat-admin@192.168.24.8
Edit the
octavia.conf
to add the project into the list of projects where Amphora security groups are on the user’s account.# List of project IDs that are allowed to have Load balancer security groups # belonging to them. amp_secgroup_allowed_projects = PROJECT_ID
Restart the Octavia worker so the new configuration loads.
controller-0$ sudo docker restart octavia_worker
Depending on your RHOSP environment, Octavia might not support UDP listeners, which means there is no support for UDP Services if Kuryr SDN is used.
1.2.3.4. Known limitations of installing with Kuryr
Using OpenShift Container Platform with Kuryr SDN has several known limitations.
1.2.3.4.1. RHOSP general limitations
OpenShift Container Platform with Kuryr SDN does not support NodePort services.
RHOSP resource limitations
- An Amphora load balancer VM is deployed per OpenShift Service that uses the default Octavia load balancer driver (Amphora driver). Creating too many Services can cause you to run out of resources.
RHOSP version limitations
Using OpenShift Container Platform with Kuryr SDN has several limitations that depend on the RHOSP version.
- Octavia RHOSP versions before 16 do not support UDP listeners. Therefore, OpenShift UDP services are not supported.
- Octavia RHOSP versions before 16 cannot listen to multiple protocols on the same port. Services that expose the same port to different protocols, like TCP and UDP, are not supported.
The OVN Octavia driver does not support listeners that use different protocols on any RHOSP version.
RHOSP environment limitations
There are limitations when using Kuryr SDN that depend on your deployment environment.
Because of Octavia’s lack of support for the UDP protocol and multiple listeners, Kuryr forces Pods to use TCP for DNS resolution if:
- The RHOSP version is earlier than 16
- The OVN Octavia driver is used
In Go versions 1.12 and earlier, applications that are compiled with CGO support disabled use UDP only. In this case, the native Go resolver does not recognize the use-vc
option in resolv.conf
, which controls whether TCP is forced for DNS resolution. As a result, UDP is still used for DNS resolution, which fails.
To ensure that TCP forcing is allowed, compile applications either with the environment variable CGO_ENABLED
set to 1
, i.e. CGO_ENABLED=1
, or ensure that the variable is absent.
In Go versions 1.13 and later, TCP is used automatically if DNS resolution using UDP fails.
musl-based containers, including Alpine-based containers, do not support the use-vc
option.
1.2.3.5. Control plane and compute machines
By default, the OpenShift Container Platform installation program stands up three control plane and compute machines.
Each machine requires:
- An instance from the RHOSP quota
- A port from the RHOSP quota
- A flavor with at least 16 GB memory, 4 vCPUs, and 25 GB storage space
Compute machines host the applications that you run on OpenShift Container Platform; aim to run as many as you can.
1.2.3.6. Bootstrap machine
During installation, a bootstrap machine is temporarily provisioned to stand up the control plane. After the production control plane is ready, the bootstrap machine is deprovisioned.
The bootstrap machine requires:
- An instance from the RHOSP quota
- A port from the RHOSP quota
- A flavor with at least 16 GB memory, 4 vCPUs, and 25 GB storage space
The installation program cannot pass certificate authority bundles to Ignition on control plane machines. Therefore, the bootstrap machine cannot retrieve Ignition configurations from Swift if your endpoint uses self-signed certificates.
1.2.4. Internet and Telemetry access for OpenShift Container Platform
In OpenShift Container Platform 4.3, you require access to the internet to install your cluster. The Telemetry service, which runs by default to provide metrics about cluster health and the success of updates, also requires internet access. If your cluster is connected to the internet, Telemetry runs automatically, and your cluster is registered to the Red Hat OpenShift Cluster Manager (OCM).
Once you confirm that your Red Hat OpenShift Cluster Manager inventory is correct, either maintained automatically by Telemetry or manually using OCM, use subscription watch to track your OpenShift Container Platform subscriptions at the account or multi-cluster level.
You must have internet access to:
- Access the Red Hat OpenShift Cluster Manager page to download the installation program and perform subscription management. If the cluster has internet access and you do not disable Telemetry, that service automatically entitles your cluster.
- Access Quay.io to obtain the packages that are required to install your cluster.
- Obtain the packages that are required to perform cluster updates.
If your cluster cannot have direct internet access, you can perform a restricted network installation on some types of infrastructure that you provision. During that process, you download the content that is required and use it to populate a mirror registry with the packages that you need to install a cluster and generate the installation program. With some installation types, the environment that you install your cluster in will not require internet access. Before you update the cluster, you update the content of the mirror registry.
1.2.5. Enabling Swift on OpenStack
OpenShift Container Platform on Red Hat OpenStack Platform (RHOSP) uses RHOSP Object Storage (Swift) to store and serve user configuration files.
Swift is operated by a user account with the swiftoperator
role.
Prerequisites
- A RHOSP administrator account on the target environment
-
On Ceph RGW, the
account in url
option must be enabled
Procedure
To enable Swift on RHOSP:
As an administrator in the RHOSP CLI, add the
swiftoperator
role to the account that will access Swift:$ openstack role add --user <user> --project <project> swiftoperator
Your RHOSP deployment can now use Swift to store and serve files.
1.2.6. Verifying external network access
The OpenShift Container Platform installer requires external network access. You must provide an external network value to it, or deployment fails. Before you run the installer, verify that a network with the External router type exists in Red Hat OpenStack Platform (RHOSP).
Prerequisites
On RHOSP, the
NeutronDhcpAgentDnsmasqDnsServers
parameter must be configured to allow DHCP agents to forward instances' DNS queries. One way to set this parameter is to:- Create a new environment file in the template directory.
Provide parameter values in the file. For example:
Sample
neutron-dhcp-agent-dnsmasq-dns-servers.yaml
fileparameter_defaults: NeutronDhcpAgentDnsmasqDnsServers: ['<DNS_server_address_1>','<DNS_server_address_2']
Include the environment file in your Overcloud deploy command. For example:
$ openstack overcloud deploy --templates -e neutron-dhcp-agent-dnsmasq-dns-servers.yaml ...
Procedure
Using the RHOSP CLI, verify the name and ID of the 'External' network:
$ openstack network list --long -c ID -c Name -c "Router Type" +--------------------------------------+----------------+-------------+ | ID | Name | Router Type | +--------------------------------------+----------------+-------------+ | 148a8023-62a7-4672-b018-003462f8d7dc | public_network | External | +--------------------------------------+----------------+-------------+
A network with an External router type appears in the network list. If at least one does not, see Creating a default floating IP network and Creating a default provider network.
If the external network’s CIDR range overlaps one of the default network ranges, you must change the matching network ranges in the install-config.yaml
file before you run the installation program.
The default network ranges are:
Network | Range |
---|---|
machineCIDR | 10.0.0.0/16 |
serviceNetwork | 172.30.0.0/16 |
clusterNetwork | 10.128.0.0/14 |
If the installation program finds multiple networks with the same name, it sets one of them at random. To avoid this behavior, create unique names for resources in RHOSP.
If the Neutron trunk service plug-in is enabled, a trunk port is created by default. For more information, see Neutron trunk port.
1.2.7. Defining parameters for the installation program
The OpenShift Container Platform installation program relies on a file called clouds.yaml
. The file describes Red Hat OpenStack Platform (RHOSP) configuration parameters, including the project name, log in information, and authorization service URLs.
Procedure
Create the
clouds.yaml
file:If your RHOSP distribution includes the Horizon web UI, generate a
clouds.yaml
file in it.ImportantRemember to add a password to the
auth
field. You can also keep secrets in a separate file fromclouds.yaml
.If your RHOSP distribution does not include the Horizon web UI, or you do not want to use Horizon, create the file yourself. For detailed information about
clouds.yaml
, see Config files in the RHOSP documentation.clouds: shiftstack: auth: auth_url: http://10.10.14.42:5000/v3 project_name: shiftstack username: shiftstack_user password: XXX user_domain_name: Default project_domain_name: Default dev-env: region_name: RegionOne auth: username: 'devuser' password: XXX project_name: 'devonly' auth_url: 'https://10.10.14.22:5001/v2.0'
Place the file that you generate in one of the following locations:
-
The value of the
OS_CLIENT_CONFIG_FILE
environment variable - The current directory
-
A Unix-specific user configuration directory, for example
~/.config/openstack/clouds.yaml
A Unix-specific site configuration directory, for example
/etc/openstack/clouds.yaml
The installation program searches for
clouds.yaml
in that order.
-
The value of the
1.2.8. Obtaining the installation program
Before you install OpenShift Container Platform, download the installation file on a local computer.
Prerequisites
- You must install the cluster from a computer that uses Linux or macOS.
- You need 500 MB of local disk space to download the installation program.
Procedure
- Access the Infrastructure Provider page on the Red Hat OpenShift Cluster Manager site. If you have a Red Hat account, log in with your credentials. If you do not, create an account.
Navigate to the page for your installation type, download the installation program for your operating system, and place the file in the directory where you will store the installation configuration files.
ImportantThe installation program creates several files on the computer that you use to install your cluster. You must keep both the installation program and the files that the installation program creates after you finish installing the cluster.
ImportantDeleting the files created by the installation program does not remove your cluster, even if the cluster failed during installation. You must complete the OpenShift Container Platform uninstallation procedures outlined for your specific cloud provider to remove your cluster entirely.
Extract the installation program. For example, on a computer that uses a Linux operating system, run the following command:
$ tar xvf <installation_program>.tar.gz
-
From the Pull Secret page on the Red Hat OpenShift Cluster Manager site, download your installation pull secret as a
.txt
file. This pull secret allows you to authenticate with the services that are provided by the included authorities, including Quay.io, which serves the container images for OpenShift Container Platform components.
1.2.9. Creating the installation configuration file
You can customize your installation of OpenShift Container Platform on Red Hat OpenStack Platform (RHOSP).
Prerequisites
- Obtain the OpenShift Container Platform installation program and the pull secret for your cluster.
Procedure
Create the
install-config.yaml
file.Run the following command:
$ ./openshift-install create install-config --dir=<installation_directory> 1
- 1
- For
<installation_directory>
, specify the directory name to store the files that the installation program creates.
ImportantSpecify an empty directory. Some installation assets, like bootstrap X.509 certificates have short expiration intervals, so you must not reuse an installation directory. If you want to reuse individual files from another cluster installation, you can copy them into your directory. However, the file names for the installation assets might change between releases. Use caution when copying installation files from an earlier OpenShift Container Platform version.
At the prompts, provide the configuration details for your cloud:
Optional: Select an SSH key to use to access your cluster machines.
NoteFor production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery on, specify an SSH key that your
ssh-agent
process uses.- Select openstack as the platform to target.
- Specify the Red Hat OpenStack Platform (RHOSP) external network name to use for installing the cluster.
- Specify the Floating IP address to use for external access to the OpenShift API.
- Specify a RHOSP flavor with at least 16 GB RAM to use for control plane and compute nodes.
- Select the base domain to deploy the cluster to. All DNS records will be sub-domains of this base and will also include the cluster name.
- Enter a name for your cluster. The name must be 14 or fewer characters long.
- Paste the pull secret that you obtained from the Pull Secret page on the Red Hat OpenShift Cluster Manager site.
-
Modify the
install-config.yaml
file. You can find more information about the available parameters in the Installation configuration parameters section. Back up the
install-config.yaml
file so that you can use it to install multiple clusters.ImportantThe
install-config.yaml
file is consumed during the installation process. If you want to reuse the file, you must back it up now.
1.2.10. Installation configuration parameters
Before you deploy an OpenShift Container Platform cluster, you provide parameter values to describe your account on the cloud platform that hosts your cluster and optionally customize your cluster’s platform. When you create the install-config.yaml
installation configuration file, you provide values for the required parameters through the command line. If you customize your cluster, you can modify the install-config.yaml
file to provide more details about the platform.
You cannot modify these parameters in the install-config.yaml
file after installation.
Parameter | Description | Values |
---|---|---|
|
The base domain of your cloud provider. This value is used to create routes to your OpenShift Container Platform cluster components. The full DNS name for your cluster is a combination of the |
A fully-qualified domain or subdomain name, such as |
|
The cloud provider to host the control plane machines. This parameter value must match the |
|
|
The cloud provider to host the worker machines. This parameter value must match the |
|
| The name of your cluster. |
A string that contains uppercase or lowercase letters, such as |
| The region to deploy your cluster in. |
A valid region for your cloud, such as |
| The pull secret that you obtained from the Pull Secret page on the Red Hat OpenShift Cluster Manager site. You use this pull secret to authenticate with the services that are provided by the included authorities, including Quay.io, which serves the container images for OpenShift Container Platform components. |
{ "auths":{ "cloud.openshift.com":{ "auth":"b3Blb=", "email":"you@example.com" }, "quay.io":{ "auth":"b3Blb=", "email":"you@example.com" } } } |
Parameter | Description | Values |
---|---|---|
| The SSH key to use to access your cluster machines. Note
For production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery on, specify an SSH key that your |
A valid, local public SSH key that you added to the |
| Whether to enable or disable FIPS mode. By default, FIPS mode is not enabled. If FIPS mode is enabled, the Red Hat Enterprise Linux CoreOS (RHCOS) machines that OpenShift Container Platform runs on bypass the default Kubernetes cryptography suite and use the cryptography modules that are provided with RHCOS instead. |
|
| How to publish the user-facing endpoints of your cluster. |
|
|
Whether to enable or disable simultaneous multithreading, or Important If you disable simultaneous multithreading, ensure that your capacity planning accounts for the dramatically decreased machine performance. |
|
| The number of compute machines, which are also known as worker machines, to provision. |
A positive integer greater than or equal to |
|
Whether to enable or disable simultaneous multithreading, or Important If you disable simultaneous multithreading, ensure that your capacity planning accounts for the dramatically decreased machine performance. |
|
| The number of control plane machines to provision. |
A positive integer greater than or equal to |
Parameter | Description | Values |
---|---|---|
| For compute machines, the size in gigabytes of the root volume. If you do not set this value, machines use ephemeral storage. |
Integer, for example |
| For compute machines, the root volume’s type. |
String, for example |
| For control plane machines, the size in gigabytes of the root volume. If you do not set this value, machines use ephemeral storage. |
Integer, for example |
| For control plane machines, the root volume’s type. |
String, for example |
| The region where the RHOSP cluster is created. |
String, for example |
|
The name of the RHOSP cloud to use from the list of clouds in the |
String, for example |
| The RHOSP external network name to be used for installation. |
String, for example |
| The RHOSP flavor to use for control plane and compute machines. |
String, for example |
| An existing floating IP address to associate with the load balancer API. |
An IP address, for example |
Parameter | Description | Values |
---|---|---|
| IP addresses for external DNS servers that cluster instances use for DNS resolution. |
A list of IP addresses as strings, for example |
| The default machine pool platform configuration. |
{ "type": "ml.large", "rootVolume": { "size": 30, "type": "performance" } } |
| The location from which the installer downloads the RHCOS image. You must set this parameter to perform an installation in a restricted network. | An HTTP or HTTPS URL, optionally with an SHA-256 checksum.
For example,
The value can also be the name of an existing Glance image, for example |
1.2.10.1. Sample customized install-config.yaml
file for RHOSP with Kuryr
To deploy with Kuryr SDN instead of the default OpenShift SDN, you must modify the install-config.yaml
file to include Kuryr
as the desired networking.networkType
and proceed with the default OpenShift SDN installation steps. This sample install-config.yaml
demonstrates all of the possible Red Hat OpenStack Platform (RHOSP) customization options.
This sample file is provided for reference only. You must obtain your install-config.yaml
file by using the installation program.
apiVersion: v1 baseDomain: example.com clusterID: os-test controlPlane: name: master platform: {} replicas: 3 compute: - name: worker platform: openstack: type: ml.large replicas: 3 metadata: name: example networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 machineCIDR: 10.0.0.0/16 serviceNetwork: - 172.30.0.0/16 networkType: Kuryr platform: openstack: region: region1 cloud: mycloud externalNetwork: external computeFlavor: m1.xlarge lbFloatingIP: 128.0.0.1 trunkSupport: true octaviaSupport: true pullSecret: '{"auths": ...}' sshKey: ssh-ed25519 AAAA...
Both trunkSupport
and octaviaSupport
are automatically discovered by the installer, so there is no need to set them. But if your environment does not meet both requirements, Kuryr SDN will not properly work. Trunks are needed to connect the Pods to the RHOSP network and Octavia is required to create the OpenShift Services.
1.2.11. Generating an SSH private key and adding it to the agent
If you want to perform installation debugging or disaster recovery on your cluster, you must provide an SSH key to both your ssh-agent
and to the installation program.
In a production environment, you require disaster recovery and debugging.
You can use this key to SSH into the master nodes as the user core
. When you deploy the cluster, the key is added to the core
user’s ~/.ssh/authorized_keys
list.
Procedure
If you do not have an SSH key that is configured for password-less authentication on your computer, create one. For example, on a computer that uses a Linux operating system, run the following command:
$ ssh-keygen -t rsa -b 4096 -N '' \ -f <path>/<file_name> 1
- 1
- Specify the path and file name, such as
~/.ssh/id_rsa
, of the SSH key. Do not specify an existing SSH key, as it will be overwritten.
Running this command generates an SSH key that does not require a password in the location that you specified.
Start the
ssh-agent
process as a background task:$ eval "$(ssh-agent -s)" Agent pid 31874
Add your SSH private key to the
ssh-agent
:$ ssh-add <path>/<file_name> 1 Identity added: /home/<you>/<path>/<file_name> (<computer_name>)
- 1
- Specify the path and file name for your SSH private key, such as
~/.ssh/id_rsa
Next steps
- When you install OpenShift Container Platform, provide the SSH public key to the installation program.
1.2.12. Enabling access to the environment
At deployment, all OpenShift Container Platform machines are created in a Red Hat OpenStack Platform (RHOSP)-tenant network. Therefore, they are not accessible directly in most RHOSP deployments.
You can configure the OpenShift Container Platform API to be accessible either with or without floating IP addresses.
1.2.12.1. Enabling access with floating IP addresses
Make OpenShift Container Platform API endpoints accessible by attaching two floating IP (FIP) addresses to them: one for the API load balancer (lb FIP
), and one for OpenShift Container Platform applications (apps FIP
).
The load balancer FIP is also used in the install-config.yaml
file.
Procedure
Using the Red Hat OpenStack Platform (RHOSP) CLI, create a new external network:
$ openstack floating ip create <external network>
Add a record that follows this pattern to your DNS server:
api.<cluster name>.<base domain> IN A <lb FIP>
NoteIf you do not control the DNS server you can add the record to your
/etc/hosts
file instead. This action makes the API accessible to you only, which is not suitable for production deployment but does allow installation for development and testing.
You can make OpenShift Container Platform resources available outside of the cluster by assigning a floating IP address and updating your firewall configuration.
1.2.12.2. Enabling access without floating IP addresses
If you cannot use floating IP addresses, the OpenShift Container Platform installation might still finish. However, the installation program fails after it times out waiting for API access.
After the installation program times out, the cluster might still initialize. After the bootstrapping processing begins, it must complete. You must edit the cluster’s networking configuration after it is deployed, however.
1.2.13. Deploy the cluster
You can install OpenShift Container Platform on a compatible cloud platform.
You can run the create cluster
command of the installation program only once, during initial installation.
Prerequisites
- Obtain the OpenShift Container Platform installation program and the pull secret for your cluster.
Procedure
Run the installation program:
$ ./openshift-install create cluster --dir=<installation_directory> \ 1 --log-level=info 2
NoteIf the cloud provider account that you configured on your host does not have sufficient permissions to deploy the cluster, the installation process stops, and the missing permissions are displayed.
When the cluster deployment completes, directions for accessing your cluster, including a link to its web console and credentials for the
kubeadmin
user, display in your terminal.ImportantThe Ignition config files that the installation program generates contain certificates that expire after 24 hours. You must keep the cluster running for 24 hours in a non-degraded state to ensure that the first certificate rotation has finished.
ImportantYou must not delete the installation program or the files that the installation program creates. Both are required to delete the cluster.
1.2.14. Verifying cluster status
To verify your OpenShift Container Platform cluster’s status during or after installation:
Procedure
In the cluster environment, export the administrator’s kubeconfig file:
$ export KUBECONFIG=<installation_directory>/auth/kubeconfig 1
- 1
- For
<installation_directory>
, specify the path to the directory that you stored the installation files in.
The
kubeconfig
file contains information about the cluster that is used by the CLI to connect a client to the correct cluster and API server.View the control plane and compute machines created after a deployment:
$ oc get nodes
View your cluster’s version:
$ oc get clusterversion
View your operators' status:
$ oc get clusteroperator
View all running Pods in the cluster:
$ oc get pods -A
1.2.15. Logging in to the cluster
You can log in to your cluster as a default system user by exporting the cluster kubeconfig
file. The kubeconfig
file contains information about the cluster that is used by the CLI to connect a client to the correct cluster and API server. The file is specific to a cluster and is created during OpenShift Container Platform installation.
Prerequisites
- Deploy an OpenShift Container Platform cluster.
-
Install the
oc
CLI.
Procedure
Export the
kubeadmin
credentials:$ export KUBECONFIG=<installation_directory>/auth/kubeconfig 1
- 1
- For
<installation_directory>
, specify the path to the directory that you stored the installation files in.
Verify you can run
oc
commands successfully using the exported configuration:$ oc whoami system:admin
1.2.16. Configuring application access with floating IP addresses
After you install OpenShift Container Platform, configure Red Hat OpenStack Platform (RHOSP) to allow application network traffic.
Prerequisites
- OpenShift Container Platform cluster must be installed
- Floating IP addresses are enabled as described in Enabling access to the environment.
Procedure
After you install the OpenShift Container Platform cluster, attach a floating IP address to the ingress port:
Show the port:
$ openstack port show <cluster name>-<clusterID>-ingress-port
Attach the port to the IP address:
$ openstack floating ip set --port <ingress port ID> <apps FIP>
Add a wildcard
A
record for*apps.
to your DNS file:*.apps.<cluster name>.<base domain> IN A <apps FIP>
If you do not control the DNS server but want to enable application access for non-production purposes, you can add these hostnames to /etc/hosts
:
<apps FIP> console-openshift-console.apps.<cluster name>.<base domain> <apps FIP> integrated-oauth-server-openshift-authentication.apps.<cluster name>.<base domain> <apps FIP> oauth-openshift.apps.<cluster name>.<base domain> <apps FIP> prometheus-k8s-openshift-monitoring.apps.<cluster name>.<base domain> <apps FIP> grafana-openshift-monitoring.apps.<cluster name>.<base domain> <apps FIP> <app name>.apps.<cluster name>.<base domain>
1.2.17. Next steps
- Customize your cluster.
- If necessary, you can opt out of remote health reporting.
- If you need to enable external access to node ports, configure ingress cluster traffic by using a NodePort.
1.3. Uninstalling a cluster on OpenStack
You can remove a cluster that you deployed to Red Hat OpenStack Platform (RHOSP).
1.3.1. Removing a cluster that uses installer-provisioned infrastructure
You can remove a cluster that uses installer-provisioned infrastructure from your cloud.
Prerequisites
- Have a copy of the installation program that you used to deploy the cluster.
- Have the files that the installation program generated when you created your cluster.
Procedure
From the computer that you used to install the cluster, run the following command:
$ ./openshift-install destroy cluster \ --dir=<installation_directory> --log-level=info 1 2
NoteYou must specify the directory that contains the cluster definition files for your cluster. The installation program requires the
metadata.json
file in this directory to delete the cluster.-
Optional: Delete the
<installation_directory>
directory and the OpenShift Container Platform installation program.
Legal Notice
Copyright © 2024 Red Hat, Inc.
OpenShift documentation is licensed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).
Modified versions must remove all Red Hat trademarks.
Portions adapted from https://github.com/kubernetes-incubator/service-catalog/ with modifications by Red Hat.
Red Hat, Red Hat Enterprise Linux, the Red Hat logo, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.