Director Installation and Usage
An end-to-end scenario on using Red Hat OpenStack Platform director to create an OpenStack cloud
Abstract
Chapter 1. Introduction to director Copy linkLink copied to clipboard!
The Red Hat OpenStack Platform (RHOSP) director is a toolset for installing and managing a complete OpenStack environment. Director is based primarily on the OpenStack project TripleO. With director you can install a fully-operational, lean, and robust RHOSP environment that can provision and control bare metal systems to use as OpenStack nodes.
Director uses two main concepts: an undercloud and an overcloud. First you install the undercloud, and then use the undercloud as a tool to install and configure the overcloud.
1.1. Undercloud Copy linkLink copied to clipboard!
The undercloud is the main management node that contains the Red Hat OpenStack Platform director toolset. It is a single-system OpenStack installation that includes components for provisioning and managing the OpenStack nodes that form your OpenStack environment (the overcloud). The components that form the undercloud have multiple functions:
- Environment planning
- The undercloud includes planning functions that you can use to create and assign certain node roles. The undercloud includes a default set of nodes: Compute, Controller, and various Storage roles. You can also design custom roles. Additionally, you can select which OpenStack Platform services to include on each node role, which provides a method to model new node types or isolate certain components on their own host.
- Bare metal system control
- The undercloud uses the out-of-band management interface, usually Intelligent Platform Management Interface (IPMI), of each node for power management control and a PXE-based service to discover hardware attributes and install OpenStack on each node. You can use this feature to provision bare metal systems as OpenStack nodes. For a full list of power management drivers, see Appendix A, Power management drivers.
- Orchestration
- The undercloud contains a set of YAML templates that represent a set of plans for your environment. The undercloud imports these plans and follows their instructions to create the resulting OpenStack environment. The plans also include hooks that you can use to incorporate your own customizations as certain points in the environment creation process.
- Undercloud components
The undercloud uses OpenStack components as its base tool set. Each component operates within a separate container on the undercloud:
- OpenStack Identity (keystone) - Provides authentication and authorization for the director components.
- OpenStack Bare Metal (ironic) and OpenStack Compute (nova) - Manages bare metal nodes.
- OpenStack Networking (neutron) and Open vSwitch - Control networking for bare metal nodes.
- OpenStack Image Service (glance) - Stores images that director writes to bare metal machines.
- OpenStack Orchestration (heat) and Puppet - Provides orchestration of nodes and configuration of nodes after director writes the overcloud image to disk.
OpenStack Telemetry (ceilometer) - Performs monitoring and data collection. Telemetry also includes the following components:
- OpenStack Telemetry Metrics (gnocchi) - Provides a time series database for metrics.
- OpenStack Telemetry Alarming (aodh) - Provide an alarming component for monitoring.
- OpenStack Telemetry Event Storage (panko) - Provides event storage for monitoring.
- OpenStack Workflow Service (mistral) - Provides a set of workflows for certain director-specific actions, such as importing and deploying plans.
- OpenStack Messaging Service (zaqar) - Provides a messaging service for the OpenStack Workflow Service.
OpenStack Object Storage (swift) - Provides object storage for various OpenStack Platform components, including:
- Image storage for OpenStack Image Service
- Introspection data for OpenStack Bare Metal
- Deployment plans for OpenStack Workflow Service
1.2. Understanding the overcloud Copy linkLink copied to clipboard!
The overcloud is the resulting Red Hat OpenStack Platform (RHOSP) environment that the undercloud creates. The overcloud consists of multiple nodes with different roles that you define based on the OpenStack Platform environment that you want to create. The undercloud includes a default set of overcloud node roles:
- Controller
Controller nodes provide administration, networking, and high availability for the OpenStack environment. A recommended OpenStack environment contains three Controller nodes together in a high availability cluster.
A default Controller node role supports the following components. Not all of these services are enabled by default. Some of these components require custom or pre-packaged environment files to enable:
- OpenStack Dashboard (horizon)
- OpenStack Identity (keystone)
- OpenStack Compute (nova) API
- OpenStack Networking (neutron)
- OpenStack Image Service (glance)
- OpenStack Block Storage (cinder)
- OpenStack Object Storage (swift)
- OpenStack Orchestration (heat)
- OpenStack Telemetry Metrics (gnocchi)
- OpenStack Telemetry Alarming (aodh)
- OpenStack Telemetry Event Storage (panko)
- OpenStack Shared File Systems (manila)
- OpenStack Bare Metal (ironic)
- MariaDB
- Open vSwitch
- Pacemaker and Galera for high availability services.
- Compute
Compute nodes provide computing resources for the OpenStack environment. You can add more Compute nodes to scale out your environment over time. A default Compute node contains the following components:
- OpenStack Compute (nova)
- KVM/QEMU
- OpenStack Telemetry (ceilometer) agent
- Open vSwitch
- Storage
Storage nodes provide storage for the OpenStack environment. The following list contains information about the various types of Storage node in RHOSP:
- Ceph Storage nodes - Used to form storage clusters. Each node contains a Ceph Object Storage Daemon (OSD). Additionally, director installs Ceph Monitor onto the Controller nodes in situations where you deploy Ceph Storage nodes as part of your environment.
Block storage (cinder) - Used as external block storage for highly available Controller nodes. This node contains the following components:
- OpenStack Block Storage (cinder) volume
- OpenStack Telemetry agents
- Open vSwitch.
Object storage (swift) - These nodes provide an external storage layer for OpenStack Swift. The Controller nodes access object storage nodes through the Swift proxy. Object storage nodes contain the following components:
- OpenStack Object Storage (swift) storage
- OpenStack Telemetry agents
- Open vSwitch.
1.3. Understanding high availability in Red Hat OpenStack Platform Copy linkLink copied to clipboard!
The Red Hat OpenStack Platform (RHOSP) director uses a Controller node cluster to provide highly available services to your OpenStack Platform environment. For each service, director installs the same components on all Controller nodes and manages the Controller nodes together as a single service. This type of cluster configuration provides a fallback in the event of operational failures on a single Controller node. This provides OpenStack users with a certain degree of continuous operation.
The OpenStack Platform director uses some key pieces of software to manage components on the Controller node:
- Pacemaker - Pacemaker is a cluster resource manager. Pacemaker manages and monitors the availability of OpenStack components across all nodes in the cluster.
- HAProxy - Provides load balancing and proxy services to the cluster.
- Galera - Replicates the RHOSP database across the cluster.
- Memcached - Provides database caching.
- From version 13 and later, you can use director to deploy High Availability for Compute Instances (Instance HA). With Instance HA you can automate evacuating instances from a Compute node when the Compute node fails.
1.4. Understanding containerization in Red Hat OpenStack Platform Copy linkLink copied to clipboard!
Each OpenStack Platform service on the undercloud and overcloud runs inside an individual Linux container on their respective node. This containerization provides a method to isolate services, maintain the environment, and upgrade Red Hat OpenStack Platform (RHOSP).
Red Hat OpenStack Platform 16.0 supports installation on the Red Hat Enterprise Linux 8.1 operating system. Red Hat Enterprise Linux 8.1 no longer includes Docker and provides a new set of tools to replace the Docker ecosystem. This means OpenStack Platform 16.0 replaces Docker with these new tools for OpenStack Platform deployment and upgrades.
- Podman
Pod Manager (Podman) is a container management tool. It implements almost all Docker CLI commands, not including commands related to Docker Swarm. Podman manages pods, containers, and container images. One of the major differences between Podman and Docker is that Podman can manage resources without a daemon running in the background.
For more information about Podman, see the Podman website.
- Buildah
Buildah specializes in building Open Containers Initiative (OCI) images, which you use in conjunction with Podman. Buildah commands replicate the contents of a Dockerfile. Buildah also provides a lower-level
coreutilsinterface to build container images, so that you do not require a Dockerfile to build containers. Buildah also uses other scripting languages to build container images without requiring a daemon.For more information about Buildah, see the Buildah website.
- Skopeo
- Skopeo provides operators with a method to inspect remote container images, which helps director collect data when it pulls images. Additional features include copying container images from one registry to another and deleting images from registries.
Red Hat supports the following methods for managing container images for your overcloud:
-
Pulling container images from the Red Hat Container Catalog to the
image-serveregistry on the undercloud and then pulling the images from theimage-serveregistry. When you pull images to the undercloud first, you avoid multiple overcloud nodes simultaneously pulling container images over an external connection. - Pulling container images from your Satellite 6 server. You can pull these images directly from the Satellite because the network traffic is internal.
This guide contains information about configuring your container image registry details and performing basic container operations.
1.5. Working with Ceph Storage in Red Hat OpenStack Platform Copy linkLink copied to clipboard!
It is common for large organizations that use Red Hat OpenStack Platform (RHOSP) to serve thousands of clients or more. Each OpenStack client is likely to have their own unique needs when consuming block storage resources. Deploying glance (images), cinder (volumes), and nova (Compute) on a single node can become impossible to manage in large deployments with thousands of clients. Scaling OpenStack externally resolves this challenge.
However, there is also a practical requirement to virtualize the storage layer with a solution like Red Hat Ceph Storage so that you can scale the RHOSP storage layer from tens of terabytes to petabytes, or even exabytes of storage. Red Hat Ceph Storage provides this storage virtualization layer with high availability and high performance while running on commodity hardware. While virtualization might seem like it comes with a performance penalty, Ceph stripes block device images as objects across the cluster, meaning that large Ceph Block Device images have better performance than a standalone disk. Ceph Block devices also support caching, copy-on-write cloning, and copy-on-read cloning for enhanced performance.
For more information about Red Hat Ceph Storage, see Red Hat Ceph Storage.
For multi-architecture clouds, Red Hat supports only pre-installed or external Ceph implementation. For more information, see Integrating an Overcloud with an Existing Red Hat Ceph Cluster and Appendix B, Red Hat OpenStack Platform for POWER.
Part I. Director installation and configuration Copy linkLink copied to clipboard!
Chapter 2. Planning your undercloud Copy linkLink copied to clipboard!
2.1. Containerized undercloud Copy linkLink copied to clipboard!
The undercloud is the node that controls the configuration, installation, and management of your final Red Hat OpenStack Platform (RHOSP) environment, which is called the overcloud. The undercloud itself uses OpenStack Platform components in the form of containers to create a toolset called director. This means that the undercloud pulls a set of container images from a registry source, generates configuration for the containers, and runs each OpenStack Platform service as a container. As a result, the undercloud provides a containerized set of services that you can use as a toolset to create and manage your overcloud.
Since both the undercloud and overcloud use containers, both use the same architecture to pull, configure, and run containers. This architecture is based on the OpenStack Orchestration service (heat) for provisioning nodes and uses Ansible to configure services and containers. It is useful to have some familiarity with heat and Ansible to help you troubleshoot issues that you might encounter.
2.2. Preparing your undercloud networking Copy linkLink copied to clipboard!
The undercloud requires access to two main networks:
- The Provisioning or Control Plane network, which is the network that director uses to provision your nodes and access them over SSH when executing Ansible configuration. This network also enables SSH access from the undercloud to overcloud nodes. The undercloud contains DHCP services for introspection and provisioning other nodes on this network, which means that no other DHCP services should exist on this network. The director configures the interface for this network.
- The External network, which enables access to OpenStack Platform repositories, container image sources, and other servers such as DNS servers or NTP servers. Use this network for standard access the undercloud from your workstation. You must manually configure an interface on the undercloud to access the external network.
The undercloud requires a minimum of 2 x 1 Gbps Network Interface Cards: one for the Provisioning or Control Plane network and one for the External network. However, it is recommended to use a 10 Gbps interface for Provisioning network traffic, especially if you want to provision a large number of nodes in your overcloud environment.
Note:
- Do not use the same Provisioning or Control Plane NIC as the one that you use to access the director machine from your workstation. The director installation creates a bridge by using the Provisioning NIC, which drops any remote connections. Use the External NIC for remote connections to the director system.
The Provisioning network requires an IP range that fits your environment size. Use the following guidelines to determine the total number of IP addresses to include in this range:
- Include at least one temporary IP address for each node that connects to the Provisioning network during introspection.
- Include at least one permanent IP address for each node that connects to the Provisioning network during deployment.
- Include an extra IP address for the virtual IP of the overcloud high availability cluster on the Provisioning network.
- Include additional IP addresses within this range for scaling the environment.
2.3. Determining environment scale Copy linkLink copied to clipboard!
Before you install the undercloud, determine the scale of your environment. Include the following factors when you plan your environment:
- How many nodes do you want to deploy in your overcloud?
- The undercloud manages each node within an overcloud. Provisioning overcloud nodes consumes resources on the undercloud. You must provide your undercloud with enough resources to adequately provision and control all of your overcloud nodes.
- How many simultaneous operations do you want the undercloud to perform?
- Most OpenStack services on the undercloud use a set of workers. Each worker performs an operation specific to that service. Multiple workers provide simultaneous operations. The default number of workers on the undercloud is determined by halving the total CPU thread count on the undercloud [1]. For example, if your undercloud has a CPU with 16 threads, then the director services spawn 8 workers by default. Director also uses a set of minimum and maximum caps by default:
| Service | Minimum | Maximum |
|---|---|---|
| OpenStack Orchestration (heat) | 4 | 24 |
| All other service | 2 | 12 |
The undercloud has the following minimum CPU and memory requirements:
- An 8-thread 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions. This provides 4 workers for each undercloud service.
A minimum of 24 GB of RAM.
-
The
ceph-ansibleplaybook consumes 1 GB resident set size (RSS) for every 10 hosts that the undercloud deploys. If you want to use a new or existing Ceph cluster in your deployment, you must provision the undercloud RAM accordingly.
-
The
To use a larger number of workers, increase the vCPUs and memory of your undercloud using the following recommendations:
- Minimum: Use 1.5 GB of memory for each thread. For example, a machine with 48 threads requires 72 GB of RAM to provide the minimum coverage for 24 heat workers and 12 workers for other services.
- Recommended: Use 3 GB of memory for each thread. For example, a machine with 48 threads requires 144 GB of RAM to provide the recommended coverage for 24 heat workers and 12 workers for other services.
2.4. Undercloud disk sizing Copy linkLink copied to clipboard!
The recommended minimum undercloud disk size is 100 GB of available disk space on the root disk:
- 20 GB for container images
- 10 GB to accommodate QCOW2 image conversion and caching during the node provisioning process
- 70 GB+ for general usage, logging, metrics, and growth
2.5. Virtualization support Copy linkLink copied to clipboard!
Red Hat only supports a virtualized undercloud on the following platforms:
| Platform | Notes |
|---|---|
| Kernel-based Virtual Machine (KVM) | Hosted by Red Hat Enterprise Linux 8, as listed on certified hypervisors. |
| Red Hat Virtualization | Hosted by Red Hat Virtualization 4.x, as listed on certified hypervisors. |
| Microsoft Hyper-V | Hosted by versions of Hyper-V as listed on the Red Hat Customer Portal Certification Catalogue. |
| VMware ESX and ESXi | Hosted by versions of ESX and ESXi as listed on the Red Hat Customer Portal Certification Catalogue. |
Red Hat OpenStack Platform director requires that the latest version of Red Hat Enterprise Linux 8 is installed as the host operating system. This means your virtualization platform must also support the underlying Red Hat Enterprise Linux version.
Virtual Machine Requirements
Resource requirements for a virtual undercloud are similar to those of a bare metal undercloud. You should consider the various tuning options when provisioning such as network model, guest CPU capabilities, storage backend, storage format, and caching mode.
Network Considerations
Note the following network considerations for your virtualized undercloud:
- Power Management
-
The undercloud VM requires access to the overcloud nodes' power management devices. This is the IP address set for the
pm_addrparameter when registering nodes. - Provisioning network
-
The NIC used for the provisioning (
ctlplane) network requires the ability to broadcast and serve DHCP requests to the NICs of the overcloud’s bare metal nodes. As a recommendation, create a bridge that connects the VM’s NIC to the same network as the bare metal NICs.
A common problem occurs when the hypervisor technology blocks the undercloud from transmitting traffic from an unknown address. - If using Red Hat Enterprise Virtualization, disable anti-mac-spoofing to prevent this. - If using VMware ESX or ESXi, allow forged transmits to prevent this. You must power off and on the director VM after you apply these settings. Rebooting the VM is not sufficient.
2.6. Character encoding configuration Copy linkLink copied to clipboard!
Red Hat OpenStack Platform has special character encoding requirements as part of the locale settings:
-
Use UTF-8 encoding on all nodes. Ensure the
LANGenvironment variable is set toen_US.UTF-8on all nodes. - Avoid using non-ASCII characters if you use Red Hat Ansible Tower to automate the creation of Red Hat OpenStack Platform resources.
2.7. Considerations when running the undercloud with a proxy Copy linkLink copied to clipboard!
If your environment uses a proxy, review these considerations to best understand the different configuration methods of integrating parts of Red Hat OpenStack Platform with a proxy and the limitations of each method.
System-wide proxy configuration
Use this method to configure proxy communication for all network traffic on the undercloud. To configure the proxy settings, edit the /etc/environment file and set the following environment variables:
- http_proxy
- The proxy that you want to use for standard HTTP requests.
- https_proxy
- The proxy that you want to use for HTTPs requests.
- no_proxy
- A comma-separated list of domains that you want to exclude from proxy communications.
The system-wide proxy method has the following limitations:
-
The
no_proxyvariable primarily uses domain names (www.example.com), domain suffixes (example.com), and domains with a wildcard (*.example.com). Most Red Hat OpenStack Platform services interpret IP addresses inno_proxybut certain services, such as container health checks, do not interpret IP addresses in theno_proxyenvironment variable due to limitations with cURL andwget. To use a system-wide proxy with the undercloud, disable container health checks with thecontainer_healthcheck_disabledparameter in theundercloud.conffile during installation. For more information, see BZ#1837458 - Container health checks fail to honor no_proxy CIDR notation. -
Some containers bind and parse the environment variables in
/etc/environmentsincorrectly, which causes problems when running these services. For more information, see BZ#1916070 - proxy configuration updates in /etc/environment files are not being picked up in containers correctly and BZ#1918408 - mistral_executor container fails to properly set no_proxy environment parameter.
dnf proxy configuration
Use this method to configure dnf to run all traffic through a proxy. To configure the proxy settings, edit the /etc/dnf/dnf.conf file and set the following parameters:
- proxy
- The URL of the proxy server.
- proxy_username
- The username that you want to use to connect to the proxy server.
- proxy_password
- The password that you want to use to connect to the proxy server.
- proxy_auth_method
- The authentication method used by the proxy server.
For more information about these options, run man dnf.conf.
The dnf proxy method has the following limitations:
-
This method provides proxy support only for
dnf. -
The
dnfproxy method does not include an option to exclude certain hosts from proxy communication.
Red Hat Subscription Manager proxy
Use this method to configure Red Hat Subscription Manager to run all traffic through a proxy. To configure the proxy settings, edit the /etc/rhsm/rhsm.conf file and set the following parameters:
- proxy_hostname
- Host for the proxy.
- proxy_scheme
- The scheme for the proxy when writing out the proxy to repo definitions.
- proxy_port
- The port for the proxy.
- proxy_username
- The username that you want to use to connect to the proxy server.
- proxy_password
- The password to use for connecting to the proxy server.
- no_proxy
- A comma-separated list of hostname suffixes for specific hosts that you want to exclude from proxy communication.
For more information about these options, run man rhsm.conf.
The Red Hat Subscription Manager proxy method has the following limitations:
- This method provides proxy support only for Red Hat Subscription Manager.
- The values for the Red Hat Subscription Manager proxy configuration override any values set for the system-wide environment variables.
Transparent proxy
If your network uses a transparent proxy to manage application layer traffic, you do not need to configure the undercloud itself to interact with the proxy because proxy management occurs automatically. A transparent proxy can help overcome limitations associated with client-based proxy configuration in Red Hat OpenStack Platform.
2.8. Undercloud repositories Copy linkLink copied to clipboard!
Red Hat OpenStack Platform 16.0 runs on Red Hat Enterprise Linux 8.1. Before enabling repositories, lock the director to a version with the subscription-manager release command:
sudo subscription-manager release --set=8.1
$ sudo subscription-manager release --set=8.1
Enable the following repositories for the installation and configuration of the undercloud.
Core repositories
The following table lists core repositories for installing the undercloud.
| Name | Repository | Description of requirement |
|---|---|---|
| Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) Extended Update Support (EUS) |
| Base operating system repository for x86_64 systems. |
| Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) Extended Update Support (EUS) |
| Contains Red Hat OpenStack Platform dependencies. |
| Red Hat Enterprise Linux 8 for x86_64 - High Availability (RPMs) Extended Update Support (EUS) |
| High availability tools for Red Hat Enterprise Linux. Used for Controller node high availability. |
| Red Hat Ansible Engine 2.8 for RHEL 8 x86_64 (RPMs) |
| Ansible Engine for Red Hat Enterprise Linux. Used to provide the latest version of Ansible. |
| Red Hat Satellite Tools for RHEL 8 Server RPMs x86_64 |
| Tools for managing hosts with Red Hat Satellite 6. |
| Red Hat OpenStack Platform 16.0 for RHEL 8 (RPMs) |
| Core Red Hat OpenStack Platform repository, which contains packages for Red Hat OpenStack Platform director. |
| Red Hat Fast Datapath for RHEL 8 (RPMS) |
| Provides Open vSwitch (OVS) packages for OpenStack Platform. |
IBM POWER repositories
The following table contains a list of repositories for Red Hat Openstack Platform on POWER PC architecture. Use these repositories in place of equivalents in the Core repositories.
| Name | Repository | Description of requirement |
|---|---|---|
| Red Hat Enterprise Linux for IBM Power, little endian - BaseOS (RPMs) |
| Base operating system repository for ppc64le systems. |
| Red Hat Enterprise Linux 8 for IBM Power, little endian - AppStream (RPMs) |
| Contains Red Hat OpenStack Platform dependencies. |
| Red Hat Enterprise Linux 8 for IBM Power, little endian - High Availability (RPMs) |
| High availability tools for Red Hat Enterprise Linux. Used for Controller node high availability. |
| Red Hat Ansible Engine 2.8 for RHEL 8 IBM Power, little endian (RPMs) |
| Ansible Engine for Red Hat Enterprise Linux. Provides the latest version of Ansible. |
| Red Hat OpenStack Platform 16.0 for RHEL 8 (RPMs) |
| Core Red Hat OpenStack Platform repository for ppc64le systems. |
Chapter 3. Preparing for director installation Copy linkLink copied to clipboard!
3.1. Preparing the undercloud Copy linkLink copied to clipboard!
Before you can install director, you must complete some basic configuration on the host machine:
- A non-root user to execute commands.
- Directories to organize images and templates.
- A resolvable hostname.
- A Red Hat subscription.
- The command line tools for image preparation and director installation.
Procedure
-
Log in to your undercloud as the
rootuser. Create the
stackuser:useradd stack
[root@director ~]# useradd stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set a password for the user:
passwd stack
[root@director ~]# passwd stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disable password requirements when using
sudo:echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack chmod 0440 /etc/sudoers.d/stack
[root@director ~]# echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack [root@director ~]# chmod 0440 /etc/sudoers.d/stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Switch to the new
stackuser:su - stack
[root@director ~]# su - stack [stack@director ~]$Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create directories for system images and heat templates:
mkdir ~/images mkdir ~/templates
[stack@director ~]$ mkdir ~/images [stack@director ~]$ mkdir ~/templatesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Director uses system images and heat templates to create the overcloud environment. Red Hat recommends creating these directories to help you organize your local file system.
Check the base and full hostname of the undercloud:
hostname hostname -f
[stack@director ~]$ hostname [stack@director ~]$ hostname -fCopy to Clipboard Copied! Toggle word wrap Toggle overflow If either of the previous commands do not report the correct fully-qualified hostname or report an error, use
hostnamectlto set a hostname:sudo hostnamectl set-hostname manager.example.com sudo hostnamectl set-hostname --transient manager.example.com
[stack@director ~]$ sudo hostnamectl set-hostname manager.example.com [stack@director ~]$ sudo hostnamectl set-hostname --transient manager.example.comCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
/etc/hostsand include an entry for the system hostname. The IP address in/etc/hostsmust match the address that you plan to use for your undercloud public API. For example, if the system is namedmanager.example.comand uses10.0.0.1for its IP address, add the following line to the/etc/hostsfile:10.0.0.1 manager.example.com manager
10.0.0.1 manager.example.com managerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Register your system either with the Red Hat Content Delivery Network or with a Red Hat Satellite. For example, run the following command to register the system to the Content Delivery Network. Enter your Customer Portal user name and password when prompted:
sudo subscription-manager register
[stack@director ~]$ sudo subscription-manager registerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Find the entitlement pool ID for Red Hat OpenStack Platform (RHOSP) director:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Locate the
Pool IDvalue and attach the Red Hat OpenStack Platform 16.0 entitlement:sudo subscription-manager attach --pool=Valid-Pool-Number-123456
[stack@director ~]$ sudo subscription-manager attach --pool=Valid-Pool-Number-123456Copy to Clipboard Copied! Toggle word wrap Toggle overflow Lock the undercloud to Red Hat Enterprise Linux 8.1:
sudo subscription-manager release --set=8.1
$ sudo subscription-manager release --set=8.1Copy to Clipboard Copied! Toggle word wrap Toggle overflow Disable all default repositories, and then enable the required Red Hat Enterprise Linux repositories:
sudo subscription-manager repos --disable=* sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms
[stack@director ~]$ sudo subscription-manager repos --disable=* [stack@director ~]$ sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpmsCopy to Clipboard Copied! Toggle word wrap Toggle overflow These repositories contain packages that the director installation requires.
Perform an update on your system to ensure that you have the latest base system packages:
sudo dnf update -y sudo reboot
[stack@director ~]$ sudo dnf update -y [stack@director ~]$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the command line tools for director installation and configuration:
sudo dnf install -y python3-tripleoclient
[stack@director ~]$ sudo dnf install -y python3-tripleoclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2. Installing ceph-ansible Copy linkLink copied to clipboard!
The ceph-ansible package is required when you use Ceph Storage with Red Hat OpenStack Platform.
If you use Red Hat Ceph Storage, or if your deployment uses an external Ceph Storage cluster, install the ceph-ansible package. For more information about integrating with an existing Ceph Storage cluster, see Integrating an Overcloud with an Existing Red Hat Ceph Cluster.
Procedure
Enable the Ceph Tools repository:
sudo subscription-manager repos --enable=rhceph-4-tools-for-rhel-8-x86_64-rpms
[stack@director ~]$ sudo subscription-manager repos --enable=rhceph-4-tools-for-rhel-8-x86_64-rpmsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the
ceph-ansiblepackage:sudo dnf install -y ceph-ansible
[stack@director ~]$ sudo dnf install -y ceph-ansibleCopy to Clipboard Copied! Toggle word wrap Toggle overflow
3.3. Preparing container images Copy linkLink copied to clipboard!
The undercloud configuration requires initial registry configuration to determine where to obtain images and how to store them. Complete the following steps to generate and customize an environment file that you can use to prepare your container images.
Procedure
- Log in to your undercloud host as the stack user.
Generate the default container image preparation file:
openstack tripleo container image prepare default \ --local-push-destination \ --output-env-file containers-prepare-parameter.yaml
$ openstack tripleo container image prepare default \ --local-push-destination \ --output-env-file containers-prepare-parameter.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command includes the following additional options:
-
--local-push-destinationsets the registry on the undercloud as the location for container images. This means the director pulls the necessary images from the Red Hat Container Catalog and pushes them to the registry on the undercloud. The director uses this registry as the container image source. To pull directly from the Red Hat Container Catalog, omit this option. --output-env-fileis an environment file name. The contents of this file include the parameters for preparing your container images. In this case, the name of the file iscontainers-prepare-parameter.yaml.NoteYou can use the same
containers-prepare-parameter.yamlfile to define a container image source for both the undercloud and the overcloud.
-
-
Modify the
containers-prepare-parameter.yamlto suit your requirements.
3.4. Container image preparation parameters Copy linkLink copied to clipboard!
The default file for preparing your containers (containers-prepare-parameter.yaml) contains the ContainerImagePrepare heat parameter. This parameter defines a list of strategies for preparing a set of images:
Each strategy accepts a set of sub-parameters that defines which images to use and what to do with the images. The following table contains information about the sub-parameters you can use with each ContainerImagePrepare strategy:
| Parameter | Description |
|---|---|
|
| List of regular expressions to exclude image names from a strategy. |
|
|
List of regular expressions to include in a strategy. At least one image name must match an existing image. All |
|
|
String to append to the tag for the destination image. For example, if you pull an image with the tag |
|
| A dictionary of image labels that filter the images that you want to modify. If an image matches the labels defined, the director includes the image in the modification process. |
|
| String of ansible role names to run during upload but before pushing the image to the destination registry. |
|
|
Dictionary of variables to pass to |
|
| Defines the namespace of the registry that you want to push images to during the upload process.
If you choose to pull container images directly from the Red Hat Container Catalog, do not set this parameter to |
|
| The source registry from where to pull the original container images. |
|
|
A dictionary of |
|
|
Use the value of specified container image labels to discover and pull the versioned tag for every image. Director inspects each container image tagged with the value that you set for |
The set parameter accepts a set of key: value definitions:
| Key | Description |
|---|---|
|
| The name of the Ceph Storage container image. |
|
| The namespace of the Ceph Storage container image. |
|
| The tag of the Ceph Storage container image. |
|
| A prefix for each OpenStack service image. |
|
| A suffix for each OpenStack service image. |
|
| The namespace for each OpenStack service image. |
|
|
The driver to use to determine which OpenStack Networking (neutron) container to use. Use a null value to set to the standard |
|
|
Sets the specific tag for all images from the source. If you use this option without specifying a |
The Red Hat Container Registry uses a specific version format to tag all Red Hat OpenStack Platform container images. This version format is {version}-{release}, which each container image stores as labels in the container metadata. This version format helps facilitate updates from one {release} to the next. For this reason, you must always use the tag_from_label: {version}-{release} parameter with the ContainerImagePrepare heat parameter. Do not only use tag on its own to to pull container images. For example, using tag by itself causes problems when performing updates because director requires a change in tag to update a container image.
The container images use multi-stream tags based on Red Hat OpenStack Platform version. This means there is no longer a latest tag.
The ContainerImageRegistryCredentials parameter maps a container registry to a username and password to authenticate to that registry.
If a container registry requires a username and password, you can use ContainerImageRegistryCredentials to include credentials with the following syntax:
In the example, replace my_username and my_password with your authentication credentials. Instead of using your individual user credentials, Red Hat recommends creating a registry service account and using those credentials to access registry.redhat.io content. For more information, see "Red Hat Container Registry Authentication".
The ContainerImageRegistryLogin parameter is used to control the registry login on the systems being deployed. This must be set to true if push_destination is set to false or not used.
3.5. Layering image preparation entries Copy linkLink copied to clipboard!
The value of the ContainerImagePrepare parameter is a YAML list. This means that you can specify multiple entries. The following example demonstrates two entries where director uses the latest version of all images except for the nova-api image, which uses the version tagged with 16.0-44:
The includes and excludes parameters use regular expressions to control image filtering for each entry. The images that match the includes strategy take precedence over excludes matches. The image name must the includes or excludes regular expression value to be considered a match.
3.6. Excluding Ceph Storage container images Copy linkLink copied to clipboard!
The default overcloud role configuration uses the default Controller, Compute, and Ceph Storage roles. However, if you use the default role configuration to deploy an overcloud without Ceph Storage nodes, director still pulls the Ceph Storage container images from the Red Hat Container Registry because the images are included as a part of the default configuration.
If your overcloud does not require Ceph Storage containers, you can configure director to not pull the Ceph Storage containers images from the Red Hat Container Registry.
Procedure
Edit the
containers-prepare-parameter.yamlfile to exclude the Ceph Storage containers:Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
excludesparameter uses regular expressions to exclude any container images that contain thecephorprometheusstrings.-
Save the
containers-prepare-parameter.yamlfile.
3.7. Obtaining container images from private registries Copy linkLink copied to clipboard!
Some container image registries require authentication to access images. In this situation, use the ContainerImageRegistryCredentials parameter in your containers-prepare-parameter.yaml environment file.
Private registries require push_destination set to true for their respective strategy in the ContainerImagePrepare.
The ContainerImageRegistryCredentials parameter uses a set of keys based on the private registry URL. Each private registry URL uses its own key and value pair to define the username (key) and password (value). This provides a method to specify credentials for multiple private registries.
The default ContainerImagePrepare parameter pulls container images from registry.redhat.io, which requires authentication.
The ContainerImageRegistryLogin parameter is used to control whether the system needs to log in to the remote registry to fetch the containers.
parameter_defaults: ... ContainerImageRegistryLogin: true
parameter_defaults:
...
ContainerImageRegistryLogin: true
You must set this value to true if push_destination is not configured for a given strategy. If push_destination is configured in a ContainerImagePrepare strategy and the ContainerImageRegistryCredentials parameter is configured, the system logs in to fetch the containers and pushes them to the remote system.
3.8. Modifying images during preparation Copy linkLink copied to clipboard!
It is possible to modify images during image preparation, and then immediately deploy with modified images. Scenarios for modifying images include:
- As part of a continuous integration pipeline where images are modified with the changes being tested before deployment.
- As part of a development workflow where local changes must be deployed for testing and development.
- When changes must be deployed but are not available through an image build pipeline. For example, adding proprietary add-ons or emergency fixes.
To modify an image during preparation, invoke an Ansible role on each image that you want to modify. The role takes a source image, makes the requested changes, and tags the result. The prepare command can push the image to the destination registry and set the heat parameters to refer to the modified image.
The Ansible role tripleo-modify-image conforms with the required role interface and provides the behaviour necessary for the modify use cases. Control the modification with the modify-specific keys in the ContainerImagePrepare parameter:
-
modify_rolespecifies the Ansible role to invoke for each image to modify. -
modify_append_tagappends a string to the end of the source image tag. This makes it obvious that the resulting image has been modified. Use this parameter to skip modification if thepush_destinationregistry already contains the modified image. Changemodify_append_tagwhenever you modify the image. -
modify_varsis a dictionary of Ansible variables to pass to the role.
To select a use case that the tripleo-modify-image role handles, set the tasks_from variable to the required file in that role.
While developing and testing the ContainerImagePrepare entries that modify images, run the image prepare command without any additional options to confirm that the image is modified as you expect:
sudo openstack tripleo container image prepare \ -e ~/containers-prepare-parameter.yaml
sudo openstack tripleo container image prepare \
-e ~/containers-prepare-parameter.yaml
3.9. Updating existing packages on container images Copy linkLink copied to clipboard!
The following example ContainerImagePrepare entry updates all packages on the images using the dnf repository configuration on the undercloud host:
3.10. Installing additional RPM files to container images Copy linkLink copied to clipboard!
You can install a directory of RPM files in your container images. This is useful for installing hotfixes, local package builds, or any package that is not available through a package repository. For example, the following ContainerImagePrepare entry installs some hotfix packages only on the nova-compute image:
3.11. Modifying container images with a custom Dockerfile Copy linkLink copied to clipboard!
For maximum flexibility, you can specify a directory containing a Dockerfile to make the required changes. When you invoke the tripleo-modify-image role, the role generates a Dockerfile.modified file that changes the FROM directive and adds extra LABEL directives. The following example runs the custom Dockerfile on the nova-compute image:
The following example shows the /home/stack/nova-custom/Dockerfile file. After you run any USER root directives, you must switch back to the original image default user:
3.12. Preparing a Satellite server for container images Copy linkLink copied to clipboard!
Red Hat Satellite 6 offers registry synchronization capabilities that you can use to pull multiple images into a Satellite server and manage them as part of an application life cycle. The Satellite also acts as a registry for other container-enabled systems to use. For more information about managing container images, see "Managing Container Images" in the Red Hat Satellite 6 Content Management Guide.
The examples in this procedure use the hammer command line tool for Red Hat Satellite 6 and an example organization called ACME. Substitute this organization for your own Satellite 6 organization.
This procedure requires authentication credentials to access container images from registry.redhat.io. Instead of using your individual user credentials, Red Hat recommends creating a registry service account and using those credentials to access registry.redhat.io content. For more information, see "Red Hat Container Registry Authentication".
Procedure
Create a list of all container images:
sudo podman search --limit 1000 "registry.redhat.io/rhosp" | grep rhosp-rhel8 | awk '{ print $2 }' | grep -v beta | sed "s/registry.redhat.io\///g" | tail -n+2 > satellite_images$ sudo podman search --limit 1000 "registry.redhat.io/rhosp" | grep rhosp-rhel8 | awk '{ print $2 }' | grep -v beta | sed "s/registry.redhat.io\///g" | tail -n+2 > satellite_imagesCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Copy the
satellite_imagesfile to a system that contains the Satellite 6hammertool. Alternatively, use the instructions in the Hammer CLI Guide to install thehammertool to the undercloud. Run the following
hammercommand to create a new product (OSP16 Containers) in your Satellite organization:hammer product create \ --organization "ACME" \ --name "OSP16 Containers"
$ hammer product create \ --organization "ACME" \ --name "OSP16 Containers"Copy to Clipboard Copied! Toggle word wrap Toggle overflow This custom product will contain your images.
Add the base container image to the product:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add the overcloud container images from the
satellite_imagesfile:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add the Ceph Storage 4 container image:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Synchronize the container images:
hammer product synchronize \ --organization "ACME" \ --name "OSP16 Containers"
$ hammer product synchronize \ --organization "ACME" \ --name "OSP16 Containers"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Wait for the Satellite server to complete synchronization.
NoteDepending on your configuration,
hammermight prompt you for your Satellite server username and password. You can configurehammerto log in automatically using a configuration file. For more information, see the "Authentication" section in the Hammer CLI Guide.-
If your Satellite 6 server uses content views, create a new content view version to incorporate the images and promote it along environments in your application life cycle. This largely depends on how you structure your application lifecycle. For example, if you have an environment called
productionin your lifecycle and you want the container images to be available in that environment, create a content view that includes the container images and promote that content view to theproductionenvironment. For more information, see "Managing Content Views". Check the available tags for the
baseimage:hammer docker tag list --repository "base" \ --organization "ACME" \ --lifecycle-environment "production" \ --content-view "myosp16" \ --product "OSP16 Containers"
$ hammer docker tag list --repository "base" \ --organization "ACME" \ --lifecycle-environment "production" \ --content-view "myosp16" \ --product "OSP16 Containers"Copy to Clipboard Copied! Toggle word wrap Toggle overflow This command displays tags for the OpenStack Platform container images within a content view for a particular environment.
Return to the undercloud and generate a default environment file that prepares images using your Satellite server as a source. Run the following example command to generate the environment file:
openstack tripleo container image prepare default \ --output-env-file containers-prepare-parameter.yaml
$ openstack tripleo container image prepare default \ --output-env-file containers-prepare-parameter.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
--output-env-fileis an environment file name. The contents of this file include the parameters for preparing your container images for the undercloud. In this case, the name of the file iscontainers-prepare-parameter.yaml.
-
Edit the
containers-prepare-parameter.yamlfile and modify the following parameters:-
push_destination- Set this totrueorfalsedepending on your chosen container image management strategy. If you set this parameter tofalse, the overcloud nodes pull images directly from the Satellite. If you set this parameter totrue, the director pulls the images from the Satellite to the undercloud registry and the overcloud pulls the images from the undercloud registry. -
namespace- The URL and port of the registry on the Satellite server. The default registry port on Red Hat Satellite is 5000. name_prefix- The prefix is based on a Satellite 6 convention. This differs depending on whether you use content views:-
If you use content views, the structure is
[org]-[environment]-[content view]-[product]-. For example:acme-production-myosp16-osp16_containers-. -
If you do not use content views, the structure is
[org]-[product]-. For example:acme-osp16_containers-.
-
If you use content views, the structure is
-
ceph_namespace,ceph_image,ceph_tag- If you use Ceph Storage, include these additional parameters to define the Ceph Storage container image location. Note thatceph_imagenow includes a Satellite-specific prefix. This prefix is the same value as thename_prefixoption.
-
The following example environment file contains Satellite-specific parameters:
You must define the containers-prepare-parameter.yaml environment file in the undercloud.conf configuration file, otherwise the undercloud uses the default values:
container_images_file = /home/stack/containers-prepare-parameter.yaml
container_images_file = /home/stack/containers-prepare-parameter.yaml
Chapter 4. Installing director Copy linkLink copied to clipboard!
4.1. Configuring director Copy linkLink copied to clipboard!
The director installation process requires certain settings in the undercloud.conf configuration file, which director reads from the home directory of the stack user. Complete the following steps to copy default template as a foundation for your configuration.
Procedure
Copy the default template to the home directory of the
stackuser’s:cp \ /usr/share/python-tripleoclient/undercloud.conf.sample \ ~/undercloud.conf
[stack@director ~]$ cp \ /usr/share/python-tripleoclient/undercloud.conf.sample \ ~/undercloud.confCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Edit the
undercloud.conffile. This file contains settings to configure your undercloud. If you omit or comment out a parameter, the undercloud installation uses the default value.
4.2. Director configuration parameters Copy linkLink copied to clipboard!
The following list contains information about parameters for configuring the undercloud.conf file. Keep all parameters within their relevant sections to avoid errors.
Defaults
The following parameters are defined in the [DEFAULT] section of the undercloud.conf file:
- additional_architectures
A list of additional (kernel) architectures that an overcloud supports. Currently the overcloud supports
ppc64learchitecture.NoteWhen you enable support for ppc64le, you must also set
ipxe_enabledtoFalse- certificate_generation_ca
-
The
certmongernickname of the CA that signs the requested certificate. Use this option only if you have set thegenerate_service_certificateparameter. If you select thelocalCA, certmonger extracts the local CA certificate to/etc/pki/ca-trust/source/anchors/cm-local-ca.pemand adds the certificate to the trust chain. - clean_nodes
- Defines whether to wipe the hard drive between deployments and after introspection.
- cleanup
-
Cleanup temporary files. Set this to
Falseto leave the temporary files used during deployment in place after you run the deployment command. This is useful for debugging the generated files or if errors occur. - container_cli
-
The CLI tool for container management. Leave this parameter set to
podman. Red Hat Enterprise Linux 8.1 only supportspodman. - container_healthcheck_disabled
-
Disables containerized service health checks. Red Hat recommends that you enable health checks and leave this option set to
false. - container_images_file
Heat environment file with container image information. This file can contain the following entries:
- Parameters for all required container images
-
The
ContainerImagePrepareparameter to drive the required image preparation. Usually the file that contains this parameter is namedcontainers-prepare-parameter.yaml.
- container_insecure_registries
-
A list of insecure registries for
podmanto use. Use this parameter if you want to pull images from another source, such as a private container registry. In most cases,podmanhas the certificates to pull container images from either the Red Hat Container Catalog or from your Satellite server if the undercloud is registered to Satellite. - container_registry_mirror
-
An optional
registry-mirrorconfigured thatpodmanuses. - custom_env_files
- Additional environment files that you want to add to the undercloud installation.
- deployment_user
-
The user who installs the undercloud. Leave this parameter unset to use the current default user
stack. - discovery_default_driver
-
Sets the default driver for automatically enrolled nodes. Requires the
enable_node_discoveryparameter to be enabled and you must include the driver in theenabled_hardware_typeslist. - enable_ironic; enable_ironic_inspector; enable_mistral; enable_nova; enable_tempest; enable_validations; enable_zaqar
-
Defines the core services that you want to enable for director. Leave these parameters set to
true. - enable_node_discovery
-
Automatically enroll any unknown node that PXE-boots the introspection ramdisk. New nodes use the
fake_pxedriver as a default but you can setdiscovery_default_driverto override. You can also use introspection rules to specify driver information for newly enrolled nodes. - enable_novajoin
-
Defines whether to install the
novajoinmetadata service in the undercloud. - enable_routed_networks
- Defines whether to enable support for routed control plane networks.
- enable_swift_encryption
- Defines whether to enable Swift encryption at-rest.
- enable_telemetry
-
Defines whether to install OpenStack Telemetry services (gnocchi, aodh, panko) in the undercloud. Set the
enable_telemetryparameter totrueif you want to install and configure telemetry services automatically. The default value isfalse, which disables telemetry on the undercloud. This parameter is required if you use other products that consume metrics data, such as Red Hat CloudForms. - enabled_hardware_types
- A list of hardware types that you want to enable for the undercloud.
- generate_service_certificate
-
Defines whether to generate an SSL/TLS certificate during the undercloud installation, which is used for the
undercloud_service_certificateparameter. The undercloud installation saves the resulting certificate/etc/pki/tls/certs/undercloud-[undercloud_public_vip].pem. The CA defined in thecertificate_generation_caparameter signs this certificate. - heat_container_image
- URL for the heat container image to use. Leave unset.
- heat_native
-
Run host-based undercloud configuration using
heat-all. Leave astrue. - hieradata_override
-
Path to
hieradataoverride file that configures Puppet hieradata on the director, providing custom configuration to services beyond theundercloud.confparameters. If set, the undercloud installation copies this file to the/etc/puppet/hieradatadirectory and sets it as the first file in the hierarchy. For more information about using this feature, see Configuring hieradata on the undercloud. - inspection_extras
-
Defines whether to enable extra hardware collection during the inspection process. This parameter requires the
python-hardwareorpython-hardware-detectpackages on the introspection image. - inspection_interface
-
The bridge that director uses for node introspection. This is a custom bridge that the director configuration creates. The
LOCAL_INTERFACEattaches to this bridge. Leave this as the defaultbr-ctlplane. - inspection_runbench
-
Runs a set of benchmarks during node introspection. Set this parameter to
trueto enable the benchmarks. This option is necessary if you intend to perform benchmark analysis when inspecting the hardware of registered nodes. - ipa_otp
-
Defines the one-time password to register the undercloud node to an IPA server. This is required when
enable_novajoinis enabled. - ipv6_address_mode
IPv6 address configuration mode for the undercloud provisioning network. The following list contains the possible values for this parameter:
- dhcpv6-stateless - Address configuration using router advertisement (RA) and optional information using DHCPv6.
- dhcpv6-stateful - Address configuration and optional information using DHCPv6.
- ipxe_enabled
-
Defines whether to use iPXE or standard PXE. The default is
true, which enables iPXE. Set this parameter tofalseto use standard PXE. - local_interface
The chosen interface for the director Provisioning NIC. This is also the device that director uses for DHCP and PXE boot services. Change this value to your chosen device. To see which device is connected, use the
ip addrcommand. For example, this is the result of anip addrcommand:Copy to Clipboard Copied! Toggle word wrap Toggle overflow In this example, the External NIC uses
em0and the Provisioning NIC usesem1, which is currently not configured. In this case, set thelocal_interfacetoem1. The configuration script attaches this interface to a custom bridge defined with theinspection_interfaceparameter.- local_ip
-
The IP address defined for the director Provisioning NIC. This is also the IP address that director uses for DHCP and PXE boot services. Leave this value as the default
192.168.24.1/24unless you use a different subnet for the Provisioning network, for example, if this IP address conflicts with an existing IP address or subnet in your environment. - local_mtu
-
The maximum transmission unit (MTU) that you want to use for the
local_interface. Do not exceed 1500 for the undercloud. - local_subnet
-
The local subnet that you want to use for PXE boot and DHCP interfaces. The
local_ipaddress should reside in this subnet. The default isctlplane-subnet. - net_config_override
-
Path to network configuration override template. If you set this parameter, the undercloud uses a JSON format template to configure the networking with
os-net-configand ignores the network parameters set inundercloud.conf. Use this parameter when you want to configure bonding or add an option to the interface. See/usr/share/python-tripleoclient/undercloud.conf.samplefor an example. - networks_file
-
Networks file to override for
heat. - output_dir
- Directory to output state, processed heat templates, and Ansible deployment files.
- overcloud_domain_name
The DNS domain name that you want to use when you deploy the overcloud.
NoteWhen you configure the overcloud, you must set the
CloudDomainparameter to a matching value. Set this parameter in an environment file when you configure your overcloud.- roles_file
- The roles file that you want to use to override the default roles file for undercloud installation. It is highly recommended to leave this parameter unset so that the director installation uses the default roles file.
- scheduler_max_attempts
- The maximum number of times that the scheduler attempts to deploy an instance. This value must be greater or equal to the number of bare metal nodes that you expect to deploy at once to avoid potential race conditions when scheduling.
- service_principal
- The Kerberos principal for the service using the certificate. Use this parameter only if your CA requires a Kerberos principal, such as in FreeIPA.
- subnets
-
List of routed network subnets for provisioning and introspection. The default value includes only the
ctlplane-subnetsubnet. For more information, see Subnets. - templates
- Heat templates file to override.
- undercloud_admin_host
-
The IP address or hostname defined for director Admin API endpoints over SSL/TLS. The director configuration attaches the IP address to the director software bridge as a routed IP address, which uses the
/32netmask. - undercloud_debug
-
Sets the log level of undercloud services to
DEBUG. Set this value totrueto enableDEBUGlog level. - undercloud_enable_selinux
-
Enable or disable SELinux during the deployment. It is highly recommended to leave this value set to
trueunless you are debugging an issue. - undercloud_hostname
- Defines the fully qualified host name for the undercloud. If set, the undercloud installation configures all system host name settings. If left unset, the undercloud uses the current host name, but you must configure all system host name settings appropriately.
- undercloud_log_file
-
The path to a log file to store the undercloud install and upgrade logs. By default, the log file is
install-undercloud.login the home directory. For example,/home/stack/install-undercloud.log. - undercloud_nameservers
- A list of DNS nameservers to use for the undercloud hostname resolution.
- undercloud_ntp_servers
- A list of network time protocol servers to help synchronize the undercloud date and time.
- undercloud_public_host
-
The IP address or hostname defined for director Public API endpoints over SSL/TLS. The director configuration attaches the IP address to the director software bridge as a routed IP address, which uses the
/32netmask. - undercloud_service_certificate
- The location and filename of the certificate for OpenStack SSL/TLS communication. Ideally, you obtain this certificate from a trusted certificate authority. Otherwise, generate your own self-signed certificate.
- undercloud_timezone
- Host timezone for the undercloud. If you do not specify a timezone, director uses the existing timezone configuration.
- undercloud_update_packages
- Defines whether to update packages during the undercloud installation.
Subnets
Each provisioning subnet is a named section in the undercloud.conf file. For example, to create a subnet called ctlplane-subnet, use the following sample in your undercloud.conf file:
You can specify as many provisioning networks as necessary to suit your environment.
- cidr
-
The network that director uses to manage overcloud instances. This is the Provisioning network, which the undercloud
neutronservice manages. Leave this as the default192.168.24.0/24unless you use a different subnet for the Provisioning network. - masquerade
Defines whether to masquerade the network defined in the
cidrfor external access. This provides the Provisioning network with a degree of network address translation (NAT) so that the Provisioning network has external access through director.NoteThe director configuration also enables IP forwarding automatically using the relevant
sysctlkernel parameter.- dhcp_start; dhcp_end
- The start and end of the DHCP allocation range for overcloud nodes. Ensure that this range contains enough IP addresses to allocate your nodes.
- dhcp_exclude
- IP addresses to exclude in the DHCP allocation range.
- dns_nameservers
-
DNS nameservers specific to the subnet. If no nameservers are defined for the subnet, the subnet uses nameservers defined in the
undercloud_nameserversparameter. - gateway
-
The gateway for the overcloud instances. This is the undercloud host, which forwards traffic to the External network. Leave this as the default
192.168.24.1unless you use a different IP address for director or want to use an external gateway directly. - host_routes
-
Host routes for the Neutron-managed subnet for the overcloud instances on this network. This also configures the host routes for the
local_subneton the undercloud. - inspection_iprange
-
Temporary IP range for nodes on this network to use during the inspection process. This range must not overlap with the range defined by
dhcp_startanddhcp_endbut must be in the same IP subnet.
Modify the values of these parameters to suit your configuration. When complete, save the file.
4.3. Configuring the undercloud with environment files Copy linkLink copied to clipboard!
You configure the main parameters for the undercloud through the undercloud.conf file. You can also perform additional undercloud configuration with an environment file that contains heat parameters.
Procedure
-
Create an environment file named
/home/stack/templates/custom-undercloud-params.yaml. Edit this file and include your heat parameters. For example, to enable debugging for certain OpenStack Platform services include the following snippet in the
custom-undercloud-params.yamlfile:parameter_defaults: Debug: True
parameter_defaults: Debug: TrueCopy to Clipboard Copied! Toggle word wrap Toggle overflow Save this file when you have finished.
Edit your
undercloud.conffile and scroll to thecustom_env_filesparameter. Edit the parameter to point to yourcustom-undercloud-params.yamlenvironment file:custom_env_files = /home/stack/templates/custom-undercloud-params.yaml
custom_env_files = /home/stack/templates/custom-undercloud-params.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteYou can specify multiple environment files using a comma-separated list.
The director installation includes this environment file during the next undercloud installation or upgrade operation.
4.4. Common heat parameters for undercloud configuration Copy linkLink copied to clipboard!
The following table contains some common heat parameters that you might set in a custom environment file for your undercloud.
| Parameter | Description |
|---|---|
|
|
Sets the undercloud |
|
|
Sets the undercloud |
|
| Enables debug mode. |
Set these parameters in your custom environment file under the parameter_defaults section:
parameter_defaults: Debug: True AdminPassword: "myp@ssw0rd!" AdminEmail: "admin@example.com"
parameter_defaults:
Debug: True
AdminPassword: "myp@ssw0rd!"
AdminEmail: "admin@example.com"
4.5. Configuring hieradata on the undercloud Copy linkLink copied to clipboard!
You can provide custom configuration for services beyond the available undercloud.conf parameters by configuring Puppet hieradata on the director.
Procedure
-
Create a hieradata override file, for example,
/home/stack/hieradata.yaml. Add the customized hieradata to the file. For example, add the following snippet to modify the Compute (nova) service parameter
force_raw_imagesfrom the default value ofTruetoFalse:nova::compute::force_raw_images: False
nova::compute::force_raw_images: FalseCopy to Clipboard Copied! Toggle word wrap Toggle overflow If there is no Puppet implementation for the parameter you want to set, then use the following method to configure the parameter:
nova::config::nova_config: DEFAULT/<parameter_name>: value: <parameter_value>nova::config::nova_config: DEFAULT/<parameter_name>: value: <parameter_value>Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:
nova::config::nova_config: DEFAULT/network_allocate_retries: value: 20 ironic/serial_console_state_timeout: value: 15nova::config::nova_config: DEFAULT/network_allocate_retries: value: 20 ironic/serial_console_state_timeout: value: 15Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
hieradata_overrideparameter in theundercloud.conffile to the path of the new/home/stack/hieradata.yamlfile:hieradata_override = /home/stack/hieradata.yaml
hieradata_override = /home/stack/hieradata.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
4.6. Configuring the undercloud for bare metal provisioning over IPv6 Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
If you have IPv6 nodes and infrastructure, you can configure the undercloud and the provisioning network to use IPv6 instead of IPv4 so that director can provision and deploy Red Hat OpenStack Platform onto IPv6 nodes. However, there are some considerations:
- Stateful DHCPv6 is available only with a limited set of UEFI firmware. For more information, see Bugzilla #1575026.
- Dual stack IPv4/6 is not available.
- Tempest validations might not perform correctly.
- IPv4 to IPv6 migration is not available during upgrades.
Modify the undercloud.conf file to enable IPv6 provisioning in Red Hat OpenStack Platform.
Prerequisites
- An IPv6 address on the undercloud. For more information, see Configuring an IPv6 address on the undercloud in the IPv6 Networking for the Overcloud guide.
Procedure
-
Copy the sample
undercloud.conffile, or modify your existingundercloud.conffile. Set the following parameter values in the
undercloud.conffile:-
Set
ipv6_address_modetodhcpv6-statelessordhcpv6-statefulif your NIC supports stateful DHCPv6 with Red Hat OpenStack Platform. For more information about stateful DHCPv6 availability, see Bugzilla #1575026. -
Set
enable_routed_networkstotrueif you do not want the undercloud to create a router on the provisioning network. In this case, the data center router must provide router advertisements. Otherwise, set this value tofalse. -
Set
local_ipto the IPv6 address of the undercloud. -
Use IPv6 addressing for the undercloud interface parameters
undercloud_public_hostandundercloud_admin_host. In the
[ctlplane-subnet]section, use IPv6 addressing in the following parameters:-
cidr -
dhcp_start -
dhcp_end -
gateway -
inspection_iprange
-
In the
[ctlplane-subnet]section, set an IPv6 nameserver for the subnet in thedns_nameserversparameter.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
-
Set
4.7. Installing director Copy linkLink copied to clipboard!
Complete the following steps to install director and perform some basic post-installation tasks.
Procedure
Run the following command to install director on the undercloud:
openstack undercloud install
[stack@director ~]$ openstack undercloud installCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command launches the director configuration script. Director installs additional packages and configures its services according to the configuration in the
undercloud.conf. This script takes several minutes to complete.The script generates two files:
-
undercloud-passwords.conf- A list of all passwords for the director services. -
stackrc- A set of initialization variables to help you access the director command line tools.
-
The script also starts all OpenStack Platform service containers automatically. You can check the enabled containers with the following command:
sudo podman ps
[stack@director ~]$ sudo podman psCopy to Clipboard Copied! Toggle word wrap Toggle overflow To initialize the
stackuser to use the command line tools, run the following command:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow The prompt now indicates that OpenStack commands authenticate and execute against the undercloud;
(undercloud) [stack@director ~]$Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The director installation is complete. You can now use the director command line tools.
4.8. Obtaining images for overcloud nodes Copy linkLink copied to clipboard!
Director requires several disk images to provision overcloud nodes:
- An introspection kernel and ramdisk for bare metal system introspection over PXE boot.
- A deployment kernel and ramdisk for system provisioning and deployment.
- An overcloud kernel, ramdisk, and full image. which form a base overcloud system that is written to the hard disk of the node.
The following procedure shows how to obtain and install these images.
4.8.1. Single CPU architecture overclouds Copy linkLink copied to clipboard!
These images and procedures are necessary for deployment of the overcloud with the default CPU architecture, x86-64.
Procedure
Source the
stackrcfile to enable the director command line tools:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the
rhosp-director-imagesandrhosp-director-images-ipapackages:sudo dnf install rhosp-director-images rhosp-director-images-ipa
(undercloud) [stack@director ~]$ sudo dnf install rhosp-director-images rhosp-director-images-ipaCopy to Clipboard Copied! Toggle word wrap Toggle overflow Extract the images archives to the
imagesdirectory in the home directory of thestackuser (/home/stack/images):cd ~/images for i in /usr/share/rhosp-director-images/overcloud-full-latest-16.0.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-16.0.tar; do tar -xvf $i; done
(undercloud) [stack@director ~]$ cd ~/images (undercloud) [stack@director images]$ for i in /usr/share/rhosp-director-images/overcloud-full-latest-16.0.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-16.0.tar; do tar -xvf $i; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow Import these images into director:
openstack overcloud image upload --image-path /home/stack/images/
(undercloud) [stack@director images]$ openstack overcloud image upload --image-path /home/stack/images/Copy to Clipboard Copied! Toggle word wrap Toggle overflow This script uploads the following images into director:
-
overcloud-full -
overcloud-full-initrd -
overcloud-full-vmlinuz
The script also installs the introspection images on the director PXE server.
-
Verify that the images uploaded successfully:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This list does not show the introspection PXE images. Director copies these files to
/var/lib/ironic/httpboot.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.8.2. Multiple CPU architecture overclouds Copy linkLink copied to clipboard!
These are the images and procedures that are necessary to deploy the overcloud to enable support of additional CPU architectures.
The following example procedure uses the ppc64le image.
Procedure
Source the
stackrcfile to enable the director command line tools:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the
rhosp-director-images-allpackage:sudo dnf install rhosp-director-images-all
(undercloud) [stack@director ~]$ sudo dnf install rhosp-director-images-allCopy to Clipboard Copied! Toggle word wrap Toggle overflow Extract the archives to an architecture specific directory in the
imagesdirectory in the home directory of thestackuser (/home/stack/images):cd ~/images for arch in x86_64 ppc64le ; do mkdir $arch ; done for arch in x86_64 ppc64le ; do for i in /usr/share/rhosp-director-images/overcloud-full-latest-16.0-${arch}.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-16.0-${arch}.tar ; do tar -C $arch -xf $i ; done ; done(undercloud) [stack@director ~]$ cd ~/images (undercloud) [stack@director images]$ for arch in x86_64 ppc64le ; do mkdir $arch ; done (undercloud) [stack@director images]$ for arch in x86_64 ppc64le ; do for i in /usr/share/rhosp-director-images/overcloud-full-latest-16.0-${arch}.tar /usr/share/rhosp-director-images/ironic-python-agent-latest-16.0-${arch}.tar ; do tar -C $arch -xf $i ; done ; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow Import these images into director:
cd ~/images openstack overcloud image upload --image-path ~/images/ppc64le --architecture ppc64le --whole-disk --http-boot /var/lib/ironic/tftpboot/ppc64le openstack overcloud image upload --image-path ~/images/x86_64/ --http-boot /var/lib/ironic/tftpboot
(undercloud) [stack@director ~]$ cd ~/images (undercloud) [stack@director images]$ openstack overcloud image upload --image-path ~/images/ppc64le --architecture ppc64le --whole-disk --http-boot /var/lib/ironic/tftpboot/ppc64le (undercloud) [stack@director images]$ openstack overcloud image upload --image-path ~/images/x86_64/ --http-boot /var/lib/ironic/tftpbootCopy to Clipboard Copied! Toggle word wrap Toggle overflow These commands upload the following images into director:
-
overcloud-full -
overcloud-full-initrd -
overcloud-full-vmlinuz -
ppc64le-bm-deploy-kernel -
ppc64le-bm-deploy-ramdisk ppc64le-overcloud-fullThe script also installs the introspection images on the director PXE server.
-
Verify that the images uploaded successfully:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This list does not show the introspection PXE images. Director copies these files to
/tftpboot.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
4.8.3. Minimal overcloud image Copy linkLink copied to clipboard!
You can use the overcloud-minimal image to provision a bare OS where you do not want to run any other Red Hat OpenStack Platform services or consume one of your subscription entitlements.
Procedure
Source the
stackrcfile to enable the director command line tools:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the
overcloud-minimalpackage:sudo dnf install rhosp-director-images-minimal
(undercloud) [stack@director ~]$ sudo dnf install rhosp-director-images-minimalCopy to Clipboard Copied! Toggle word wrap Toggle overflow Extract the images archives to the
imagesdirectory in the home directory of thestackuser (/home/stack/images):cd ~/images tar xf /usr/share/rhosp-director-images/overcloud-minimal-latest-16.0.tar
(undercloud) [stack@director ~]$ cd ~/images (undercloud) [stack@director images]$ tar xf /usr/share/rhosp-director-images/overcloud-minimal-latest-16.0.tarCopy to Clipboard Copied! Toggle word wrap Toggle overflow Import the images into director:
openstack overcloud image upload --image-path /home/stack/images/ --os-image-name overcloud-minimal.qcow2
(undercloud) [stack@director images]$ openstack overcloud image upload --image-path /home/stack/images/ --os-image-name overcloud-minimal.qcow2Copy to Clipboard Copied! Toggle word wrap Toggle overflow This script uploads the following images into director:
-
overcloud-minimal -
overcloud-minimal-initrd -
overcloud-minimal-vmlinuz
-
Verify that the images uploaded successfully:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The default overcloud-full.qcow2 image is a flat partition image. However, you can also import and use whole disk images. For more information, see Chapter 22, Creating whole disk images.
4.9. Setting a nameserver for the control plane Copy linkLink copied to clipboard!
If you intend for the overcloud to resolve external hostnames, such as cdn.redhat.com, set a nameserver on the overcloud nodes. For a standard overcloud without network isolation, the nameserver is defined using the undercloud control plane subnet. Complete the following procedure to define nameservers for the environment.
Procedure
Source the
stackrcfile to enable the director command line tools:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the nameservers for the
ctlplane-subnetsubnet:openstack subnet set --dns-nameserver [nameserver1-ip] --dns-nameserver [nameserver2-ip] ctlplane-subnet
(undercloud) [stack@director images]$ openstack subnet set --dns-nameserver [nameserver1-ip] --dns-nameserver [nameserver2-ip] ctlplane-subnetCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
--dns-nameserveroption for each nameserver.View the subnet to verify the nameserver:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
If you aim to isolate service traffic onto separate networks, the overcloud nodes use the DnsServers parameter in your network environment files.
4.10. Updating the undercloud configuration Copy linkLink copied to clipboard!
If you need to change the undercloud configuration to suit new requirements, you can make changes to your undercloud configuration after installation, edit the relevant configuration files and re-run the openstack undercloud install command.
Procedure
Modify the undercloud configuration files. For example, edit the
undercloud.conffile and add theidrachardware type to the list of enabled hardware types:enabled_hardware_types = ipmi,redfish,idrac
enabled_hardware_types = ipmi,redfish,idracCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack undercloud installcommand to refresh your undercloud with the new changes:openstack undercloud install
[stack@director ~]$ openstack undercloud installCopy to Clipboard Copied! Toggle word wrap Toggle overflow Wait until the command runs to completion.
Initialize the
stackuser to use the command line tools,:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow The prompt now indicates that OpenStack commands authenticate and execute against the undercloud:
(undercloud) [stack@director ~]$Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that director has applied the new configuration. For this example, check the list of enabled hardware types:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The undercloud re-configuration is complete.
4.11. Undercloud container registry Copy linkLink copied to clipboard!
Red Hat Enterprise Linux 8.1 no longer includes the docker-distribution package, which installed a Docker Registry v2. To maintain the compatibility and the same level of feature, the director installation creates an Apache web server with a vhost called image-serve to provide a registry. This registry also uses port 8787/TCP with SSL disabled. The Apache-based registry is not containerized, which means that you must run the following command to restart the registry:
sudo systemctl restart httpd
$ sudo systemctl restart httpd
You can find the container registry logs in the following locations:
- /var/log/httpd/image_serve_access.log
- /var/log/httpd/image_serve_error.log.
The image content is served from /var/lib/image-serve. This location uses a specific directory layout and apache configuration to implement the pull function of the registry REST API.
The Apache-based registry does not support podman push nor buildah push commands, which means that you cannot push container images using traditional methods. To modify images during deployment, use the container preparation workflow, such as the ContainerImagePrepare parameter. To manage container images, use the container management commands:
- sudo openstack tripleo container image list
- Lists all images stored on the registry.
- sudo openstack tripleo container image show
- Show metadata for a specific image on the registry.
- sudo openstack tripleo container image push
- Push an image from a remote registry to the undercloud registry.
- sudo openstack tripleo container image delete
- Delete an image from the registry.
You must run all container image management commands with sudo level permissions.
4.12. Next steps Copy linkLink copied to clipboard!
- Install an undercloud minion to scale undercloud services. See Chapter 5, Installing undercloud minions.
- Perform basic overcloud configuration, including registering nodes, inspecting them, and then tagging them into various node roles. For more information, see Chapter 7, Configuring a basic overcloud with CLI tools.
Chapter 5. Installing undercloud minions Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
5.1. Undercloud minion Copy linkLink copied to clipboard!
An undercloud minion provides additional heat-engine and ironic-conductor services on a separate host. These additional services support the undercloud with orchestration and provisioning operations. The distribution of undercloud operations across multiple hosts provides more resources to run an overcloud deployment, which can result in potentially faster and larger deployments.
5.2. Undercloud minion requirements Copy linkLink copied to clipboard!
The scaled heat-engine and ironic-conductor services on an undercloud minion use a set of workers. Each worker performs operations specific to that service. Multiple workers provide simultaneous operations. The default number of workers on the minion is determined by halving the total CPU thread count of the minion host. In this instance, total thread count is the number of CPU cores multiplied by the hyper-threading value. For example, if your minion has a CPU with 16 threads, then the minion spawns 8 workers for each service by default. The minion also uses a set of minimum and maximum caps by default:
| Service | Minimum | Maximum |
|---|---|---|
|
| 4 | 24 |
|
| 2 | 12 |
An undercloud minion has the following minimum CPU and memory requirements:
- An 8-thread 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions. This processor provides 4 workers for each undercloud service.
- A minimum of 16 GB of RAM.
To use a larger number of workers, increase the vCPUs and memory count on the undercloud using a ratio of 2 GB of RAM for each CPU thread. For example, a machine with 48 threads must have 96 GB of RAM. This provides coverage for 24 heat-engine workers and 12 ironic-conductor workers.
5.3. Preparing a minion Copy linkLink copied to clipboard!
Before you can install a minion, you must complete some basic configuration on the host machine:
- A non-root user to execute commands.
- A resolvable hostname
- A Red Hat subscription
- The command line tools for image preparation and minion installation
Procedure
-
Log in to the minion host as the
rootuser. Create the
stackuser:useradd stack
[root@minion ~]# useradd stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set a password for the
stackuser:passwd stack
[root@minion ~]# passwd stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disable password requirements when using
sudo:echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack chmod 0440 /etc/sudoers.d/stack
[root@minion ~]# echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack [root@minion ~]# chmod 0440 /etc/sudoers.d/stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow Switch to the new
stackuser:su - stack
[root@minion ~]# su - stack [stack@minion ~]$Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the base and full hostname of the minion:
hostname hostname -f
[stack@minion ~]$ hostname [stack@minion ~]$ hostname -fCopy to Clipboard Copied! Toggle word wrap Toggle overflow If either of the previous commands do not report the correct fully-qualified hostname or report an error, use
hostnamectlto set a hostname:sudo hostnamectl set-hostname minion.example.com sudo hostnamectl set-hostname --transient minion.example.com
[stack@minion ~]$ sudo hostnamectl set-hostname minion.example.com [stack@minion ~]$ sudo hostnamectl set-hostname --transient minion.example.comCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
/etc/hostsfile and include an entry for the system hostname. For example, if the system is namedminion.example.comand uses the IP address10.0.0.1, add the following line to the/etc/hostsfile:10.0.0.1 minion.example.com manager
10.0.0.1 minion.example.com managerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Register your system either with the Red Hat Content Delivery Network or Red Hat Satellite. For example, run the following command to register the system to the Content Delivery Network. Enter your Customer Portal user name and password when prompted:
sudo subscription-manager register
[stack@minion ~]$ sudo subscription-manager registerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Find the entitlement pool ID for Red Hat OpenStack Platform (RHOSP) director:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Locate the
Pool IDvalue and attach the Red Hat OpenStack Platform 16.0 entitlement:sudo subscription-manager attach --pool=Valid-Pool-Number-123456
[stack@minion ~]$ sudo subscription-manager attach --pool=Valid-Pool-Number-123456Copy to Clipboard Copied! Toggle word wrap Toggle overflow Disable all default repositories, and then enable the required Red Hat Enterprise Linux repositories:
sudo subscription-manager repos --disable=* sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms
[stack@minion ~]$ sudo subscription-manager repos --disable=* [stack@minion ~]$ sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpmsCopy to Clipboard Copied! Toggle word wrap Toggle overflow These repositories contain packages that the minion installation requires.
Perform an update on your system to ensure that you have the latest base system packages:
sudo dnf update -y sudo reboot
[stack@minion ~]$ sudo dnf update -y [stack@minion ~]$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow Install the command line tools for minion installation and configuration:
sudo dnf install -y python3-tripleoclient
[stack@minion ~]$ sudo dnf install -y python3-tripleoclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.4. Copying the undercloud configuration files to the minion Copy linkLink copied to clipboard!
The minion requires some configuration files from the undercloud so that the minion installation can configure the minion services and register them with director:
-
tripleo-undercloud-outputs.yaml -
tripleo-undercloud-passwords.yaml
Procedure
-
Log in to your undercloud as the
stackuser. Copy the files from the undercloud to the minion:
scp ~/tripleo-undercloud-outputs.yaml ~/tripleo-undercloud-passwords.yaml stack@<minion-host>:~/.
$ scp ~/tripleo-undercloud-outputs.yaml ~/tripleo-undercloud-passwords.yaml stack@<minion-host>:~/.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<minion-host>with the hostname or IP address of the minion.
5.5. Copying the undercloud certificate authority Copy linkLink copied to clipboard!
If the undercloud uses SSL/TLS for endpoint encryption, the minion host must contain the certificate authority that signed the undercloud SSL/TLS certificates. Depending on your undercloud configuration, this certificate authority is one of the following:
- An external certificate authority whose certificate is preloaded on the minion host. No action is required.
-
A director-generated self-signed certificate authority, which the director creates at
/etc/pki/ca-trust/source/anchors/cm-local-ca.pem. Copy this file to the minion host and include the file as a part of the trusted certificate authorities for the minion host. This procedure uses this file as an example. -
A custom self-signed certificate authority, which you create with OpenSSL. Examples in this document refer to this file as
ca.crt.pem. Copy this file to the minion host and include the file as a part of the trusted certificate authorities for the minion host.
Procedure
-
Log in to the minion host as the
rootuser. Copy the certificate authority file from the undercloud to the minion:
scp \ root@<undercloud-host>:/etc/pki/ca-trust/source/anchors/cm-local-ca.pem \ /etc/pki/ca-trust/source/anchors/undercloud-ca.pem[root@minion ~]# scp \ root@<undercloud-host>:/etc/pki/ca-trust/source/anchors/cm-local-ca.pem \ /etc/pki/ca-trust/source/anchors/undercloud-ca.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<undercloud-host>with the hostname or IP address of the undercloud.Update the trusted certificate authorities for the minion host:
update-ca-trust enable update-ca-trust extract
[root@minion ~]# update-ca-trust enable [root@minion ~]# update-ca-trust extractCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.6. Configuring the minion Copy linkLink copied to clipboard!
The minion installation process requires certain settings in the minion.conf configuration file, which the minion reads from the home directory of the stack user. Complete the following steps to use the default template as a foundation for your configuration.
Procedure
-
Log in to the minion host as the
stackuser. Copy the default template to the home directory of the
stackuser:cp \ /usr/share/python-tripleoclient/minion.conf.sample \ ~/minion.conf
[stack@minion ~]$ cp \ /usr/share/python-tripleoclient/minion.conf.sample \ ~/minion.confCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
minion.conffile. This file contains settings to configure your minion. If you omit or comment out a parameter, the minion installation uses the default value. Review the following recommended parameters:-
minion_hostname, which you set to the hostname of the minion. -
minion_local_interface, which you set to the interface that connects to the undercloud through the Provisioning Network. -
minion_local_ip, which you set to a free IP address on the Provisioning Network. -
minion_nameservers, which you set to the DNS nameservers so that the minion can resolve hostnames. -
enable_ironic_conductor, which defines whether to enable theironic-conductorservice. -
enable_heat_engine, which defines whether to enable theheat-engineservice.
-
The default minion.conf file enables only the heat-engine service on the minion. To enable the ironic-conductor service, set the enable_ironic_conductor parameter to true.
5.7. Minion configuration parameters Copy linkLink copied to clipboard!
The following list contains information about parameters for configuring the minion.conf file. Keep all parameters within their relevant sections to avoid errors.
Defaults
The following parameters are defined in the [DEFAULT] section of the minion.conf file:
- cleanup
-
Cleanup temporary files. Set this parmaeter to
Falseto leave the temporary files used during deployment in place after the command is run. This is useful for debugging the generated files or if errors occur. - container_cli
-
The CLI tool for container management. Leave this parameter set to
podman. Red Hat Enterprise Linux 8.1 only supportspodman. - container_healthcheck_disabled
-
Disables containerized service health checks. Red Hat recommends that you enable health checks and leave this option set to
false. - container_images_file
Heat environment file with container image information. This file can contain the following entries:
- Parameters for all required container images
-
The
ContainerImagePrepareparameter to drive the required image preparation. Usually the file that contains this parameter is namedcontainers-prepare-parameter.yaml.
- container_insecure_registries
-
A list of insecure registries for
podmanto use. Use this parameter if you want to pull images from another source, such as a private container registry. In most cases,podmanhas the certificates to pull container images from either the Red Hat Container Catalog or from your Satellite server if the minion is registered to Satellite. - container_registry_mirror
-
An optional
registry-mirrorconfigured thatpodmanuses. - custom_env_files
- Additional environment file that you want to add to the minion installation.
- deployment_user
-
The user who installs the minion. Leave this parameter unset to use the current default user
stack. - enable_heat_engine
-
Defines whether to install the heat engine on the minion. The default is
true. - enable_ironic_conductor
-
Defines whether to install the ironic conductor service on the minion. The default value is
false. Set this value totrueto enable the ironic conductor service. - heat_container_image
- URL for the heat container image that you want to use. Leave unset.
- heat_native
-
Use native heat templates. Leave as
true. - hieradata_override
-
Path to
hieradataoverride file that configures Puppet hieradata on the director, providing custom configuration to services beyond theminion.confparameters. If set, the minion installation copies this file to the/etc/puppet/hieradatadirectory and sets it as the first file in the hierarchy. - minion_debug
-
Set this value to
trueto enable theDEBUGlog level for minion services. - minion_enable_selinux
-
Enable or disable SELinux during the deployment. It is highly recommended to leave this value set to
trueunless you are debugging an issue. - minion_enable_validations
- Enable validation services on the minion.
- minion_hostname
- Defines the fully qualified host name for the minion. If set, the minion installation configures all system host name settings. If left unset, the minion uses the current host name, but you must configure all system host name settings appropriately.
- minion_local_interface
The chosen interface for the Provisioning NIC on the undercloud. This is also the device that the minion uses for DHCP and PXE boot services. Change this value to your chosen device. To see which device is connected, use the
ip addrcommand. For example, this is the result of anip addrcommand:Copy to Clipboard Copied! Toggle word wrap Toggle overflow In this example, the External NIC uses
eth0and the Provisioning NIC useseth1, which is currently not configured. In this case, set thelocal_interfacetoeth1. The configuration script attaches this interface to a custom bridge defined with theinspection_interfaceparameter.- minion_local_ip
-
The IP address defined for the Provisioning NIC on the undercloud. This is also the IP address that the minion uses for DHCP and PXE boot services. Leave this value as the default
192.168.24.1/24unless you use a different subnet for the Provisioning network, for example, if the default IP address conflicts with an existing IP address or subnet in your environment. - minion_local_mtu
-
The maximum transmission unit (MTU) that you want to use for the
local_interface. Do not exceed 1500 for the minion. - minion_log_file
-
The path to a log file where you want to store the minion install and upgrade logs. By default, the log file is
install-minion.login the home directory. For example,/home/stack/install-minion.log. - minion_nameservers
- A list of DNS nameservers to use for the minion hostname resolution.
- minion_ntp_servers
- A list of network time protocol servers to help synchronize the minion date and time.
- minion_password_file
-
The file that contains the passwords for the minion to connect to undercloud services. Leave this parameter set to the
tripleo-undercloud-passwords.yamlfile copied from the undercloud. - minion_service_certificate
- The location and filename of the certificate for OpenStack SSL/TLS communication. Ideally, you obtain this certificate from a trusted certificate authority. Otherwise, generate your own self-signed certificate.
- minion_timezone
- Host timezone for the minion. If you do not specify a timezone, the minion uses the existing timezone configuration.
- minion_undercloud_output_file
-
The file that contains undercloud configuration information that the minion can use to connect to undercloud services. Leave this parameter set to the
tripleo-undercloud-outputs.yamlfile copied from the undercloud. - net_config_override
-
The path to a network configuration override template. If you set this parameter, the minion uses a JSON format template to configure the networking with
os-net-configand ignores the network parameters set inminion.conf. See/usr/share/python-tripleoclient/minion.conf.samplefor an example. - networks_file
-
Networks file to override for
heat. - output_dir
- Directory to output state, processed heat templates, and Ansible deployment files.
- roles_file
- The roles file that you want to use to override the default roles file for minion installation. It is highly recommended to leave this parameter unset so that the minion installation uses the default roles file.
- templates
- Heat templates file to override.
5.8. Installing the minion Copy linkLink copied to clipboard!
Complete the following steps to install the minion.
Procedure
-
Log in to the minion host as the
stackuser. Run the following command to install the minion:
openstack undercloud minion install
[stack@minion ~]$ openstack undercloud minion installCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command launches the configuration script for the minion, installs additional packages, and configures minion services according to the configuration in the
minion.conffile. This script takes several minutes to complete.
5.9. Verifying the minion installation Copy linkLink copied to clipboard!
Complete the following steps to confirm a successful minion installation.
Procedure
-
Log in to your undercloud as the
stackuser. Source the
stackrcfile:source ~/stackrc
[stack@director ~]$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow If you enabled the heat engine service on the minion, verify that the
heat-engineservice from the minion appears on the undercloud service list:$ openstack orchestration service list
[stack@director ~]$ $ openstack orchestration service listCopy to Clipboard Copied! Toggle word wrap Toggle overflow The command output displays a table with
heat-engineworkers for both the undercloud and any minions.If you enabled the ironic conductor service on the minion, verify that the
ironic-conductorservice from the minion appears on the undercloud service list:$ openstack baremetal conductor list
[stack@director ~]$ $ openstack baremetal conductor listCopy to Clipboard Copied! Toggle word wrap Toggle overflow The command output displays a table with
ironic-conductorservices for both the undercloud and any minions.
5.10. Next Steps Copy linkLink copied to clipboard!
- Perform basic overcloud configuration, including registering nodes, inspecting nodes, and tagging nodes into various node roles. For more information, see Chapter 7, Configuring a basic overcloud with CLI tools.
Part II. Basic overcloud deployment Copy linkLink copied to clipboard!
Chapter 6. Planning your overcloud Copy linkLink copied to clipboard!
The following section contains some guidelines for planning various aspects of your Red Hat OpenStack Platform (RHOSP) environment. This includes defining node roles, planning your network topology, and storage.
6.1. Node roles Copy linkLink copied to clipboard!
Director includes the following default node types to build your overcloud:
- Controller
Provides key services for controlling your environment. This includes the dashboard (horizon), authentication (keystone), image storage (glance), networking (neutron), orchestration (heat), and high availability services. A Red Hat OpenStack Platform (RHOSP) environment requires three Controller nodes for a highly available production-level environment.
NoteUse environments with one Controller node only for testing purposes, not for production. Environments with two Controller nodes or more than three Controller nodes are not supported.
- Compute
- A physical server that acts as a hypervisor and contains the processing capabilities required to run virtual machines in the environment. A basic RHOSP environment requires at least one Compute node.
- Ceph Storage
- A host that provides Red Hat Ceph Storage. Additional Ceph Storage hosts scale into a cluster. This deployment role is optional.
- Swift Storage
- A host that provides external object storage to the OpenStack Object Storage (swift) service. This deployment role is optional.
The following table contains some examples of different overclouds and defines the node types for each scenario.
| Controller | Compute | Ceph Storage | Swift Storage | Total | |
| Small overcloud | 3 | 1 | - | - | 4 |
| Medium overcloud | 3 | 3 | - | - | 6 |
| Medium overcloud with additional object storage | 3 | 3 | - | 3 | 9 |
| Medium overcloud with Ceph Storage cluster | 3 | 3 | 3 | - | 9 |
In addition, consider whether to split individual services into custom roles. For more information about the composable roles architecture, see "Composable Services and Custom Roles" in the Advanced Overcloud Customization guide.
6.2. Overcloud networks Copy linkLink copied to clipboard!
It is important to plan the networking topology and subnets in your environment so that you can map roles and services to communicate with each other correctly. Red Hat OpenStack Platform (RHOSP) uses the Openstack Networking (neutron) service, which operates autonomously and manages software-based networks, static and floating IP addresses, and DHCP.
By default, director configures nodes to use the Provisioning / Control Plane for connectivity. However, it is possible to isolate network traffic into a series of composable networks, that you can customize and assign services.
In a typical RHOSP installation, the number of network types often exceeds the number of physical network links. To connect all the networks to the proper hosts, the overcloud uses VLAN tagging to deliver more than one network on each interface. Most of the networks are isolated subnets but some networks require a Layer 3 gateway to provide routing for Internet access or infrastructure network connectivity. If you use VLANs to isolate your network traffic types, you must use a switch that supports 802.1Q standards to provide tagged VLANs.
It is recommended that you deploy a project network (tunneled with GRE or VXLAN) even if you intend to use a neutron VLAN mode with tunneling disabled at deployment time. This requires minor customization at deployment time and leaves the option available to use tunnel networks as utility networks or virtualization networks in the future. You still create Tenant networks using VLANs, but you can also create VXLAN tunnels for special-use networks without consuming tenant VLANs. It is possible to add VXLAN capability to a deployment with a Tenant VLAN, but it is not possible to add a Tenant VLAN to an existing overcloud without causing disruption.
Director also includes a set of templates that you can use to configure NICs with isolated composable networks. The following configurations are the default configurations:
- Single NIC configuration - One NIC for the Provisioning network on the native VLAN and tagged VLANs that use subnets for the different overcloud network types.
- Bonded NIC configuration - One NIC for the Provisioning network on the native VLAN and two NICs in a bond for tagged VLANs for the different overcloud network types.
- Multiple NIC configuration - Each NIC uses a subnet for a different overcloud network type.
You can also create your own templates to map a specific NIC configuration.
The following details are also important when you consider your network configuration:
- During the overcloud creation, you refer to NICs using a single name across all overcloud machines. Ideally, you should use the same NIC on each overcloud node for each respective network to avoid confusion. For example, use the primary NIC for the Provisioning network and the secondary NIC for the OpenStack services.
- Set all overcloud systems to PXE boot off the Provisioning NIC, and disable PXE boot on the External NIC and any other NICs on the system. Also ensure that the Provisioning NIC has PXE boot at the top of the boot order, ahead of hard disks and CD/DVD drives.
- All overcloud bare metal systems require a supported power management interface, such as an Intelligent Platform Management Interface (IPMI), so that director can control the power management of each node.
- Make a note of the following details for each overcloud system: the MAC address of the Provisioning NIC, the IP address of the IPMI NIC, IPMI username, and IPMI password. This information is useful later when you configure the overcloud nodes.
- If an instance must be accessible from the external internet, you can allocate a floating IP address from a public network and associate the floating IP with an instance. The instance retains its private IP but network traffic uses NAT to traverse through to the floating IP address. Note that a floating IP address can be assigned only to a single instance rather than multiple private IP addresses. However, the floating IP address is reserved for use only by a single tenant, which means that the tenant can associate or disassociate the floating IP address with a particular instance as required. This configuration exposes your infrastructure to the external internet and you must follow suitable security practices.
- To mitigate the risk of network loops in Open vSwitch, only a single interface or a single bond can be a member of a given bridge. If you require multiple bonds or interfaces, you can configure multiple bridges.
- Red Hat recommends using DNS hostname resolution so that your overcloud nodes can connect to external services, such as the Red Hat Content Delivery Network and network time servers.
You can virtualize the overcloud control plane if you are using Red Hat Virtualization (RHV). For more information, see Creating virtualized control planes.
6.3. Overcloud storage Copy linkLink copied to clipboard!
Using LVM on a guest instance that uses a back end cinder-volume of any driver or back-end type results in issues with performance, volume visibility and availability, and data corruption. Use an LVM filter to mitigate these issues. For more information, see section 2.1 Back Ends in the Storage Guide and KCS article 3213311, "Using LVM on a cinder volume exposes the data to the compute host."
Director includes different storage options for the overcloud environment:
- Ceph Storage nodes
Director creates a set of scalable storage nodes using Red Hat Ceph Storage. The overcloud uses these nodes for the following storage types:
- Images - The Image service (glance) manages images for virtual machines. Images are immutable. OpenStack treats images as binary blobs and downloads them accordingly. You can use the Image service (glance) to store images in a Ceph Block Device.
- Volumes - OpenStack manages volumes with the Block Storage service (cinder). The Block Storage service (cinder) volumes are block devices. OpenStack uses volumes to boot virtual machines, or to attach volumes to running virtual machines. You can use the Block Storage serivce to boot a virtual machine using a copy-on-write clone of an image.
- File Systems - Openstack manages shared file systems with the Shared File Systems service (manila). Shares are backed by file systems. You can use manila to manage shares backed by a CephFS file system with data on the Ceph Storage nodes.
Guest Disks - Guest disks are guest operating system disks. By default, when you boot a virtual machine with the Compute service (nova), the virtual machine disk appears as a file on the filesystem of the hypervisor (usually under
/var/lib/nova/instances/<uuid>/). Every virtual machine inside Ceph can be booted without using the Block Storage service (cinder). As a result, you can perform maintenance operations easily with the live-migration process. Additionally, if your hypervisor fails, it is also convenient to triggernova evacuateand run the virtual machine elsewhere.ImportantFor information about supported image formats, see the Image Service chapter in the Instances and Images Guide.
For more information about Ceph Storage, see the Red Hat Ceph Storage Architecture Guide.
- Swift Storage nodes
- Director creates an external object storage node. This is useful in situations where you need to scale or replace Controller nodes in your overcloud environment but need to retain object storage outside of a high availability cluster.
6.4. Overcloud security Copy linkLink copied to clipboard!
Your OpenStack Platform implementation is only as secure as your environment. Follow good security principles in your networking environment to ensure that you control network access properly:
- Use network segmentation to mitigate network movement and isolate sensitive data. A flat network is much less secure.
- Restrict services access and ports to a minimum.
- Enforce proper firewall rules and password usage.
- Ensure that SELinux is enabled.
For more information about securing your system, see the following Red Hat guides:
- Security Hardening for Red Hat Enterprise Linux 8
- Using SELinux for Red Hat Enterprise Linux 8
6.5. Overcloud high availability Copy linkLink copied to clipboard!
To deploy a highly-available overcloud, director configures multiple Controller, Compute and Storage nodes to work together as a single cluster. In case of node failure, an automated fencing and re-spawning process is triggered based on the type of node that failed. For more information about overcloud high availability architecture and services, see High Availability Deployment and Usage.
Deploying a highly available overcloud without STONITH is not supported. You must configure a STONITH device for each node that is a part of the Pacemaker cluster in a highly available overcloud. For more information on STONITH and Pacemaker, see Fencing in a Red Hat High Availability Cluster and Support Policies for RHEL High Availability Clusters.
You can also configure high availability for Compute instances with director (Instance HA). This high availability mechanism automates evacuation and re-spawning of instances on Compute nodes in case of node failure. The requirements for Instance HA are the same as the general overcloud requirements, but you must perform a few additional steps to prepare your environment for the deployment. For more information about Instance HA and installation instructions, see the High Availability for Compute Instances guide.
6.6. Controller node requirements Copy linkLink copied to clipboard!
Controller nodes host the core services in a Red Hat OpenStack Platform environment, such as the Dashboard (horizon), the back-end database server, the Identity service (keystone) authentication, and high availability services.
- Processor
- 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions.
- Memory
The minimum amount of memory is 32 GB. However, the amount of recommended memory depends on the number of vCPUs, which is based on the number of CPU cores multiplied by hyper-threading value. Use the following calculations to determine your RAM requirements:
Controller RAM minimum calculation:
- Use 1.5 GB of memory for each vCPU. For example, a machine with 48 vCPUs should have 72 GB of RAM.
Controller RAM recommended calculation:
- Use 3 GB of memory for each vCPU. For example, a machine with 48 vCPUs should have 144 GB of RAM
For more information about measuring memory requirements, see "Red Hat OpenStack Platform Hardware Requirements for Highly Available Controllers" on the Red Hat Customer Portal.
- Disk Storage and layout
A minimum amount of 40 GB storage is required if the Object Storage service (swift) is not running on the Controller nodes. However, the Telemetry and Object Storage services are both installed on the Controllers, with both configured to use the root disk. These defaults are suitable for deploying small overclouds built on commodity hardware. These environments are typical of proof-of-concept and test environments. You can use these defaults to deploy overclouds with minimal planning, but they offer little in terms of workload capacity and performance.
In an enterprise environment, however, the defaults could cause a significant bottleneck because Telemetry accesses storage constantly. This results in heavy disk I/O usage, which severely impacts the performance of all other Controller services. In this type of environment, you must plan your overcloud and configure it accordingly.
Red Hat provides several configuration recommendations for both Telemetry and Object Storage. For more information, see Deployment Recommendations for Specific Red Hat OpenStack Platform Services.
- Network Interface Cards
- A minimum of 2 x 1 Gbps Network Interface Cards. Use additional network interface cards for bonded interfaces or to delegate tagged VLAN traffic.
- Power management
- Each Controller node requires a supported power management interface, such as an Intelligent Platform Management Interface (IPMI) functionality, on the server motherboard.
- Virtualization support
- Red Hat supports virtualized Controller nodes only on Red Hat Virtualization platforms. For more information, see Virtualized control planes.
6.7. Compute node requirements Copy linkLink copied to clipboard!
Compute nodes are responsible for running virtual machine instances after they are launched. Compute nodes must support hardware virtualization. Compute nodes must also have enough memory and disk space to support the requirements of the virtual machine instances that they host.
- Processor
- 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions, and the AMD-V or Intel VT hardware virtualization extensions enabled. It is recommended that this processor has a minimum of 4 cores.
- IBM POWER 8 processor.
- Memory
- A minimum of 6 GB of RAM. Add additional RAM to this requirement based on the amount of memory that you intend to make available to virtual machine instances.
- Disk space
- A minimum of 40 GB of available disk space.
- Network Interface Cards
- A minimum of one 1 Gbps Network Interface Cards, although it is recommended to use at least two NICs in a production environment. Use additional network interface cards for bonded interfaces or to delegate tagged VLAN traffic.
- Power management
- Each Compute node requires a supported power management interface, such as an Intelligent Platform Management Interface (IPMI) functionality, on the server motherboard.
6.8. Ceph Storage node requirements Copy linkLink copied to clipboard!
Ceph Storage nodes are responsible for providing object storage in a Red Hat OpenStack Platform environment.
- Placement Groups (PGs)
- Ceph uses placement groups to facilitate dynamic and efficient object tracking at scale. In the case of OSD failure or cluster rebalancing, Ceph can move or replicate a placement group and its contents, which means a Ceph cluster can re-balance and recover efficiently. The default placement group count that director creates is not always optimal so it is important to calculate the correct placement group count according to your requirements. You can use the placement group calculator to calculate the correct count: Placement Groups (PGs) per Pool Calculator
- Processor
- 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions.
- Memory
- Red Hat typically recommends a baseline of 16 GB of RAM per OSD host, with an additional 2 GB of RAM per OSD daemon.
- Disk layout
Sizing is dependent on your storage requirements. Red Hat recommends that your Ceph Storage node configuration includes three or more disks in a layout similar to the following example:
-
/dev/sda- The root disk. The director copies the main overcloud image to the disk. Ensure that the disk has a minimum of 40 GB of available disk space. -
/dev/sdb- The journal disk. This disk divides into partitions for Ceph OSD journals. For example,/dev/sdb1,/dev/sdb2, and/dev/sdb3. The journal disk is usually a solid state drive (SSD) to aid with system performance. /dev/sdcand onward - The OSD disks. Use as many disks as necessary for your storage requirements.NoteRed Hat OpenStack Platform director uses
ceph-ansible, which does not support installing the OSD on the root disk of Ceph Storage nodes. This means that you need at least two disks for a supported Ceph Storage node.
-
- Network Interface Cards
- A minimum of one 1 Gbps Network Interface Cards, although Red Hat recommends that you use at least two NICs in a production environment. Use additional network interface cards for bonded interfaces or to delegate tagged VLAN traffic. Red Hat recommends that you use a 10 Gbps interface for storage nodes, especially if you want to create an OpenStack Platform environment that serves a high volume of traffic.
- Power management
- Each Controller node requires a supported power management interface, such as Intelligent Platform Management Interface (IPMI) functionality on the motherboard of the server.
For more information about installing an overcloud with a Ceph Storage cluster, see the Deploying an Overcloud with Containerized Red Hat Ceph guide.
6.9. Object Storage node requirements Copy linkLink copied to clipboard!
Object Storage nodes provide an object storage layer for the overcloud. The Object Storage proxy is installed on Controller nodes. The storage layer requires bare metal nodes with multiple disks on each node.
- Processor
- 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions.
- Memory
- Memory requirements depend on the amount of storage space. Use at minimum 1 GB of memory for each 1 TB of hard disk space. For optimal performance, it is recommended to use 2 GB for each 1 TB of hard disk space, especially for workloads with files smaller than 100GB.
- Disk space
Storage requirements depend on the capacity needed for the workload. It is recommended to use SSD drives to store the account and container data. The capacity ratio of account and container data to objects is approximately 1 per cent. For example, for every 100TB of hard drive capacity, provide 1TB of SSD capacity for account and container data.
However, this depends on the type of stored data. If you want to store mostly small objects, provide more SSD space. For large objects (videos, backups), use less SSD space.
- Disk layout
The recommended node configuration requires a disk layout similar to the following example:
-
/dev/sda- The root disk. Director copies the main overcloud image to the disk. -
/dev/sdb- Used for account data. -
/dev/sdc- Used for container data. -
/dev/sddand onward - The object server disks. Use as many disks as necessary for your storage requirements.
-
- Network Interface Cards
- A minimum of 2 x 1 Gbps Network Interface Cards. Use additional network interface cards for bonded interfaces or to delegate tagged VLAN traffic.
- Power management
- Each Controller node requires a supported power management interface, such as an Intelligent Platform Management Interface (IPMI) functionality, on the server motherboard.
6.10. Overcloud repositories Copy linkLink copied to clipboard!
Red Hat OpenStack Platform 16.0 runs on Red Hat Enterprise Linux 8.1. After overcloud deployment, lock each host to a specific version with the subscription-manager release command:
sudo subscription-manager release --set=8.1
$ sudo subscription-manager release --set=8.1
You must enable the following repositories to install and configure the overcloud.
Core repositories
The following table lists core repositories for installing the overcloud.
| Name | Repository | Description of requirement |
|---|---|---|
| Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) Extended Update Support (EUS) |
| Base operating system repository for x86_64 systems. |
| Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) Extended Update Support (EUS) |
| Contains Red Hat OpenStack Platform dependencies. |
| Red Hat Enterprise Linux 8 for x86_64 - High Availability (RPMs) Extended Update Support (EUS) |
| High availability tools for Red Hat Enterprise Linux. Used for Controller node high availability. |
| Red Hat Ansible Engine 2.8 for RHEL 8 x86_64 (RPMs) |
| Ansible Engine for Red Hat Enterprise Linux. Used to provide the latest version of Ansible. |
| Advanced Virtualization for RHEL 8 x86_64 (RPMs) |
| Provides virtualization packages for OpenStack Platform. |
| Red Hat Satellite Tools for RHEL 8 Server RPMs x86_64 |
| Tools for managing hosts with Red Hat Satellite 6. |
| Red Hat OpenStack Platform 16.0 for RHEL 8 (RPMs) |
| Core Red Hat OpenStack Platform repository. |
| Red Hat Fast Datapath for RHEL 8 (RPMS) |
| Provides Open vSwitch (OVS) packages for OpenStack Platform. |
Ceph repositories
The following table lists Ceph Storage related repositories for the overcloud.
| Name | Repository | Description of Requirement |
|---|---|---|
| Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) |
| Base operating system repository for x86_64 systems. |
| Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) |
| Contains Red Hat OpenStack Platform dependencies. |
| Red Hat Enterprise Linux 8 for x86_64 - High Availability (RPMs) |
| High availability tools for Red Hat Enterprise Linux. |
| Red Hat Ansible Engine 2.8 for RHEL 8 x86_64 (RPMs) |
| Ansible Engine for Red Hat Enterprise Linux. Used to provide the latest version of Ansible. |
| Red Hat OpenStack Platform 16.0 Director Deployment Tools for RHEL 8 x86_64 (RPMs) |
| Packages to help director configure Ceph Storage nodes. |
| Red Hat Ceph Storage OSD 4 for RHEL 8 x86_64 (RPMs) |
| (For Ceph Storage Nodes) Repository for Ceph Storage Object Storage daemon. Installed on Ceph Storage nodes. |
| Red Hat Ceph Storage MON 4 for RHEL 8 x86_64 (RPMs) |
| (For Ceph Storage Nodes) Repository for Ceph Storage Monitor daemon. Installed on Controller nodes in OpenStack environments using Ceph Storage nodes. |
| Red Hat Ceph Storage Tools 4 for RHEL 8 x86_64 (RPMs) |
| Provides tools for nodes to communicate with the Ceph Storage cluster. Enable this repository for all nodes when you deploy an overcloud with a Ceph Storage cluster or when you integrate your overcloud with an existing Ceph Storage cluster. |
Real Time repositories
The following table lists repositories for Real Time Compute (RTC) functionality.
| Name | Repository | Description of requirement |
|---|---|---|
| Red Hat Enterprise Linux 8 for x86_64 - Real Time (RPMs) |
|
Repository for Real Time KVM (RT-KVM). Contains packages to enable the real time kernel. Enable this repository for all Compute nodes targeted for RT-KVM. NOTE: You need a separate subscription to a |
| Red Hat Enterprise Linux 8 for x86_64 - Real Time for NFV (RPMs) |
|
Repository for Real Time KVM (RT-KVM) for NFV. Contains packages to enable the real time kernel. Enable this repository for all NFV Compute nodes targeted for RT-KVM. NOTE: You need a separate subscription to a |
IBM POWER repositories
The following table lists repositories for Openstack Platform on POWER PC architecture. Use these repositories in place of equivalents in the Core repositories.
| Name | Repository | Description of requirement |
|---|---|---|
| Red Hat Enterprise Linux for IBM Power, little endian - BaseOS (RPMs) |
| Base operating system repository for ppc64le systems. |
| Red Hat Enterprise Linux 8 for IBM Power, little endian - AppStream (RPMs) |
| Contains Red Hat OpenStack Platform dependencies. |
| Red Hat Enterprise Linux 8 for IBM Power, little endian - High Availability (RPMs) |
| High availability tools for Red Hat Enterprise Linux. Used for Controller node high availability. |
| Red Hat Ansible Engine 2.8 for RHEL 8 IBM Power, little endian (RPMs) |
| Ansible Engine for Red Hat Enterprise Linux. Used to provide the latest version of Ansible. |
| Red Hat OpenStack Platform 16.0 for RHEL 8 (RPMs) |
| Core Red Hat OpenStack Platform repository for ppc64le systems. |
6.11. Provisioning methods Copy linkLink copied to clipboard!
There are three main methods that you can use to provision the nodes for your Red Hat OpenStack Platform environment:
- Provisioning with director
-
Red Hat OpenStack Platform director is the standard provisioning method. In this scenario, the
openstack overcloud deploycommand performs both the provisioning and the configuration of your deployment. For more information about the standard provisioning and deployment method, see Chapter 7, Configuring a basic overcloud with CLI tools. - Provisioning with the OpenStack Bare Metal (ironic) service
In this scenario, you can separate the provisioning and configuration stages of the standard director deployment into two distinct processes. This is useful if you want to mitigate some of the risk involved with the standard director deployment and identify points of failure more efficiently. For more information about this scenario, see Chapter 8, Provisioning bare metal nodes before deploying the overcloud.
ImportantThis feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
- Provisioning with an external tool
In this scenario, director controls the overcloud configuration on nodes that you pre-provision with an external tool. This is useful if you want to create an overcloud without power management control, use networks that have DHCP/PXE boot restrictions, or if you want to use nodes that have a custom partitioning layout that does not rely on the QCOW2
overcloud-fullimage. This scenario does not use the OpenStack Compute (nova), OpenStack Bare Metal (ironic), or OpenStack Image (glance) services for managing nodes.For more information about this scenario, see Chapter 9, Configuring a basic overcloud with pre-provisioned nodes.
You cannot combine pre-provisioned nodes with director-provisioned nodes.
Chapter 7. Configuring a basic overcloud with CLI tools Copy linkLink copied to clipboard!
This chapter contains basic configuration procedures to deploy an OpenStack Platform environment using the CLI tools. An overcloud with a basic configuration contains no custom features. However, you can add advanced configuration options to this basic overcloud and customize it to your specifications using the instructions in the Advanced Overcloud Customization guide.
7.1. Registering nodes for the overcloud Copy linkLink copied to clipboard!
Director requires a node definition template, which you create manually. This template uses a JSON or YAML format, and contains the hardware and power management details for your nodes.
Procedure
Create a template that lists your nodes. Use the following JSON and YAML template examples to understand how to structure your node definition template:
Example JSON template
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example YAML template
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This template contains the following attributes:
- name
- The logical name for the node.
- pm_type
The power management driver that you want to use. This example uses the IPMI driver (
ipmi).NoteIPMI is the preferred supported power management driver. For more information about supported power management types and their options, see Appendix A, Power management drivers. If these power management drivers do not work as expected, use IPMI for your power management.
- pm_user; pm_password
- The IPMI username and password.
- pm_addr
- The IP address of the IPMI device.
- pm_port (Optional)
- The port to access the specific IPMI device.
- mac
- (Optional) A list of MAC addresses for the network interfaces on the node. Use only the MAC address for the Provisioning NIC of each system.
- cpu
- (Optional) The number of CPUs on the node.
- memory
- (Optional) The amount of memory in MB.
- disk
- (Optional) The size of the hard disk in GB.
- arch
(Optional) The system architecture.
ImportantWhen building a multi-architecture cloud, the
archkey is mandatory to distinguish nodes usingx86_64andppc64learchitectures.
After you create the template, run the following commands to verify the formatting and syntax:
source ~/stackrc (undercloud) $ openstack overcloud node import --validate-only ~/nodes.json
$ source ~/stackrc (undercloud) $ openstack overcloud node import --validate-only ~/nodes.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow Save the file to the home directory of the
stackuser (/home/stack/nodes.json), then run the following commands to import the template to director:(undercloud) $ openstack overcloud node import ~/nodes.json
(undercloud) $ openstack overcloud node import ~/nodes.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command registers each node from the template into director.
Wait for the node registration and configuration to complete. When complete, confirm that director has successfully registered the nodes:
(undercloud) $ openstack baremetal node list
(undercloud) $ openstack baremetal node listCopy to Clipboard Copied! Toggle word wrap Toggle overflow
7.2. Validating the introspection requirements Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
Run the pre-introspection validation group to check the introspection requirements.
Procedure
Source the
stackrcfile.source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack tripleo validator runcommand with the --group pre-introspection option:openstack tripleo validator run --group pre-introspection
$ openstack tripleo validator run --group pre-introspectionCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Review the results of the validation report.
A FAILED validation does not prevent you from deploying or running Red Hat OpenStack Platform. However, a FAILED validation can indicate a potential issue with a production environment.
7.3. Inspecting the hardware of nodes Copy linkLink copied to clipboard!
Director can run an introspection process on each node. This process boots an introspection agent over PXE on each node. The introspection agent collects hardware data from the node and sends the data back to director. Director then stores this introspection data in the OpenStack Object Storage (swift) service running on director. Director uses hardware information for various purposes such as profile tagging, benchmarking, and manual root disk assignment.
Procedure
Run the following command to inspect the hardware attributes of each node:
(undercloud) $ openstack overcloud node introspect --all-manageable --provide
(undercloud) $ openstack overcloud node introspect --all-manageable --provideCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Use the
--all-manageableoption to introspect only the nodes that are in a managed state. In this example, all nodes are in a managed state. -
Use the
--provideoption to reset all nodes to anavailablestate after introspection.
-
Use the
Monitor the introspection progress logs in a separate terminal window:
(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/ironic-inspector.log
(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/ironic-inspector.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantEnsure that this process runs to completion. This process usually takes 15 minutes for bare metal nodes.
After the introspection completes, all nodes change to an available state.
7.4. Tagging nodes into profiles Copy linkLink copied to clipboard!
After you register and inspect the hardware of each node, tag the nodes into specific profiles. These profile tags match your nodes to flavors, which assigns the flavors to deployment roles. The following example shows the relationships across roles, flavors, profiles, and nodes for Controller nodes:
| Type | Description |
|---|---|
| Role |
The |
| Flavor |
The |
| Profile |
The |
| Node |
You also apply the |
Default profile flavors compute, control, swift-storage, ceph-storage, and block-storage are created during undercloud installation and are usable without modification in most environments.
Procedure
To tag a node into a specific profile, add a
profileoption to theproperties/capabilitiesparameter for each node. For example, to tag your nodes to use Controller and Compute profiles respectively, use the following commands:(undercloud) $ openstack baremetal node set --property capabilities='profile:control,boot_option:local' 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0 (undercloud) $ openstack baremetal node set --property capabilities='profile:compute,boot_option:local' 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13
(undercloud) $ openstack baremetal node set --property capabilities='profile:control,boot_option:local' 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0 (undercloud) $ openstack baremetal node set --property capabilities='profile:compute,boot_option:local' 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13Copy to Clipboard Copied! Toggle word wrap Toggle overflow The addition of the
profile:controlandprofile:computeoptions tag the two nodes into each respective profiles.These commands also set the
boot_option:localparameter, which defines how each node boots.After you complete node tagging, check the assigned profiles or possible profiles:
(undercloud) $ openstack overcloud profiles list
(undercloud) $ openstack overcloud profiles listCopy to Clipboard Copied! Toggle word wrap Toggle overflow
7.5. Setting UEFI boot mode Copy linkLink copied to clipboard!
The default boot mode is the legacy BIOS mode. Newer systems might require UEFI boot mode instead of the legacy BIOS mode. Complete the following steps to change the boot mode to UEFI mode.
Procedure
Set the following parameters in your
undercloud.conffile:ipxe_enabled = True inspection_enable_uefi = True
ipxe_enabled = True inspection_enable_uefi = TrueCopy to Clipboard Copied! Toggle word wrap Toggle overflow Save the
undercloud.conffile and run the undercloud installation:openstack undercloud install
$ openstack undercloud installCopy to Clipboard Copied! Toggle word wrap Toggle overflow Wait until the installation script completes.
Set the boot mode to
uefifor each registered node. For example, to add or replace the existingboot_modeparameters in thecapabilitiesproperty, run the following command:NODE=<NODE NAME OR ID> ; openstack baremetal node set --property capabilities="boot_mode:uefi,$(openstack baremetal node show $NODE -f json -c properties | jq -r .properties.capabilities | sed "s/boot_mode:[^,]*,//g")" $NODE
$ NODE=<NODE NAME OR ID> ; openstack baremetal node set --property capabilities="boot_mode:uefi,$(openstack baremetal node show $NODE -f json -c properties | jq -r .properties.capabilities | sed "s/boot_mode:[^,]*,//g")" $NODECopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteCheck that you have retained the
profileandboot_optioncapabilities:openstack baremetal node show r530-12 -f json -c properties | jq -r .properties.capabilities
$ openstack baremetal node show r530-12 -f json -c properties | jq -r .properties.capabilitiesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the boot mode to
uefifor each flavor:openstack flavor set --property capabilities:boot_mode='uefi' control
$ openstack flavor set --property capabilities:boot_mode='uefi' controlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
7.6. Enabling virtual media boot Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
You can use Redfish virtual media boot to supply a boot image to the Baseboard Management Controller (BMC) of a node so that the BMC can insert the image into one of the virtual drives. The node can then boot from the virtual drive into the operating system that exists in the image.
Redfish hardware types support booting deploy, rescue, and user images over virtual media. The Bare Metal service (ironic) uses kernel and ramdisk images associated with a node to build bootable ISO images for UEFI or BIOS boot modes at the moment of node deployment. The major advantage of virtual media boot is that you can eliminate the TFTP image transfer phase of PXE and use HTTP GET, or other methods, instead.
To boot a node with the redfish hardware type over virtual media, set the boot interface to redfish-virtual-media and, for UEFI nodes, define the EFI System Partition (ESP) image. Then configure an enrolled node to use Redfish virtual media boot.
Prerequisites
-
Redfish driver enabled in the
enabled_hardware_typesparameter in theundercloud.conffile. - A bare metal node registered and enrolled.
- IPA and instance images in the Image Service (glance).
- For UEFI nodes, you must also have an EFI system partition image (ESP) available in the Image Service (glance).
- A bare metal flavor.
- A network for cleaning and provisioning.
Sushy library installed:
sudo yum install sushy
$ sudo yum install sushyCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Procedure
Set the Bare Metal service (ironic) boot interface to
redfish-virtual-media:openstack baremetal node set --boot-interface redfish-virtual-media $NODE_NAME
$ openstack baremetal node set --boot-interface redfish-virtual-media $NODE_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
$NODE_NAMEwith the name of the node.For UEFI nodes, set the boot mode to
uefi:NODE=<NODE NAME OR ID> ; openstack baremetal node set --property capabilities="boot_mode:uefi,$(openstack baremetal node show $NODE -f json -c properties | jq -r .properties.capabilities | sed "s/boot_mode:[^,]*,//g")" $NODE
NODE=<NODE NAME OR ID> ; openstack baremetal node set --property capabilities="boot_mode:uefi,$(openstack baremetal node show $NODE -f json -c properties | jq -r .properties.capabilities | sed "s/boot_mode:[^,]*,//g")" $NODECopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
$NODEwith the name of the node.NoteFor BIOS nodes, do not complete this step.
For UEFI nodes, define the EFI System Partition (ESP) image:
openstack baremetal node set --driver-info bootloader=$ESP $NODE_NAME
$ openstack baremetal node set --driver-info bootloader=$ESP $NODE_NAMECopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
$ESPwith the glance image UUID or URL for the ESP image, and replace$NODE_NAMEwith the name of the node.NoteFor BIOS nodes, do not complete this step.
Create a port on the bare metal node and associate the port with the MAC address of the NIC on the bare metal node:
openstack baremetal port create --pxe-enabled True --node $UUID $MAC_ADDRESS
$ openstack baremetal port create --pxe-enabled True --node $UUID $MAC_ADDRESSCopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
$UUIDwith the UUID of the bare metal node, and replace$MAC_ADDRESSwith the MAC address of the NIC on the bare metal node.
7.7. Defining the root disk for multi-disk clusters Copy linkLink copied to clipboard!
Director must identify the root disk during provisioning in the case of nodes with multiple disks. For example, most Ceph Storage nodes use multiple disks. By default, the director writes the overcloud image to the root disk during the provisioning process
There are several properties that you can define to help the director identify the root disk:
-
model(String): Device identifier. -
vendor(String): Device vendor. -
serial(String): Disk serial number. -
hctl(String): Host:Channel:Target:Lun for SCSI. -
size(Integer): Size of the device in GB. -
wwn(String): Unique storage identifier. -
wwn_with_extension(String): Unique storage identifier with the vendor extension appended. -
wwn_vendor_extension(String): Unique vendor storage identifier. -
rotational(Boolean): True for a rotational device (HDD), otherwise false (SSD). -
name(String): The name of the device, for example: /dev/sdb1.
Use the name property only for devices with persistent names. Do not use name to set the root disk for any other devices because this value can change when the node boots.
Complete the following steps to specify the root device using its serial number.
Procedure
Check the disk information from the hardware introspection of each node. Run the following command to display the disk information of a node:
(undercloud) $ openstack baremetal introspection data save 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0 | jq ".inventory.disks"
(undercloud) $ openstack baremetal introspection data save 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0 | jq ".inventory.disks"Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example, the data for one node might show three disks:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack baremetal node set --property root_device=command to set the root disk for a node. Include the most appropriate hardware attribute value to define the root disk.(undercloud) $ openstack baremetal node set --property root_device=’{“serial”:”<serial_number>”}' <node-uuid>(undercloud) $ openstack baremetal node set --property root_device=’{“serial”:”<serial_number>”}' <node-uuid>Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example, to set the root device to disk 2, which has the serial number
61866da04f380d001ea4e13c12e36ad6run the following command:
(undercloud) $ openstack baremetal node set --property root_device='{"serial": "61866da04f380d001ea4e13c12e36ad6"}' 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0
(undercloud) $ openstack baremetal node set --property root_device='{"serial": "61866da04f380d001ea4e13c12e36ad6"}' 1a4e30da-b6dc-499d-ba87-0bd8a3819bc0
+
Ensure that you configure the BIOS of each node to include booting from the root disk that you choose. Configure the boot order to boot from the network first, then to boot from the root disk.
Director identifies the specific disk to use as the root disk. When you run the openstack overcloud deploy command, director provisions and writes the overcloud image to the root disk.
7.8. Using the overcloud-minimal image to avoid using a Red Hat subscription entitlement Copy linkLink copied to clipboard!
By default, director writes the QCOW2 overcloud-full image to the root disk during the provisioning process. The overcloud-full image uses a valid Red Hat subscription. However, you can also use the overcloud-minimal image, for example, to provision a bare OS where you do not want to run any other OpenStack services and consume your subscription entitlements.
A common use case for this occurs when you want to provision nodes with only Ceph daemons. For this and similar use cases, you can use the overcloud-minimal image option to avoid reaching the limit of your paid Red Hat subscriptions. For information about how to obtain the overcloud-minimal image, see Obtaining images for overcloud nodes.
A Red Hat OpenStack Platform subscription contains Open vSwitch (OVS), but core services, such as OVS, are not available when you use the overcloud-minimal image. OVS is not required to deploy Ceph Storage nodes. Instead of using ovs_bond to define bonds, use linux_bond. For more information about linux_bond, see Linux bonding options.
Procedure
To configure director to use the
overcloud-minimalimage, create an environment file that contains the following image definition:parameter_defaults: <roleName>Image: overcloud-minimal
parameter_defaults: <roleName>Image: overcloud-minimalCopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<roleName>with the name of the role and appendImageto the name of the role. The following example shows anovercloud-minimalimage for Ceph storage nodes:parameter_defaults: CephStorageImage: overcloud-minimal
parameter_defaults: CephStorageImage: overcloud-minimalCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Pass the environment file to the
openstack overcloud deploycommand.
The overcloud-minimal image supports only standard Linux bridges and not OVS because OVS is an OpenStack service that requires a Red Hat OpenStack Platform subscription entitlement.
7.9. Creating architecture specific roles Copy linkLink copied to clipboard!
When building a multi-architecture cloud, you must add any architecture specific roles to the roles_data.yaml file. The following example includes the ComputePPC64LE role along with the default roles:
openstack overcloud roles generate \
--roles-path /usr/share/openstack-tripleo-heat-templates/roles -o ~/templates/roles_data.yaml \
Controller Compute ComputePPC64LE BlockStorage ObjectStorage CephStorage
openstack overcloud roles generate \
--roles-path /usr/share/openstack-tripleo-heat-templates/roles -o ~/templates/roles_data.yaml \
Controller Compute ComputePPC64LE BlockStorage ObjectStorage CephStorage
The Creating a Custom Role File section has information on roles.
7.10. Environment files Copy linkLink copied to clipboard!
The undercloud includes a set of heat templates that form the plan for your overcloud creation. You can customize aspects of the overcloud with environment files, which are YAML-formatted files that override parameters and resources in the core heat template collection. You can include as many environment files as necessary. However, the order of the environment files is important because the parameters and resources that you define in subsequent environment files take precedence. Use the following list as an example of the environment file order:
- The number of nodes and the flavors for each role. It is vital to include this information for overcloud creation.
- The location of the container images for containerized OpenStack services.
Any network isolation files, starting with the initialization file (
environments/network-isolation.yaml) from the heat template collection, then your custom NIC configuration file, and finally any additional network configurations. For more information, see the following chapters in the Advanced Overcloud Customization guide:- Any external load balancing environment files if you are using an external load balancer. For more information, see External Load Balancing for the Overcloud.
- Any storage environment files such as Ceph Storage, NFS, or iSCSI.
- Any environment files for Red Hat CDN or Satellite registration.
- Any other custom environment files.
Red Hat recommends that you organize your custom environment files in a separate directory, such as the templates directory.
For more information about customizing advanced features for your overcloud, see the Advanced Overcloud Customization guide.
A basic overcloud uses local LVM storage for block storage, which is not a supported configuration. It is recommended to use an external storage solution, such as Red Hat Ceph Storage, for block storage.
The environment file extension must be .yaml or .template, or it will not be treated as a custom template resource.
The next few sections contain information about creating some environment files necessary for your overcloud.
7.11. Creating an environment file that defines node counts and flavors Copy linkLink copied to clipboard!
By default, director deploys an overcloud with 1 Controller node and 1 Compute node using the baremetal flavor. However, this is only suitable for a proof-of-concept deployment. You can override the default configuration by specifying different node counts and flavors. For a small-scale production environment, deploy at least 3 Controller nodes and 3 Compute nodes, and assign specific flavors to ensure that the nodes have the appropriate resource specifications. Complete the following steps to create an environment file named node-info.yaml that stores the node counts and flavor assignments.
Procedure
Create a
node-info.yamlfile in the/home/stack/templates/directory:(undercloud) $ touch /home/stack/templates/node-info.yaml
(undercloud) $ touch /home/stack/templates/node-info.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the file to include the node counts and flavors that you need. This example contains 3 Controller nodes and 3 Compute nodes:
parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 3 ComputeCount: 3
parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 3 ComputeCount: 3Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.12. Creating an environment file for undercloud CA trust Copy linkLink copied to clipboard!
If your undercloud uses TLS and the Certificate Authority (CA) is not publicly trusted, you can use the CA for SSL endpoint encryption that the undercloud operates. To ensure that the undercloud endpoints are accessible to the rest of your deployment, configure your overcloud nodes to trust the undercloud CA.
For this approach to work, your overcloud nodes must have a network route to the public endpoint on the undercloud. It is likely that you must apply this configuration for deployments that rely on spine-leaf networking.
There are two types of custom certificates you can use in the undercloud:
-
User-provided certificates - This definition applies when you have provided your own certificate. This can be from your own CA, or it can be self-signed. This is passed using the
undercloud_service_certificateoption. In this case, you must either trust the self-signed certificate, or the CA (depending on your deployment). -
Auto-generated certificates - This definition applies when you use
certmongerto generate the certificate using its own local CA. Enable auto-generated certificates with thegenerate_service_certificateoption in theundercloud.conffile. In this case, director generates a CA certificate at/etc/pki/ca-trust/source/anchors/cm-local-ca.pemand the director configures the undercloud’s HAProxy instance to use a server certificate. Add the CA certificate to theinject-trust-anchor-hiera.yamlfile to present the certificate to OpenStack Platform.
This example uses a self-signed certificate located in /home/stack/ca.crt.pem. If you use auto-generated certificates, use /etc/pki/ca-trust/source/anchors/cm-local-ca.pem instead.
Procedure
Open the certificate file and copy only the certificate portion. Do not include the key:
vi /home/stack/ca.crt.pem
$ vi /home/stack/ca.crt.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow The certificate portion you need looks similar to this shortened example:
-----BEGIN CERTIFICATE----- MIIDlTCCAn2gAwIBAgIJAOnPtx2hHEhrMA0GCSqGSIb3DQEBCwUAMGExCzAJBgNV BAYTAlVTMQswCQYDVQQIDAJOQzEQMA4GA1UEBwwHUmFsZWlnaDEQMA4GA1UECgwH UmVkIEhhdDELMAkGA1UECwwCUUUxFDASBgNVBAMMCzE5Mi4xNjguMC4yMB4XDTE3 -----END CERTIFICATE-----
-----BEGIN CERTIFICATE----- MIIDlTCCAn2gAwIBAgIJAOnPtx2hHEhrMA0GCSqGSIb3DQEBCwUAMGExCzAJBgNV BAYTAlVTMQswCQYDVQQIDAJOQzEQMA4GA1UEBwwHUmFsZWlnaDEQMA4GA1UECgwH UmVkIEhhdDELMAkGA1UECwwCUUUxFDASBgNVBAMMCzE5Mi4xNjguMC4yMB4XDTE3 -----END CERTIFICATE-----Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new YAML file called
/home/stack/inject-trust-anchor-hiera.yamlwith the following contents, and include the certificate you copied from the PEM file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The certificate string must follow the PEM format.
The CAMap parameter might contain other certificates relevant to SSL/TLS configuration.
Director copies the CA certificate to each overcloud node during the overcloud deployment. As a result, each node trusts the encryption presented by the undercloud’s SSL endpoints. For more information about environment files, see Section 7.15, “Including environment files in an overcloud deployment”.
7.13. Deployment command Copy linkLink copied to clipboard!
The final stage in creating your OpenStack environment is to run the openstack overcloud deploy command to create the overcloud. Before you run this command, familiarize yourself with key options and how to include custom environment files.
Do not run openstack overcloud deploy as a background process. The overcloud creation might hang mid-deployment if you run it as a background process.
7.14. Deployment command options Copy linkLink copied to clipboard!
The following table lists the additional parameters for the openstack overcloud deploy command.
| Parameter | Description |
|---|---|
|
|
The directory that contains the heat templates that you want to deploy. If blank, the deployment command uses the default template location at |
|
| The name of the stack that you want to create or update |
|
| The deployment timeout duration in minutes |
|
| The virtualization type that you want to use for hypervisors |
|
|
The Network Time Protocol (NTP) server that you want to use to synchronize time. You can also specify multiple NTP servers in a comma-separated list, for example: |
|
|
Defines custom values for the environment variable |
|
|
Defines the SSH user to access the overcloud nodes. Normally SSH access occurs through the |
|
| Defines the key path for SSH access to overcloud nodes. |
|
| Defines the network name that you want to use for SSH access to overcloud nodes. |
|
|
Extra environment files that you want to pass to the overcloud deployment. You can specify this option more than once. Note that the order of environment files that you pass to the |
|
| A directory that contains environment files that you want to include in deployment. The deployment command processes these environment files in numerical order, then alphabetical order. |
|
|
Defines the roles file and overrides the default |
|
|
Defines the networks file and overrides the default network_data.yaml in the |
|
|
Defines the plan Environment file and overrides the default |
|
| Use this option if you do not want to delete temporary files after deployment, and log their location. |
|
| Use this option if you want to update the plan without performing the actual deployment. |
|
| The overcloud creation process performs a set of pre-deployment checks. This option exits if any non-fatal errors occur from the pre-deployment checks. It is advisable to use this option as any errors can cause your deployment to fail. |
|
| The overcloud creation process performs a set of pre-deployment checks. This option exits if any non-critical warnings occur from the pre-deployment checks. openstack-tripleo-validations |
|
| Use this option if you want to perform a validation check on the overcloud without creating the overcloud. |
|
|
Use this option to run external validations from the |
|
| Use this option to skip the overcloud post-deployment configuration. |
|
| Use this option to force the overcloud post-deployment configuration. |
|
|
Use this option if you do not want the deployment command to generate a unique identifier for the |
|
| The path to a YAML file with arguments and parameters. |
|
| Use this option if you want to disable password generation for the overcloud services. |
|
|
Use this option if you want to deploy pre-provisioned overcloud nodes. Used in conjunction with |
|
|
Use this option if you want to disable the |
|
|
Use this option if you want to disable the overcloud stack creation and only run the |
|
|
The directory that you want to use for saved |
|
|
The path to an Ansible configuration file. The configuration in the file overrides any configuration that |
|
|
The timeout duration in minutes that you want to use for |
Run the following command to view a full list of options:
(undercloud) $ openstack help overcloud deploy
(undercloud) $ openstack help overcloud deploy
Some command line parameters are outdated or deprecated in favor of using heat template parameters, which you include in the parameter_defaults section in an environment file. The following table maps deprecated parameters to their heat template equivalents.
| Parameter | Description | Heat template parameter |
|---|---|---|
|
| The number of Controller nodes to scale out |
|
|
| The number of Compute nodes to scale out |
|
|
| The number of Ceph Storage nodes to scale out |
|
|
| The number of Block Storage (cinder) nodes to scale out |
|
|
| The number of Object Storage (swift) nodes to scale out |
|
|
| The flavor that you want to use for Controller nodes |
|
|
| The flavor that you want to use for Compute nodes |
|
|
| The flavor that you want to use for Ceph Storage nodes |
|
|
| The flavor that you want to use for Block Storage (cinder) nodes |
|
|
| The flavor that you want to use for Object Storage (swift) nodes |
|
|
| The overcloud creation process performs a set of pre-deployment checks. This option exits if any fatal errors occur from the pre-deployment checks. It is advisable to use this option because any errors can cause your deployment to fail. | No parameter mapping |
|
|
Disable the pre-deployment validations entirely. These validations were built-in pre-deployment validations, which have been replaced with external validations from the | No parameter mapping |
|
|
Run deployment using the | No parameter mapping |
|
| Use this option to register overcloud nodes to the Customer Portal or Satellite 6. |
|
|
|
Use this option to define the registration method that you want to use for the overcloud nodes. |
|
|
| The organization that you want to use for registration. |
|
|
| Use this option to register the system even if it is already registered. |
|
|
|
The base URL of the Satellite server to register overcloud nodes. Use the Satellite HTTP URL and not the HTTPS URL for this parameter. For example, use http://satellite.example.com and not https://satellite.example.com. The overcloud creation process uses this URL to determine whether the server is a Red Hat Satellite 5 or Red Hat Satellite 6 server. If the server is a Red Hat Satellite 6 server, the overcloud obtains the |
|
|
| Use this option to define the activation key that you want to use for registration. |
|
These parameters are scheduled for removal in a future version of Red Hat OpenStack Platform.
7.15. Including environment files in an overcloud deployment Copy linkLink copied to clipboard!
Use the -e option to include an environment file to customize your overcloud. You can include as many environment files as necessary. However, the order of the environment files is important because the parameters and resources that you define in subsequent environment files take precedence. Use the following list as an example of the environment file order:
- The number of nodes and the flavors for each role. It is vital to include this information for overcloud creation.
- The location of the container images for containerized OpenStack services.
Any network isolation files, starting with the initialization file (
environments/network-isolation.yaml) from the heat template collection, then your custom NIC configuration file, and finally any additional network configurations. For more information, see the following chapters in the Advanced Overcloud Customization guide:- Any external load balancing environment files if you are using an external load balancer. For more information, see External Load Balancing for the Overcloud.
- Any storage environment files such as Ceph Storage, NFS, or iSCSI.
- Any environment files for Red Hat CDN or Satellite registration.
- Any other custom environment files.
Any environment files that you add to the overcloud using the -e option become part of the stack definition of the overcloud.
The following command is an example of how to start the overcloud creation using environment files defined earlier in this scenario:
(undercloud) $ openstack overcloud deploy --templates \ -e /home/stack/templates/node-info.yaml\ -e /home/stack/containers-prepare-parameter.yaml \ -e /home/stack/inject-trust-anchor-hiera.yaml -r /home/stack/templates/roles_data.yaml \
(undercloud) $ openstack overcloud deploy --templates \
-e /home/stack/templates/node-info.yaml\
-e /home/stack/containers-prepare-parameter.yaml \
-e /home/stack/inject-trust-anchor-hiera.yaml
-r /home/stack/templates/roles_data.yaml \
This command contains the following additional options:
- --templates
-
Creates the overcloud using the heat template collection in
/usr/share/openstack-tripleo-heat-templatesas a foundation. - -e /home/stack/templates/node-info.yaml
- Adds an environment file to define how many nodes and which flavors to use for each role.
- -e /home/stack/containers-prepare-parameter.yaml
- Adds the container image preparation environment file. You generated this file during the undercloud installation and can use the same file for your overcloud creation.
- -e /home/stack/inject-trust-anchor-hiera.yaml
- Adds an environment file to install a custom certificate in the undercloud.
- -r /home/stack/templates/roles_data.yaml
- (Optional) The generated roles data if you use custom roles or want to enable a multi architecture cloud. For more information, see Section 7.9, “Creating architecture specific roles”.
Director requires these environment files for re-deployment and post-deployment functions. Failure to include these files can result in damage to your overcloud.
To modify the overcloud configuration at a later stage, perform the following actions:
- Modify parameters in the custom environment files and heat templates.
-
Run the
openstack overcloud deploycommand again with the same environment files.
Do not edit the overcloud configuration directly because director overrides any manual configuration when you update the overcloud stack.
7.16. Validating the deployment requirements Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
Run the pre-deployment validation group to check the deployment requirements.
Procedure
Source the
stackrcfile.source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow This validation requires a copy of your overcloud plan. Upload your overcloud plan with all necessary environment files. To upload your plan only, run the
openstack overcloud deploycommand with the--update-plan-onlyoption:openstack overcloud deploy --templates \ -e environment-file1.yaml \ -e environment-file2.yaml \ ... --update-plan-only$ openstack overcloud deploy --templates \ -e environment-file1.yaml \ -e environment-file2.yaml \ ... --update-plan-onlyCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack tripleo validator runcommand with the --group pre-deployment option:openstack tripleo validator run --group pre-deployment
$ openstack tripleo validator run --group pre-deploymentCopy to Clipboard Copied! Toggle word wrap Toggle overflow If the overcloud uses a plan name that is different to the default
overcloudname, set the plan name with the --planoption:openstack tripleo validator run --group pre-deployment \ --plan myovercloud$ openstack tripleo validator run --group pre-deployment \ --plan myovercloudCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Review the results of the validation report.
A FAILED validation does not prevent you from deploying or running Red Hat OpenStack Platform. However, a FAILED validation can indicate a potential issue with a production environment.
7.17. Overcloud deployment output Copy linkLink copied to clipboard!
When the overcloud creation completes, director provides a recap of the Ansible plays that were executed to configure the overcloud:
Director also provides details to access your overcloud.
7.18. Accessing the overcloud Copy linkLink copied to clipboard!
The director generates a script to configure and help authenticate interactions with your overcloud from the undercloud. The director saves this file, overcloudrc, in the home directory of the stack user. Run the following command to use this file:
(undercloud) $ source ~/overcloudrc
(undercloud) $ source ~/overcloudrc
This command loads the environment variables that are necessary to interact with your overcloud from the undercloud CLI. The command prompt changes to indicate this:
(overcloud) $
(overcloud) $
To return to interacting with the undercloud, run the following command:
(overcloud) $ source ~/stackrc (undercloud) $
(overcloud) $ source ~/stackrc
(undercloud) $
Each node in the overcloud also contains a heat-admin user. The stack user has SSH access to this user on each node. To access a node over SSH, find the IP address of the node that you want to access:
(undercloud) $ openstack server list
(undercloud) $ openstack server list
Then connect to the node using the heat-admin user and the IP address of the node:
(undercloud) $ ssh heat-admin@192.168.24.23
(undercloud) $ ssh heat-admin@192.168.24.23
7.19. Validating the post-deployment state Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
Run the post-deployment validation group to check the post-deployment state.
Procedure
Source the
stackrcfile.source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack tripleo validator runcommand with the --group post-deployment option:openstack tripleo validator run --group post-deployment
$ openstack tripleo validator run --group post-deploymentCopy to Clipboard Copied! Toggle word wrap Toggle overflow If the overcloud uses a plan name that is different to the default
overcloudname, set the plan name with the --planoption:openstack tripleo validator run --group post-deployment \ --plan myovercloud$ openstack tripleo validator run --group post-deployment \ --plan myovercloudCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Review the results of the validation report.
A FAILED validation does not prevent you from deploying or running Red Hat OpenStack Platform. However, a FAILED validation can indicate a potential issue with a production environment.
7.20. Next steps Copy linkLink copied to clipboard!
This concludes the creation of the overcloud using the command line tools. For more information about post-creation functions, see Chapter 11, Performing overcloud post-installation tasks.
Chapter 8. Provisioning bare metal nodes before deploying the overcloud Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
The overcloud deployment process contains two primary operations:
- Provisioning nodes
- Deploying the overcloud
You can mitigate some of the risk involved with this process and identify points of failure more efficiently if you separate these operations into distinct processes:
Provision your bare metal nodes.
- Create a node definition file in yaml format.
- Run the provisioning command, including the node definition file.
Deploy your overcloud.
- Run the deployment command, including the heat environment file that the provisioning command generates.
The provisioning process provisions your nodes and generates a heat environment file that contains various node specifications, including node count, predictive node placement, custom images, and custom NICs. When you deploy your overcloud, include this file in the deployment command.
You cannot combine pre-provisioned nodes with director-provisioned nodes.
8.1. Registering nodes for the overcloud Copy linkLink copied to clipboard!
Director requires a node definition template, which you create manually. This template uses a JSON or YAML format, and contains the hardware and power management details for your nodes.
Procedure
Create a template that lists your nodes. Use the following JSON and YAML template examples to understand how to structure your node definition template:
Example JSON template
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example YAML template
Copy to Clipboard Copied! Toggle word wrap Toggle overflow This template contains the following attributes:
- name
- The logical name for the node.
- pm_type
The power management driver that you want to use. This example uses the IPMI driver (
ipmi).NoteIPMI is the preferred supported power management driver. For more information about supported power management types and their options, see Appendix A, Power management drivers. If these power management drivers do not work as expected, use IPMI for your power management.
- pm_user; pm_password
- The IPMI username and password.
- pm_addr
- The IP address of the IPMI device.
- pm_port (Optional)
- The port to access the specific IPMI device.
- mac
- (Optional) A list of MAC addresses for the network interfaces on the node. Use only the MAC address for the Provisioning NIC of each system.
- cpu
- (Optional) The number of CPUs on the node.
- memory
- (Optional) The amount of memory in MB.
- disk
- (Optional) The size of the hard disk in GB.
- arch
(Optional) The system architecture.
ImportantWhen building a multi-architecture cloud, the
archkey is mandatory to distinguish nodes usingx86_64andppc64learchitectures.
After you create the template, run the following commands to verify the formatting and syntax:
source ~/stackrc (undercloud) $ openstack overcloud node import --validate-only ~/nodes.json
$ source ~/stackrc (undercloud) $ openstack overcloud node import --validate-only ~/nodes.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow Save the file to the home directory of the
stackuser (/home/stack/nodes.json), then run the following commands to import the template to director:(undercloud) $ openstack overcloud node import ~/nodes.json
(undercloud) $ openstack overcloud node import ~/nodes.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command registers each node from the template into director.
Wait for the node registration and configuration to complete. When complete, confirm that director has successfully registered the nodes:
(undercloud) $ openstack baremetal node list
(undercloud) $ openstack baremetal node listCopy to Clipboard Copied! Toggle word wrap Toggle overflow
8.2. Inspecting the hardware of nodes Copy linkLink copied to clipboard!
Director can run an introspection process on each node. This process boots an introspection agent over PXE on each node. The introspection agent collects hardware data from the node and sends the data back to director. Director then stores this introspection data in the OpenStack Object Storage (swift) service running on director. Director uses hardware information for various purposes such as profile tagging, benchmarking, and manual root disk assignment.
Procedure
Run the following command to inspect the hardware attributes of each node:
(undercloud) $ openstack overcloud node introspect --all-manageable --provide
(undercloud) $ openstack overcloud node introspect --all-manageable --provideCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Use the
--all-manageableoption to introspect only the nodes that are in a managed state. In this example, all nodes are in a managed state. -
Use the
--provideoption to reset all nodes to anavailablestate after introspection.
-
Use the
Monitor the introspection progress logs in a separate terminal window:
(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/ironic-inspector.log
(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/ironic-inspector.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantEnsure that this process runs to completion. This process usually takes 15 minutes for bare metal nodes.
After the introspection completes, all nodes change to an available state.
8.3. Provisioning bare metal nodes Copy linkLink copied to clipboard!
Create a new YAML file ~/overcloud-baremetal-deploy.yaml, define the quantity and attributes of the bare metal nodes that you want to deploy, and assign overcloud roles to these nodes. The provisioning process creates a heat environment file that you can include in your openstack overcloud deploy command.
Prerequisites
- A successful undercloud installation. For more information, see Section 4.7, “Installing director”.
- Bare metal nodes introspected and available for provisioning and deployment. For more information, see Section 8.1, “Registering nodes for the overcloud” and Section 8.2, “Inspecting the hardware of nodes”.
Procedure
Source the
stackrcundercloud credential file:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new
~/overcloud-baremetal-deploy.yamlfile and define the node count for each role that you want to provision. For example, to provision three Controller nodes and three Compute nodes, use the following syntax:- name: Controller count: 3 - name: Compute count: 3
- name: Controller count: 3 - name: Compute count: 3Copy to Clipboard Copied! Toggle word wrap Toggle overflow In the
~/overcloud-baremetal-deploy.yamlfile, define any predictive node placements, custom images, custom NICs, or other attributes that you want to assign to your nodes. For example, use the following example syntax to provision three Controller nodes on nodesnode00,node01, andnode02, and three Compute nodes onnode04,node05, andnode06:Copy to Clipboard Copied! Toggle word wrap Toggle overflow By default, the provisioning process uses the
overcloud-fullimage. You can use theimageattribute in theinstancesparameter to define a custom image:Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can also override the default parameter values with the
defaultsparameter to avoid manual node definitions for each node entry:Copy to Clipboard Copied! Toggle word wrap Toggle overflow For more information about the parameters, attributes, and values that you can use in your node definition file, see Section 8.6, “Bare metal node provisioning attributes”.
Run the provisioning command, specifying the
~/overcloud-baremetal-deploy.yamlfile and defining an output file with the--outputoption:(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yaml
(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow The provisioning process generates a heat environment file with the name that you specify in the
--outputoption. This file contains your node definitions. When you deploy the overcloud, include this file in the deployment command.In a separate terminal, monitor your nodes to verify that they provision successfully. The provisioning process changes the node state from
availabletoactive:(undercloud) $ watch openstack baremetal node list
(undercloud) $ watch openstack baremetal node listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
metalsmithtool to obtain a unified view of your nodes, including allocations and neutron ports:(undercloud) $ metalsmith list
(undercloud) $ metalsmith listCopy to Clipboard Copied! Toggle word wrap Toggle overflow You can also use the
openstack baremetal allocationcommand to verify association of nodes to hostnames, and to obtain IP addresses for the provisioned nodes:(undercloud) $ openstack baremetal allocation list
(undercloud) $ openstack baremetal allocation listCopy to Clipboard Copied! Toggle word wrap Toggle overflow
When your nodes are provisioned successfully, you can deploy the overcloud. For more information, see Chapter 9, Configuring a basic overcloud with pre-provisioned nodes.
8.4. Scaling up bare metal nodes Copy linkLink copied to clipboard!
To increase the count of bare metal nodes in an existing overcloud, increment the node count in the ~/overcloud-baremetal-deploy.yaml file and redeploy the overcloud.
Prerequisites
- A successful undercloud installation. For more information, see Section 4.7, “Installing director”.
- A successful overcloud deployment. For more information, see Chapter 9, Configuring a basic overcloud with pre-provisioned nodes.
- Bare metal nodes introspected and available for provisioning and deployment. For more information, see Section 8.1, “Registering nodes for the overcloud” and Section 8.2, “Inspecting the hardware of nodes”.
Procedure
Source the
stackrcundercloud credential file:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
~/overcloud-baremetal-deploy.yamlfile that you used to provision your bare metal nodes, and increment thecountparameter for the roles that you want to scale up. For example, if your overcloud contains three Compute nodes, use the following snippet to increase the Compute node count to 10:- name: Controller count: 3 - name: Compute count: 10
- name: Controller count: 3 - name: Compute count: 10Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can also add predictive node placement with the
instancesparameter. For more information about the parameters and attributes that are available, see Section 8.6, “Bare metal node provisioning attributes”.Run the provisioning command, specifying the
~/overcloud-baremetal-deploy.yamlfile and defining an output file with the--outputoption:(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yaml
(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Monitor the provisioning progress with the
openstack baremetal node listcommand. Deploy the overcloud, including the
~/overcloud-baremetal-deployed.yamlfile that the provisioning command generates, along with any other environment files relevant to your deployment:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
8.5. Scaling down bare metal nodes Copy linkLink copied to clipboard!
Tag the nodes that you want to delete from the stack in the ~/overcloud-baremetal-deploy.yaml file, redeploy the overcloud, and then include this file in the openstack overcloud node delete command with the --baremetal-deployment option.
Prerequisites
- A successful undercloud installation. For more information, see Section 4.7, “Installing director”.
- A successful overcloud deployment. For more information, see Chapter 9, Configuring a basic overcloud with pre-provisioned nodes.
- At least one bare metal node that you want to remove from the stack.
Procedure
Source the
stackrcundercloud credential file:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
~/overcloud-baremetal-deploy.yamlfile that you used to provision your bare metal nodes, and decrement thecountparameter for the roles that you want to scale down. You must also define the following attributes for each node that you want to remove from the stack:- The name of the node.
- The hostname that is associated with the node.
The attribute
provisioned: false.For example, to remove the node
overcloud-controller-1from the stack, include the following snippet in your~/overcloud-baremetal-deploy.yamlfile:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Run the provisioning command, specifying the
~/overcloud-baremetal-deploy.yamlfile and defining an output file with the--outputoption:(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yaml
(undercloud) $ sudo openstack overcloud node provision \ --stack stack \ --output ~/overcloud-baremetal-deployed.yaml \ ~/overcloud-baremetal-deploy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Redeploy the overcloud and include the
~/overcloud-baremetal-deployed.yamlfile that the provisioning command generates, along with any other environment files relevant to your deployment:Copy to Clipboard Copied! Toggle word wrap Toggle overflow After you redeploy the overcloud, the nodes that you define with the
provisioned: falseattribute are no longer present in the stack. However, these nodes are still running in a provisioned state.NoteIf you want to remove a node from the stack temporarily, you can deploy the overcloud with the attribute
provisioned: falseand then redeploy the overcloud with the attributeprovisioned: trueto return the node to the stack.Run the
openstack overcloud node deletecommand, including the~/overcloud-baremetal-deploy.yamlfile with the--baremetal-deploymentoption.(undercloud) $ sudo openstack overcloud node delete \ --stack stack \ --baremetal-deployment ~/overcloud-baremetal-deploy.yaml
(undercloud) $ sudo openstack overcloud node delete \ --stack stack \ --baremetal-deployment ~/overcloud-baremetal-deploy.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteDo not include the nodes that you want to remove from the stack as command arguments in the
openstack overcloud node deletecommand.
8.6. Bare metal node provisioning attributes Copy linkLink copied to clipboard!
Use the following tables to understand the parameters, attributes, and values that are available for you to use when you provision bare metal nodes with the openstack baremetal node provision command.
| Parameter | Value |
|---|---|
| name | Mandatory role name |
| count |
The number of nodes that you want to provision for this role. The default value is |
| defaults |
A dictionary of default values for |
| instances |
A dictionary of values that you can use to specify attributes for specific nodes. For more information about supported properties in the |
| hostname_format |
Overrides the default hostname format for this role. The default format uses the lower case role name. For example, the default format for the Controller role is |
Example syntax
In the following example, the name refers to the logical name of the node, and the hostname refers to the generated hostname which is derived from the overcloud stack name, the role, and an incrementing index. All Controller servers use a default custom image overcloud-full-custom and are on predictive nodes. One of the Compute servers is placed predictively on node04 with custom host name overcloud-compute-special, and the other 99 Compute servers are on nodes allocated automatically from the pool of available nodes:
| Parameter | Value |
|---|---|
| hostname |
If the hostname complies with the |
| name | The name of the node that you want to provision. |
| image |
Details of the image that you want to provision onto the node. For more information about supported properties in the |
| capabilities | Selection criteria to match the node capabilities. |
| nics |
List of dictionaries that represent requested NICs. For more information about supported properties in the |
| profile | Selection criteria to use Advanced Profile Matching. |
| provisioned |
Boolean to determine whether this node is provisioned or unprovisioned. The default value is |
| resource_class |
Selection criteria to match the resource class of the node. The default value is |
| root_size_gb |
Size of the root partition in GiB. The default value is |
| swap_size_mb | Size of the swap partition in MiB. |
| traits | A list of traits as selection criteria to match the node traits. |
Example syntax
In the following example, all Controller servers use a custom default overcloud image overcloud-full-custom. The Controller server overcloud-controller-0 is placed predictively on node00 and has custom root and swap sizes. The other two Controller servers are on nodes allocated automatically from the pool of available nodes, and have default root and swap sizes:
| Parameter | Value |
|---|---|
| href |
Glance image reference or URL of the root partition or whole disk image. URL schemes supported are |
| checksum | When the href is a URL, this value must be the SHA512 checksum of the root partition or whole disk image. |
| kernel | Glance image reference or URL of the kernel image. Use this property only for partition images. |
| ramdisk | Glance image reference or URL of the ramdisk image. Use this property only for partition images. |
Example syntax
In the following example, all three Controller servers are on nodes allocated automatically from the pool of available nodes. All Controller servers in this environment use a default custom image overcloud-full-custom:
| Parameter | Value |
|---|---|
| fixed_ip | The specific IP address that you want to use for this NIC. |
| network | The neutron network where you want to create the port for this NIC. |
| subnet | The neutron subnet where you want to create the port for this NIC. |
| port | Existing Neutron port to use instead of creating a new port. |
Example syntax
In the following example, all three Controller servers are on nodes allocated automatically from the pool of available nodes. All Controller servers in this environment use a default custom image overcloud-full-custom and have specific networking requirements:
Chapter 9. Configuring a basic overcloud with pre-provisioned nodes Copy linkLink copied to clipboard!
This chapter contains basic configuration procedures that you can use to configure a Red Hat OpenStack Platform (RHOSP) environment with pre-provisioned nodes. This scenario differs from the standard overcloud creation scenarios in several ways:
- You can provision nodes with an external tool and let the director control the overcloud configuration only.
- You can use nodes without relying on the director provisioning methods. This is useful if you want to create an overcloud without power management control, or use networks with DHCP/PXE boot restrictions.
- The director does not use OpenStack Compute (nova), OpenStack Bare Metal (ironic), or OpenStack Image (glance) to manage nodes.
- Pre-provisioned nodes can use a custom partitioning layout that does not rely on the QCOW2 overcloud-full image.
This scenario includes only basic configuration with no custom features. However, you can add advanced configuration options to this basic overcloud and customize it to your specifications with the instructions in the Advanced Overcloud Customization guide.
You cannot combine pre-provisioned nodes with director-provisioned nodes.
9.1. Pre-provisioned node requirements Copy linkLink copied to clipboard!
Before you begin deploying an overcloud with pre-provisioned nodes, ensure that the following configuration is present in your environment:
- The director node that you created in Chapter 4, Installing director.
- A set of bare metal machines for your nodes. The number of nodes required depends on the type of overcloud you intend to create. These machines must comply with the requirements set for each node type. These nodes require Red Hat Enterprise Linux 8.1 or later installed as the host operating system. Red Hat recommends using the latest version available.
- One network connection for managing the pre-provisioned nodes. This scenario requires uninterrupted SSH access to the nodes for orchestration agent configuration.
One network connection for the Control Plane network. There are two main scenarios for this network:
Using the Provisioning Network as the Control Plane, which is the default scenario. This network is usually a layer-3 (L3) routable network connection from the pre-provisioned nodes to director. The examples for this scenario use following IP address assignments:
Expand Table 9.1. Provisioning Network IP assignments Node name IP address Director
192.168.24.1
Controller 0
192.168.24.2
Compute 0
192.168.24.3
- Using a separate network. In situations where the director’s Provisioning network is a private non-routable network, you can define IP addresses for nodes from any subnet and communicate with director over the Public API endpoint. For more information about the requirements for this scenario, see Section 9.6, “Using a separate network for pre-provisioned nodes”.
- All other network types in this example also use the Control Plane network for OpenStack services. However, you can create additional networks for other network traffic types.
-
If any nodes use Pacemaker resources, the service user
haclusterand the service grouphaclientmust have a UID/GID of 189. This is due to CVE-2018-16877. If you installed Pacemaker together with the operating system, the installation creates these IDs automatically. If the ID values are set incorrectly, follow the steps in the article OpenStack minor update / fast-forward upgrade can fail on the controller nodes at pacemaker step with "Could not evaluate: backup_cib" to change the ID values. -
To prevent some services from binding to an incorrect IP address and causing deployment failures, make sure that the
/etc/hostsfile does not include thenode-name=127.0.0.1mapping.
9.2. Creating a user on pre-provisioned nodes Copy linkLink copied to clipboard!
When you configure an overcloud with pre-provisioned nodes, director requires SSH access to the overcloud nodes as the stack user. To create the stack user, complete the following steps.
Procedure
On each overcloud node, create the
stackuser and set a password. For example, run the following commands on the Controller node:useradd stack passwd stack # specify a password
[root@controller-0 ~]# useradd stack [root@controller-0 ~]# passwd stack # specify a passwordCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disable password requirements for this user when using
sudo:echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack chmod 0440 /etc/sudoers.d/stack
[root@controller-0 ~]# echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack [root@controller-0 ~]# chmod 0440 /etc/sudoers.d/stackCopy to Clipboard Copied! Toggle word wrap Toggle overflow After you create and configure the
stackuser on all pre-provisioned nodes, copy thestackuser’s public SSH key from the director node to each overcloud node. For example, to copy the director’s public SSH key to the Controller node, run the following command:ssh-copy-id stack@192.168.24.2
[stack@director ~]$ ssh-copy-id stack@192.168.24.2Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.3. Registering the operating system for pre-provisioned nodes Copy linkLink copied to clipboard!
Each node requires access to a Red Hat subscription. Complete the following steps on each node to register your nodes with the Red Hat Content Delivery Network.
Procedure
Run the registration command and enter your Customer Portal user name and password when prompted:
sudo subscription-manager register
[root@controller-0 ~]# sudo subscription-manager registerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Find the entitlement pool for the Red Hat OpenStack Platform 16:
sudo subscription-manager list --available --all --matches="Red Hat OpenStack"
[root@controller-0 ~]# sudo subscription-manager list --available --all --matches="Red Hat OpenStack"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the pool ID located in the previous step to attach the Red Hat OpenStack Platform 16 entitlements:
sudo subscription-manager attach --pool=pool_id
[root@controller-0 ~]# sudo subscription-manager attach --pool=pool_idCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disable all default repositories:
sudo subscription-manager repos --disable=*
[root@controller-0 ~]# sudo subscription-manager repos --disable=*Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enable the required Red Hat Enterprise Linux repositories.
For x86_64 systems, run:
sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=rhceph-4-osd-for-rhel-8-x86_64-rpms --enable=rhceph-4-mon-for-rhel-8-x86_64-rpms --enable=rhceph-4-tools-for-rhel-8-x86_64-rpms --enable=advanced-virt-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms
[root@controller-0 ~]# sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-eus-rpms --enable=rhel-8-for-x86_64-appstream-eus-rpms --enable=rhel-8-for-x86_64-highavailability-eus-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms --enable=openstack-16-for-rhel-8-x86_64-rpms --enable=rhceph-4-osd-for-rhel-8-x86_64-rpms --enable=rhceph-4-mon-for-rhel-8-x86_64-rpms --enable=rhceph-4-tools-for-rhel-8-x86_64-rpms --enable=advanced-virt-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpmsCopy to Clipboard Copied! Toggle word wrap Toggle overflow For POWER systems, run:
sudo subscription-manager repos --enable=rhel-8-for-ppc64le-baseos-rpms --enable=rhel-8-for-ppc64le-appstream-rpms --enable=rhel-8-for-ppc64le-highavailability-rpms --enable=ansible-2.8-for-rhel-8-ppc64le-rpms --enable=openstack-16-for-rhel-8-ppc64le-rpms --enable=advanced-virt-for-rhel-8-ppc64le-rpms
[root@controller-0 ~]# sudo subscription-manager repos --enable=rhel-8-for-ppc64le-baseos-rpms --enable=rhel-8-for-ppc64le-appstream-rpms --enable=rhel-8-for-ppc64le-highavailability-rpms --enable=ansible-2.8-for-rhel-8-ppc64le-rpms --enable=openstack-16-for-rhel-8-ppc64le-rpms --enable=advanced-virt-for-rhel-8-ppc64le-rpmsCopy to Clipboard Copied! Toggle word wrap Toggle overflow
ImportantEnable only the repositories listed. Additional repositories can cause package and software conflicts. Do not enable any additional repositories.
Update your system to ensure you have the latest base system packages:
sudo dnf update -y sudo reboot
[root@controller-0 ~]# sudo dnf update -y [root@controller-0 ~]# sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow
The node is now ready to use for your overcloud.
9.4. Configuring SSL/TLS access to director Copy linkLink copied to clipboard!
If the director uses SSL/TLS, the pre-provisioned nodes require the certificate authority file used to sign the director’s SSL/TLS certificates. If you use your own certificate authority, perform the following actions on each overcloud node.
Procedure
-
Copy the certificate authority file to the
/etc/pki/ca-trust/source/anchors/directory on each pre-provisioned node. Run the following command on each overcloud node:
sudo update-ca-trust extract
[root@controller-0 ~]# sudo update-ca-trust extractCopy to Clipboard Copied! Toggle word wrap Toggle overflow
These steps ensure that the overcloud nodes can access the director’s Public API over SSL/TLS.
9.5. Configuring networking for the control plane Copy linkLink copied to clipboard!
The pre-provisioned overcloud nodes obtain metadata from director using standard HTTP requests. This means all overcloud nodes require L3 access to either:
-
The director Control Plane network, which is the subnet that you define with the
network_cidrparameter in yourundercloud.conffile. The overcloud nodes require either direct access to this subnet or routable access to the subnet. -
The director Public API endpoint, that you specify with the
undercloud_public_hostparameter in yourundercloud.conffile. This option is available if you do not have an L3 route to the Control Plane or if you want to use SSL/TLS communication. For more information about configuring your overcloud nodes to use the Public API endpoint, see Section 9.6, “Using a separate network for pre-provisioned nodes”.
Director uses the Control Plane network to manage and configure a standard overcloud. For an overcloud with pre-provisioned nodes, your network configuration might require some modification to accommodate communication between the director and the pre-provisioned nodes.
Using network isolation
You can use network isolation to group services to use specific networks, including the Control Plane. There are multiple network isolation strategies in the the Advanced Overcloud Customization guide. You can also define specific IP addresses for nodes on the Control Plane. For more information about isolating networks and creating predictable node placement strategies, see the following sections in the Advanced Overcloud Customizations guide:
If you use network isolation, ensure that your NIC templates do not include the NIC used for undercloud access. These templates can reconfigure the NIC, which introduces connectivity and configuration problems during deployment.
Assigning IP addresses
If you do not use network isolation, you can use a single Control Plane network to manage all services. This requires manual configuration of the Control Plane NIC on each node to use an IP address within the Control Plane network range. If you are using the director Provisioning network as the Control Plane, ensure that the overcloud IP addresses that you choose are outside of the DHCP ranges for both provisioning (dhcp_start and dhcp_end) and introspection (inspection_iprange).
During standard overcloud creation, director creates OpenStack Networking (neutron) ports and automatically assigns IP addresses to the overcloud nodes on the Provisioning / Control Plane network. However, this can cause director to assign different IP addresses to the ones that you configure manually for each node. In this situation, use a predictable IP address strategy to force director to use the pre-provisioned IP assignments on the Control Plane.
For example, you can use an environment file ctlplane-assignments.yaml with the following IP assignments to implement a predictable IP strategy:
In this example, the OS::TripleO::DeployedServer::ControlPlanePort resource passes a set of parameters to director and defines the IP assignments of your pre-provisioned nodes. Use the DeployedServerPortMap parameter to define the IP addresses and subnet CIDRs that correspond to each overcloud node. The mapping defines the following attributes:
-
The name of the assignment, which follows the format
<node_hostname>-<network>where the<node_hostname>value matches the short host name for the node, and<network>matches the lowercase name of the network. For example:controller-0-ctlplaneforcontroller-0.example.comandcompute-0-ctlplaneforcompute-0.example.com. The IP assignments, which use the following parameter patterns:
-
fixed_ips/ip_address- Defines the fixed IP addresses for the control plane. Use multipleip_addressparameters in a list to define multiple IP addresses. -
subnets/cidr- Defines the CIDR value for the subnet.
-
A later section in this chapter uses the resulting environment file (ctlplane-assignments.yaml) as part of the openstack overcloud deploy command.
9.6. Using a separate network for pre-provisioned nodes Copy linkLink copied to clipboard!
By default, director uses the Provisioning network as the overcloud Control Plane. However, if this network is isolated and non-routable, nodes cannot communicate with the director Internal API during configuration. In this situation, you might need to define a separate network for the nodes and configure them to communicate with the director over the Public API.
There are several requirements for this scenario:
- The overcloud nodes must accommodate the basic network configuration from Section 9.5, “Configuring networking for the control plane”.
- You must enable SSL/TLS on the director for Public API endpoint usage. For more information, see Section 4.2, “Director configuration parameters” and Chapter 18, Configuring custom SSL/TLS certificates.
-
You must define an accessible fully qualified domain name (FQDN) for director. This FQDN must resolve to a routable IP address for the director. Use the
undercloud_public_hostparameter in theundercloud.conffile to set this FQDN.
The examples in this section use IP address assignments that differ from the main scenario:
| Node Name | IP address or FQDN |
|---|---|
| Director (Internal API) | 192.168.24.1 (Provisioning Network and Control Plane) |
| Director (Public API) | 10.1.1.1 / director.example.com |
| Overcloud Virtual IP | 192.168.100.1 |
| Controller 0 | 192.168.100.2 |
| Compute 0 | 192.168.100.3 |
The following sections provide additional configuration for situations that require a separate network for overcloud nodes.
IP address assignments
The method for IP assignments is similar to Section 9.5, “Configuring networking for the control plane”. However, since the Control Plane is not routable from the deployed servers, you must use the DeployedServerPortMap parameter to assign IP addresses from your chosen overcloud node subnet, including the virtual IP address to access the Control Plane. The following example is a modified version of the ctlplane-assignments.yaml environment file from Section 9.5, “Configuring networking for the control plane” that accommodates this network architecture:
- 1
- The
RedisVipPortresource is mapped tonetwork/ports/noop.yaml. This mapping is necessary because the default Redis VIP address comes from the Control Plane. In this situation, use anoopto disable this Control Plane mapping. - 2
- The
EC2MetadataIpandControlPlaneDefaultRouteparameters are set to the value of the Control Plane virtual IP address. The default NIC configuration templates require these parameters and you must set them to use a pingable IP address to pass the validations performed during deployment. Alternatively, customize the NIC configuration so that they do not require these parameters.
9.7. Mapping pre-provisioned node hostnames Copy linkLink copied to clipboard!
When you configure pre-provisioned nodes, you must map heat-based hostnames to their actual hostnames so that ansible-playbook can reach a resolvable host. Use the HostnameMap to map these values.
Procedure
Create an environment file, for example
hostname-map.yaml, and include theHostnameMapparameter and the hostname mappings. Use the following syntax:parameter_defaults: HostnameMap: [HEAT HOSTNAME]: [ACTUAL HOSTNAME] [HEAT HOSTNAME]: [ACTUAL HOSTNAME]parameter_defaults: HostnameMap: [HEAT HOSTNAME]: [ACTUAL HOSTNAME] [HEAT HOSTNAME]: [ACTUAL HOSTNAME]Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
[HEAT HOSTNAME]usually conforms to the following convention:[STACK NAME]-[ROLE]-[INDEX]:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Save the
hostname-map.yamlfile.
9.8. Configuring Ceph Storage for pre-provisioned nodes Copy linkLink copied to clipboard!
Complete the following steps on the undercloud host to configure ceph-ansible for nodes that are already deployed.
Procedure
On the undercloud host, create an environment variable,
OVERCLOUD_HOSTS, and set the variable to a space-separated list of IP addresses of the overcloud hosts that you want to use as Ceph clients:export OVERCLOUD_HOSTS="192.168.1.8 192.168.1.42"
export OVERCLOUD_HOSTS="192.168.1.8 192.168.1.42"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
enable-ssh-admin.shscript to configure a user on the overcloud nodes that Ansible can use to configure Ceph clients:bash /usr/share/openstack-tripleo-heat-templates/deployed-server/scripts/enable-ssh-admin.sh
bash /usr/share/openstack-tripleo-heat-templates/deployed-server/scripts/enable-ssh-admin.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow
When you run the openstack overcloud deploy command, Ansible configures the hosts that you define in the OVERCLOUD_HOSTS variable as Ceph clients.
9.9. Creating the overcloud with pre-provisioned nodes Copy linkLink copied to clipboard!
The overcloud deployment uses the standard CLI methods from Section 7.13, “Deployment command”. For pre-provisioned nodes, the deployment command requires some additional options and environment files from the core heat template collection:
-
--disable-validations- Use this option to disable basic CLI validations for services not used with pre-provisioned infrastructure. If you do not disable these validations, the deployment fails. -
environments/deployed-server-environment.yaml- Include this environment file to create and configure the pre-provisioned infrastructure. This environment file substitutes theOS::Nova::Serverresources withOS::Heat::DeployedServerresources.
The following command is an example overcloud deployment command with the environment files specific to the pre-provisioned architecture:
The --overcloud-ssh-user and --overcloud-ssh-key options are used to SSH into each overcloud node during the configuration stage, create an initial tripleo-admin user, and inject an SSH key into /home/tripleo-admin/.ssh/authorized_keys. To inject the SSH key, specify the credentials for the initial SSH connection with --overcloud-ssh-user and --overcloud-ssh-key (defaults to ~/.ssh/id_rsa). To limit exposure to the private key that you specify with the --overcloud-ssh-key option, director never passes this key to any API service, such as heat or the Workflow service (mistral), and only the director openstack overcloud deploy command uses this key to enable access for the tripleo-admin user.
9.10. Overcloud deployment output Copy linkLink copied to clipboard!
When the overcloud creation completes, director provides a recap of the Ansible plays that were executed to configure the overcloud:
Director also provides details to access your overcloud.
9.11. Accessing the overcloud Copy linkLink copied to clipboard!
The director generates a script to configure and help authenticate interactions with your overcloud from the undercloud. The director saves this file, overcloudrc, in the home directory of the stack user. Run the following command to use this file:
(undercloud) $ source ~/overcloudrc
(undercloud) $ source ~/overcloudrc
This command loads the environment variables that are necessary to interact with your overcloud from the undercloud CLI. The command prompt changes to indicate this:
(overcloud) $
(overcloud) $
To return to interacting with the undercloud, run the following command:
(overcloud) $ source ~/stackrc (undercloud) $
(overcloud) $ source ~/stackrc
(undercloud) $
Each node in the overcloud also contains a heat-admin user. The stack user has SSH access to this user on each node. To access a node over SSH, find the IP address of the node that you want to access:
(undercloud) $ openstack server list
(undercloud) $ openstack server list
Then connect to the node using the heat-admin user and the IP address of the node:
(undercloud) $ ssh heat-admin@192.168.24.23
(undercloud) $ ssh heat-admin@192.168.24.23
9.12. Scaling pre-provisioned nodes Copy linkLink copied to clipboard!
The process for scaling pre-provisioned nodes is similar to the standard scaling procedures in Chapter 15, Scaling overcloud nodes. However, the process to add new pre-provisioned nodes differs because pre-provisioned nodes do not use the standard registration and management process from OpenStack Bare Metal (ironic) and OpenStack Compute (nova).
Scaling up pre-provisioned nodes
When scaling up the overcloud with pre-provisioned nodes, you must configure the orchestration agent on each node to correspond to the director node count.
Perform the following actions to scale up overcloud nodes:
- Prepare the new pre-provisioned nodes according to Section 9.1, “Pre-provisioned node requirements”.
- Scale up the nodes. For more information, see Chapter 15, Scaling overcloud nodes.
- After you execute the deployment command, wait until the director creates the new node resources and launches the configuration.
Scaling down pre-provisioned nodes
When scaling down the overcloud with pre-provisioned nodes, follow the scale down instructions in Chapter 15, Scaling overcloud nodes.
In most scaling operations, you must obtain the UUID value of the node that you want to remove and pass this value to the openstack overcloud node delete command. To obtain this UUID, list the resources for the specific role:
openstack stack resource list overcloud -c physical_resource_id -c stack_name -n5 --filter type=OS::TripleO::<RoleName>Server
$ openstack stack resource list overcloud -c physical_resource_id -c stack_name -n5 --filter type=OS::TripleO::<RoleName>Server
Replace <RoleName> with the name of the role that you want to scale down. For example, for the ComputeDeployedServer role, run the following command:
openstack stack resource list overcloud -c physical_resource_id -c stack_name -n5 --filter type=OS::TripleO::ComputeDeployedServerServer
$ openstack stack resource list overcloud -c physical_resource_id -c stack_name -n5 --filter type=OS::TripleO::ComputeDeployedServerServer
Use the stack_name column in the command output to identify the UUID associated with each node. The stack_name includes the integer value of the index of the node in the heat resource group:
The indices 0, 1, or 2 in the stack_name column correspond to the node order in the heat resource group. Pass the corresponding UUID value from the physical_resource_id column to openstack overcloud node delete command.
After you remove overcloud nodes from the stack, power off these nodes. In a standard deployment, the bare metal services on the director control this function. However, with pre-provisioned nodes, you must either manually shut down these nodes or use the power management control for each physical system. If you do not power off the nodes after removing them from the stack, they might remain operational and reconnect as part of the overcloud environment.
After you power off the removed nodes, reprovision them to a base operating system configuration so that they do not unintentionally join the overcloud in the future
Do not attempt to reuse nodes previously removed from the overcloud without first reprovisioning them with a fresh base operating system. The scale down process only removes the node from the overcloud stack and does not uninstall any packages.
9.13. Removing a pre-provisioned overcloud Copy linkLink copied to clipboard!
To remove an entire overcloud that uses pre-provisioned nodes, see Section 12.5, “Removing the overcloud” for the standard overcloud remove procedure. After you remove the overcloud, power off all nodes and reprovision them to a base operating system configuration.
Do not attempt to reuse nodes previously removed from the overcloud without first reprovisioning them with a fresh base operating system. The removal process only deletes the overcloud stack and does not uninstall any packages.
9.14. Next steps Copy linkLink copied to clipboard!
This concludes the creation of the overcloud using pre-provisioned nodes. For post-creation functions, see Chapter 11, Performing overcloud post-installation tasks.
Chapter 10. Deploying multiple overclouds Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
You can use a single undercloud node to deploy and manage multiple overclouds. Each overcloud is a unique heat stack that does not share stack resources. This can be useful for environments where a 1:1 ratio of underclouds to overclouds creates an unmanageable amount of overhead. For example, Edge, multi-site, and multi-product environments.
The overcloud environments in the multi-overcloud scenario are completely separate, and you can use the source command to switch between the environments. If you use Ironic for bare metal provisioning, all overclouds must be on the same provisioning network. If it is not possible to use the same provisioning network, you can use the deployed servers method to deploy multiple overclouds with routed networks. In this scenario, you must ensure that the value in the HostnameMap parameter matches the stack name for each overcloud.
Use the following workflow to understand the basic process:
- Deploying the undercloud
- Deploy the undercloud as normal. For more information, see Part I, “Director installation and configuration”.
- Deploying the first overcloud
- Deploy the first overcloud as normal. For more information, see Part II, “Basic overcloud deployment”.
- Deploying additional overclouds
-
Create a new set of environment files for the new overcloud. Run the deployment command, and specify the core heat templates together with the new configuration files and a new
stackname.
10.1. Deploying additional overclouds Copy linkLink copied to clipboard!
In this example, overcloud-one is the existing overcloud. Complete the following steps to deploy a new overcloud overcloud-two.
Prerequisites
Before you begin to deploy additional overclouds, ensure that your environment contains the following configurations:
- Successful undercloud and overcloud deployments.
- Nodes available for your additional overcloud.
- Custom networks for additional overclouds so that each overcloud has a unique network in the resulting stack.
Procedure
Create a new directory for the additional overcloud that you want to deploy:
mkdir ~/overcloud-two
$ mkdir ~/overcloud-twoCopy to Clipboard Copied! Toggle word wrap Toggle overflow In the new directory, create new environment files specific to the requirements of the additional overcloud, and copy any relevant environment files from the existing overcloud:
cp network-data.yaml ~/overcloud-two/network-data.yaml cp network-environment.yaml ~/overcloud-two/network-environment.yaml
$ cp network-data.yaml ~/overcloud-two/network-data.yaml $ cp network-environment.yaml ~/overcloud-two/network-environment.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Modify the environment files according to the specification of the new overcloud. For example, the existing overcloud has the name
overcloud-oneand uses the VLANs that you define in thenetwork-data.yamlenvironment file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow The new overcloud has the name
overcloud-twoand uses different VLANs. Edit the~/overcloud-two/network-data.yamlenvironment file and include the new VLAN IDs for each subnet. You must also define a uniquename_lowervalue, and set theservice_net_map_replaceattribute to the name of the network that you want to replace:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Modify the following parameters in the
~/overcloud-two/network-environment.yamlfile:-
Enter a unique value in the
{'provider:physical_network'}attribute of theExternalNetValueSpecsparameter so thatovercloud-twohas a distinct external network, and define the network type with the'provider:network_type'attribute. -
Set the
ExternalInterfaceDefaultRouteparameter to the IP address of the gateway for the external network so that the overcloud has external access. Set the
DnsServersparameter to the IP address of your DNS server so that the overcloud can reach the DNS server.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
-
Enter a unique value in the
Run the
openstack overcloud deploycommand. Specify the core heat template collection with the--templatesoption, a newstackname with the--stackoption, and any new environment files from the~/overcloud-twodirectory:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Each overcloud has a unique credential file. In this example, the deployment process creates overcloud-onerc for overcloud-one, and overcloud-tworc for overcloud-two. To interact with either overcloud, you must source the appropriate credential file. For example, to source the credential for the first overcloud, run the following command:
source overcloud-onerc
$ source overcloud-onerc
10.2. Managing multiple overclouds Copy linkLink copied to clipboard!
Each overcloud that you deploy uses the same set of core heat templates /usr/share/openstack-tripleo-heat-templates. Red Hat recommends that you do not modify or duplicate these templates, because using a non-standard set of core templates can introduce issues with updates and upgrades.
Instead, for ease of management when you deploy or maintain multiple overclouds, create separate directories of environment files specific to each cloud. When you run the deploy command for each cloud, include the core heat templates together with the cloud-specific environment files that you create separately. For example, create the following directories for the undercloud and two overclouds:
~stack/undercloud- Contains the environment files specific to the undercloud.
~stack/overcloud-one- Contains the environment files specific to the first overcloud.
~stack/overcloud-two- Contains the environment files specific to the second overcloud.
When you deploy or redeploy overcloud-one or overcloud-two, include the core heat templates in the deploy command with the --templates option, and then specify any additional environment files from the cloud-specific environment file directories.
Alternatively, create a repository in a version control system and use branches for each deployment. For more information, see the Using Customized Core Heat Templates section of the Advanced Overcloud Customization guide.
Use the following command to view a list of overcloud plans that are available:
openstack overcloud plan list
$ openstack overcloud plan list
Use the following command to view a list of overclouds that are currently deployed:
openstack stack list
$ openstack stack list
Part III. Post deployment operations Copy linkLink copied to clipboard!
Chapter 11. Performing overcloud post-installation tasks Copy linkLink copied to clipboard!
This chapter contains information about tasks to perform immediately after you create your overcloud. These tasks ensure your overcloud is ready to use.
11.1. Checking overcloud deployment status Copy linkLink copied to clipboard!
To check the deployment status of the overcloud, use the openstack overcloud status command. This command returns the result of all deployment steps.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the deployment status command:
openstack overcloud status
$ openstack overcloud statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow The output of this command displays the status of the overcloud:
+-----------+---------------------+---------------------+-------------------+ | Plan Name | Created | Updated | Deployment Status | +-----------+---------------------+---------------------+-------------------+ | overcloud | 2018-05-03 21:24:50 | 2018-05-03 21:27:59 | DEPLOY_SUCCESS | +-----------+---------------------+---------------------+-------------------+
+-----------+---------------------+---------------------+-------------------+ | Plan Name | Created | Updated | Deployment Status | +-----------+---------------------+---------------------+-------------------+ | overcloud | 2018-05-03 21:24:50 | 2018-05-03 21:27:59 | DEPLOY_SUCCESS | +-----------+---------------------+---------------------+-------------------+Copy to Clipboard Copied! Toggle word wrap Toggle overflow If your overcloud uses a different name, use the
--planargument to select an overcloud with a different name:openstack overcloud status --plan my-deployment
$ openstack overcloud status --plan my-deploymentCopy to Clipboard Copied! Toggle word wrap Toggle overflow
11.2. Creating basic overcloud flavors Copy linkLink copied to clipboard!
Validation steps in this guide assume that your installation contains flavors. If you have not already created at least one flavor, complete the following steps to create a basic set of default flavors that have a range of storage and processing capabilities:
Procedure
Source the
overcloudrcfile:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack flavor createcommand to create a flavor. Use the following options to specify the hardware requirements for each flavor:- --disk
- Defines the hard disk space for a virtual machine volume.
- --ram
- Defines the RAM required for a virtual machine.
- --vcpus
- Defines the quantity of virtual CPUs for a virtual machine.
The following example creates the default overcloud flavors:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Use $ openstack flavor create --help to learn more about the openstack flavor create command.
11.3. Creating a default tenant network Copy linkLink copied to clipboard!
The overcloud requires a default Tenant network so that virtual machines can communicate internally.
Procedure
Source the
overcloudrcfile:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the default Tenant network:
(overcloud) $ openstack network create default
(overcloud) $ openstack network create defaultCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a subnet on the network:
(overcloud) $ openstack subnet create default --network default --gateway 172.20.1.1 --subnet-range 172.20.0.0/16
(overcloud) $ openstack subnet create default --network default --gateway 172.20.1.1 --subnet-range 172.20.0.0/16Copy to Clipboard Copied! Toggle word wrap Toggle overflow Confirm the created network:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
These commands create a basic Networking service (neutron) network named default. The overcloud automatically assigns IP addresses from this network to virtual machines using an internal DHCP mechanism.
11.4. Creating a default floating IP network Copy linkLink copied to clipboard!
To access your virtual machines from outside of the overcloud, you must configure an external network that provides floating IP addresses to your virtual machines.
This procedure contains two examples. Use the example that best suits your environment:
- Native VLAN (flat network)
- Non-Native VLAN (VLAN network)
Both of these examples involve creating a network with the name public. The overcloud requires this specific name for the default floating IP pool. This name is also important for the validation tests in Section 11.7, “Validating the overcloud”.
By default, Openstack Networking (neutron) maps a physical network name called datacentre to the the br-ex bridge on your host nodes. You connect the public overcloud network to the physical datacentre and this provides a gateway through the br-ex bridge.
Prerequisites
- A dedicated interface or native VLAN for the floating IP network.
Procedure
Source the
overcloudrcfile:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
publicnetwork:Create a
flatnetwork for a native VLAN connection:(overcloud) $ openstack network create public --external --provider-network-type flat --provider-physical-network datacentre
(overcloud) $ openstack network create public --external --provider-network-type flat --provider-physical-network datacentreCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
vlannetwork for non-native VLAN connections:(overcloud) $ openstack network create public --external --provider-network-type vlan --provider-physical-network datacentre --provider-segment 201
(overcloud) $ openstack network create public --external --provider-network-type vlan --provider-physical-network datacentre --provider-segment 201Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
--provider-segmentoption to define the VLAN that you want to use. In this example, the VLAN is201.
Create a subnet with an allocation pool for floating IP addresses. In this example, the IP range is
10.1.1.51to10.1.1.250:(overcloud) $ openstack subnet create public --network public --dhcp --allocation-pool start=10.1.1.51,end=10.1.1.250 --gateway 10.1.1.1 --subnet-range 10.1.1.0/24
(overcloud) $ openstack subnet create public --network public --dhcp --allocation-pool start=10.1.1.51,end=10.1.1.250 --gateway 10.1.1.1 --subnet-range 10.1.1.0/24Copy to Clipboard Copied! Toggle word wrap Toggle overflow Ensure that this range does not conflict with other IP addresses in your external network.
11.5. Creating a default provider network Copy linkLink copied to clipboard!
A provider network is another type of external network connection that routes traffic from private tenant networks to external infrastructure network. The provider network is similar to a floating IP network but the provider network uses a logical router to connect private networks to the provider network.
This procedure contains two examples. Use the example that best suits your environment:
- Native VLAN (flat network)
- Non-Native VLAN (VLAN network)
By default, Openstack Networking (neutron) maps a physical network name called datacentre to the the br-ex bridge on your host nodes. You connect the public overcloud network to the physical datacentre and this provides a gateway through the br-ex bridge.
Procedure
Source the
overcloudrcfile:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the
providernetwork:Create a
flatnetwork for a native VLAN connection:(overcloud) $ openstack network create provider --external --provider-network-type flat --provider-physical-network datacentre --share
(overcloud) $ openstack network create provider --external --provider-network-type flat --provider-physical-network datacentre --shareCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
vlannetwork for non-native VLAN connections:(overcloud) $ openstack network create provider --external --provider-network-type vlan --provider-physical-network datacentre --provider-segment 201 --share
(overcloud) $ openstack network create provider --external --provider-network-type vlan --provider-physical-network datacentre --provider-segment 201 --shareCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
--provider-segmentoption to define the VLAN that you want to use. In this example, the VLAN is201.
These example commands create a shared network. It is also possible to specify a tenant instead of specifying
--shareso that only the tenant has access to the new network.+ If you mark a provider network as external, only the operator may create ports on that network.
Add a subnet to the
providernetwork to provide DHCP services:(overcloud) $ openstack subnet create provider-subnet --network provider --dhcp --allocation-pool start=10.9.101.50,end=10.9.101.100 --gateway 10.9.101.254 --subnet-range 10.9.101.0/24
(overcloud) $ openstack subnet create provider-subnet --network provider --dhcp --allocation-pool start=10.9.101.50,end=10.9.101.100 --gateway 10.9.101.254 --subnet-range 10.9.101.0/24Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a router so that other networks can route traffic through the provider network:
(overcloud) $ openstack router create external
(overcloud) $ openstack router create externalCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set the external gateway for the router to the
providernetwork:(overcloud) $ openstack router set --external-gateway provider external
(overcloud) $ openstack router set --external-gateway provider externalCopy to Clipboard Copied! Toggle word wrap Toggle overflow Attach other networks to this router. For example, run the following command to attach a subnet
subnet1to the router:(overcloud) $ openstack router add subnet external subnet1
(overcloud) $ openstack router add subnet external subnet1Copy to Clipboard Copied! Toggle word wrap Toggle overflow This command adds
subnet1to the routing table and allows traffic from virtual machines usingsubnet1to route to the provider network.
11.6. Creating additional bridge mappings Copy linkLink copied to clipboard!
Floating IP networks can use any bridge, not just br-ex, provided that you complete the following prerequisite actions:
-
Set the
NeutronExternalNetworkBridgeparameter to"''"in your network environment file. Map the additional bridge during deployment. For example, to map a new bridge called
br-floatingto thefloatingphysical network, include theNeutronBridgeMappingsparameter in an environment file:parameter_defaults: NeutronBridgeMappings: "datacentre:br-ex,floating:br-floating"
parameter_defaults: NeutronBridgeMappings: "datacentre:br-ex,floating:br-floating"Copy to Clipboard Copied! Toggle word wrap Toggle overflow
With this method, you can create separate external networks after creating the overcloud. For example, to create a floating IP network that maps to the floating physical network, run the following commands:
source ~/overcloudrc (overcloud) $ openstack network create public --external --provider-physical-network floating --provider-network-type vlan --provider-segment 105 (overcloud) $ openstack subnet create public --network public --dhcp --allocation-pool start=10.1.2.51,end=10.1.2.250 --gateway 10.1.2.1 --subnet-range 10.1.2.0/24
$ source ~/overcloudrc
(overcloud) $ openstack network create public --external --provider-physical-network floating --provider-network-type vlan --provider-segment 105
(overcloud) $ openstack subnet create public --network public --dhcp --allocation-pool start=10.1.2.51,end=10.1.2.250 --gateway 10.1.2.1 --subnet-range 10.1.2.0/24
11.7. Validating the overcloud Copy linkLink copied to clipboard!
The overcloud uses the OpenStack Integration Test Suite (tempest) tool set to conduct a series of integration tests. This section contains information about preparations for running the integration tests. For full instructions about how to use the OpenStack Integration Test Suite, see the OpenStack Integration Test Suite Guide.
The Integration Test Suite requires a few post-installation steps to ensure successful tests.
Procedure
If you run this test from the undercloud, ensure that the undercloud host has access to the Internal API network on the overcloud. For example, add a temporary VLAN on the undercloud host to access the Internal API network (ID: 201) using the 172.16.0.201/24 address:
source ~/stackrc (undercloud) $ sudo ovs-vsctl add-port br-ctlplane vlan201 tag=201 -- set interface vlan201 type=internal (undercloud) $ sudo ip l set dev vlan201 up; sudo ip addr add 172.16.0.201/24 dev vlan201
$ source ~/stackrc (undercloud) $ sudo ovs-vsctl add-port br-ctlplane vlan201 tag=201 -- set interface vlan201 type=internal (undercloud) $ sudo ip l set dev vlan201 up; sudo ip addr add 172.16.0.201/24 dev vlan201Copy to Clipboard Copied! Toggle word wrap Toggle overflow Before you run the OpenStack Integration Test Suite, ensure that the
heat_stack_ownerrole exists in your overcloud:Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the role does not exist, create it:
(overcloud) $ openstack role create heat_stack_owner
(overcloud) $ openstack role create heat_stack_ownerCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Run the integration tests as described in the OpenStack Integration Test Suite Guide.
After completing the validation, remove any temporary connections to the overcloud Internal API. In this example, use the following commands to remove the previously created VLAN on the undercloud:
source ~/stackrc (undercloud) $ sudo ovs-vsctl del-port vlan201
$ source ~/stackrc (undercloud) $ sudo ovs-vsctl del-port vlan201Copy to Clipboard Copied! Toggle word wrap Toggle overflow
11.8. Protecting the overcloud from removal Copy linkLink copied to clipboard!
Heat contains a set of default policies in code that you can override by creating the /var/lib/config-data/puppet-generated/heat/etc/heat/policy.json file and adding customized rules. Add the following policy to deny all users the permissions necessary to delete the overcloud.
{"stacks:delete": "rule:deny_everybody"}
{"stacks:delete": "rule:deny_everybody"}
This prevents removal of the overcloud with the heat client. To allow removal of the overcloud, delete the custom policy and save /var/lib/config-data/puppet-generated/heat/etc/heat/policy.json.
Chapter 12. Performing basic overcloud administration tasks Copy linkLink copied to clipboard!
This chapter contains information about basic tasks you might need to perform during the lifecycle of your overcloud.
12.1. Managing containerized services Copy linkLink copied to clipboard!
Red Hat OpenStack Platform (RHOSP) runs services in containers on the undercloud and overcloud nodes. In certain situations, you might need to control the individual services on a host. This section contains information about some common commands you can run on a node to manage containerized services.
Listing containers and images
To list running containers, run the following command:
sudo podman ps
$ sudo podman ps
To include stopped or failed containers in the command output, add the --all option to the command:
sudo podman ps --all
$ sudo podman ps --all
To list container images, run the following command:
sudo podman images
$ sudo podman images
Inspecting container properties
To view the properties of a container or container images, use the podman inspect command. For example, to inspect the keystone container, run the following command:
sudo podman inspect keystone
$ sudo podman inspect keystone
Managing containers with Systemd services
Previous versions of OpenStack Platform managed containers with Docker and its daemon. In OpenStack Platform 15, the Systemd services interface manages the lifecycle of the containers. Each container is a service and you run Systemd commands to perform specific operations for each container.
It is not recommended to use the Podman CLI to stop, start, and restart containers because Systemd applies a restart policy. Use Systemd service commands instead.
To check a container status, run the systemctl status command:
To stop a container, run the systemctl stop command:
sudo systemctl stop tripleo_keystone
$ sudo systemctl stop tripleo_keystone
To start a container, run the systemctl start command:
sudo systemctl start tripleo_keystone
$ sudo systemctl start tripleo_keystone
To restart a container, run the systemctl restart command:
sudo systemctl restart tripleo_keystone
$ sudo systemctl restart tripleo_keystone
Because no daemon monitors the containers status, Systemd automatically restarts most containers in these situations:
-
Clean exit code or signal, such as running
podman stopcommand. - Unclean exit code, such as the podman container crashing after a start.
- Unclean signals.
- Timeout if the container takes more than 1m 30s to start.
For more information about Systemd services, see the systemd.service documentation.
Any changes to the service configuration files within the container revert after restarting the container. This is because the container regenerates the service configuration based on files on the local file system of the node in /var/lib/config-data/puppet-generated/. For example, if you edit /etc/keystone/keystone.conf within the keystone container and restart the container, the container regenerates the configuration using /var/lib/config-data/puppet-generated/keystone/etc/keystone/keystone.conf on the local file system of the node, which overwrites any the changes that were made within the container before the restart.
Monitoring podman containers with Systemd timers
The Systemd timers interface manages container health checks. Each container has a timer that runs a service unit that executes health check scripts.
To list all OpenStack Platform containers timers, run the systemctl list-timers command and limit the output to lines containing tripleo:
To check the status of a specific container timer, run the systemctl status command for the healthcheck service:
To stop, start, restart, and show the status of a container timer, run the relevant systemctl command against the .timer Systemd resource. For example, to check the status of the tripleo_keystone_healthcheck.timer resource, run the following command:
sudo systemctl status tripleo_keystone_healthcheck.timer ● tripleo_keystone_healthcheck.timer - keystone container healthcheck Loaded: loaded (/etc/systemd/system/tripleo_keystone_healthcheck.timer; enabled; vendor preset: disabled) Active: active (waiting) since Fri 2019-02-15 23:53:18 UTC; 2 days ago
$ sudo systemctl status tripleo_keystone_healthcheck.timer
● tripleo_keystone_healthcheck.timer - keystone container healthcheck
Loaded: loaded (/etc/systemd/system/tripleo_keystone_healthcheck.timer; enabled; vendor preset: disabled)
Active: active (waiting) since Fri 2019-02-15 23:53:18 UTC; 2 days ago
If the healthcheck service is disabled but the timer for that service is present and enabled, it means that the check is currently timed out, but will be run according to timer. You can also start the check manually.
The podman ps command does not show the container health status.
Checking container logs
OpenStack Platform 16 introduces a new logging directory /var/log/containers/stdout that contains the standard output (stdout) all of the containers, and standard errors (stderr) consolidated in one single file for each container.
Paunch and the container-puppet.py script configure podman containers to push their outputs to the /var/log/containers/stdout directory, which creates a collection of all logs, even for the deleted containers, such as container-puppet-* containers.
The host also applies log rotation to this directory, which prevents huge files and disk space issues.
In case a container is replaced, the new container outputs to the same log file, because podman uses the container name instead of container ID.
You can also check the logs for a containerized service with the podman logs command. For example, to view the logs for the keystone container, run the following command:
sudo podman logs keystone
$ sudo podman logs keystone
Accessing containers
To enter the shell for a containerized service, use the podman exec command to launch /bin/bash. For example, to enter the shell for the keystone container, run the following command:
sudo podman exec -it keystone /bin/bash
$ sudo podman exec -it keystone /bin/bash
To enter the shell for the keystone container as the root user, run the following command:
sudo podman exec --user 0 -it <NAME OR ID> /bin/bash
$ sudo podman exec --user 0 -it <NAME OR ID> /bin/bash
To exit the container, run the following command:
exit
# exit
12.2. Modifying the overcloud environment Copy linkLink copied to clipboard!
You can modify the overcloud to add additional features or alter existing operations. To modify the overcloud, make modifications to your custom environment files and heat templates, then rerun the openstack overcloud deploy command from your initial overcloud creation. For example, if you created an overcloud using Section 7.13, “Deployment command”, rerun the following command:
Director checks the overcloud stack in heat, and then updates each item in the stack with the environment files and heat templates. Director does not recreate the overcloud, but rather changes the existing overcloud.
Removing parameters from custom environment files does not revert the parameter value to the default configuration. You must identify the default value from the core heat template collection in /usr/share/openstack-tripleo-heat-templates and set the value in your custom environment file manually.
If you want to include a new environment file, add it to the openstack overcloud deploy command with the`-e` option. For example:
This command includes the new parameters and resources from the environment file into the stack.
It is not advisable to make manual modifications to the overcloud configuration because director might overwrite these modifications later.
12.3. Importing virtual machines into the overcloud Copy linkLink copied to clipboard!
You can migrate virtual machines from an existing OpenStack environment to your Red Hat OpenStack Platform (RHOSP) environment.
Procedure
On the existing OpenStack environment, create a new image by taking a snapshot of a running server and download the image:
openstack server image create instance_name --name image_name openstack image save image_name --file exported_vm.qcow2
$ openstack server image create instance_name --name image_name $ openstack image save image_name --file exported_vm.qcow2Copy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the exported image to the undercloud node:
scp exported_vm.qcow2 stack@192.168.0.2:~/.
$ scp exported_vm.qcow2 stack@192.168.0.2:~/.Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Log in to the undercloud as the
stackuser. Source the
overcloudrcfile:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the exported image into the overcloud:
(overcloud) $ openstack image create imported_image --file exported_vm.qcow2 --disk-format qcow2 --container-format bare
(overcloud) $ openstack image create imported_image --file exported_vm.qcow2 --disk-format qcow2 --container-format bareCopy to Clipboard Copied! Toggle word wrap Toggle overflow Launch a new instance:
(overcloud) $ openstack server create imported_instance --key-name default --flavor m1.demo --image imported_image --nic net-id=net_id
(overcloud) $ openstack server create imported_instance --key-name default --flavor m1.demo --image imported_image --nic net-id=net_idCopy to Clipboard Copied! Toggle word wrap Toggle overflow
These commands copy each virtual machine disk from the existing OpenStack environment to the new Red Hat OpenStack Platform. QCOW snapshots lose their original layering system.
This process migrates all instances from a Compute node. You can now perform maintenance on the node without any instance downtime. To return the Compute node to an enabled state, run the following command:
source ~/overcloudrc (overcloud) $ openstack compute service set [hostname] nova-compute --enable
$ source ~/overcloudrc
(overcloud) $ openstack compute service set [hostname] nova-compute --enable
12.4. Running the dynamic inventory script Copy linkLink copied to clipboard!
Director can run Ansible-based automation in your Red Hat OpenStack Platform (RHOSP) environment. Director uses the tripleo-ansible-inventory command to generate a dynamic inventory of nodes in your environment.
Procedure
To view a dynamic inventory of nodes, run the
tripleo-ansible-inventorycommand after sourcingstackrc:source ~/stackrc (undercloud) $ tripleo-ansible-inventory --list
$ source ~/stackrc (undercloud) $ tripleo-ansible-inventory --listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
--listoption to return details about all hosts. This command outputs the dynamic inventory in a JSON format:{"overcloud": {"children": ["controller", "compute"], "vars": {"ansible_ssh_user": "heat-admin"}}, "controller": ["192.168.24.2"], "undercloud": {"hosts": ["localhost"], "vars": {"overcloud_horizon_url": "http://192.168.24.4:80/dashboard", "overcloud_admin_password": "abcdefghijklm12345678", "ansible_connection": "local"}}, "compute": ["192.168.24.3"]}{"overcloud": {"children": ["controller", "compute"], "vars": {"ansible_ssh_user": "heat-admin"}}, "controller": ["192.168.24.2"], "undercloud": {"hosts": ["localhost"], "vars": {"overcloud_horizon_url": "http://192.168.24.4:80/dashboard", "overcloud_admin_password": "abcdefghijklm12345678", "ansible_connection": "local"}}, "compute": ["192.168.24.3"]}Copy to Clipboard Copied! Toggle word wrap Toggle overflow To execute Ansible playbooks on your environment, run the
ansiblecommand and include the full path of the dynamic inventory tool using the-ioption. For example:(undercloud) $ ansible [HOSTS] -i /bin/tripleo-ansible-inventory [OTHER OPTIONS]
(undercloud) $ ansible [HOSTS] -i /bin/tripleo-ansible-inventory [OTHER OPTIONS]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
[HOSTS]with the type of hosts that you want to use to use:-
controllerfor all Controller nodes -
computefor all Compute nodes -
overcloudfor all overcloud child nodes. For example,controllerandcomputenodes -
undercloudfor the undercloud -
"*"for all nodes
-
Replace
[OTHER OPTIONS]with additional Ansible options.-
Use the
--ssh-extra-args='-o StrictHostKeyChecking=no'option to bypass confirmation on host key checking. -
Use the
-u [USER]option to change the SSH user that executes the Ansible automation. The default SSH user for the overcloud is automatically defined using theansible_ssh_userparameter in the dynamic inventory. The-uoption overrides this parameter. -
Use the
-m [MODULE]option to use a specific Ansible module. The default iscommand, which executes Linux commands. -
Use the
-a [MODULE_ARGS]option to define arguments for the chosen module.
-
Use the
Custom Ansible automation on the overcloud is not part of the standard overcloud stack. Subsequent execution of the openstack overcloud deploy command might override Ansible-based configuration for OpenStack Platform services on overcloud nodes.
12.5. Removing the overcloud Copy linkLink copied to clipboard!
To remove the overcloud, complete the following steps:
Delete an existing overcloud:
source ~/stackrc (undercloud) $ openstack overcloud delete overcloud
$ source ~/stackrc (undercloud) $ openstack overcloud delete overcloudCopy to Clipboard Copied! Toggle word wrap Toggle overflow Confirm that the overcloud is no longer present in the output of the
openstack stack listcommand:(undercloud) $ openstack stack list
(undercloud) $ openstack stack listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Deletion takes a few minutes.
- When the deletion completes, follow the standard steps in the deployment scenarios to recreate your overcloud.
Chapter 13. Configuring the overcloud with Ansible Copy linkLink copied to clipboard!
Ansible is the main method to apply the overcloud configuration. This chapter provides information about how to interact with the overcloud Ansible configuration.
Although director generates the Ansible playbooks automatically, it is a good idea to familiarize yourself with Ansible syntax. For more information about using Ansible, see https://docs.ansible.com/.
Ansible also uses the concept of roles, which are different to OpenStack Platform director roles. Ansible roles form reusable components of playbooks, whereas director roles contain mappings of OpenStack services to node types.
13.1. Ansible-based overcloud configuration (config-download) Copy linkLink copied to clipboard!
The config-download feature is the method that director uses to configure the overcloud. Director uses config-download in conjunction with OpenStack Orchestration (heat) and OpenStack Workflow Service (mistral) to generate the software configuration and apply the configuration to each overcloud node. Although heat creates all deployment data from SoftwareDeployment resources to perform the overcloud installation and configuration, heat does not apply any of the configuration. Heat only provides the configuration data through the heat API. When director creates the stack, a mistral workflow queries the heat API to obtain the configuration data, generate a set of Ansible playbooks, and applies the Ansible playbooks to the overcloud.
As a result, when you run the openstack overcloud deploy command, the following process occurs:
-
Director creates a new deployment plan based on
openstack-tripleo-heat-templatesand includes any environment files and parameters to customize the plan. - Director uses heat to interpret the deployment plan and create the overcloud stack and all descendant resources. This includes provisioning nodes with the OpenStack Bare Metal service (ironic).
- Heat also creates the software configuration from the deployment plan. Director compiles the Ansible playbooks from this software configuration.
-
Director generates a temporary user (
tripleo-admin) on the overcloud nodes specifically for Ansible SSH access. - Director downloads the heat software configuration and generates a set of Ansible playbooks using heat outputs.
-
Director applies the Ansible playbooks to the overcloud nodes using
ansible-playbook.
13.2. config-download working directory Copy linkLink copied to clipboard!
Director generates a set of Ansible playbooks for the config-download process. These playbooks are stored in a working directory in the /var/lib/mistral/. This directory is named after the name of the overcloud, which is overcloud by default.
The working directory contains a set of sub-directories named after each overcloud role. These sub-directories contain all tasks relevant to the configuration of the nodes in the overcloud role. These sub-directories also contain additional sub-directories named after each specific node. These sub-directories contain node-specific variables to apply to the overcloud role tasks. As a result, the overcloud roles within the working directory use the following structure:
Each working directory is a local Git repository that records changes after each deployment operation. Use the local Git repositories to track configuration changes between each deployment.
13.3. Enabling access to config-download working directories Copy linkLink copied to clipboard!
The mistral user in the OpenStack Workflow service (mistral) containers own all files in the /var/lib/mistral/ working directories. You can grant the stack user on the undercloud access to all files in this directory. This helps with performing certain operations within the directory.
Procedure
Use the
setfaclcommand to grant thestackuser on the undercloud access to the files in the/var/lib/mistraldirectory:sudo setfacl -R -m u:stack:rwx /var/lib/mistral
$ sudo setfacl -R -m u:stack:rwx /var/lib/mistralCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command retains
mistraluser access to the directory.
13.4. Checking config-download log Copy linkLink copied to clipboard!
During the config-download process, Ansible creates a log file on the undercloud in the config-download working directory.
Procedure
View the log with the
lesscommand within theconfig-downloadworking directory. The following example uses theovercloudworking directory:less /var/lib/mistral/overcloud/ansible.log
$ less /var/lib/mistral/overcloud/ansible.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow
13.5. Separating the provisioning and configuration processes Copy linkLink copied to clipboard!
The openstack overcloud deploy command runs the heat-based provisioning process and then the config-download configuration process. You can also run the command to execute each process individually.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the deployment command with the
--stack-onlyoption. Include any environment files required for your overcloud:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the provisioning process completes.
Enable SSH access from the undercloud to the overcloud for the
tripleo-adminuser. Theconfig-downloadprocess uses thetripleo-adminuser to perform the Ansible-based configuration:openstack overcloud admin authorize
$ openstack overcloud admin authorizeCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the deployment command with the
--config-download-onlyoption. Include any environment files required for your overcloud:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the configuration process completes.
13.6. Running config-download manually Copy linkLink copied to clipboard!
The working directory in /var/lib/mistral/overcloud contains the playbooks and scripts necessary to interact with ansible-playbook directly. This procedure shows how to interact with these files.
Procedure
Change to the directory of the Ansible playbook::
cd /var/lib/mistral/overcloud/
$ cd /var/lib/mistral/overcloud/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
ansible-playbook-command.shcommand to reproduce the deployment:./ansible-playbook-command.sh
$ ./ansible-playbook-command.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow You can pass additional Ansible arguments to this script, which are then passed unchanged to the
ansible-playbookcommand. This means that you can use other Ansible features, such as check mode (--check), limiting hosts (--limit), or overriding variables (-e). For example:./ansible-playbook-command.sh --limit Controller
$ ./ansible-playbook-command.sh --limit ControllerCopy to Clipboard Copied! Toggle word wrap Toggle overflow The working directory contains a playbook called
deploy_steps_playbook.yaml, which runs the overcloud configuration. To view this playbook, run the following command:less deploy_steps_playbook.yaml
$ less deploy_steps_playbook.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow The playbook uses various task files contained in the working directory. Some task files are common to all OpenStack Platform roles and some are specific to certain OpenStack Platform roles and servers.
The working directory also contains sub-directories that correspond to each role that you define in your overcloud
roles_datafile. For example:ls Controller/
$ ls Controller/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Each OpenStack Platform role directory also contains sub-directories for individual servers of that role type. The directories use the composable role hostname format:
ls Controller/overcloud-controller-0
$ ls Controller/overcloud-controller-0Copy to Clipboard Copied! Toggle word wrap Toggle overflow The Ansible tasks are tagged. To see the full list of tags, use the CLI argument
--list-tagsforansible-playbook:ansible-playbook -i tripleo-ansible-inventory.yaml --list-tags deploy_steps_playbook.yaml
$ ansible-playbook -i tripleo-ansible-inventory.yaml --list-tags deploy_steps_playbook.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Then apply tagged configuration using the
--tags,--skip-tags, or--start-at-taskwith theansible-playbook-command.shscript:./ansible-playbook-command.sh --tags overcloud
$ ./ansible-playbook-command.sh --tags overcloudCopy to Clipboard Copied! Toggle word wrap Toggle overflow When
config-downloadconfigures Ceph, Ansible executesceph-ansiblefrom within theconfig-download external_deploy_steps_tasksplaybook. When you runconfig-downloadmanually, the second Ansible execution does not inherit thessh_argsargument. To pass Ansible environment variables to this execution, use a heat environment file. For example:parameter_defaults: CephAnsibleEnvironmentVariables: ANSIBLE_HOST_KEY_CHECKING: 'False' ANSIBLE_PRIVATE_KEY_FILE: '/home/stack/.ssh/id_rsa'parameter_defaults: CephAnsibleEnvironmentVariables: ANSIBLE_HOST_KEY_CHECKING: 'False' ANSIBLE_PRIVATE_KEY_FILE: '/home/stack/.ssh/id_rsa'Copy to Clipboard Copied! Toggle word wrap Toggle overflow
When you use ansible-playbook CLI arguments such as --tags, --skip-tags, or --start-at-task, do not run or apply deployment configuration out of order. These CLI arguments are a convenient way to rerun previously failed tasks or to iterate over an initial deployment. However, to guarantee a consistent deployment, you must run all tasks from deploy_steps_playbook.yaml in order.
13.7. Performing Git operations on the working directory Copy linkLink copied to clipboard!
The config-download working directory is a local Git repository. Every time a deployment operation runs, director adds a Git commit to the working directory with the relevant changes. You can perform Git operations to view configuration for the deployment at different stages and compare the configuration with different deployments.
Be aware of the limitations of the working directory. For example, if you use Git to revert to a previous version of the config-download working directory, this action affects only the configuration in the working directory. It does not affect the following configurations:
- The overcloud data schema: Applying a previous version of the working directory software configuration does not undo data migration and schema changes.
- The hardware layout of the overcloud: Reverting to previous software configuration does not undo changes related to overcloud hardware, such as scaling up or down.
-
The heat stack: Reverting to earlier revisions of the working directory has no effect on the configuration stored in the heat stack. The heat stack creates a new version of the software configuration that applies to the overcloud. To make permanent changes to the overcloud, modify the environment files applied to the overcloud stack before you rerun the
openstack overcloud deploycommand.
Complete the following steps to compare different commits of the config-download working directory.
Procedure
Change to the
config-downloadworking directory for your overcloud. In this example, the working directory is for the overcloud namedovercloud:cd /var/lib/mistral/overcloud
$ cd /var/lib/mistral/overcloudCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
git logcommand to list the commits in your working directory. You can also format the log output to show the date:git log --format=format:"%h%x09%cd%x09" a7e9063 Mon Oct 8 21:17:52 2018 +1000 dfb9d12 Fri Oct 5 20:23:44 2018 +1000 d0a910b Wed Oct 3 19:30:16 2018 +1000 ...
$ git log --format=format:"%h%x09%cd%x09" a7e9063 Mon Oct 8 21:17:52 2018 +1000 dfb9d12 Fri Oct 5 20:23:44 2018 +1000 d0a910b Wed Oct 3 19:30:16 2018 +1000 ...Copy to Clipboard Copied! Toggle word wrap Toggle overflow By default, the most recent commit appears first.
Run the
git diffcommand against two commit hashes to see all changes between the deployments:git diff a7e9063 dfb9d12
$ git diff a7e9063 dfb9d12Copy to Clipboard Copied! Toggle word wrap Toggle overflow
13.8. Creating config-download files manually Copy linkLink copied to clipboard!
You can generate your own config-download files outside of the standard workflow. For example, you can generate the overcloud heat stack using the --stack-only option with the openstack overcloud deploy command so that you can apply the configuration separately. Complete the following steps to create your own config-download files manually.
Procedure
Generate the
config-downloadfiles:openstack overcloud config download \ --name overcloud \ --config-dir ~/config-download
$ openstack overcloud config download \ --name overcloud \ --config-dir ~/config-downloadCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
--nameis the name of the overcloud that you want to use for the Ansible file export. -
--config-diris the location where you want to save theconfig-downloadfiles.
-
Change to the directory that contains your
config-downloadfiles:cd ~/config-download
$ cd ~/config-downloadCopy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a static inventory file:
tripleo-ansible-inventory \ --ansible_ssh_user heat-admin \ --static-yaml-inventory inventory.yaml
$ tripleo-ansible-inventory \ --ansible_ssh_user heat-admin \ --static-yaml-inventory inventory.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Use the config-download files and the static inventory file to perform a configuration. To execute the deployment playbook, run the ansible-playbook command:
ansible-playbook \ -i inventory.yaml \ --private-key ~/.ssh/id_rsa \ --become \ ~/config-download/deploy_steps_playbook.yaml
$ ansible-playbook \
-i inventory.yaml \
--private-key ~/.ssh/id_rsa \
--become \
~/config-download/deploy_steps_playbook.yaml
To generate an overcloudrc file manually from this configuration, run the following command:
13.9. config-download top level files Copy linkLink copied to clipboard!
The following file are important top level files within a config-download working directory.
Ansible configuration and execution
The following files are specific to configuring and executing Ansible within the config-download working directory.
- ansible.cfg
-
Configuration file used when running
ansible-playbook. - ansible.log
-
Log file from the last run of
ansible-playbook. - ansible-errors.json
- JSON structured file that contains any deployment errors.
- ansible-playbook-command.sh
-
Executable script to rerun the
ansible-playbookcommand from the last deployment operation. - ssh_private_key
- Private SSH key that Ansible uses to access the overcloud nodes.
- tripleo-ansible-inventory.yaml
- Ansible inventory file that contains hosts and variables for all the overcloud nodes.
- overcloud-config.tar.gz
- Archive of the working directory.
Playbooks
The following files are playbooks within the config-download working directory.
- deploy_steps_playbook.yaml
- Main deployment steps. This playbook performs the main configuration operations for your overcloud.
- pre_upgrade_rolling_steps_playbook.yaml
- Pre upgrade steps for major upgrade
- upgrade_steps_playbook.yaml
- Major upgrade steps.
- post_upgrade_steps_playbook.yaml
- Post upgrade steps for major upgrade.
- update_steps_playbook.yaml
- Minor update steps.
- fast_forward_upgrade_playbook.yaml
- Fast forward upgrade tasks. Use this playbook only when you want to upgrade from one long-life version of Red Hat OpenStack Platform to the next.
13.10. config-download tags Copy linkLink copied to clipboard!
The playbooks use tagged tasks to control the tasks that they apply to the overcloud. Use tags with the ansible-playbook CLI arguments --tags or --skip-tags to control which tasks to execute. The following list contains information about the tags that are enabled by default:
- facts
- Fact gathering operations.
- common_roles
- Ansible roles common to all nodes.
- overcloud
- All plays for overcloud deployment.
- pre_deploy_steps
-
Deployments that happen before the
deploy_stepsoperations. - host_prep_steps
- Host preparation steps.
- deploy_steps
- Deployment steps.
- post_deploy_steps
-
Steps that happen after the
deploy_stepsoperations. - external
- All external deployment tasks.
- external_deploy_steps
- External deployment tasks that run on the undercloud only.
13.11. config-download deployment steps Copy linkLink copied to clipboard!
The deploy_steps_playbook.yaml playbook configures the overcloud. This playbook applies all software configuration that is necessary to deploy a full overcloud based on the overcloud deployment plan.
This section contains a summary of the different Ansible plays used within this playbook. The play names in this section are the same names that are used within the playbook and that are displayed in the ansible-playbook output. This section also contains information about the Ansible tags that are set on each play.
- Gather facts from undercloud
Fact gathering for the undercloud node.
Tags:
facts- Gather facts from overcloud
Fact gathering for the overcloud nodes.
Tags:
facts- Load global variables
Loads all variables from
global_vars.yaml.Tags:
always- Common roles for TripleO servers
Applies common Ansible roles to all overcloud nodes, including tripleo-bootstrap for installing bootstrap packages, and tripleo-ssh-known-hosts for configuring ssh known hosts.
Tags:
common_roles- Overcloud deploy step tasks for step 0
Applies tasks from the deploy_steps_tasks template interface.
Tags:
overcloud,deploy_steps- Server deployments
Applies server-specific heat deployments for configuration such as networking and hieradata. Includes NetworkDeployment, <Role>Deployment, <Role>AllNodesDeployment, etc.
Tags:
overcloud,pre_deploy_steps- Host prep steps
Applies tasks from the host_prep_steps template interface.
Tags:
overcloud,host_prep_steps- External deployment step [1,2,3,4,5]
Applies tasks from the external_deploy_steps_tasks template interface. Ansible runs these tasks only against the undercloud node.
Tags:
external,external_deploy_steps- Overcloud deploy step tasks for [1,2,3,4,5]
Applies tasks from the deploy_steps_tasks template interface.
Tags:
overcloud,deploy_steps- Overcloud common deploy step tasks [1,2,3,4,5]
Applies the common tasks performed at each step, including puppet host configuration,
container-puppet.py, and paunch (container configuration).Tags:
overcloud,deploy_steps- Server Post Deployments
Applies server specific heat deployments for configuration performed after the 5-step deployment process.
Tags:
overcloud,post_deploy_steps- External deployment Post Deploy tasks
Applies tasks from the external_post_deploy_steps_tasks template interface. Ansible runs these tasks only against the undercloud node.
Tags:
external,external_deploy_steps
13.12. Next Steps Copy linkLink copied to clipboard!
You can now continue your regular overcloud operations.
Chapter 14. Using the validation framework Copy linkLink copied to clipboard!
This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
Red Hat OpenStack Platform includes a validation framework that you can use to verify the requirements and functionality of the undercloud and overcloud. The framework includes two types of validations:
-
Manual Ansible-based validations, which you execute through the
openstack tripleo validatorcommand set. - Automatic in-flight validations, which execute during the deployment process.
14.1. Ansible-based validations Copy linkLink copied to clipboard!
During the installation of Red Hat OpenStack Platform director, director also installs a set of playbooks from the openstack-tripleo-validations package. Each playbook contains tests for certain system requirements and a set of groups that define when to run the test:
no-op- Validations that run a no-op (no operation) task to verify to workflow functions correctly. These validations run on both the undercloud and overcloud.
prep-
Validations that check the hardware configuration of the undercloud node. Run these validation before you run the
openstack undercloud installcommand. openshift-on-openstack- Validations that check that the environment meets the requirements to be able to deploy OpenShift on OpenStack.
pre-introspection- Validations to run before the nodes introspection using Ironic Inspector.
pre-deployment-
Validations to run before the
openstack overcloud deploycommand. post-deployment- Validations to run after the overcloud deployment has finished.
pre-upgrade- Validations to validate your OpenStack deployment before an upgrade.
post-upgrade- Validations to validate your OpenStack deployment after an upgrade.
14.2. Listing validations Copy linkLink copied to clipboard!
Run the openstack tripleo validator list command to list the different types of validations available.
Procedure
Source the
stackrcfile.source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the
openstack tripleo validator listcommand:To list all validations, run the command without any options:
openstack tripleo validator list
$ openstack tripleo validator listCopy to Clipboard Copied! Toggle word wrap Toggle overflow To list validations in a group, run the command with the
--groupoption:openstack tripleo validator list --group prep
$ openstack tripleo validator list --group prepCopy to Clipboard Copied! Toggle word wrap Toggle overflow
For a full list of options, run openstack tripleo validator list --help.
14.3. Running validations Copy linkLink copied to clipboard!
To run a validation or validation group, use the openstack tripleo validator run command. To see a full list of options, use the openstack tripleo validator run --help command.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Enter the
openstack tripleo validator runcommand:To run a single validation, enter the command with the
--validationoption and the name of the validation. For example, to check the undercloud memory requirements, enter--validation undercloud-ram:openstack tripleo validator run --validation undercloud-ram
$ openstack tripleo validator run --validation undercloud-ramCopy to Clipboard Copied! Toggle word wrap Toggle overflow To run all validations in a group, enter the command with the
--groupoption:openstack tripleo validator run --group prep
$ openstack tripleo validator run --group prepCopy to Clipboard Copied! Toggle word wrap Toggle overflow
14.4. In-flight validations Copy linkLink copied to clipboard!
Red Hat OpenStack Platform includes in-flight validations in the templates of composable services. In-flight validations verify the operational status of services at key steps of the overcloud deployment process.
In-flight validations run automatically as part of the deployment process. Some in-flight validations also use the roles from the openstack-tripleo-validations package.
Chapter 15. Scaling overcloud nodes Copy linkLink copied to clipboard!
Do not use openstack server delete to remove nodes from the overcloud. Follow the procedures in this section to remove and replace nodes correctly.
If you want to add or remove nodes after the creation of the overcloud, you must update the overcloud.
Use the following table to determine support for scaling each node type:
| Node type | Scale up? | Scale down? | Notes |
| Controller | N | N | You can replace Controller nodes using the procedures in Chapter 16, Replacing Controller nodes. |
| Compute | Y | Y | |
| Ceph Storage nodes | Y | N | You must have at least 1 Ceph Storage node from the initial overcloud creation. |
| Object Storage nodes | Y | Y |
Ensure that you have at least 10 GB free space before you scale the overcloud. This free space accommodates image conversion and caching during the node provisioning process.
15.1. Adding nodes to the overcloud Copy linkLink copied to clipboard!
Complete the following steps to add more nodes to the director node pool.
Procedure
Create a new JSON file (
newnodes.json) that contains details of the new node that you want to register:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the following command to register the new nodes:
source ~/stackrc (undercloud) $ openstack overcloud node import newnodes.json
$ source ~/stackrc (undercloud) $ openstack overcloud node import newnodes.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow After you register the new nodes, run the following commands to launch the introspection process for each new node:
(undercloud) $ openstack baremetal node manage [NODE UUID] (undercloud) $ openstack overcloud node introspect [NODE UUID] --provide
(undercloud) $ openstack baremetal node manage [NODE UUID] (undercloud) $ openstack overcloud node introspect [NODE UUID] --provideCopy to Clipboard Copied! Toggle word wrap Toggle overflow This process detects and benchmarks the hardware properties of the nodes.
Configure the image properties for the node:
(undercloud) $ openstack overcloud node configure [NODE UUID]
(undercloud) $ openstack overcloud node configure [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow
15.2. Increasing node counts for roles Copy linkLink copied to clipboard!
Complete the following steps to scale overcloud nodes for a specific role, such as a Compute node.
Procedure
Tag each new node with the role you want. For example, to tag a node with the Compute role, run the following command:
(undercloud) $ openstack baremetal node set --property capabilities='profile:compute,boot_option:local' [NODE UUID]
(undercloud) $ openstack baremetal node set --property capabilities='profile:compute,boot_option:local' [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow To scale the overcloud, you must edit the environment file that contains your node counts and re-deploy the overcloud. For example, to scale your overcloud to 5 Compute nodes, edit the
ComputeCountparameter:parameter_defaults: ... ComputeCount: 5 ...
parameter_defaults: ... ComputeCount: 5 ...Copy to Clipboard Copied! Toggle word wrap Toggle overflow Rerun the deployment command with the updated file, which in this example is called
node-info.yaml:(undercloud) $ openstack overcloud deploy --templates -e /home/stack/templates/node-info.yaml [OTHER_OPTIONS]
(undercloud) $ openstack overcloud deploy --templates -e /home/stack/templates/node-info.yaml [OTHER_OPTIONS]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Ensure that you include all environment files and options from your initial overcloud creation. This includes the same scale parameters for non-Compute nodes.
- Wait until the deployment operation completes.
15.3. Removing Compute nodes Copy linkLink copied to clipboard!
There might be situations where you need to remove Compute nodes from the overcloud. For example, you might need to replace a problematic Compute node.
Before you remove a Compute node from the overcloud, migrate the workload from the node to other Compute nodes. For more information, see Migrating virtual machine instances between Compute nodes.
Prerequisites
-
The Placement service package
python3-osc-placementinstalled on the undercloud.
Procedure
Source the overcloud configuration:
source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Disable the Compute service on the outgoing node on the overcloud to prevent the node from scheduling new instances:
(overcloud) $ openstack compute service list (overcloud) $ openstack compute service set <hostname> nova-compute --disable
(overcloud) $ openstack compute service list (overcloud) $ openstack compute service set <hostname> nova-compute --disableCopy to Clipboard Copied! Toggle word wrap Toggle overflow TipUse the
--disable-reasonoption to add a short explanation on why the service is being disabled. This is useful if you intend to redeploy the Compute service at a later point.Source the undercloud configuration:
(overcloud) $ source ~/stackrc
(overcloud) $ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the UUID of the overcloud stack:
(undercloud) $ openstack stack list
(undercloud) $ openstack stack listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the UUIDs or hostnames of the nodes that you want to delete:
(undercloud) $ openstack server list
(undercloud) $ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Redeploy the overcloud with the
--update-plan-onlyoption, including all of the environment files that are relevant to your deployment:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Delete the nodes from the stack:
openstack overcloud node delete --stack [STACK_UUID] --templates -e [ENVIRONMENT_FILE] [NODE1_UUID] [NODE2_UUID] [NODE3_UUID]
$ openstack overcloud node delete --stack [STACK_UUID] --templates -e [ENVIRONMENT_FILE] [NODE1_UUID] [NODE2_UUID] [NODE3_UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace [node] with a UUID or hostname of a node.
ImportantDo not use a mix of UUIDs and hostnames. Use either only UUIDs or only hostnames.
Ensure that the
openstack overcloud node deletecommand runs to completion:(undercloud) $ openstack stack list
(undercloud) $ openstack stack listCopy to Clipboard Copied! Toggle word wrap Toggle overflow The status of the
overcloudstack showsUPDATE_COMPLETEwhen the delete operation is complete.ImportantIf you intend to redeploy the Compute service with the same host name, you must use the existing service records for the redeployed node. If this is the case, skip the remaining steps in this procedure, and proceed with the instructions detailed in Redeploying the Compute service using the same host name.
Remove the Compute service from the node:
(undercloud) $ source ~/overcloudrc (overcloud) $ openstack compute service list (overcloud) $ openstack compute service delete <service-id>
(undercloud) $ source ~/overcloudrc (overcloud) $ openstack compute service list (overcloud) $ openstack compute service delete <service-id>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the Open vSwitch agent from the node:
(overcloud) $ openstack network agent list (overcloud) $ openstack network agent delete <openvswitch-agent-id>
(overcloud) $ openstack network agent list (overcloud) $ openstack network agent delete <openvswitch-agent-id>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the deleted Compute service as a resource provider from the Placement service:
(overcloud) $ openstack resource provider list (overcloud) $ openstack resource provider delete <uuid>
(overcloud) $ openstack resource provider list (overcloud) $ openstack resource provider delete <uuid>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Decrease the
ComputeCountparameter in the environment file that contains your node counts. This file is usually namednode-info.yaml. For example, decrease the node count from five nodes to three nodes if you removed two nodes:parameter_defaults: ... ComputeCount: 3 ...
parameter_defaults: ... ComputeCount: 3 ...Copy to Clipboard Copied! Toggle word wrap Toggle overflow Decreasing the node count ensures director does not provision any new nodes when you run
openstack overcloud deploy.
You can remove the node from the overcloud and re-provision it for other purposes.
Redeploying the Compute service using the same host name
To redeploy a disabled Compute service, re-enable it after you redeploy a Compute node with the same host name.
Procedure
Remove the deleted Compute service as a resource provider from the Placement service:
(undercloud) $ source ~/overcloudrc (overcloud) $ openstack resource provider list (overcloud) $ openstack resource provider delete <uuid>
(undercloud) $ source ~/overcloudrc (overcloud) $ openstack resource provider list (overcloud) $ openstack resource provider delete <uuid>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the status of the Compute service:
(overcloud) $ openstack compute service list --long ... | ID | Binary | Host | Zone | Status | State | Updated At | Disabled Reason | | 80 | nova-compute | compute-1.localdomain | nova | disabled | up | 2018-07-13T14:35:04.000000 | gets re-provisioned | ...
(overcloud) $ openstack compute service list --long ... | ID | Binary | Host | Zone | Status | State | Updated At | Disabled Reason | | 80 | nova-compute | compute-1.localdomain | nova | disabled | up | 2018-07-13T14:35:04.000000 | gets re-provisioned | ...Copy to Clipboard Copied! Toggle word wrap Toggle overflow When the service state of the redeployed Compute node changes to
up, re-enable the service:(overcloud) $ openstack compute service set compute-1.localdomain nova-compute --enable
(overcloud) $ openstack compute service set compute-1.localdomain nova-compute --enableCopy to Clipboard Copied! Toggle word wrap Toggle overflow
15.4. Replacing Ceph Storage nodes Copy linkLink copied to clipboard!
You can use director to replace Ceph Storage nodes in a director-created cluster. For more information, see the Deploying an Overcloud with Containerized Red Hat Ceph guide.
15.5. Replacing Object Storage nodes Copy linkLink copied to clipboard!
Follow the instructions in this section to understand how to replace Object Storage nodes without impact to the integrity of the cluster. This example involves a three-node Object Storage cluster in which you want to replace the node overcloud-objectstorage-1 node. The goal of the procedure is to add one more node and then remove the overcloud-objectstorage-1 node. The new node replaces the overcloud-objectstorage-1 node.
Procedure
Increase the Object Storage count using the
ObjectStorageCountparameter. This parameter is usually located innode-info.yaml, which is the environment file that contains your node counts:parameter_defaults: ObjectStorageCount: 4
parameter_defaults: ObjectStorageCount: 4Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
ObjectStorageCountparameter defines the quantity of Object Storage nodes in your environment. In this example, scale the quantity of Object Storage nodes from3to4.Run the deployment command with the updated
ObjectStorageCountparameter:source ~/stackrc (undercloud) $ openstack overcloud deploy --templates -e node-info.yaml ENVIRONMENT_FILES
$ source ~/stackrc (undercloud) $ openstack overcloud deploy --templates -e node-info.yaml ENVIRONMENT_FILESCopy to Clipboard Copied! Toggle word wrap Toggle overflow - After the deployment command completes, the overcloud contains an additional Object Storage node.
Replicate data to the new node. Before you remove a node, in this case,
overcloud-objectstorage-1, wait for a replication pass to finish on the new node. Check the replication pass progress in the/var/log/swift/swift.logfile. When the pass finishes, the Object Storage service should log entries similar to the following example:Mar 29 08:49:05 localhost object-server: Object replication complete. Mar 29 08:49:11 localhost container-server: Replication run OVER Mar 29 08:49:13 localhost account-server: Replication run OVER
Mar 29 08:49:05 localhost object-server: Object replication complete. Mar 29 08:49:11 localhost container-server: Replication run OVER Mar 29 08:49:13 localhost account-server: Replication run OVERCopy to Clipboard Copied! Toggle word wrap Toggle overflow To remove the old node from the ring, reduce the
ObjectStorageCountparameter to omit the old node. In this example, reduce theObjectStorageCountparameter to3:parameter_defaults: ObjectStorageCount: 3
parameter_defaults: ObjectStorageCount: 3Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new environment file named
remove-object-node.yaml. This file identifies and removes the specified Object Storage node. The following content specifies the removal ofovercloud-objectstorage-1:parameter_defaults: ObjectStorageRemovalPolicies: [{'resource_list': ['1']}]parameter_defaults: ObjectStorageRemovalPolicies: [{'resource_list': ['1']}]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Include both the
node-info.yamlandremove-object-node.yamlfiles in the deployment command:(undercloud) $ openstack overcloud deploy --templates -e node-info.yaml ENVIRONMENT_FILES -e remove-object-node.yaml
(undercloud) $ openstack overcloud deploy --templates -e node-info.yaml ENVIRONMENT_FILES -e remove-object-node.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Director deletes the Object Storage node from the overcloud and updates the rest of the nodes on the overcloud to accommodate the node removal.
Include all environment files and options from your initial overcloud creation. This includes the same scale parameters for non-Compute nodes.
15.6. Blacklisting nodes Copy linkLink copied to clipboard!
You can exclude overcloud nodes from receiving an updated deployment. This is useful in scenarios where you want to scale new nodes and exclude existing nodes from receiving an updated set of parameters and resources from the core heat template collection. This means that the blacklisted nodes are isolated from the effects of the stack operation.
Use the DeploymentServerBlacklist parameter in an environment file to create a blacklist.
Setting the blacklist
The DeploymentServerBlacklist parameter is a list of server names. Write a new environment file, or add the parameter value to an existing custom environment file and pass the file to the deployment command:
parameter_defaults:
DeploymentServerBlacklist:
- overcloud-compute-0
- overcloud-compute-1
- overcloud-compute-2
parameter_defaults:
DeploymentServerBlacklist:
- overcloud-compute-0
- overcloud-compute-1
- overcloud-compute-2
The server names in the parameter value are the names according to OpenStack Orchestration (heat), not the actual server hostnames.
Include this environment file with your openstack overcloud deploy command:
source ~/stackrc (undercloud) $ openstack overcloud deploy --templates \ -e server-blacklist.yaml \ [OTHER OPTIONS]
$ source ~/stackrc
(undercloud) $ openstack overcloud deploy --templates \
-e server-blacklist.yaml \
[OTHER OPTIONS]
Heat blacklists any servers in the list from receiving updated heat deployments. After the stack operation completes, any blacklisted servers remain unchanged. You can also power off or stop the os-collect-config agents during the operation.
- Exercise caution when you blacklist nodes. Only use a blacklist if you fully understand how to apply the requested change with a blacklist in effect. It is possible to create a hung stack or configure the overcloud incorrectly when you use the blacklist feature. For example, if cluster configuration changes apply to all members of a Pacemaker cluster, blacklisting a Pacemaker cluster member during this change can cause the cluster to fail.
-
When you add servers to the blacklist, further changes to those nodes are not supported until you remove the server from the blacklist. This includes updates, upgrades, scale up, scale down, and node replacement. For example, when you blacklist existing Compute nodes while scaling out the overcloud with new Compute nodes, the blacklisted nodes miss the information added to
/etc/hostsand/etc/ssh/ssh_known_hosts. This can cause live migration to fail, depending on the destination host. The Compute nodes are updated with the information added to/etc/hostsand/etc/ssh/ssh_known_hostsduring the next overcloud deployment where they are no longer blacklisted.
Clearing the blacklist
To clear the blacklist for subsequent stack operations, edit the DeploymentServerBlacklist to use an empty array:
parameter_defaults: DeploymentServerBlacklist: []
parameter_defaults:
DeploymentServerBlacklist: []
Do not omit the DeploymentServerBlacklist parameter. If you omit the parameter, the overcloud deployment uses the previously saved value.
Chapter 16. Replacing Controller nodes Copy linkLink copied to clipboard!
In certain circumstances a Controller node in a high availability cluster might fail. In these situations, you must remove the node from the cluster and replace it with a new Controller node.
Complete the steps in this section to replace a Controller node. The Controller node replacement process involves running the openstack overcloud deploy command to update the overcloud with a request to replace a Controller node.
The following procedure applies only to high availability environments. Do not use this procedure if you are using only one Controller node.
16.1. Preparing for Controller replacement Copy linkLink copied to clipboard!
Before you replace an overcloud Controller node, it is important to check the current state of your Red Hat OpenStack Platform environment. Checking the current state can help avoid complications during the Controller replacement process. Use the following list of preliminary checks to determine if it is safe to perform a Controller node replacement. Run all commands for these checks on the undercloud.
Procedure
Check the current status of the
overcloudstack on the undercloud:source stackrc (undercloud) $ openstack stack list --nested
$ source stackrc (undercloud) $ openstack stack list --nestedCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
overcloudstack and its subsequent child stacks should have either aCREATE_COMPLETEorUPDATE_COMPLETE.Install the database client tools:
(undercloud) $ sudo dnf -y install mariadb
(undercloud) $ sudo dnf -y install mariadbCopy to Clipboard Copied! Toggle word wrap Toggle overflow Configure root user access to the database:
(undercloud) $ sudo cp /var/lib/config-data/puppet-generated/mysql/root/.my.cnf /root/.
(undercloud) $ sudo cp /var/lib/config-data/puppet-generated/mysql/root/.my.cnf /root/.Copy to Clipboard Copied! Toggle word wrap Toggle overflow Perform a backup of the undercloud databases:
(undercloud) $ mkdir /home/stack/backup (undercloud) $ sudo mysqldump --all-databases --quick --single-transaction | gzip > /home/stack/backup/dump_db_undercloud.sql.gz
(undercloud) $ mkdir /home/stack/backup (undercloud) $ sudo mysqldump --all-databases --quick --single-transaction | gzip > /home/stack/backup/dump_db_undercloud.sql.gzCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check that your undercloud contains 10 GB free storage to accommodate for image caching and conversion when you provision the new node:
(undercloud) $ df -h
(undercloud) $ df -hCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the status of Pacemaker on the running Controller nodes. For example, if 192.168.0.47 is the IP address of a running Controller node, use the following command to view the Pacemaker status:
(undercloud) $ ssh heat-admin@192.168.0.47 'sudo pcs status'
(undercloud) $ ssh heat-admin@192.168.0.47 'sudo pcs status'Copy to Clipboard Copied! Toggle word wrap Toggle overflow The output shows all services that are running on the existing nodes and that are stopped on the failed node.
Check the following parameters on each node of the overcloud MariaDB cluster:
-
wsrep_local_state_comment: Synced wsrep_cluster_size: 2Use the following command to check these parameters on each running Controller node. In this example, the Controller node IP addresses are 192.168.0.47 and 192.168.0.46:
(undercloud) $ for i in 192.168.24.6 192.168.24.7 ; do echo "*** $i ***" ; ssh heat-admin@$i "sudo podman exec \$(sudo podman ps --filter name=galera-bundle -q) mysql -e \"SHOW STATUS LIKE 'wsrep_local_state_comment'; SHOW STATUS LIKE 'wsrep_cluster_size';\""; done
(undercloud) $ for i in 192.168.24.6 192.168.24.7 ; do echo "*** $i ***" ; ssh heat-admin@$i "sudo podman exec \$(sudo podman ps --filter name=galera-bundle -q) mysql -e \"SHOW STATUS LIKE 'wsrep_local_state_comment'; SHOW STATUS LIKE 'wsrep_cluster_size';\""; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow
-
Check the RabbitMQ status. For example, if 192.168.0.47 is the IP address of a running Controller node, use the following command to view the RabbitMQ status:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo podman exec \$(sudo podman ps -f name=rabbitmq-bundle -q) rabbitmqctl cluster_status"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo podman exec \$(sudo podman ps -f name=rabbitmq-bundle -q) rabbitmqctl cluster_status"Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
running_nodeskey should show only the two available nodes and not the failed node.If fencing is enabled, disable it. For example, if 192.168.0.47 is the IP address of a running Controller node, use the following command to check the status of fencing:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs property show stonith-enabled"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs property show stonith-enabled"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the following command to disable fencing:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs property set stonith-enabled=false"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs property set stonith-enabled=false"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the Compute services are active on the director node:
(undercloud) $ openstack hypervisor list
(undercloud) $ openstack hypervisor listCopy to Clipboard Copied! Toggle word wrap Toggle overflow The output should show all non-maintenance mode nodes as
up.Ensure all undercloud containers are running:
(undercloud) $ sudo podman ps
(undercloud) $ sudo podman psCopy to Clipboard Copied! Toggle word wrap Toggle overflow
16.2. Removing a Ceph Monitor daemon Copy linkLink copied to clipboard!
If your Controller node is running a Ceph monitor service, complete the following steps to remove the ceph-mon daemon..
Adding a new Controller node to the cluster also adds a new Ceph monitor daemon automatically.
Procedure
Connect to the Controller node that you want to replace and become the root user:
ssh heat-admin@192.168.0.47 sudo su -
# ssh heat-admin@192.168.0.47 # sudo su -Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIf the Controller node is unreachable, skip steps 1 and 2 and continue the procedure at step 3 on any working Controller node.
Stop the monitor:
systemctl stop ceph-mon@<monitor_hostname>
# systemctl stop ceph-mon@<monitor_hostname>Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example:
systemctl stop ceph-mon@overcloud-controller-1
# systemctl stop ceph-mon@overcloud-controller-1Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Disconnect from the Controller node that you want to replace.
Connect to one of the existing Controller nodes.
ssh heat-admin@192.168.0.46 sudo su -
# ssh heat-admin@192.168.0.46 # sudo su -Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the monitor from the cluster:
sudo podman exec -it ceph-mon-controller-0 ceph mon remove overcloud-controller-1
# sudo podman exec -it ceph-mon-controller-0 ceph mon remove overcloud-controller-1Copy to Clipboard Copied! Toggle word wrap Toggle overflow On all Controller nodes, remove the v1 and v2 monitor entries from
/etc/ceph/ceph.conf. For example, if you remove controller-1, then remove the IPs and hostname for controller-1.Before:
mon host = [v2:172.18.0.21:3300,v1:172.18.0.21:6789],[v2:172.18.0.22:3300,v1:172.18.0.22:6789],[v2:172.18.0.24:3300,v1:172.18.0.24:6789] mon initial members = overcloud-controller-2,overcloud-controller-1,overcloud-controller-0
mon host = [v2:172.18.0.21:3300,v1:172.18.0.21:6789],[v2:172.18.0.22:3300,v1:172.18.0.22:6789],[v2:172.18.0.24:3300,v1:172.18.0.24:6789] mon initial members = overcloud-controller-2,overcloud-controller-1,overcloud-controller-0Copy to Clipboard Copied! Toggle word wrap Toggle overflow After:
mon host = [v2:172.18.0.21:3300,v1:172.18.0.21:6789],[v2:172.18.0.24:3300,v1:172.18.0.24:6789] mon initial members = overcloud-controller-2,overcloud-controller-0
mon host = [v2:172.18.0.21:3300,v1:172.18.0.21:6789],[v2:172.18.0.24:3300,v1:172.18.0.24:6789] mon initial members = overcloud-controller-2,overcloud-controller-0Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteDirector updates the
ceph.conffile on the relevant overcloud nodes when you add the replacement Controller node. Normally, director manages this configuration file exclusively and you should not edit the file manually. However, you can edit the file manually if you want to ensure consistency in case the other nodes restart before you add the new node.(Optional) Archive the monitor data and save the archive on another server:
mv /var/lib/ceph/mon/<cluster>-<daemon_id> /var/lib/ceph/mon/removed-<cluster>-<daemon_id>
# mv /var/lib/ceph/mon/<cluster>-<daemon_id> /var/lib/ceph/mon/removed-<cluster>-<daemon_id>Copy to Clipboard Copied! Toggle word wrap Toggle overflow
16.3. Preparing the cluster for Controller node replacement Copy linkLink copied to clipboard!
Before you replace the old node, you must ensure that Pacemaker is not running on the node and then remove that node from the Pacemaker cluster.
Procedure
To view the list of IP addresses for the Controller nodes, run the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the old node is still reachable, log in to one of the remaining nodes and stop pacemaker on the old node. For this example, stop pacemaker on overcloud-controller-1:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs status | grep -w Online | grep -w overcloud-controller-1" (undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster stop overcloud-controller-1"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs status | grep -w Online | grep -w overcloud-controller-1" (undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster stop overcloud-controller-1"Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIn case the old node is physically unavailable or stopped, it is not necessary to perform the previous operation, as pacemaker is already stopped on that node.
After you stop Pacemaker on the old node, delete the old node from the pacemaker cluster. The following example command logs in to
overcloud-controller-0to removeovercloud-controller-1:(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster node remove overcloud-controller-1"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster node remove overcloud-controller-1"Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the node that that you want to replace is unreachable (for example, due to a hardware failure), run the
pcscommand with additional--skip-offlineand--forceoptions to forcibly remove the node from the cluster:(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster node remove overcloud-controller-1 --skip-offline --force"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs cluster node remove overcloud-controller-1 --skip-offline --force"Copy to Clipboard Copied! Toggle word wrap Toggle overflow After you remove the old node from the pacemaker cluster, remove the node from the list of known hosts in pacemaker:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs host deauth overcloud-controller-1"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs host deauth overcloud-controller-1"Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can run this command whether the node is reachable or not.
The overcloud database must continue to run during the replacement procedure. To ensure that Pacemaker does not stop Galera during this procedure, select a running Controller node and run the following command on the undercloud with the IP address of the Controller node:
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs resource unmanage galera-bundle"
(undercloud) $ ssh heat-admin@192.168.0.47 "sudo pcs resource unmanage galera-bundle"Copy to Clipboard Copied! Toggle word wrap Toggle overflow
16.4. Replacing a Controller node Copy linkLink copied to clipboard!
To replace a Controller node, identify the index of the node that you want to replace.
- If the node is a virtual node, identify the node that contains the failed disk and restore the disk from a backup. Ensure that the MAC address of the NIC used for PXE boot on the failed server remains the same after disk replacement.
- If the node is a bare metal node, replace the disk, prepare the new disk with your overcloud configuration, and perform a node introspection on the new hardware.
- If the node is a part of a high availability cluster with fencing, you might need recover the Galera nodes separately. For more information, see the article How Galera works and how to rescue Galera clusters in the context of Red Hat OpenStack Platform.
Complete the following example steps to replace the the overcloud-controller-1 node with the overcloud-controller-3 node. The overcloud-controller-3 node has the ID 75b25e9a-948d-424a-9b3b-f0ef70a6eacf.
To replace the node with an existing bare metal node, enable maintenance mode on the outgoing node so that the director does not automatically reprovision the node.
Replacement of an overcloud Controller might cause swift rings to become inconsistent across nodes. This can result in decreased availability of Object Storage service. This is a known issue. If this happens, log in to the previously existing Controller node using SSH, deploy the updated rings, and restart the Object Storage containers:
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the index of the
overcloud-controller-1node:INSTANCE=$(openstack server list --name overcloud-controller-1 -f value -c ID)
$ INSTANCE=$(openstack server list --name overcloud-controller-1 -f value -c ID)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the bare metal node associated with the instance:
NODE=$(openstack baremetal node list -f csv --quote minimal | grep $INSTANCE | cut -f1 -d,)
$ NODE=$(openstack baremetal node list -f csv --quote minimal | grep $INSTANCE | cut -f1 -d,)Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the node to maintenance mode:
openstack baremetal node maintenance set $NODE
$ openstack baremetal node maintenance set $NODECopy to Clipboard Copied! Toggle word wrap Toggle overflow If the Controller node is a virtual node, run the following command on the Controller host to replace the virtual disk from a backup:
cp <VIRTUAL_DISK_BACKUP> /var/lib/libvirt/images/<VIRTUAL_DISK>
$ cp <VIRTUAL_DISK_BACKUP> /var/lib/libvirt/images/<VIRTUAL_DISK>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<VIRTUAL_DISK_BACKUP>with the path to the backup of the failed virtual disk, and replace<VIRTUAL_DISK>with the name of the virtual disk that you want to replace.If you do not have a backup of the outgoing node, you must use a new virtualized node.
If the Controller node is a bare metal node, complete the following steps to replace the disk with a new bare metal disk:
- Replace the physical hard drive or solid state drive.
- Prepare the node with the same configuration as the failed node.
List unassociated nodes and identify the ID of the new node:
openstack baremetal node list --unassociated
$ openstack baremetal node list --unassociatedCopy to Clipboard Copied! Toggle word wrap Toggle overflow Tag the new node with the
controlprofile:(undercloud) $ openstack baremetal node set --property capabilities='profile:control,boot_option:local' 75b25e9a-948d-424a-9b3b-f0ef70a6eacf
(undercloud) $ openstack baremetal node set --property capabilities='profile:control,boot_option:local' 75b25e9a-948d-424a-9b3b-f0ef70a6eacfCopy to Clipboard Copied! Toggle word wrap Toggle overflow
16.5. Triggering the Controller node replacement Copy linkLink copied to clipboard!
Complete the following steps to remove the old Controller node and replace it with a new Controller node.
Procedure
Determine the UUID of the node that you want to remove and store it in the
NODEIDvariable. Ensure that you replace NODE_NAME with the name of the node that you want to remove:NODEID=$(openstack server list -f value -c ID --name NODE_NAME)
$ NODEID=$(openstack server list -f value -c ID --name NODE_NAME)Copy to Clipboard Copied! Toggle word wrap Toggle overflow To identify the Heat resource ID, enter the following command:
openstack stack resource show overcloud ControllerServers -f json -c attributes | jq --arg NODEID "$NODEID" -c '.attributes.value | keys[] as $k | if .[$k] == $NODEID then "Node index \($k) for \(.[$k])" else empty end'
$ openstack stack resource show overcloud ControllerServers -f json -c attributes | jq --arg NODEID "$NODEID" -c '.attributes.value | keys[] as $k | if .[$k] == $NODEID then "Node index \($k) for \(.[$k])" else empty end'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the following environment file
~/templates/remove-controller.yamland include the node index of the Controller node that you want to remove:parameters: ControllerRemovalPolicies: [{'resource_list': ['NODE_INDEX']}]parameters: ControllerRemovalPolicies: [{'resource_list': ['NODE_INDEX']}]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Enter the overcloud deployment command, and include the
remove-controller.yamlenvironment file with any other environment files relevant to your environment:(undercloud) $ openstack overcloud deploy --templates \ -e /home/stack/templates/remove-controller.yaml \ -e /home/stack/templates/node-info.yaml \ [OTHER OPTIONS](undercloud) $ openstack overcloud deploy --templates \ -e /home/stack/templates/remove-controller.yaml \ -e /home/stack/templates/node-info.yaml \ [OTHER OPTIONS]Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteInclude
-e ~/templates/remove-controller.yamlonly for this instance of the deployment command. Remove this environment file from subsequent deployment operations.Director removes the old node, creates a new node, and updates the overcloud stack. You can check the status of the overcloud stack with the following command:
(undercloud) $ openstack stack list --nested
(undercloud) $ openstack stack list --nestedCopy to Clipboard Copied! Toggle word wrap Toggle overflow When the deployment command completes, director shows that the old node is replaced with the new node:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The new node now hosts running control plane services.
16.6. Cleaning up after Controller node replacement Copy linkLink copied to clipboard!
After you complete the node replacement, complete the following steps to finalize the Controller cluster.
Procedure
- Log into a Controller node.
Enable Pacemaker management of the Galera cluster and start Galera on the new node:
sudo pcs resource refresh galera-bundle sudo pcs resource manage galera-bundle
[heat-admin@overcloud-controller-0 ~]$ sudo pcs resource refresh galera-bundle [heat-admin@overcloud-controller-0 ~]$ sudo pcs resource manage galera-bundleCopy to Clipboard Copied! Toggle word wrap Toggle overflow Perform a final status check to ensure that the services are running correctly:
sudo pcs status
[heat-admin@overcloud-controller-0 ~]$ sudo pcs statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIf any services have failed, use the
pcs resource refreshcommand to resolve and restart the failed services.Exit to director:
exit
[heat-admin@overcloud-controller-0 ~]$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow Source the
overcloudrcfile so that you can interact with the overcloud:source ~/overcloudrc
$ source ~/overcloudrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the network agents in your overcloud environment:
(overcloud) $ openstack network agent list
(overcloud) $ openstack network agent listCopy to Clipboard Copied! Toggle word wrap Toggle overflow If any agents appear for the old node, remove them:
(overcloud) $ for AGENT in $(openstack network agent list --host overcloud-controller-1.localdomain -c ID -f value) ; do openstack network agent delete $AGENT ; done
(overcloud) $ for AGENT in $(openstack network agent list --host overcloud-controller-1.localdomain -c ID -f value) ; do openstack network agent delete $AGENT ; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow If necessary, add your router to the L3 agent host on the new node. Use the following example command to add a router named
r1to the L3 agent using the UUID 2d1c1dc1-d9d4-4fa9-b2c8-f29cd1a649d4:(overcloud) $ openstack network agent add router --l3 2d1c1dc1-d9d4-4fa9-b2c8-f29cd1a649d4 r1
(overcloud) $ openstack network agent add router --l3 2d1c1dc1-d9d4-4fa9-b2c8-f29cd1a649d4 r1Copy to Clipboard Copied! Toggle word wrap Toggle overflow Because compute services for the removed node still exist in the overcloud, you must remove them. First, check the compute services for the removed node:
source ~/overcloudrc (overcloud) $ openstack compute service list --host overcloud-controller-1.localdomain
[stack@director ~]$ source ~/overcloudrc (overcloud) $ openstack compute service list --host overcloud-controller-1.localdomainCopy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the compute services for the removed node:
(overcloud) $ for SERVICE in $(openstack compute service list --host overcloud-controller-1.localdomain -c ID -f value ) ; do openstack compute service delete $SERVICE ; done
(overcloud) $ for SERVICE in $(openstack compute service list --host overcloud-controller-1.localdomain -c ID -f value ) ; do openstack compute service delete $SERVICE ; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 17. Rebooting nodes Copy linkLink copied to clipboard!
You might need to reboot the nodes in the undercloud and overcloud. Use the following procedures to understand how to reboot different node types.
- If you reboot all nodes in one role, it is advisable to reboot each node individually. If you reboot all nodes in a role simultaneously, service downtime can occurduring the reboot operation.
- If you reboot all nodes in your OpenStack Platform environment, reboot the nodes in the following sequential order:
Recommended node reboot order
- Reboot the undercloud node.
- Reboot Controller and other composable nodes.
- Reboot standalone Ceph MON nodes.
- Reboot Ceph Storage nodes.
- Reboot Compute nodes.
17.1. Rebooting the undercloud node Copy linkLink copied to clipboard!
Complete the following steps to reboot the undercloud node.
Procedure
-
Log in to the undercloud as the
stackuser. Reboot the undercloud:
sudo reboot
$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the node boots.
17.2. Rebooting Controller and composable nodes Copy linkLink copied to clipboard!
Complete the following steps to reboot Controller nodes and standalone nodes based on composable roles, excluding Compute nodes and Ceph Storage nodes.
Procedure
- Log in to the node that you want to reboot.
Optional: If the node uses Pacemaker resources, stop the cluster:
sudo pcs cluster stop
[heat-admin@overcloud-controller-0 ~]$ sudo pcs cluster stopCopy to Clipboard Copied! Toggle word wrap Toggle overflow Reboot the node:
sudo reboot
[heat-admin@overcloud-controller-0 ~]$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the node boots.
Check the services. For example:
If the node uses Pacemaker services, check that the node has rejoined the cluster:
sudo pcs status
[heat-admin@overcloud-controller-0 ~]$ sudo pcs statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow If the node uses Systemd services, check that all services are enabled:
sudo systemctl status
[heat-admin@overcloud-controller-0 ~]$ sudo systemctl statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow If the node uses containerized services, check that all containers on the node are active:
sudo podman ps
[heat-admin@overcloud-controller-0 ~]$ sudo podman psCopy to Clipboard Copied! Toggle word wrap Toggle overflow
17.3. Rebooting standalone Ceph MON nodes Copy linkLink copied to clipboard!
Complete the following steps to reboot standalone Ceph MON nodes.
Procedure
- Log in to a Ceph MON node.
Reboot the node:
sudo reboot
$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the node boots and rejoins the MON cluster.
Repeat these steps for each MON node in the cluster.
17.4. Rebooting a Ceph Storage (OSD) cluster Copy linkLink copied to clipboard!
Complete the following steps to reboot a cluster of Ceph Storage (OSD) nodes.
Procedure
Log in to a Ceph MON or Controller node and disable Ceph Storage cluster rebalancing temporarily:
sudo podman exec -it ceph-mon-controller-0 ceph osd set noout sudo podman exec -it ceph-mon-controller-0 ceph osd set norebalance
$ sudo podman exec -it ceph-mon-controller-0 ceph osd set noout $ sudo podman exec -it ceph-mon-controller-0 ceph osd set norebalanceCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Select the first Ceph Storage node that you want to reboot and log in to the node.
Reboot the node:
sudo reboot
$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the node boots.
Log in to the node and check the cluster status:
sudo podman exec -it ceph-mon-controller-0 ceph status
$ sudo podman exec -it ceph-mon-controller-0 ceph statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check that the
pgmapreports allpgsas normal (active+clean).- Log out of the node, reboot the next node, and check its status. Repeat this process until you have rebooted all Ceph storage nodes.
When complete, log into a Ceph MON or Controller node and re-enable cluster rebalancing:
sudo podman exec -it ceph-mon-controller-0 ceph osd unset noout sudo podman exec -it ceph-mon-controller-0 ceph osd unset norebalance
$ sudo podman exec -it ceph-mon-controller-0 ceph osd unset noout $ sudo podman exec -it ceph-mon-controller-0 ceph osd unset norebalanceCopy to Clipboard Copied! Toggle word wrap Toggle overflow Perform a final status check to verify that the cluster reports
HEALTH_OK:sudo podman exec -it ceph-mon-controller-0 ceph status
$ sudo podman exec -it ceph-mon-controller-0 ceph statusCopy to Clipboard Copied! Toggle word wrap Toggle overflow
17.5. Rebooting Compute nodes Copy linkLink copied to clipboard!
Complete the following steps to reboot Compute nodes. To ensure minimal downtime of instances in your Red Hat OpenStack Platform environment, this procedure also includes instructions about migrating instances from the Compute node that you want to reboot. This involves the following workflow:
- Decide whether to migrate instances to another Compute node before rebooting the node.
- Select and disable the Compute node you want to reboot so that it does not provision new instances.
- Migrate the instances to another Compute node.
- Reboot the empty Compute node.
- Enable the empty Compute node.
Prerequisites
Before you reboot the Compute node, you must decide whether to migrate instances to another Compute node while the node is rebooting.
If for some reason you cannot or do not want to migrate the instances, you can set the following core template parameters to control the state of the instances after the Compute node reboots:
NovaResumeGuestsStateOnHostBoot-
Determines whether to return instances to the same state on the Compute node after reboot. When set to
False, the instances will remain down and you must start them manually. Default value is:False NovaResumeGuestsShutdownTimeout-
Number of seconds to wait for an instance to shut down before rebooting. It is not recommended to set this value to
0. Default value is: 300 NovaResumeGuestsShutdownTimeout-
Number of seconds to wait for an instance to shut down before rebooting. It is not recommended to set this value to
0. Default value is: 300
For more information about overcloud parameters and their usage, see Overcloud Parameters.
Procedure
-
Log in to the undercloud as the
stackuser. List all Compute nodes and their UUIDs:
source ~/stackrc (undercloud) $ openstack server list --name compute
$ source ~/stackrc (undercloud) $ openstack server list --name computeCopy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the UUID of the Compute node that you want to reboot.
From the undercloud, select a Compute node. Disable the node:
source ~/overcloudrc (overcloud) $ openstack compute service list (overcloud) $ openstack compute service set [hostname] nova-compute --disable
$ source ~/overcloudrc (overcloud) $ openstack compute service list (overcloud) $ openstack compute service set [hostname] nova-compute --disableCopy to Clipboard Copied! Toggle word wrap Toggle overflow List all instances on the Compute node:
(overcloud) $ openstack server list --host [hostname] --all-projects
(overcloud) $ openstack server list --host [hostname] --all-projectsCopy to Clipboard Copied! Toggle word wrap Toggle overflow - If you decide not to migrate instances, skip to this step.
If you decide to migrate the instances to another Compute node, use one of the following commands:
Migrate the instance to a different host:
(overcloud) $ openstack server migrate [instance-id] --live [target-host]--wait
(overcloud) $ openstack server migrate [instance-id] --live [target-host]--waitCopy to Clipboard Copied! Toggle word wrap Toggle overflow Let
nova-schedulerautomatically select the target host:(overcloud) $ nova live-migration [instance-id]
(overcloud) $ nova live-migration [instance-id]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Live migrate all instances at once:
nova host-evacuate-live [hostname]
$ nova host-evacuate-live [hostname]Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteThe
novacommand might cause some deprecation warnings, which are safe to ignore.
- Wait until migration completes.
Confirm that the migration was successful:
(overcloud) $ openstack server list --host [hostname] --all-projects
(overcloud) $ openstack server list --host [hostname] --all-projectsCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Continue to migrate instances until none remain on the chosen Compute node.
Log in to the Compute node and reboot the node:
sudo reboot
[heat-admin@overcloud-compute-0 ~]$ sudo rebootCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Wait until the node boots.
Re-enable the Compute node:
source ~/overcloudrc (overcloud) $ openstack compute service set [hostname] nova-compute --enable
$ source ~/overcloudrc (overcloud) $ openstack compute service set [hostname] nova-compute --enableCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check that the Compute node is enabled:
(overcloud) $ openstack compute service list
(overcloud) $ openstack compute service listCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Part IV. Additional director operations and configuration Copy linkLink copied to clipboard!
Chapter 18. Configuring custom SSL/TLS certificates Copy linkLink copied to clipboard!
You can configure the undercloud to use SSL/TLS for communication over public endpoints. However, if want to you use a SSL certificate with your own certificate authority, you must complete the following configuration steps.
18.1. Initializing the signing host Copy linkLink copied to clipboard!
The signing host is the host that generates and signs new certificates with a certificate authority. If you have never created SSL certificates on the chosen signing host, you might need to initialize the host so that it can sign new certificates.
Procedure
The
/etc/pki/CA/index.txtfile contains records of all signed certificates. Check if this file exists. If it does not exist, create an empty file:sudo touch /etc/pki/CA/index.txt
$ sudo touch /etc/pki/CA/index.txtCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
/etc/pki/CA/serialfile identifies the next serial number to use for the next certificate to sign. Check if this file exists. If the file does not exist, create a new file with a new starting value:echo '1000' | sudo tee /etc/pki/CA/serial
$ echo '1000' | sudo tee /etc/pki/CA/serialCopy to Clipboard Copied! Toggle word wrap Toggle overflow
18.2. Creating a certificate authority Copy linkLink copied to clipboard!
Normally you sign your SSL/TLS certificates with an external certificate authority. In some situations, you might want to use your own certificate authority. For example, you might want to have an internal-only certificate authority.
Procedure
- Generate a key and certificate pair to act as the certificate authority:
openssl genrsa -out ca.key.pem 4096 openssl req -key ca.key.pem -new -x509 -days 7300 -extensions v3_ca -out ca.crt.pem
$ openssl genrsa -out ca.key.pem 4096
$ openssl req -key ca.key.pem -new -x509 -days 7300 -extensions v3_ca -out ca.crt.pem
-
The
openssl reqcommand requests certain details about your authority. Enter these details at the prompt.
These commands create a certificate authority file called ca.crt.pem.
18.3. Adding the certificate authority to clients Copy linkLink copied to clipboard!
For any external clients aiming to communicate using SSL/TLS, copy the certificate authority file to each client that requires access to your Red Hat OpenStack Platform environment.
Procedure
Copy the certificate authority to the client system:
sudo cp ca.crt.pem /etc/pki/ca-trust/source/anchors/
$ sudo cp ca.crt.pem /etc/pki/ca-trust/source/anchors/Copy to Clipboard Copied! Toggle word wrap Toggle overflow After you copy the certificate authority file to each client, run the following command on each client to add the certificate to the certificate authority trust bundle:
sudo update-ca-trust extract
$ sudo update-ca-trust extractCopy to Clipboard Copied! Toggle word wrap Toggle overflow
18.4. Creating an SSL/TLS key Copy linkLink copied to clipboard!
Enabling SSL/TLS on an OpenStack environment requires an SSL/TLS key to generate your certificates.
Procedure
Run the following command to generate the SSL/TLS key (
server.key.pem):openssl genrsa -out server.key.pem 2048
$ openssl genrsa -out server.key.pem 2048Copy to Clipboard Copied! Toggle word wrap Toggle overflow
18.5. Creating an SSL/TLS certificate signing request Copy linkLink copied to clipboard!
Complete the following steps to create a certificate signing request.
Procedure
Copy the default OpenSSL configuration file:
cp /etc/pki/tls/openssl.cnf .
$ cp /etc/pki/tls/openssl.cnf .Copy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the new
openssl.cnffile and configure the SSL parameters that you want to use for director. An example of the types of parameters to modify include:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the
commonName_defaultto one of the following entries:-
If you are using an IP address to access director over SSL/TLS, use the
undercloud_public_hostparameter in theundercloud.conffile. If you are using a fully qualified domain name to access director over SSL/TLS, use the domain name.
Add
subjectAltName = @alt_namesto thev3_reqsection.Edit the
alt_namessection to include the following entries:-
IP- A list of IP addresses that clients use to access director over SSL. -
DNS- A list of domain names that clients use to access director over SSL. Also include the Public API IP address as a DNS entry at the end of thealt_namessection.
NoteFor more information about
openssl.cnf, run theman openssl.cnfcommand.-
If you are using an IP address to access director over SSL/TLS, use the
Run the following command to generate a certificate signing request (
server.csr.pem):openssl req -config openssl.cnf -key server.key.pem -new -out server.csr.pem
$ openssl req -config openssl.cnf -key server.key.pem -new -out server.csr.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Ensure that you include your OpenStack SSL/TLS key with the
-keyoption.
This command generates a server.csr.pem file, which is the certificate signing request. Use this file to create your OpenStack SSL/TLS certificate.
18.6. Creating the SSL/TLS certificate Copy linkLink copied to clipboard!
To generate the SSL/TLS certificate for your OpenStack environment, the following files must be present:
openssl.cnf- The customized configuration file that specifies the v3 extensions.
server.csr.pem- The certificate signing request to generate and sign the certificate with a certificate authority.
ca.crt.pem- The certificate authority, which signs the certificate.
ca.key.pem- The certificate authority private key.
Procedure
Run the following command to create a certificate for your undercloud or overcloud:
sudo openssl ca -config openssl.cnf -extensions v3_req -days 3650 -in server.csr.pem -out server.crt.pem -cert ca.crt.pem -keyfile ca.key.pem
$ sudo openssl ca -config openssl.cnf -extensions v3_req -days 3650 -in server.csr.pem -out server.crt.pem -cert ca.crt.pem -keyfile ca.key.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command uses the following options:
-config-
Use a custom configuration file, which is the
openssl.cnffile with v3 extensions. -extensions v3_req- Enabled v3 extensions.
-days- Defines how long in days until the certificate expires.
-in'- The certificate signing request.
-out- The resulting signed certificate.
-cert- The certificate authority file.
-keyfile- The certificate authority private key.
This command creates a new certificate named server.crt.pem. Use this certificate in conjunction with your OpenStack SSL/TLS key
18.7. Adding the certificate to the undercloud Copy linkLink copied to clipboard!
Complete the following steps to add your OpenStack SSL/TLS certificate to the undercloud trust bundle.
Procedure
Run the following command to combine the certificate and key:
cat server.crt.pem server.key.pem > undercloud.pem
$ cat server.crt.pem server.key.pem > undercloud.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command creates a
undercloud.pemfile.Copy the
undercloud.pemfile to a location within your/etc/pkidirectory and set the necessary SELinux context so that HAProxy can read it:sudo mkdir /etc/pki/undercloud-certs sudo cp ~/undercloud.pem /etc/pki/undercloud-certs/. sudo semanage fcontext -a -t etc_t "/etc/pki/undercloud-certs(/.*)?" sudo restorecon -R /etc/pki/undercloud-certs
$ sudo mkdir /etc/pki/undercloud-certs $ sudo cp ~/undercloud.pem /etc/pki/undercloud-certs/. $ sudo semanage fcontext -a -t etc_t "/etc/pki/undercloud-certs(/.*)?" $ sudo restorecon -R /etc/pki/undercloud-certsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Add the
undercloud.pemfile location to theundercloud_service_certificateoption in theundercloud.conffile:undercloud_service_certificate = /etc/pki/undercloud-certs/undercloud.pem
undercloud_service_certificate = /etc/pki/undercloud-certs/undercloud.pemCopy to Clipboard Copied! Toggle word wrap Toggle overflow Add the certificate authority that signed the certificate to the list of trusted Certificate Authorities on the undercloud so that different services within the undercloud have access to the certificate authority:
sudo cp ca.crt.pem /etc/pki/ca-trust/source/anchors/ sudo update-ca-trust extract
$ sudo cp ca.crt.pem /etc/pki/ca-trust/source/anchors/ $ sudo update-ca-trust extractCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 19. Additional introspection operations Copy linkLink copied to clipboard!
19.1. Performing individual node introspection Copy linkLink copied to clipboard!
To perform a single introspection on an available node, run the following commands to set the node to management mode and perform the introspection:
(undercloud) $ openstack baremetal node manage [NODE UUID] (undercloud) $ openstack overcloud node introspect [NODE UUID] --provide
(undercloud) $ openstack baremetal node manage [NODE UUID]
(undercloud) $ openstack overcloud node introspect [NODE UUID] --provide
After the introspection completes, the node changes to an available state.
19.2. Performing node introspection after initial introspection Copy linkLink copied to clipboard!
After an initial introspection, all nodes enter an available state due to the --provide option. To perform introspection on all nodes after the initial introspection, set all nodes to a manageable state and run the bulk introspection command:
(undercloud) $ for node in $(openstack baremetal node list --fields uuid -f value) ; do openstack baremetal node manage $node ; done (undercloud) $ openstack overcloud node introspect --all-manageable --provide
(undercloud) $ for node in $(openstack baremetal node list --fields uuid -f value) ; do openstack baremetal node manage $node ; done
(undercloud) $ openstack overcloud node introspect --all-manageable --provide
After the introspection completes, all nodes change to an available state.
19.3. Performing network introspection for interface information Copy linkLink copied to clipboard!
Network introspection retrieves link layer discovery protocol (LLDP) data from network switches. The following commands show a subset of LLDP information for all interfaces on a node, or full information for a particular node and interface. This can be useful for troubleshooting. Director enables LLDP data collection by default.
To get a list of interfaces on a node, run the following command:
(undercloud) $ openstack baremetal introspection interface list [NODE UUID]
(undercloud) $ openstack baremetal introspection interface list [NODE UUID]
For example:
To view interface data and switch port information, run the following command:
(undercloud) $ openstack baremetal introspection interface show [NODE UUID] [INTERFACE]
(undercloud) $ openstack baremetal introspection interface show [NODE UUID] [INTERFACE]
For example:
Retrieving hardware introspection details
The Bare Metal service hardware-inspection-extras feature is enabled by default, and you can use it to retrieve hardware details for overcloud configuration. For more information about the inspection_extras parameter in the undercloud.conf file, see Configuring the Director.
For example, the numa_topology collector is part of the hardware-inspection extras and includes the following information for each NUMA node:
- RAM (in kilobytes)
- Physical CPU cores and their sibling threads
- NICs associated with the NUMA node
To retrieve the information listed above, substitute <UUID> with the UUID of the bare-metal node to complete the following command:
openstack baremetal introspection data save <UUID> | jq .numa_topology
# openstack baremetal introspection data save <UUID> | jq .numa_topology
The following example shows the retrieved NUMA information for a bare-metal node:
Chapter 20. Automatically discovering bare metal nodes Copy linkLink copied to clipboard!
You can use auto-discovery to register overcloud nodes and generate their metadata, without the need to create an instackenv.json file. This improvement can help to reduce the time it takes to collect information about a node. For example, if you use auto-discovery, you do not to collate the IPMI IP addresses and subsequently create the instackenv.json.
20.1. Prerequisites Copy linkLink copied to clipboard!
- You have configured all overcloud nodes BMCs to be accessible to director through the IPMI.
- You have configured all overcloud nodes to PXE boot from the NIC that is connected to the undercloud control plane network.
20.2. Enabling auto-discovery Copy linkLink copied to clipboard!
Enable Bare Metal auto-discovery in the
undercloud.conffile:enable_node_discovery = True discovery_default_driver = ipmi
enable_node_discovery = True discovery_default_driver = ipmiCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
enable_node_discovery- When enabled, any node that boots the introspection ramdisk using PXE is enrolled in the Bare Metal service (ironic) automatically. -
discovery_default_driver- Sets the driver to use for discovered nodes. For example,ipmi.
-
Add your IPMI credentials to ironic:
Add your IPMI credentials to a file named
ipmi-credentials.json. Replace theSampleUsername,RedactedSecurePassword, andbmc_addressvalues in this example to suit your environment:Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Import the IPMI credentials file into ironic:
openstack baremetal introspection rule import ipmi-credentials.json
$ openstack baremetal introspection rule import ipmi-credentials.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow
20.3. Testing auto-discovery Copy linkLink copied to clipboard!
- Power on the required nodes.
Run the
openstack baremetal node listcommand. You should see the new nodes listed in anenrolledstate:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Set the resource class for each node:
for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node set $NODE --resource-class baremetal ; done
$ for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node set $NODE --resource-class baremetal ; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow Configure the kernel and ramdisk for each node:
for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node manage $NODE ; done $ openstack overcloud node configure --all-manageable
$ for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node manage $NODE ; done $ openstack overcloud node configure --all-manageableCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set all nodes to available:
for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node provide $NODE ; done
$ for NODE in `openstack baremetal node list -c UUID -f value` ; do openstack baremetal node provide $NODE ; doneCopy to Clipboard Copied! Toggle word wrap Toggle overflow
20.4. Using rules to discover different vendor hardware Copy linkLink copied to clipboard!
If you have a heterogeneous hardware environment, you can use introspection rules to assign credentials and remote management credentials. For example, you might want a separate discovery rule to handle your Dell nodes that use DRAC:
Create a file named
dell-drac-rules.jsonwith the following contents:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace the user name and password values in this example to suit your environment:
Import the rule into ironic:
openstack baremetal introspection rule import dell-drac-rules.json
$ openstack baremetal introspection rule import dell-drac-rules.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 21. Configuring automatic profile tagging Copy linkLink copied to clipboard!
The introspection process performs a series of benchmark tests. The director saves the data from these tests. You can create a set of policies that use this data in various ways:
- The policies can identify underperforming or unstable nodes and isolate these nodes from use in the overcloud.
- The policies can define whether to tag nodes into specific profiles automatically.
21.1. Policy file syntax Copy linkLink copied to clipboard!
Policy files use a JSON format that contains a set of rules. Each rule defines a description, a condition, and an action. A description is a plain text description of the rule, a condition defines an evaluation using a key-value pattern, and an action is the performance of the condition. .Description
A description is a plain text description of the rule.
Example:
"description": "A new rule for my node tagging policy"
"description": "A new rule for my node tagging policy"
Conditions
A condition defines an evaluation using the following key-value pattern:
- field
Defines the field to evaluate:
-
memory_mb- The amount of memory for the node in MB. -
cpus- The total number of threads for the node CPU. -
cpu_arch- The architecture of the node CPU. -
local_gb- The total storage space of the node root disk.
-
- op
Defines the operation to use for the evaluation. This includes the following attributes:
-
eq- Equal to -
ne- Not equal to -
lt- Less than -
gt- Greater than -
le- Less than or equal to -
ge- Greater than or equal to -
in-net- Checks that an IP address is in a given network -
matches- Requires a full match against a given regular expression -
contains- Requires a value to contain a given regular expression -
is-empty- Checks thatfieldis empty
-
- invert
- Boolean value to define whether to invert the result of the evaluation.
- multiple
Defines the evaluation to use if multiple results exist. This parameter includes the following attributes:
-
any- Requires any result to match -
all- Requires all results to match -
first- Requires the first result to match
-
- value
- Defines the value in the evaluation. If the field and operation result in the value, the condition return a true result. Otherwise, the condition returns a false result.
Example:
Actions
If a condition is true, the policy performs an action. The action uses the action key and additional keys depending on the value of action:
-
fail- Fails the introspection. Requires amessageparameter for the failure message. -
set-attribute- Sets an attribute on an ironic node. Requires apathfield, which is the path to an ironic attribute (for example,/driver_info/ipmi_address), and avalueto set. -
set-capability- Sets a capability on an ironic node. Requiresnameandvaluefields, which are the name and the value for a new capability. This replaces the existing value for this capability. For example, use this to define node profiles. -
extend-attribute- The same asset-attributebut treats the existing value as a list and appends value to it. If the optionaluniqueparameter is set to True, nothing is added if the given value is already in a list.
Example:
21.2. Policy file example Copy linkLink copied to clipboard!
The following is an example JSON file (rules.json) that contains introspection rules:
This example consists of three rules:
- Fail introspection if memory is lower than 4096 MiB. You can apply these types of rules if you want to exclude certain nodes from your cloud.
- Nodes with a hard drive size 1 TiB and bigger are assigned the swift-storage profile unconditionally.
-
Nodes with a hard drive less than 1 TiB but more than 40 GiB can be either Compute or Controller nodes. You can assign two capabilities (
compute_profileandcontrol_profile) so that theopenstack overcloud profiles matchcommand can later make the final choice. For this process to succeed, you must remove the existing profile capability, otherwise the existing profile capability has priority.
The profile matching rules do not change any other nodes.
Using introspection rules to assign the profile capability always overrides the existing value. However, [PROFILE]_profile capabilities are ignored for nodes that already have a profile capability.
21.3. Importing policy files Copy linkLink copied to clipboard!
To import policy files to director, complete the following steps.
Procedure
Import the policy file into director:
openstack baremetal introspection rule import rules.json
$ openstack baremetal introspection rule import rules.jsonCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the introspection process:
openstack overcloud node introspect --all-manageable
$ openstack overcloud node introspect --all-manageableCopy to Clipboard Copied! Toggle word wrap Toggle overflow After introspection completes, check the nodes and their assigned profiles:
openstack overcloud profiles list
$ openstack overcloud profiles listCopy to Clipboard Copied! Toggle word wrap Toggle overflow If you made a mistake in introspection rules, run the following command to delete all rules:
openstack baremetal introspection rule purge
$ openstack baremetal introspection rule purgeCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 22. Creating whole disk images Copy linkLink copied to clipboard!
The main overcloud image is a flat partition image that contains no partitioning information or bootloader. Director uses a separate kernel and ramdisk when it boots nodes and creates a basic partitioning layout when it writes the overcloud image to disk. However, you can create a whole disk image, which includes a partitioning layout, bootloader, and hardened security.
The following process uses the director image building feature. Red Hat only supports images that use the guidelines contained in this section. Custom images built outside of these specifications are not supported.
22.1. Security hardening measures Copy linkLink copied to clipboard!
The whole disk image includes extra security hardening measures necessary for Red Hat OpenStack Platform deployments where security is an important feature.
Security recommendations for image creation
-
The
/tmpdirectory is mounted on a separate volume or partition and has therw,nosuid,nodev,noexec, andrelatimeflags. -
The
/var,/var/logand the/var/log/auditdirectories are mounted on separate volumes or partitions, with therwandrelatimeflags. -
The
/homedirectory is mounted on a separate partition or volume and has therw,nodev, andrelatimeflags. Include the following changes to the
GRUB_CMDLINE_LINUXsetting:-
To enable auditing, add the
audit=1kernel boot flag. -
To disable the kernel support for USB using boot loader configuration, add
nousb. -
To remove the insecure boot flags, set
crashkernel=auto.
-
To enable auditing, add the
-
Blacklist insecure modules (
usb-storage,cramfs,freevxfs,jffs2,hfs,hfsplus,squashfs,udf,vfat) and prevent these modules from loading. -
Remove any insecure packages (
kdumpinstalled bykexec-toolsandtelnet) from the image because they are installed by default.
22.2. Whole disk image workflow Copy linkLink copied to clipboard!
To build a whole disk image, complete the following workflow:
- Download a base Red Hat Enterprise Linux 8 image.
- Set the environment variables specific to registration.
- Customize the image by modifying the partition schema and the size.
- Create the image.
- Upload the image to director.
22.3. Downloading the base cloud image Copy linkLink copied to clipboard!
Before you build a whole disk image, you must download an existing cloud image of Red Hat Enterprise Linux to use as a basis.
Procedure
Navigate to the Red Hat Customer Portal:
- Click DOWNLOADS on the top menu.
Click Red Hat Enterprise Linux 8.
NoteEnter your customer Customer Portal login details if a prompt appears.
Select the KVM Guest Image that you want to download. For example, the KVM Guest Image for the latest Red Hat Enterprise Linux is available on the following page:
22.4. Disk image environment variables Copy linkLink copied to clipboard!
As a part of the disk image building process, the director requires a base image and registration details to obtain packages for the new overcloud image. Define these attributes with the following Linux environment variables.
The image building process temporarily registers the image with a Red Hat subscription and unregisters the system when the image building process completes.
To build a disk image, set Linux environment variables that suit your environment and requirements:
- DIB_LOCAL_IMAGE
- Sets the local image that you want to use as the basis for your whole disk image.
- REG_ACTIVATION_KEY
- Use an activation key instead of login details as part of the registration process.
- REG_AUTO_ATTACH
- Defines whether to attach the most compatible subscription automatically.
- REG_BASE_URL
-
The base URL of the content delivery server that contains packages for the image. The default Customer Portal Subscription Management process uses
https://cdn.redhat.com. If you use a Red Hat Satellite 6 server, set this parameter to the base URL of your Satellite server. - REG_ENVIRONMENT
- Registers to an environment within an organization.
- REG_METHOD
-
Sets the method of registration. Use
portalto register a system to the Red Hat Customer Portal. Usesatelliteto register a system with Red Hat Satellite 6. - REG_ORG
- The organization where you want to register the images.
- REG_POOL_ID
- The pool ID of the product subscription information.
- REG_PASSWORD
- Sets the password for the user account that registers the image.
- REG_REPOS
A comma-separated string of repository names. Each repository in this string is enabled through
subscription-manager.Use the following repositories for a security hardened whole disk image:
-
rhel-8-for-x86_64-baseos-eus-rpms -
rhel-8-for-x86_64-appstream-eus-rpms -
rhel-8-for-x86_64-highavailability-eus-rpms -
ansible-2.8-for-rhel-8-x86_64-rpms -
openstack-16-for-rhel-8-x86_64-rpms
-
- REG_SAT_URL
- The base URL of the Satellite server to register overcloud nodes. Use the Satellite HTTP URL and not the HTTPS URL for this parameter. For example, use http://satellite.example.com and not https://satellite.example.com.
- REG_SERVER_URL
-
Sets the host name of the subscription service to use. The default host name is for the Red Hat Customer Portal at
subscription.rhn.redhat.com. If you use a Red Hat Satellite 6 server, set this parameter to the host name of your Satellite server. - REG_USER
- Sets the user name for the account that registers the image.
Use the following set of example commands to export a set of environment variables and temporarily register a local QCOW2 image to the Red Hat Customer Portal:
22.5. Customizing the disk layout Copy linkLink copied to clipboard!
The default security hardened image size is 20G and uses predefined partitioning sizes. However, you must modify the partitioning layout to accommodate overcloud container images. Complete the steps in the following sections to increase the image size to 40G. You can modify the partitioning layout and disk size to further suit your needs.
To modify the partitioning layout and disk size, perform the following steps:
-
Modify the partitioning schema using the
DIB_BLOCK_DEVICE_CONFIGenvironment variable. -
Modify the global size of the image by updating the
DIB_IMAGE_SIZEenvironment variable.
22.6. Modifying the partitioning schema Copy linkLink copied to clipboard!
You can modify the partitioning schema to alter the partitioning size, create new partitions, or remove existing partitions. Use the following environment variable to define a new partitioning schema:
export DIB_BLOCK_DEVICE_CONFIG='<yaml_schema_with_partitions>'
$ export DIB_BLOCK_DEVICE_CONFIG='<yaml_schema_with_partitions>'
The following YAML structure represents the modified logical volume partitioning layout to accommodate enough space to pull overcloud container images:
Use this sample YAML content as a basis for the partition schema of your image. Modify the partition sizes and layout to suit your needs.
You must define the correct partition sizes for the image because you cannot resize them after the deployment.
22.7. Modifying the image size Copy linkLink copied to clipboard!
The global sum of the modified partitioning schema might exceed the default disk size (20G). In this situation, you might need to modify the image size. To modify the image size, edit the configuration files that create the image.
Procedure
Create a copy of the
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-python3.yaml:cp /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-python3.yaml \ /home/stack/overcloud-hardened-images-python3-custom.yaml
# cp /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-python3.yaml \ /home/stack/overcloud-hardened-images-python3-custom.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteFor UEFI whole disk images, use
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-uefi-python3.yaml.Edit the
DIB_IMAGE_SIZEin the configuration file and adjust the values as necessary:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Adjust this value to the new total disk size.
- Save the file.
When you deploy the overcloud, the director creates a RAW version of the overcloud image. This means your undercloud must have enough free space to accommodate the RAW image. For example, if you set the security hardened image size to 40G, you must have 40G of space available on the undercloud hard disk.
When director writes the image to the physical disk, it creates a 64MB configuration drive primary partition at the end of the disk. When you create your whole disk image, ensure that the size of the physical disk accommodates this extra partition.
22.8. Building the whole disk image Copy linkLink copied to clipboard!
After you set the environment variables and customize the image, create the image using the openstack overcloud image build command.
Procedure
Run the
openstack overcloud image buildcommand with all necessary configuration files.openstack overcloud image build \ --image-name overcloud-hardened-full \ --config-file /home/stack/overcloud-hardened-images-python3-custom.yaml \ --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel8.yaml
# openstack overcloud image build \ --image-name overcloud-hardened-full \ --config-file /home/stack/overcloud-hardened-images-python3-custom.yaml \1 --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel8.yaml2 Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- This is the custom configuration file that contains the new disk size. If you are not using a different custom disk size, use the original
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-python3.yamlfile instead. For standard UEFI whole disk images, useovercloud-hardened-images-uefi-python3.yaml. - 2
- For UEFI whole disk images, use
overcloud-hardened-images-uefi-rhel8.yaml.
This command creates an image called
overcloud-hardened-full.qcow2, which contains all the necessary security features.
22.9. Uploading the whole disk image Copy linkLink copied to clipboard!
Upload the image to the OpenStack Image (glance) service and start using it from the Red Hat OpenStack Platform director. To upload a security hardened image, complete the following steps:
Rename the newly generated image and move the image to your
imagesdirectory:mv overcloud-hardened-full.qcow2 ~/images/overcloud-full.qcow2
# mv overcloud-hardened-full.qcow2 ~/images/overcloud-full.qcow2Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove all the old overcloud images:
openstack image delete overcloud-full openstack image delete overcloud-full-initrd openstack image delete overcloud-full-vmlinuz
# openstack image delete overcloud-full # openstack image delete overcloud-full-initrd # openstack image delete overcloud-full-vmlinuzCopy to Clipboard Copied! Toggle word wrap Toggle overflow Upload the new overcloud image:
openstack overcloud image upload --image-path /home/stack/images --whole-disk
# openstack overcloud image upload --image-path /home/stack/images --whole-diskCopy to Clipboard Copied! Toggle word wrap Toggle overflow
If you want to replace an existing image with the security hardened image, use the --update-existing flag. This flag overwrites the original overcloud-full image with a new security hardened image.
Chapter 23. Configuring Direct Deploy Copy linkLink copied to clipboard!
When provisioning nodes, director mounts the overcloud base operating system image on an iSCSI mount and then copies the image to disk on each node. Direct deploy is an alternative method that writes disk images from a HTTP location directly to disk on bare metal nodes.
23.1. Configuring the direct deploy interface on the undercloud Copy linkLink copied to clipboard!
The iSCSI deploy interface is the default deploy interface. However, you can enable the direct deploy interface to download an image from a HTTP location to the target disk.
Your overcloud node memory tmpfs must have at least 8GB of RAM.
Procedure
Create or modify a custom environment file
/home/stack/undercloud_custom_env.yamland specify theIronicDefaultDeployInterface.parameter_defaults: IronicDefaultDeployInterface: direct
parameter_defaults: IronicDefaultDeployInterface: directCopy to Clipboard Copied! Toggle word wrap Toggle overflow By default, the Bare Metal service (ironic) agent on each node obtains the image stored in the Object Storage service (swift) through a HTTP link. Alternatively, ironic can stream this image directly to the node through the
ironic-conductorHTTP server. To change the service that provides the image, set theIronicImageDownloadSourcetohttpin the/home/stack/undercloud_custom_env.yamlfile:parameter_defaults: IronicDefaultDeployInterface: direct IronicImageDownloadSource: http
parameter_defaults: IronicDefaultDeployInterface: direct IronicImageDownloadSource: httpCopy to Clipboard Copied! Toggle word wrap Toggle overflow Include the custom environment file in the
DEFAULTsection of theundercloud.conffile.custom_env_files = /home/stack/undercloud_custom_env.yaml
custom_env_files = /home/stack/undercloud_custom_env.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Perform the undercloud installation:
openstack undercloud install
$ openstack undercloud installCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 24. Creating virtualized control planes Copy linkLink copied to clipboard!
A virtualized control plane is a control plane located on virtual machines (VMs) rather than on bare metal. Use a virtualized control plane reduce the number of bare metal machines that you require for the control plane.
This chapter explains how to virtualize your Red Hat OpenStack Platform (RHOSP) control plane for the overcloud using RHOSP and Red Hat Virtualization.
24.1. Virtualized control plane architecture Copy linkLink copied to clipboard!
Use director to provision an overcloud using Controller nodes that are deployed in a Red Hat Virtualization cluster. You can then deploy these virtualized controllers as the virtualized control plane nodes.
Virtualized Controller nodes are supported only on Red Hat Virtualization.
The following architecture diagram illustrates how to deploy a virtualized control plane. Distribute the overcloud with the Controller nodes running on VMs on Red Hat Virtualization and run the Compute and Storage nodes on bare metal.
Run the OpenStack virtualized undercloud on Red Hat Virtualization.
Virtualized control plane architecture
The OpenStack Bare Metal Provisioning service (ironic) includes a driver for Red Hat Virtualization VMs, staging-ovirt. You can use this driver to manage virtual nodes within a Red Hat Virtualization environment. You can also use it to deploy overcloud controllers as virtual machines within a Red Hat Virtualization environment.
24.2. Benefits and limitations of virtualizing your RHOSP overcloud control plane Copy linkLink copied to clipboard!
Although there are a number of benefits to virtualizing your RHOSP overcloud control plane, this is not an option in every configuration.
Benefits
Virtualizing the overloud control plane has a number of benefits that prevent downtime and improve performance.
- You can allocate resources to the virtualized controllers dynamically, using hot add and hot remove to scale CPU and memory as required. This prevents downtime and facilitates increased capacity as the platform grows.
- You can deploy additional infrastructure VMs on the same Red Hat Virtualization cluster. This minimizes the server footprint in the data center and maximizes the efficiency of the physical nodes.
- You can use composable roles to define more complex RHOSP control planes and allocate resources to specific components of the control plane.
- You can maintain systems without service interruption with the VM live migration feature.
- You can integrate third-party or custom tools that Red Hat Virtualization supports.
Limitations
Virtualized control planes limit the types of configurations that you can use.
- Virtualized Ceph Storage nodes and Compute nodes are not supported.
- Block Storage (cinder) image-to-volume is not supported for back ends that use Fiber Channel. Red Hat Virtualization does not support N_Port ID Virtualization (NPIV). Therefore, Block Storage (cinder) drivers that need to map LUNs from a storage back end to the controllers, where cinder-volume runs by default, do not work. You must create a dedicated role for cinder-volume instead of including it on the virtualized controllers. For more information, see Composable Services and Custom Roles.
24.3. Provisioning virtualized controllers using the Red Hat Virtualization driver Copy linkLink copied to clipboard!
Complete the following steps to provision a virtualized RHOSP control plane for the overcloud using RHOSP and Red Hat Virtualization.
Prerequisites
- You must have a 64-bit x86 processor with support for the Intel 64 or AMD64 CPU extensions.
You must have the following software already installed and configured:
- Red Hat Virtualization. For more information, see Red Hat Virtualization Documentation Suite.
- Red Hat OpenStack Platform (RHOSP). For more information, see Director Installation and Usage.
- You must have the virtualized Controller nodes prepared in advance. These requirements are the same as for bare metal Controller nodes. For more information, see Controller Node Requirements.
- You must have the bare metal nodes being used as overcloud Compute nodes, and the storage nodes, prepared in advance. For hardware specifications, see the Compute Node Requirements and Ceph Storage Node Requirements. To deploy overcloud Compute nodes on POWER (ppc64le) hardware, see Red Hat OpenStack Platform for POWER.
- You must have the logical networks created, and your cluster of host networks ready to use network isolation with multiple networks. For more information, see Logical Networks.
- You must have the internal BIOS clock of each node set to UTC to prevent issues with future-dated file timestamps when hwclock synchronizes the BIOS clock before applying the timezone offset.
To avoid performance bottlenecks, use composable roles and keep the data plane services on the bare metal Controller nodes.
Procedure
To enable the
staging-ovirtdriver in director, add the driver to theenabled_hardware_typesparameter in theundercloud.confconfiguration file:enabled_hardware_types = ipmi,redfish,ilo,idrac,staging-ovirt
enabled_hardware_types = ipmi,redfish,ilo,idrac,staging-ovirtCopy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the undercloud contains the
staging-ovirtdriver:openstack baremetal driver list
(undercloud) [stack@undercloud ~]$ openstack baremetal driver listCopy to Clipboard Copied! Toggle word wrap Toggle overflow If you have configured the undercloud correctly, this command returns the following result:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the overcloud node definition template, for example,
nodes.json, to register the VMs hosted on Red Hat Virtualization with director. For more information, see Registering Nodes for the Overcloud. Use the following key:value pairs to define aspects of the VMs that you want to deploy with your overcloud:Expand Table 24.1. Configuring the VMs for the overcloud Key Set to this value pm_typeOpenStack Bare Metal Provisioning (ironic) service driver for oVirt/RHV VMs,
staging-ovirt.pm_userRed Hat Virtualization Manager username.
pm_passwordRed Hat Virtualization Manager password.
pm_addrHostname or IP of the Red Hat Virtualization Manager server.
pm_vm_nameName of the virtual machine in Red Hat Virtualization Manager where the controller is created.
For example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure one Controller on each Red Hat Virtualization Host
- Configure an affinity group in Red Hat Virtualization with "soft negative affinity" to ensure high availability is implemented for your controller VMs. For more information, see Affinity Groups.
- Open the Red Hat Virtualization Manager interface, and use it to map each VLAN to a separate logical vNIC in the controller VMs. For more information, see Logical Networks.
-
Set
no_filterin the vNIC of the director and controller VMs, and restart the VMs, to disable the MAC spoofing filter on the networks attached to the controller VMs. For more information, see Virtual Network Interface Cards. Deploy the overcloud to include the new virtualized controller nodes in your environment:
openstack overcloud deploy --templates
(undercloud) [stack@undercloud ~]$ openstack overcloud deploy --templatesCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Part V. Troubleshooting and tips Copy linkLink copied to clipboard!
Chapter 25. Troubleshooting director errors Copy linkLink copied to clipboard!
Errors can occur at certain stages of the director processes. This section contains some information about diagnosing common problems.
25.1. Troubleshooting node registration Copy linkLink copied to clipboard!
Issues with node registration usually occur due to issues with incorrect node details. In these situations, validate the template file containing your node details and correct the imported node details.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the node import command with the
--validate-onlyoption. This option validates your node template without performing an import:(undercloud) $ openstack overcloud node import --validate-only ~/nodes.json Waiting for messages on queue 'tripleo' with no timeout. Successfully validated environment file
(undercloud) $ openstack overcloud node import --validate-only ~/nodes.json Waiting for messages on queue 'tripleo' with no timeout. Successfully validated environment fileCopy to Clipboard Copied! Toggle word wrap Toggle overflow To fix incorrect details with imported nodes, run the
openstack baremetalcommands to update node details. The following example shows how to change networking details:Identify the assigned port UUID for the imported node:
source ~/stackrc (undercloud) $ openstack baremetal port list --node [NODE UUID]
$ source ~/stackrc (undercloud) $ openstack baremetal port list --node [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the MAC address:
(undercloud) $ openstack baremetal port set --address=[NEW MAC] [PORT UUID]
(undercloud) $ openstack baremetal port set --address=[NEW MAC] [PORT UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure a new IPMI address on the node:
(undercloud) $ openstack baremetal node set --driver-info ipmi_address=[NEW IPMI ADDRESS] [NODE UUID]
(undercloud) $ openstack baremetal node set --driver-info ipmi_address=[NEW IPMI ADDRESS] [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow
25.2. Troubleshooting hardware introspection Copy linkLink copied to clipboard!
You must run the introspection process to completion. However, ironic-inspector times out after a default one hour period if the inspection ramdisk does not respond. Sometimes this indicates a bug in the inspection ramdisk but usually this time-out occurs due to an environment misconfiguration, particularly BIOS boot settings.
To diagnose and resolve common environment misconfiguration issues, complete the following steps:
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Director uses OpenStack Object Storage (swift) to save the hardware data that it obtains during the introspection process. If this service is not running, the introspection can fail. Check all services related to OpenStack Object Storage to ensure that the service is running:
(undercloud) $ sudo systemctl list-units tripleo_swift*
(undercloud) $ sudo systemctl list-units tripleo_swift*Copy to Clipboard Copied! Toggle word wrap Toggle overflow Ensure that your nodes are in a
manageablestate. The introspection does not inspect nodes in anavailablestate, which is meant for deployment. If you want to inspect nodes that are in anavailablestate, change the node status tomanageablestate before introspection:(undercloud) $ openstack baremetal node manage [NODE UUID]
(undercloud) $ openstack baremetal node manage [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure temporary access to the introspection ramdisk. You can provide either a temporary password or an SSH key to access the node during introspection debugging. Complete the following procedure to configure ramdisk access:
Run the
openssl passwd -1command with a temporary password to generate an MD5 hash:(undercloud) $ openssl passwd -1 mytestpassword $1$enjRSyIw$/fYUpJwr6abFy/d.koRgQ/
(undercloud) $ openssl passwd -1 mytestpassword $1$enjRSyIw$/fYUpJwr6abFy/d.koRgQ/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Edit the
/var/lib/ironic/httpboot/inspector.ipxefile, find the line starting withkernel, and append therootpwdparameter and the MD5 hash:kernel http://192.2.0.1:8088/agent.kernel ipa-inspection-callback-url=http://192.168.0.1:5050/v1/continue ipa-inspection-collectors=default,extra-hardware,logs systemd.journald.forward_to_console=yes BOOTIF=${mac} ipa-debug=1 ipa-inspection-benchmarks=cpu,mem,disk rootpwd="$1$enjRSyIw$/fYUpJwr6abFy/d.koRgQ/" selinux=0kernel http://192.2.0.1:8088/agent.kernel ipa-inspection-callback-url=http://192.168.0.1:5050/v1/continue ipa-inspection-collectors=default,extra-hardware,logs systemd.journald.forward_to_console=yes BOOTIF=${mac} ipa-debug=1 ipa-inspection-benchmarks=cpu,mem,disk rootpwd="$1$enjRSyIw$/fYUpJwr6abFy/d.koRgQ/" selinux=0Copy to Clipboard Copied! Toggle word wrap Toggle overflow Alternatively, append your public SSH key to the
sshkeyparameter.NoteInclude quotation marks for both the
rootpwdandsshkeyparameters.
Run the introspection on the node:
(undercloud) $ openstack overcloud node introspect [NODE UUID] --provide
(undercloud) $ openstack overcloud node introspect [NODE UUID] --provideCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
--provideoption to change the node state toavailableafter the introspection completes.Identify the IP address of the node from the
dnsmasqlogs:(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/dnsmasq.log
(undercloud) $ sudo tail -f /var/log/containers/ironic-inspector/dnsmasq.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow If an error occurs, access the node using the root user and temporary access details:
ssh root@192.168.24.105
$ ssh root@192.168.24.105Copy to Clipboard Copied! Toggle word wrap Toggle overflow Access the node during introspection to run diagnostic commands and troubleshoot the introspection failure.
To stop the introspection process, run the following command:
(undercloud) $ openstack baremetal introspection abort [NODE UUID]
(undercloud) $ openstack baremetal introspection abort [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can also wait until the process times out.
NoteRed Hat OpenStack Platform director retries introspection three times after the initial abort. Run the
openstack baremetal introspection abortcommand at each attempt to abort the introspection completely.
25.3. Troubleshooting workflows and executions Copy linkLink copied to clipboard!
The OpenStack Workflow (mistral) service groups multiple OpenStack tasks into workflows. Red Hat OpenStack Platform uses a set of these workflows to perform common functions across the director, including bare metal node control, validations, plan management, and overcloud deployment.
For example, when you run the openstack overcloud deploy command, the OpenStack Workflow service executes two workflows. The first workflow uploads the deployment plan:
Removing the current plan files Uploading new plan files Started Mistral Workflow. Execution ID: aef1e8c6-a862-42de-8bce-073744ed5e6b Plan updated
Removing the current plan files
Uploading new plan files
Started Mistral Workflow. Execution ID: aef1e8c6-a862-42de-8bce-073744ed5e6b
Plan updated
The second workflow starts the overcloud deployment:
The OpenStack Workflow service uses the following objects to track the workflow:
- Actions
- A particular instruction that OpenStack performs when an associated task runs. Examples include running shell scripts or performing HTTP requests. Some OpenStack components have in-built actions that OpenStack Workflow uses.
- Tasks
- Defines the action to run and the result of running the action. These tasks usually have actions or other workflows associated with them. When a task completes, the workflow directs to another task, usually depending on whether the task succeeded or failed.
- Workflows
- A set of tasks grouped together and executed in a specific order.
- Executions
- Defines a particular action, task, or workflow running.
OpenStack Workflow also provides robust logging of executions, which helps to identify issues with certain command failures. For example, if a workflow execution fails, you can identify the point of failure.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow List the workflow executions that have the failed state
ERROR:(undercloud) $ openstack workflow execution list | grep "ERROR"
(undercloud) $ openstack workflow execution list | grep "ERROR"Copy to Clipboard Copied! Toggle word wrap Toggle overflow Get the UUID of the failed workflow execution (for example,
dffa96b0-f679-4cd2-a490-4769a3825262) and view the execution and output:(undercloud) $ openstack workflow execution show dffa96b0-f679-4cd2-a490-4769a3825262 (undercloud) $ openstack workflow execution output show dffa96b0-f679-4cd2-a490-4769a3825262
(undercloud) $ openstack workflow execution show dffa96b0-f679-4cd2-a490-4769a3825262 (undercloud) $ openstack workflow execution output show dffa96b0-f679-4cd2-a490-4769a3825262Copy to Clipboard Copied! Toggle word wrap Toggle overflow These commands return information about the failed task in the execution. The
openstack workflow execution showcommand also displays the workflow that was used for the execution (for example,tripleo.plan_management.v1.publish_ui_logs_to_swift). You can view the full workflow definition with the following command:(undercloud) $ openstack workflow definition show tripleo.plan_management.v1.publish_ui_logs_to_swift
(undercloud) $ openstack workflow definition show tripleo.plan_management.v1.publish_ui_logs_to_swiftCopy to Clipboard Copied! Toggle word wrap Toggle overflow This is useful for identifying where in the workflow a particular task occurs.
View action executions and their results using a similar command syntax:
(undercloud) $ openstack action execution list (undercloud) $ openstack action execution show 8a68eba3-0fec-4b2a-adc9-5561b007e886 (undercloud) $ openstack action execution output show 8a68eba3-0fec-4b2a-adc9-5561b007e886
(undercloud) $ openstack action execution list (undercloud) $ openstack action execution show 8a68eba3-0fec-4b2a-adc9-5561b007e886 (undercloud) $ openstack action execution output show 8a68eba3-0fec-4b2a-adc9-5561b007e886Copy to Clipboard Copied! Toggle word wrap Toggle overflow This is useful for identifying a specific action that causes issues.
25.4. Troubleshooting overcloud creation and deployment Copy linkLink copied to clipboard!
The initial creation of the overcloud occurs with the OpenStack Orchestration (heat) service. If an overcloud deployment fails, use the OpenStack clients and service log files to diagnose the failed deployment.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the deployment failures command:
openstack overcloud failures
$ openstack overcloud failuresCopy to Clipboard Copied! Toggle word wrap Toggle overflow Run the following command to display the details of the failure:
(undercloud) $ openstack stack failures list <OVERCLOUD_NAME> --long
(undercloud) $ openstack stack failures list <OVERCLOUD_NAME> --longCopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<OVERCLOUD_NAME>with the name of your overcloud.Run the following command to identify the stacks that failed:
(undercloud) $ openstack stack list --nested --property status=FAILED
(undercloud) $ openstack stack list --nested --property status=FAILEDCopy to Clipboard Copied! Toggle word wrap Toggle overflow
25.5. Troubleshooting node provisioning Copy linkLink copied to clipboard!
The OpenStack Orchestration (heat) service controls the provisioning process. If node provisioning fails, use the OpenStack clients and service log files to diagnose the issues.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the bare metal service to see all registered nodes and their current status:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow All nodes available for provisioning should have the following states set:
-
Maintenance set to
False. -
Provision State set to
availablebefore provisioning.
The following table outlines some common provisioning failure scenarios.
-
Maintenance set to
| Problem | Cause | Solution |
|---|---|---|
|
Maintenance sets itself to | The director cannot access the power management for the nodes. | Check the credentials for node power management. |
|
Provision State is set to | The problem occurred before bare metal deployment started. | Check the node details including the profile and flavor mapping. Check that the node hardware details are within the requirements for the flavor. |
|
Provision State is set to | The node provisioning process has not yet finished for this node. | Wait until this status changes. Otherwise, connect to the virtual console of the node and check the output. |
|
Provision State is | The node provisioning has finished successfully and there is a problem during the post-deployment configuration step. | Diagnose the node configuration process. Connect to the virtual console of the node and check the output. |
|
Provision State is | Node provisioning has failed. |
View the bare metal node details with the |
25.6. Troubleshooting IP address conflicts during provisioning Copy linkLink copied to clipboard!
Introspection and deployment tasks fail if the destination hosts are allocated an IP address that is already in use. To prevent these failures, you can perform a port scan of the Provisioning network to determine whether the discovery IP range and host IP range are free.
Procedure
Install
nmap:sudo dnf install nmap
$ sudo dnf install nmapCopy to Clipboard Copied! Toggle word wrap Toggle overflow Use
nmapto scan the IP address range for active addresses. This example scans the 192.168.24.0/24 range, replace this with the IP subnet of the Provisioning network (using CIDR bitmask notation):sudo nmap -sn 192.168.24.0/24
$ sudo nmap -sn 192.168.24.0/24Copy to Clipboard Copied! Toggle word wrap Toggle overflow Review the output of the
nmapscan. For example, you should see the IP address of the undercloud, and any other hosts that are present on the subnet:Copy to Clipboard Copied! Toggle word wrap Toggle overflow If any of the active IP addresses conflict with the IP ranges in undercloud.conf, you must either change the IP address ranges or release the IP addresses before you introspect or deploy the overcloud nodes.
25.7. Troubleshooting "No Valid Host Found" errors Copy linkLink copied to clipboard!
Sometimes the /var/log/nova/nova-conductor.log contains the following error:
NoValidHost: No valid host was found. There are not enough hosts available.
NoValidHost: No valid host was found. There are not enough hosts available.
This error occurs when the Compute Scheduler cannot find a bare metal node that is suitable for booting the new instance. This usually means that there is a mismatch between resources that the Compute service expects to find and resources that the Bare Metal service advertised to Compute. To check that there is a mismatch error, complete the following steps:
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check that the introspection succeeded on the node. If the introspection fails, check that each node contains the required ironic node properties:
(undercloud) $ openstack baremetal node show [NODE UUID]
(undercloud) $ openstack baremetal node show [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check that the
propertiesJSON field has valid values for keyscpus,cpu_arch,memory_mbandlocal_gb.Ensure that the Compute flavor that is mapped to the node does not exceed the node properties for the required number of nodes:
(undercloud) $ openstack flavor show [FLAVOR NAME]
(undercloud) $ openstack flavor show [FLAVOR NAME]Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Run the
openstack baremetal node listcommand to ensure that there are sufficient nodes in the available state. Nodes inmanageablestate usually signify a failed introspection. Run the
openstack baremetal node listcommand and ensure that the nodes are not in maintenance mode. If a node changes to maintenance mode automatically, the likely cause is an issue with incorrect power management credentials. Check the power management credentials and then remove maintenance mode:(undercloud) $ openstack baremetal node maintenance unset [NODE UUID]
(undercloud) $ openstack baremetal node maintenance unset [NODE UUID]Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
If you are using automatic profile tagging, check that you have enough nodes that correspond to each flavor and profile. Run the
openstack baremetal node showcommand on a node and check thecapabilitieskey in thepropertiesfield. For example, a node tagged for the Compute role contains theprofile:computevalue. You must wait for node information to propagate from Bare Metal to Compute after introspection. However, if you performed some steps manually, there might be a short period of time when nodes are not available to the Compute service (nova). Use the following command to check the total resources in your system:
(undercloud) $ openstack hypervisor stats show
(undercloud) $ openstack hypervisor stats showCopy to Clipboard Copied! Toggle word wrap Toggle overflow
25.8. Troubleshooting overcloud configuration Copy linkLink copied to clipboard!
OpenStack Platform director uses Ansible to configure the overcloud. Complete the following steps to diagnose Ansible playbook errors (config-download) on the overcloud.
Procedure
Ensure that the
stackuser has access to the files in the/var/lib/mistraldirectory on theundercloud:sudo setfacl -R -m u:stack:rwx /var/lib/mistral
$ sudo setfacl -R -m u:stack:rwx /var/lib/mistralCopy to Clipboard Copied! Toggle word wrap Toggle overflow This command retains
mistraluser access to the directory.Change to the working directory for the
config-downloadfiles. This is usually/var/lib/mistral/overcloud/.cd /var/lib/mistral/overcloud/
$ cd /var/lib/mistral/overcloud/Copy to Clipboard Copied! Toggle word wrap Toggle overflow Search the
ansible.logfile for the point of failure.less ansible.log
$ less ansible.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow Make a note of the step that failed.
-
Find the step that failed in the
config-downloadplaybooks within the working directory to identify the action that ocurred.
25.9. Troubleshooting container configuration Copy linkLink copied to clipboard!
OpenStack Platform director uses paunch to launch containers, podman to manage containers, and puppet to create container configuration. This procedure shows how to diagnose a container when errors occur.
Accessing the host
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Get the IP address of the node with the container failure.
(undercloud) $ openstack server list
(undercloud) $ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Log in to the node:
(undercloud) $ ssh heat-admin@192.168.24.60
(undercloud) $ ssh heat-admin@192.168.24.60Copy to Clipboard Copied! Toggle word wrap Toggle overflow Change to the root user:
sudo -i
$ sudo -iCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Identifying failed containers
View all containers:
podman ps --all
$ podman ps --allCopy to Clipboard Copied! Toggle word wrap Toggle overflow Identify the failed container. The failed container usually exits with a non-zero status.
Checking container logs
Each container retains standard output from its main process. Use this output as a log to help determine what actually occurs during a container run. For example, to view the log for the
keystonecontainer, run the following command:sudo podman logs keystone
$ sudo podman logs keystoneCopy to Clipboard Copied! Toggle word wrap Toggle overflow In most cases, this log contains information about the cause of a container failure.
The host also retains the
stdoutlog for the failed service. You can find thestdoutlogs in/var/log/containers/stdouts/. For example, to view the log for a failedkeystonecontainer, run the following command:cat /var/log/containers/stdouts/keystone.log
$ cat /var/log/containers/stdouts/keystone.logCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Inspecting containers
In some situations, you might need to verify information about a container. For example, use the following command to view keystone container data:
sudo podman inspect keystone
$ sudo podman inspect keystone
This command returns a JSON object containing low-level configuration data. You can pipe the output to the jq command to parse specific data. For example, to view the container mounts for the keystone container, run the following command:
sudo podman inspect keystone | jq .[0].Mounts
$ sudo podman inspect keystone | jq .[0].Mounts
You can also use the --format option to parse data to a single line, which is useful for running commands against sets of container data. For example, to recreate the options used to run the keystone container, use the following inspect command with the --format option:
sudo podman inspect --format='{{range .Config.Env}} -e "{{.}}" {{end}} {{range .Mounts}} -v {{.Source}}:{{.Destination}}:{{ join .Options "," }}{{end}} -ti {{.Config.Image}}' keystone
$ sudo podman inspect --format='{{range .Config.Env}} -e "{{.}}" {{end}} {{range .Mounts}} -v {{.Source}}:{{.Destination}}:{{ join .Options "," }}{{end}} -ti {{.Config.Image}}' keystone
The --format option uses Go syntax to create queries.
Use these options in conjunction with the podman run command to recreate the container for troubleshooting purposes:
OPTIONS=$( sudo podman inspect --format='{{range .Config.Env}} -e "{{.}}" {{end}} {{range .Mounts}} -v {{.Source}}:{{.Destination}}{{if .Mode}}:{{.Mode}}{{end}}{{end}} -ti {{.Config.Image}}' keystone )
sudo podman run --rm $OPTIONS /bin/bash
$ OPTIONS=$( sudo podman inspect --format='{{range .Config.Env}} -e "{{.}}" {{end}} {{range .Mounts}} -v {{.Source}}:{{.Destination}}{{if .Mode}}:{{.Mode}}{{end}}{{end}} -ti {{.Config.Image}}' keystone )
$ sudo podman run --rm $OPTIONS /bin/bash
Running commands in a container
In some cases, you might need to obtain information from within a container through a specific Bash command. In this situation, use the following podman command to execute commands within a running container. For example, run the podman exec command to run a command inside the keystone container:
sudo podman exec -ti keystone <COMMAND>
$ sudo podman exec -ti keystone <COMMAND>
The -ti options run the command through an interactive pseudoterminal.
Replace <COMMAND> with the command you want to run. For example, each container has a health check script to verify the service connection. You can run the health check script for keystone with the following command:
sudo podman exec -ti keystone /openstack/healthcheck
$ sudo podman exec -ti keystone /openstack/healthcheck
To access the container shell, run podman exec using /bin/bash as the command you want to run inside the container:
sudo podman exec -ti keystone /bin/bash
$ sudo podman exec -ti keystone /bin/bash
Viewing a container filesystem
To view the file system for the failed container, run the
podman mountcommand. For example, to view the file system for a failedkeystonecontainer, run the following command:podman mount keystone
$ podman mount keystoneCopy to Clipboard Copied! Toggle word wrap Toggle overflow This provides a mounted location to view the filesystem contents:
/var/lib/containers/storage/overlay/78946a109085aeb8b3a350fc20bd8049a08918d74f573396d7358270e711c610/merged
/var/lib/containers/storage/overlay/78946a109085aeb8b3a350fc20bd8049a08918d74f573396d7358270e711c610/mergedCopy to Clipboard Copied! Toggle word wrap Toggle overflow This is useful for viewing the Puppet reports within the container. You can find these reports in the
var/lib/puppet/directory within the container mount.
Exporting a container
When a container fails, you might need to investigate the full contents of the file. In this case, you can export the full file system of a container as a tar archive. For example, to export the keystone container file system, run the following command:
sudo podman export keystone -o keystone.tar
$ sudo podman export keystone -o keystone.tar
This command creates the keystone.tar archive, which you can extract and explore.
25.10. Troubleshooting Compute node failures Copy linkLink copied to clipboard!
Compute nodes use the Compute service to perform hypervisor-based operations. This means the main diagnosis for Compute nodes revolves around this service.
Procedure
Source the
stackrcfile:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Get the IP address of the Compute node that contains the failure:
(undercloud) $ openstack server list
(undercloud) $ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Log in to the node:
(undercloud) $ ssh heat-admin@192.168.24.60
(undercloud) $ ssh heat-admin@192.168.24.60Copy to Clipboard Copied! Toggle word wrap Toggle overflow Change to the root user:
sudo -i
$ sudo -iCopy to Clipboard Copied! Toggle word wrap Toggle overflow View the status of the container:
sudo podman ps -f name=nova_compute
$ sudo podman ps -f name=nova_computeCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
The primary log file for Compute nodes is
/var/log/containers/nova/nova-compute.log. If issues occur with Compute node communication, use this file to begin the diagnosis. - If you perform maintenance on the Compute node, migrate the existing instances from the host to an operational Compute node, then disable the node.
25.11. Creating an sosreport Copy linkLink copied to clipboard!
If you need to contact Red Hat for support with Red Hat OpenStack Platform, you might need to generate an sosreport. For more information about creating an sosreport, see:
25.12. Log locations Copy linkLink copied to clipboard!
Use the following logs to gather information about the undercloud and overcloud when you troubleshoot issues.
| Information | Log location |
|---|---|
| Containerized service logs |
|
| Standard output from containerized services |
|
| Ansible configuration logs |
|
| Information | Log location |
|---|---|
|
Command history for |
|
| Undercloud installation log |
|
| Information | Log location |
|---|---|
| Cloud-Init Log |
|
| High availability log |
|
Chapter 26. Tips for undercloud and overcloud services Copy linkLink copied to clipboard!
This section provides advice on tuning and managing specific OpenStack services on the undercloud.
26.1. Review the database flush intervals Copy linkLink copied to clipboard!
Some services use a cron container to flush old content from the database.
- OpenStack Identity (keystone): Flush expired tokens.
- OpenStack Orchestration (heat): Flush expired deleted template data.
- OpenStack Compute (nova): Flush expired deleted instance data.
The default flush periods for each service are listed in this table:
| Service | Database content flushed | Default flush period |
|---|---|---|
| OpenStack Identity (keystone) | Expired tokens | Every hour |
| OpenStack Orchestration (heat) | Deleted template data that has expired and is older than 30 days | Every day |
| OpenStack Compute (nova) | Archive deleted instance data | Every day |
| OpenStack Compute (nova) | Flush archived data older than 14 days | Every day |
The following tables outline the parameters that you can use to control these cron jobs.
| Parameter | Description |
|---|---|
|
|
Cron to purge expired tokens - Minute. The default value is: |
|
|
Cron to purge expired tokens - Hour. The default value is: |
|
|
Cron to purge expired tokens - Month Day. The default value is: |
|
|
Cron to purge expired tokens - Month. The default value is: |
|
|
Cron to purge expired tokens - Week Day. The default value is: |
| Parameter | Description |
|---|---|
|
|
Cron to purge database entries marked as deleted and older than $age - Age. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Age type. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Minute. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Hour. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Month Day. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Month. The default value is: |
|
|
Cron to purge database entries marked as deleted and older than $age - Week Day. The default value is: |
| Parameter | Description |
|
|
Cron to move deleted instances to another table - Max Rows. The default value is: |
|
|
Purge shadow tables immediately after scheduled archiving. The default value is: |
|
|
Cron to move deleted instances to another table - Minute. The default value is: |
|
|
Cron to move deleted instances to another table - Hour. The default value is: |
|
|
Cron to move deleted instances to another table - Month Day. The default value is: |
|
|
Cron to move deleted instances to another table - Month. The default value is: |
|
|
Cron to move deleted instances to another table - Week Day. The default value is: |
|
|
Cron to move deleted instances to another table - Until complete. The default value is: |
|
|
Cron to purge shadow tables - Age This will define the retention policy when purging the shadow tables in days. 0 means, purge data older than today in shadow tables. The default value is: |
|
|
Cron to purge shadow tables - Minute. The default value is: |
|
|
Cron to purge shadow tables - Hour. The default value is: |
|
|
Cron to purge shadow tables - Month Day. The default value is: |
|
|
Cron to purge shadow tables - Month. The default value is: |
|
|
Cron to purge shadow tables - Week Day. The default value is: |
To adjust these intervals, create an environment file that contains your token flush interval for the respective services and add this file to the custom_env_files parameter in your undercloud.conf file. For example, to change the OpenStack Identity (keystone) token flush to 30 minutes, use the following snippets
keystone-cron.yaml
parameter_defaults: KeystoneCronTokenFlushMinute: '0/30'
parameter_defaults:
KeystoneCronTokenFlushMinute: '0/30'
undercloud.yaml
custom_env_files: keystone-cron.yaml
custom_env_files: keystone-cron.yaml
Then rerun the openstack undercloud install command.
openstack undercloud install
$ openstack undercloud install
You can also use these parameters for your overcloud. For more information, see the Overcloud Parameters guide.
26.2. Tuning deployment performance Copy linkLink copied to clipboard!
OpenStack Platform director uses OpenStack Orchestration (heat) to conduct the main deployment and provisioning functions. Heat uses a series of workers to execute deployment tasks. To calculate the default number of workers, the director heat configuration halves the total CPU thread count of the undercloud. [2]. For example, if your undercloud has a CPU with 16 threads, heat spawns 8 workers by default. The director configuration also uses a minimum and maximum cap by default:
| Service | Minimum | Maximum |
|---|---|---|
| OpenStack Orchestration (heat) | 4 | 24 |
However, you can set the number of workers manually with the HeatWorkers parameter in an environment file:
heat-workers.yaml
parameter_defaults: HeatWorkers: 16
parameter_defaults:
HeatWorkers: 16
undercloud.yaml
custom_env_files: heat-workers.yaml
custom_env_files: heat-workers.yaml
26.3. Running swift-ring-builder in a container Copy linkLink copied to clipboard!
To manage your Object Storage (swift) rings, use the swift-ring-builder commands inside the server containers:
-
swift_object_server -
swift_container_server -
swift_account_server
For example, to view information about your swift object rings, run the following command:
sudo podman exec -ti -u swift swift_object_server swift-ring-builder /etc/swift/object.builder
$ sudo podman exec -ti -u swift swift_object_server swift-ring-builder /etc/swift/object.builder
You can run this command on both the undercloud and overcloud nodes.
26.4. Changing the SSL/TLS cipher rules for HAProxy Copy linkLink copied to clipboard!
If you enabled SSL/TLS in the undercloud (see Section 4.2, “Director configuration parameters”), you might want to harden the SSL/TLS ciphers and rules that are used with the HAProxy configuration. This hardening helps to avoid SSL/TLS vulnerabilities, such as the POODLE vulnerability.
Set the following hieradata using the hieradata_override undercloud configuration option:
- tripleo::haproxy::ssl_cipher_suite
- The cipher suite to use in HAProxy.
- tripleo::haproxy::ssl_options
- The SSL/TLS rules to use in HAProxy.
For example, you might want to use the following cipher and rules:
-
Cipher:
ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS -
Rules:
no-sslv3 no-tls-tickets
Create a hieradata override file (haproxy-hiera-overrides.yaml) with the following content:
tripleo::haproxy::ssl_cipher_suite: ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS tripleo::haproxy::ssl_options: no-sslv3 no-tls-tickets
tripleo::haproxy::ssl_cipher_suite: ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
tripleo::haproxy::ssl_options: no-sslv3 no-tls-tickets
The cipher collection is one continuous line.
Set the hieradata_override parameter in the undercloud.conf file to use the hieradata override file you created before you ran openstack undercloud install:
[DEFAULT] ... hieradata_override = haproxy-hiera-overrides.yaml ...
[DEFAULT]
...
hieradata_override = haproxy-hiera-overrides.yaml
...
Part VI. Appendices Copy linkLink copied to clipboard!
Appendix A. Power management drivers Copy linkLink copied to clipboard!
Although IPMI is the main method that director uses for power management control, director also supports other power management types. This appendix contains a list of the power management features that director supports. Use these power management settings when you register nodes for the overcloud. For more information, see Registering nodes for the overcloud.
A.1. Intelligent Platform Management Interface (IPMI) Copy linkLink copied to clipboard!
The standard power management method when you use a baseboard management controller (BMC).
- pm_type
-
Set this option to
ipmi. - pm_user; pm_password
- The IPMI username and password.
- pm_addr
- The IP address of the IPMI controller.
- pm_port (Optional)
- The port to connect to the IPMI controller.
A.2. Redfish Copy linkLink copied to clipboard!
A standard RESTful API for IT infrastructure developed by the Distributed Management Task Force (DMTF)
- pm_type
-
Set this option to
redfish. - pm_user; pm_password
- The Redfish username and password.
- pm_addr
- The IP address of the Redfish controller.
- pm_system_id
-
The canonical path to the system resource. This path must include the root service, version, and the path/unqiue ID for the system. For example:
/redfish/v1/Systems/CX34R87. - redfish_verify_ca
-
If the Redfish service in your baseboard management controller (BMC) is not configured to use a valid TLS certificate signed by a recognized certificate authority (CA), the Redfish client in ironic fails to connect to the BMC. Set the
redfish_verify_caoption tofalseto mute the error. However, be aware that disabling BMC authentication compromises the access security of your BMC.
A.3. Dell Remote Access Controller (DRAC) Copy linkLink copied to clipboard!
DRAC is an interface that provides out-of-band remote management features including power management and server monitoring.
- pm_type
-
Set this option to
idrac. - pm_user; pm_password
- The DRAC username and password.
- pm_addr
- The IP address of the DRAC host.
A.4. Integrated Lights-Out (iLO) Copy linkLink copied to clipboard!
iLO from Hewlett-Packard is an interface that provides out-of-band remote management features including power management and server monitoring.
- pm_type
-
Set this option to
ilo. - pm_user; pm_password
- The iLO username and password.
- pm_addr
The IP address of the iLO interface.
-
To enable this driver, add
iloto theenabled_hardware_typesoption in yourundercloud.confand rerunopenstack undercloud install. Director also requires an additional set of utilities for iLo. Install the
python3-proliantutilspackage and restart theopenstack-ironic-conductorservice:sudo dnf install python3-proliantutils sudo systemctl restart openstack-ironic-conductor.service
$ sudo dnf install python3-proliantutils $ sudo systemctl restart openstack-ironic-conductor.serviceCopy to Clipboard Copied! Toggle word wrap Toggle overflow - HP nodes must have a minimum ILO firmware version of 1.85 (May 13 2015) for successful introspection. Director has been successfully tested with nodes using this ILO firmware version.
- Using a shared iLO port is not supported.
-
To enable this driver, add
A.5. Fujitsu Integrated Remote Management Controller (iRMC) Copy linkLink copied to clipboard!
Fujitsu iRMC is a Baseboard Management Controller (BMC) with integrated LAN connection and extended functionality. This driver focuses on the power management for bare metal systems connected to the iRMC.
iRMC S4 or higher is required.
- pm_type
-
Set this option to
irmc. - pm_user; pm_password
- The username and password for the iRMC interface.
- pm_addr
- The IP address of the iRMC interface.
- pm_port (Optional)
- The port to use for iRMC operations. The default is 443.
- pm_auth_method (Optional)
-
The authentication method for iRMC operations. Use either
basicordigest. The default isbasic - pm_client_timeout (Optional)
- Timeout (in seconds) for iRMC operations. The default is 60 seconds.
- pm_sensor_method (Optional)
Sensor data retrieval method. Use either
ipmitoolorscci. The default isipmitool.-
To enable this driver, add
irmcto theenabled_hardware_typesoption in yourundercloud.confand rerun theopenstack undercloud installcommand. If you enable SCCI as the sensor method, you must also install an additional set of utilities. Install the
python3-scciclientpackage and restart theopenstack-ironic-conductorservice:dnf install python3-scciclient sudo systemctl restart openstack-ironic-conductor.service
$ dnf install python3-scciclient $ sudo systemctl restart openstack-ironic-conductor.serviceCopy to Clipboard Copied! Toggle word wrap Toggle overflow
-
To enable this driver, add
A.6. Red Hat Virtualization Copy linkLink copied to clipboard!
This driver provides control over virtual machines in Red Hat Virtualization (RHV) through its RESTful API.
- pm_type
-
Set this option to
staging-ovirt. - pm_user; pm_password
-
The username and password for your RHV environment. The username also includes the authentication provider. For example:
admin@internal. - pm_addr
- The IP address of the RHV REST API.
- pm_vm_name
- The name of the virtual machine to control.
- mac
A list of MAC addresses for the network interfaces on the node. Use only the MAC address for the Provisioning NIC of each system.
-
To enable this driver, add
staging-ovirtto theenabled_hardware_typesoption in yourundercloud.confand rerun theopenstack undercloud installcommand.
-
To enable this driver, add
A.7. manual-management Driver Copy linkLink copied to clipboard!
Use the manual-management driver to control bare metal devices that do not have power management. Director does not control the registered bare metal devices, and you must perform manual power operations at certain points in the introspection and deployment processes.
This option is available only for testing and evaluation purposes. It is not recommended for Red Hat OpenStack Platform enterprise environments.
- pm_type
Set this option to
manual-management.- This driver does not use any authentication details because it does not control power management.
-
To enable this driver, add
manual-managementto theenabled_hardware_typesoption in yourundercloud.confand rerun theopenstack undercloud installcommand. -
In your
instackenv.jsonnode inventory file, set thepm_typetomanual-managementfor the nodes that you want to manage manually. -
When performing introspection on nodes, manually start the nodes after running the
openstack overcloud node introspectcommand. -
When performing overcloud deployment, check the node status with the
openstack baremetal node listcommand. Wait until the node status changes fromdeployingtodeploy wait-callbackand then manually start the nodes. -
After the overcloud provisioning process completes, reboot the nodes. To check the completion of provisioning, check the node status with the
openstack baremetal node listcommand, wait until the node status changes toactive, then manually reboot all overcloud nodes.
Appendix B. Red Hat OpenStack Platform for POWER Copy linkLink copied to clipboard!
In a new Red Hat OpenStack Platform installation, you can deploy overcloud Compute nodes on POWER (ppc64le) hardware. For the Compute node cluster, you can use the same architecture, or use a combination of x86_64 and ppc64le systems. The undercloud, Controller nodes, Ceph Storage nodes, and all other systems are supported only on x86_64 hardware.
B.1. Ceph Storage Copy linkLink copied to clipboard!
When you configure access to external Ceph in a multi-architecture cloud, set the CephAnsiblePlaybook parameter to /usr/share/ceph-ansible/site.yml.sample and include your client key and other Ceph-specific parameters.
For example:
parameter_defaults: CephAnsiblePlaybook: /usr/share/ceph-ansible/site.yml.sample CephClientKey: AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ== CephClusterFSID: 4b5c8c0a-ff60-454b-a1b4-9747aa737d19 CephExternalMonHost: 172.16.1.7, 172.16.1.8
parameter_defaults:
CephAnsiblePlaybook: /usr/share/ceph-ansible/site.yml.sample
CephClientKey: AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ==
CephClusterFSID: 4b5c8c0a-ff60-454b-a1b4-9747aa737d19
CephExternalMonHost: 172.16.1.7, 172.16.1.8
B.2. Composable services Copy linkLink copied to clipboard!
The following services typically form part of the Controller node and are available for use in custom roles as Technology Preview:
- Block Storage service (cinder)
- Image service (glance)
- Identity service (keystone)
- Networking service (neutron)
- Object Storage service (swift)
Red Hat does not support features in Technology Preview.
For more information about composable services, see composable services and custom roles in the Advanced Overcloud Customization guide. Use the following example to understand how to move the listed services from the Controller node to a dedicated ppc64le node: