Chapter 11. Configuring Containerized Compute Nodes

download PDF

The director provides an option to integrate services from OpenStack’s containerization project (kolla) into the Overcloud’s Compute nodes. This includes creating Compute nodes that use Red Hat Enterprise Linux Atomic Host as a base operating system and individual containers to run different OpenStack services.


Containerized Compute nodes are a Technology Preview feature. Technology Preview features are not fully supported under Red Hat Subscription Service Level Agreements (SLAs), may not be functionally complete, and are not intended for production use. However, these features provide early access to upcoming product innovations, enabling customers to test functionality and provide feedback during the development process. For more information on the support scope for features marked as technology previews, see

The director’s core Heat template collection includes environment files to aid the configuration of containerized Compute nodes. These files include:

  • docker.yaml - The main environment file for configuring containerized Compute nodes.
  • docker-network.yaml - The environment file for containerized Compute nodes networking without network isolation.
  • docker-network-isolation.yaml - The environment file for containerized Compute nodes using network isolation.

11.1. Increasing the Stack Depth

To accommodate the number of resource stacks in the containerized compute Heat templates, you should increase the stack depth for OpenStack Orchestration (heat) on the undercloud. Use the following steps to increase the stack depth:

  1. Edit the /etc/heat/heat.conf and search for the max_nested_stack_depth parameter. Increase this parameter’s value to 10:

    max_nested_stack_depth = 10

    Save this file.

  2. Restart the OpenStack Orchestration (heat) service:

    sudo systemctl restart openstack-heat-engine.service

Undercloud minor and major version updates can revert changes to the /etc/heat/heat.conf file. If necessary, set the heat::engine::max_nested_stack_depth hieradata to ensure the stack depth is permanent. To set undercloud hieradata, point the hieradata_override parameter in your undercloud.conf file to a file containing the custom hieradata.

11.2. Examining the Containerized Compute Environment File (docker.yaml)

The docker.yaml file is the main environment file for the containerized Compute node configuration. It includes the entries in the resource_registry:

  OS::TripleO::ComputePostDeployment: ../docker/compute-post.yaml
  OS::TripleO::NodeUserData: ../docker/firstboot/install_docker_agents.yaml
Provides a Heat template that uses custom configuration on first boot. In this case, it installs the openstack-heat-docker-agents container on the Compute nodes when they first boot. This container provides a set of initialization scripts to configure the containerized Compute node and Heat hooks to communicate with the director.

Provides a Heat template with a set of post-configuration resources for Compute nodes. This includes a software configuration resource that provides a set of tags to Puppet:

    type: OS::Heat::SoftwareConfig
      group: puppet
        enable_hiera: True
        enable_facter: False
        tags: package,file,concat,file_line,nova_config,neutron_config,neutron_agent_ovs,neutron_plugin_ml2
      - name: tripleo::packages::enable_install
        type: Boolean
        default: True
      - name: result
        get_file: ../puppet/manifests/overcloud_compute.pp

These tags define the Puppet modules to pass to the openstack-heat-docker-agents container.

The docker.yaml file includes a parameter called NovaImage that replaces the standard overcloud-full image with a different image (atomic-image) when provisioning Compute nodes. See in Section 11.3, “Uploading the Atomic Host Image” for instructions on uploading this new image.

The docker.yaml file also includes a parameter_defaults section that defines the Docker registry and images to use for our Compute node services. You can modify this section to use a local registry instead of the default See Section 11.4, “Using a Local Registry” for instructions on configuring a local registry.

11.3. Uploading the Atomic Host Image

The director requires a copy of the Cloud Image for Red Hat Enterprise Linux 7 Atomic Host imported into its image store as atomic-image. This is because the Compute node requires this image for the base OS during the provisioning phase of the Overcloud creation.

Download a copy of the Cloud Image from the Red Hat Enterprise Linux 7 Atomic Host product page ( and save it to the images subdirectory in the stack user’s home directory.

Once the image download completes, import the image into the director as the stack user.

$ glance image-create --name atomic-image --file ~/images/rhel-atomic-cloud-7.2-12.x86_64.qcow2 --disk-format qcow2 --container-format bare

This imports the image alongside the other Overcloud images.

$ glance image-list
| ID                                   | Name                   |
| 27b5bad7-f8b2-4dd8-9f69-32dfe84644cf | atomic-image           |
| 08c116c6-8913-427b-b5b0-b55c18a01888 | bm-deploy-kernel       |
| aec4c104-0146-437b-a10b-8ebc351067b9 | bm-deploy-ramdisk      |
| 9012ce83-4c63-4cd7-a976-0c972be747cd | overcloud-full         |
| 376e95df-c1c1-4f2a-b5f3-93f639eb9972 | overcloud-full-initrd  |
| 0b5773eb-4c64-4086-9298-7f28606b68af | overcloud-full-vmlinuz |

11.4. Using a Local Registry

The default configuration uses Red Hat’s container registry for image downloads. However, as an optional step, you can use a local registry to conserve bandwidth during the Overcloud creation process.

You can use an existing local registry or install a new one. To install a new registry, use the instructions in "Get Started with Docker Formatted Container Images" in Getting Started with Containers.

Pull the required images into your registry:

$ sudo docker pull
$ sudo docker pull
$ sudo docker pull
$ sudo docker pull
$ sudo docker pull
$ sudo docker pull
$ sudo docker pull

After pulling the images, tag them with the proper registry host:

$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/
$ sudo docker tag localhost:8787/

Push them to the registry:

$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/
$ sudo docker push localhost:8787/

Create a copy of the main docker.yaml environment file in the templates subdirectory:

$ cp /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml ~/templates/.

Edit the file and modify the resource_registry to use absolute paths:

  OS::TripleO::ComputePostDeployment: /usr/share/openstack-tripleo-heat-templates/docker/compute-post.yaml
  OS::TripleO::NodeUserData: /usr/share/openstack-tripleo-heat-templates/docker/firstboot/install_docker_agents.yaml

Set DockerNamespace in parameter_defaults to your registry URL. Also set DockerNamespaceIsRegistry to true For example:

  DockerNamespaceIsRegistry: true

Your local registry now has the required docker images and the containerized Compute configuration is now set to use that registry.

11.5. Including Environment Files in the Overcloud Deployment

When running the Overcloud creation, include the main environment file (docker.yaml) and the network environment file (docker-network.yaml) for the containerized Compute nodes along with the openstack overcloud deploy command. For example:

$ openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-network.yaml [OTHER OPTIONS] ...

The containerized Compute nodes also function in an Overcloud with network isolation. This also requires the main environment file along with the network isolation file (docker-network-isolation.yaml). Add these files before the network isolation files from Chapter 7, Isolating Networks. For example:

openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml [OTHER OPTIONS] ...

The director creates an Overcloud with containerized Compute nodes.

Red Hat logoGithubRedditYoutubeTwitter


Try, buy, & sell


About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.