Chapter 5. Creating and managing resources for bare-metal instances
As a cloud operator you can create and manage resources for bare-metal workloads and enable your cloud users to create bare-metal instances.
You can create the following resources for bare-metal workloads:
- Bare-metal instances
- Images for bare-metal instances
- Virtual network interfaces (VIFs) for bare-metal nodes
- Port groups
You can perform the following resource management tasks:
- Manual node cleaning
- Attach a virtual network interface (VIF) to a bare-metal instance
5.1. Prerequisites Copy linkLink copied to clipboard!
- The RHOSO environment includes the Bare Metal Provisioning service. For more information, see Enabling the Bare Metal Provisioning service (ironic).
-
You are logged on to a workstation that has access to the RHOCP cluster, as a user with
cluster-adminprivileges. -
The
occommand line tool is installed on the workstation.
5.2. Launching bare-metal instances Copy linkLink copied to clipboard!
You can launch a bare-metal instance by using the OpenStack Client CLI.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the bare-metal instance:
openstack server create \ --nic net-id=<network_uuid> \ --flavor baremetal \ --image <image_uuid> \ myBareMetalInstance
$ openstack server create \ --nic net-id=<network_uuid> \ --flavor baremetal \ --image <image_uuid> \ myBareMetalInstanceCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<network_uuid>with the unique identifier for the network that you created to use with the Bare Metal Provisioning service. -
Replace
<image_uuid>with the unique identifier for the image that has the software profile that your instance requires.
-
Replace
Check the status of the instance:
openstack server list --name myBareMetalInstance
$ openstack server list --name myBareMetalInstanceCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.3. Images for launching bare-metal instances Copy linkLink copied to clipboard!
A Red Hat OpenStack Services on OpenShift (RHOSO) environment that includes the Bare Metal Provisioning service (ironic) requires two sets of images:
-
Deploy images: The deploy images are the
agent.ramdiskandagent.kernelimages that the Bare Metal Provisioning agent (ironic-python-agent) requires to boot the RAM disk over the network and copy the user image to the disk. User images: The images the cloud user uses to provision their bare-metal instances. The user image consists of a
kernelimage, aramdiskimage, and amainimage. The main image is either a root partition, or a whole-disk image:- Whole-disk image: An image that contains the partition table and boot loader.
- Root partition image: Contains only the root partition of the operating system.
Compatible whole-disk RHEL guest images should work without modification. To create your own custom disk image, see Creating RHEL KVM or RHOSP-compatible images in Creating and managing images.
5.4. Booting an ISO image directly for use as a RAM disk Copy linkLink copied to clipboard!
You can boot a bare-metal instance from a RAM disk or an ISO image if you want to boot an instance with PXE, iPXE, or Virtual Media, and use the instance memory for local storage. This is useful for advanced scientific and ephemeral workloads where writing an image to the local storage is not required or desired.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Specify
ramdiskas the deploy interface for the bare-metal node that boots from an ISO image:openstack baremetal node set --deploy-interface ramdisk
$ openstack baremetal node set --deploy-interface ramdiskCopy to Clipboard Copied! Toggle word wrap Toggle overflow TipYou can configure the deploy interface when you create the bare-metal node by adding
--deploy-interface ramdiskto theopenstack baremetal node createcommand. For information on how to create a bare-metal node, see Enrolling a bare-metal node manually.Update the bare-metal node to boot an ISO image:
openstack baremetal node set <node_UUID> \ --instance-info boot_iso=<boot_iso_url>$ openstack baremetal node set <node_UUID> \ --instance-info boot_iso=<boot_iso_url>Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<node_UUID>with the UUID of the bare-metal node that you want to boot from an ISO image. Replace
<boot_iso_url>with the URL of the boot ISO file. You can specify the boot ISO file URL by using one of the following methods:- HTTP or HTTPS URL
- File path URL
- Image service (glance) object UUID
-
Replace
Deploy the bare-metal node as an ISO image:
openstack baremetal node deploy <node_UUID>
$ openstack baremetal node deploy <node_UUID>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.5. Creating the virtual network interfaces (VIFs) for bare-metal instances Copy linkLink copied to clipboard!
Cloud users can attach their bare-metal instances to the network interfaces you create for the bare-metal workloads. You must create the virtual network interfaces (VIFs) for the cloud user to select for attachment.
5.5.1. Bare Metal Provisioning service virtual network interfaces (VIFs) Copy linkLink copied to clipboard!
The Bare Metal Provisioning service (ironic) uses the Networking service (neutron) to manage the attachment state of the virtual network interfaces (VIFs). A VIF is a Networking service port, referred to by the port ID, which is a UUID value. A VIF can be available across a limited number of physical networks, dependent upon the cloud’s operating configuration and operating constraints.
The Bare Metal Provisioning service can also attach the bare-metal instance to a separate provider network to improve the overall operational security.
Each VIF must be attached to a port or port group, therefore the maximum number of VIFs is determined by the number of configured and available ports represented in the Bare Metal Provisioning service.
The network interface is one of the driver interfaces that manages the network switching for bare-metal instances. The type of network interface you create influences the operation of your bare-metal workloads. The following network interfaces are available to use with the Bare Metal Provisioning service:
-
noop: Used for standalone deployments, and does not perform any network switching. -
flat: Places all nodes into a single provider network that is pre-configured on the Networking service and physical equipment. Nodes remain physically connected to this network during their entire life cycle. The supplied VIF attachment record is updated with new DHCP records as needed. When using this network interface, the VIF needs to be created on the same network that the bare-metal node is physically attached to. -
neutron: Provides tenant-defined networking through the Networking service, separating tenant networks from each other and from the provisioning and cleaning provider networks. Nodes move between these networks during their life cycle. This interface requires Networking service support for the switches attached to the bare-metal instances so they can be programmed. This interface requires the ML2 plugin OVN mechanism driver or other SDN integrations to facilitate port configuration on the network. Use theneutroninterface when your environment uses IPv6.
5.5.2. How the Bare Metal Provisioning service manages VIFs when provisioning a bare-metal node Copy linkLink copied to clipboard!
When provisioning, by default the Bare Metal Provisioning service (ironic) attempts to attach all PXE-enabled ports to the provisioning network. If you have neutron.add_all_ports enabled, then the Bare Metal Provisioning service attempts to bind all ports to the required service network beyond the Bare Metal Provisioning service ports with pxe_enabled set to True.
After the bare-metal nodes are provisioned, and before the bare-metal nodes are moved to the ACTIVE provisioning state, the previously attached ports are unbound. The process for unbinding is dependent on the network interface:
-
flat: All the requested VIFs with all binding configurations in all states are unbound. -
neutron: The VIFs requested by the cloud user are attached to the bare-metal node for the first time, because the VIFs that the Bare Metal Provisioning service created were being deleted during the provisioning process.
The same flow and logic applies to the cleaning, service, and rescue processes.
5.5.3. Creating a virtual network interface (VIF) for bare-metal nodes Copy linkLink copied to clipboard!
Use the Networking service (neutron) to create the port that serves as the virtual network interface (VIF). If you are using the neutron network interface, then you must also create a physical connection to the underlying physical network by creating a Bare Metal Provisioning service (ironic) port with a binding profile. The binding profile is required by the Networking service’s ML2 mechanism driver when a VIF is attached to a bare-metal instance. The binding profile includes the VNIC_BAREMETAL port type, the bare-metal node UUID, and local link connection information that identifies the tenant network that the ML2 mechanism driver must attach to the physical bare-metal port.
The binding profile information is populated through the introspection process by using LLDP data that is broadcast from the switches, therefore the switches must have LLDP enabled. You need to manually set or update the binding profile when there is a physical networking change, for example, when a bare-metal port’s cable has been moved to a different port on a switch, or the switch has been replaced.
Decoding LLDP data is performed as a best effort action. Some switch vendors, or changes in switch vendor firmware might impact field decoding.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create the virtual network interface (VIF):
openstack port create --network <network> <name>
$ openstack port create --network <network> <name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you are using the
neutronnetwork interface, then create a Bare Metal Provisioning service port with the binding profile information:Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<switch_mac_address>with the MAC address or OpenFlow-based datapath_id of the switch. -
Replace
<switch_hostname>with the name of the bare-metal node that hosts the switch. -
Replace
<switch_port_for_connection>with the port ID on the switch, for example,Gig0/1, orrep0-0. -
Replace
<phys_net>with the name of the physical network you want to associate with the bare-metal port. The Bare Metal Provisioning service uses the physical network to map the Networking service virtual ports to physical ports and port groups. If not set then any VIF is mapped to that port when there no bare-metal port with a suitable physical network assignment exists.
-
Replace
Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.6. Configuring port groups in the Bare Metal Provisioning service Copy linkLink copied to clipboard!
Port group functionality for bare-metal nodes is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should be used only for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.
Port groups (bonds) provide a method to aggregate multiple network interfaces into a single "bonded" interface. Port group configuration always takes precedence over an individual port configuration. During interface attachment, port groups have a higher priority than the ports, so they are used first. Currently, it is not possible to specify preference for port or port group in an interface attachment request. If a port group is available, the interface attachment will use it. Port groups that do not have any ports are ignored.
If a port group has a physical network, then all the ports in that port group must have the same physical network. The Bare Metal Provisioning service uses configdrive to support configuration of port groups in the instances.
Bare Metal Provisioning service API version 1.26 and later supports port group configuration.
To configure port groups in a bare metal deployment, you must configure the port groups on the switches manually. You must ensure that the mode and properties on the switch correspond to the mode and properties on the bare metal side as the naming can vary on the switch.
You cannot use port groups for provisioning and cleaning if you need to boot a deployment using iPXE.
With port group fallback, all the ports in a port group can fallback to individual switch ports when a connection fails. Based on whether a switch supports port group fallback or not, you can use the --support-standalone-ports and --unsupport-standalone-ports options.
5.6.1. Prerequisites Copy linkLink copied to clipboard!
- The RHOSO environment includes the Bare Metal Provisioning service. For more information, see Enabling the Bare Metal Provisioning service (ironic).
-
You are logged on to a workstation that has access to the RHOCP cluster, as a user with
cluster-adminprivileges. -
The
occommand line tool is installed on the workstation.
5.6.2. Configuring port groups in the Bare Metal Provisioning service Copy linkLink copied to clipboard!
Create a port group to aggregate multiple network interfaces into a single bonded interface.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Create a port group:
openstack baremetal port group create \ --node <node_uuid> --name <group_name> \ [--address <mac_address>] [--mode <mode>] \ --property miimon=100 --property xmit_hash_policy="layer2+3"
$ openstack baremetal port group create \ --node <node_uuid> --name <group_name> \ [--address <mac_address>] [--mode <mode>] \ --property miimon=100 --property xmit_hash_policy="layer2+3" [--support-standalone-ports]Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<node_uuid>with the UUID of the node that this port group belongs to. -
Replace
<group_name>with the name for this port group. -
Optional: Replace
<mac_address>with the MAC address for the port group. If you do not specify an address, the deployed instance port group address is the same as the Networking service port. If you do not attach the Networking service port, the port group configuration fails. -
Optional: Replace
<mode>with mode of the port group. - Specify if the group supports fallback to standalone ports.
NoteYou must configure port groups manually in standalone mode either in the image or by generating the
configdriveand adding it to the node’sinstance_info. Ensure that you havecloud-initversion 0.7.7 or later for the port group configuration to work.-
Replace
Associate a port with a port group:
During port creation:
openstack baremetal port create --node <node_uuid> --address <mac_address> --port-group <group_name>
$ openstack baremetal port create --node <node_uuid> --address <mac_address> --port-group <group_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow During port update:
openstack baremetal port set <port_uuid> --port-group <group_uuid>
$ openstack baremetal port set <port_uuid> --port-group <group_uuid>Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Boot an instance by providing an image that has
cloud-initor supports bonding.To check if the port group is configured properly, run the following command:
cat /proc/net/bonding/bondX
# cat /proc/net/bonding/bondXCopy to Clipboard Copied! Toggle word wrap Toggle overflow Here,
Xis a number thatcloud-initgenerates automatically for each configured port group, starting with a0and incremented by one for each configured port group.Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.7. Cleaning nodes manually Copy linkLink copied to clipboard!
The Bare Metal Provisioning service (ironic) cleans nodes automatically when they are unprovisioned to prepare them for provisioning. You can perform manual cleaning on specific nodes as required. Node cleaning has two modes:
- Metadata only clean: Removes partitions from all disks on the node. The metadata only mode of cleaning is faster than a full clean, but less secure because it erases only partition tables. Use this mode only on trusted tenant environments.
- Full clean: Removes all data from all disks, using either ATA secure erase or by shredding. A full clean can take several hours to complete.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check the current state of the node:
openstack baremetal node show \ -f value -c provision_state <node>
$ openstack baremetal node show \ -f value -c provision_state <node>Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<node>with the name or UUID of the node to clean.
-
Replace
If the node is not in the
manageablestate, then set it tomanageable:openstack baremetal node manage <node>
$ openstack baremetal node manage <node>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Clean the node:
openstack baremetal node clean <node> \ --clean-steps '[{"interface": "deploy", "step": "<clean_mode>"}]'$ openstack baremetal node clean <node> \ --clean-steps '[{"interface": "deploy", "step": "<clean_mode>"}]'Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<node>with the name or UUID of the node to clean. Replace
<clean_mode>with the type of cleaning to perform on the node:-
erase_devices: Performs a full clean. -
erase_devices_metadata: Performs a metadata only clean.
-
-
Replace
Wait for the clean to complete, then check the status of the node:
-
manageable: The clean was successful, and the node is ready to provision. -
clean failed: The clean was unsuccessful. Inspect thelast_errorfield for the cause of failure.
-
Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.8. Attaching a virtual network interface (VIF) to a bare-metal instance Copy linkLink copied to clipboard!
To attach a bare-metal instance to the bare-metal network interface, the cloud user can use the Compute service (nova) or the Bare Metal Provisioning service (ironic).
Compute service: Cloud users use the
openstack server add networkcommand. For more information, see Attaching a network to an instance.Note===
- When using the Compute service you must explicitly declare the port when creating the instance. When the Compute service makes a request to the Bare Metal Provisioning service to create an instance, the Compute service attempts to record all the VIFs the user requested to be attached in the Bare Metal Provisioning service to generate the metadata.
- You cannot specify which physical port to attach a VIF to when using the Compute service.If you want to explicitly declare which port to map to, then instead use the Bare Metal Provisioning service to create the attachment. ===
-
Bare Metal Provisioning service: Cloud users use the
openstack baremetal node vif attachcommand to attach a VIF to a bare-metal instance. For more information about virtual network interfaces (VIFs), see Bare Metal Provisioning service virtual network interfaces (VIFs).
The following procedure uses the Bare Metal Provisioning service to attach a bare-metal instance to a network. The Bare Metal Provisioning service creates the VIF attachment by using the UUID of the port you created with the Networking service .
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve the UUID of the bare-metal instance you want to attach the VIF to:
openstack server list
$ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Retrieve the UUID of the VIF you want to attach to your node:
openstack port list
$ openstack port listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: Retrieve the UUID of the bare-metal port you want to map the VIF to:
openstack baremetal port list
$ openstack baremetal port listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Attach the VIF to your bare-metal instance:
openstack baremetal node vif attach [--port-uuid <port_uuid>] \ <node> <vif_id>
$ openstack baremetal node vif attach [--port-uuid <port_uuid>] \ <node> <vif_id>Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Optional: Replace
<port_uuid>with the UUID of the bare-metal port to attach the VIF to. -
Replace
<node>with the name or UUID of the bare-metal instance you want to attach the VIF to. -
Replace
<vif_id>with the name or UUID of the VIF to attach to the bare-metal instance.
-
Optional: Replace
Exit the
openstackclientpod:exit
$ exitCopy to Clipboard Copied! Toggle word wrap Toggle overflow
5.8.1. How the Bare Metal Provisioning service attaches the VIF to a bare-metal instance Copy linkLink copied to clipboard!
When a cloud user requests that a virtual network interface (VIF) is attached to their bare-metal instance by using the openstack baremetal node vif attach command without a declared port or port group preference, the Bare Metal Provisioning service (ironic) selects a suitable unattached port or port group by evaluating the following criteria in order:
- Ports or port groups do not have a physical network or have a physical network that matches one of the VIF’s available physical networks.
- Prefer ports and port groups that have a physical network to ports and port groups that do not have a physical network.
- Prefer port groups to ports.
- Prefer ports with PXE enabled.
When the Bare Metal Provisioning service attaches any VIF to a bare-metal instance it explicitly sets the MAC address for the physical port to which the VIF is bound. If a node is already in an ACTIVE state, then the Networking service (neutron) updates the VIF attachment.
When the Bare Metal Provisioning service unbinds the VIF, it makes a request to the Networking service to reset the assigned MAC address to avoid conflicts with the Networking service’s unique hardware MAC address requirement.
5.8.2. Attaching and detaching virtual network interfaces Copy linkLink copied to clipboard!
The Bare Metal Provisioning service has an API that you can use to manage the mapping between virtual network interfaces. For example, the interfaces in the Networking service (neutron) and your physical interfaces (NICs). You can configure these interfaces for each bare-metal node to set the virtual network interface (VIF) to physical network interface (PIF) mapping logic.
Procedure
Access the remote shell for the
OpenStackClientpod from your workstation:oc rsh -n openstack openstackclient
$ oc rsh -n openstack openstackclientCopy to Clipboard Copied! Toggle word wrap Toggle overflow List the VIF IDs that are connected to the bare-metal node:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<node>with the name or UUID of the bare-metal node.
-
Replace
After the VIF is attached, the Bare Metal Provisioning service updates the virtual port in the Networking service with the MAC address of the physical port. Check this port address:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new port on the network where you created the bare-metal node:
openstack port create --network baremetal --fixed-ip ip-address=192.168.24.24 <port_name>
$ openstack port create --network baremetal --fixed-ip ip-address=192.168.24.24 <port_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the port from the bare-metal instance it was attached to:
openstack server remove port <instance_name> 4475bc5a-6f6e-466d-bcb6-6c2dce0fba16
$ openstack server remove port <instance_name> 4475bc5a-6f6e-466d-bcb6-6c2dce0fba16Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check that the IP address no longer exists on the list:
openstack server list
$ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check if there are VIFs attached to the node:
openstack baremetal node vif list <node> openstack port list
$ openstack baremetal node vif list <node> $ openstack port listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Add the newly created port:
openstack server add port <instance_name> <port_name>
$ openstack server add port <instance_name> <port_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that the new IP address shows the new port:
openstack server list
$ openstack server listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check if the VIF ID is the UUID of the new port:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check if the Networking service port MAC address is updated and matches one of the Bare Metal Provisioning service ports:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Reboot the bare-metal node so that it recognizes the new IP address:
openstack server reboot overcloud-baremetal-0
$ openstack server reboot overcloud-baremetal-0Copy to Clipboard Copied! Toggle word wrap Toggle overflow After you detach or attach interfaces, the bare-metal OS removes, adds, or modifies the network interfaces that have changed. When you replace a port, a DHCP request obtains the new IP address, but this might take some time because the old DHCP lease is still valid. To initiate these changes immediately, reboot the bare-metal node.