Red Hat Summit Red Hat SummitSupportKonsoleEntwicklerDemo startenKontakt

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 6. Installer-provisioned postinstallation configuration

After successfully deploying an installer-provisioned cluster, consider the following postinstallation procedures.

6.1. Optional: Configuring NTP for disconnected clusters
Link kopieren

OpenShift Container Platform installs the chrony Network Time Protocol (NTP) service on the cluster nodes. Use the following procedure to configure NTP servers on the control plane nodes and configure compute nodes as NTP clients of the control plane nodes after a successful deployment.

Configuring NTP for disconnected clusters
View larger image
Configuring NTP for disconnected clusters

OpenShift Container Platform nodes must agree on a date and time to run properly. When compute nodes retrieve the date and time from the NTP servers on the control plane nodes, it enables the installation and operation of clusters that are not connected to a routable network and thereby do not have access to a higher stratum NTP server.

Procedure

  1. Install Butane on your installation host by using the following command: 

    $ sudo dnf -y install butane
    Copy to Clipboard Toggle word wrap

  2. Create a Butane config, 99-master-chrony-conf-override.bu, including the contents of the chrony.conf file for the control plane nodes.

    Note

    See "Creating machine configs with Butane" for information about Butane.

    Butane config example

    variant: openshift
version: 4.16.0
metadata:
  name: 99-master-chrony-conf-override
  labels:
    machineconfiguration.openshift.io/role: master
storage:
  files:
    - path: /etc/chrony.conf
      mode: 0644
      overwrite: true
      contents:
        inline: |
          # Use public servers from the pool.ntp.org project.
          # Please consider joining the pool (https://www.pool.ntp.org/join.html).

          # The Machine Config Operator manages this file
          server openshift-master-0.<cluster-name>.<domain> iburst
    1 
    
          server openshift-master-1.<cluster-name>.<domain> iburst
          server openshift-master-2.<cluster-name>.<domain> iburst

          stratumweight 0
          driftfile /var/lib/chrony/drift
          rtcsync
          makestep 10 3
          bindcmdaddress 127.0.0.1
          bindcmdaddress ::1
          keyfile /etc/chrony.keys
          commandkey 1
          generatecommandkey
          noclientlog
          logchange 0.5
          logdir /var/log/chrony

          # Configure the control plane nodes to serve as local NTP servers
          # for all compute nodes, even if they are not in sync with an
          # upstream NTP server.

          # Allow NTP client access from the local network.
          allow all
          # Serve time even if not synchronized to a time source.
          local stratum 3 orphan
    Copy to Clipboard Toggle word wrap

    1
    You must replace <cluster-name> with the name of the cluster and replace <domain> with the fully qualified domain name.

  3. Use Butane to generate a MachineConfig object file, 99-master-chrony-conf-override.yaml, containing the configuration to be delivered to the control plane nodes: 

    $ butane 99-master-chrony-conf-override.bu -o 99-master-chrony-conf-override.yaml
    Copy to Clipboard Toggle word wrap

  4. Create a Butane config, 99-worker-chrony-conf-override.bu, including the contents of the chrony.conf file for the compute nodes that references the NTP servers on the control plane nodes.

    Butane config example

    variant: openshift
version: 4.16.0
metadata:
  name: 99-worker-chrony-conf-override
  labels:
    machineconfiguration.openshift.io/role: worker
storage:
  files:
    - path: /etc/chrony.conf
      mode: 0644
      overwrite: true
      contents:
        inline: |
          # The Machine Config Operator manages this file.
          server openshift-master-0.<cluster-name>.<domain> iburst
    1 
    
          server openshift-master-1.<cluster-name>.<domain> iburst
          server openshift-master-2.<cluster-name>.<domain> iburst

          stratumweight 0
          driftfile /var/lib/chrony/drift
          rtcsync
          makestep 10 3
          bindcmdaddress 127.0.0.1
          bindcmdaddress ::1
          keyfile /etc/chrony.keys
          commandkey 1
          generatecommandkey
          noclientlog
          logchange 0.5
          logdir /var/log/chrony
    Copy to Clipboard Toggle word wrap

    1
    You must replace <cluster-name> with the name of the cluster and replace <domain> with the fully qualified domain name.

  5. Use Butane to generate a MachineConfig object file, 99-worker-chrony-conf-override.yaml, containing the configuration to be delivered to the worker nodes: 

    $ butane 99-worker-chrony-conf-override.bu -o 99-worker-chrony-conf-override.yaml
    Copy to Clipboard Toggle word wrap

  6. Apply the 99-master-chrony-conf-override.yaml policy to the control plane nodes. 

    $ oc apply -f 99-master-chrony-conf-override.yaml
    Copy to Clipboard Toggle word wrap

    Example output

    machineconfig.machineconfiguration.openshift.io/99-master-chrony-conf-override created
    Copy to Clipboard Toggle word wrap

  7. Apply the 99-worker-chrony-conf-override.yaml policy to the compute nodes. 

    $ oc apply -f 99-worker-chrony-conf-override.yaml
    Copy to Clipboard Toggle word wrap

    Example output

    machineconfig.machineconfiguration.openshift.io/99-worker-chrony-conf-override created
    Copy to Clipboard Toggle word wrap

  8. Check the status of the applied NTP settings. 

    $ oc describe machineconfigpool
    Copy to Clipboard Toggle word wrap

6.2. Enabling a provisioning network after installation
Link kopieren

The assisted installer and installer-provisioned installation for bare metal clusters provide the ability to deploy a cluster without a provisioning network. This capability is for scenarios such as proof-of-concept clusters or deploying exclusively with Redfish virtual media when each node’s baseboard management controller is routable via the baremetal network.

You can enable a provisioning network after installation using the Cluster Baremetal Operator (CBO).

Prerequisites

  • A dedicated physical network must exist, connected to all worker and control plane nodes.
  • You must isolate the native, untagged physical network.
  • The network cannot have a DHCP server when the provisioningNetwork configuration setting is set to Managed.
  • You can omit the provisioningInterface setting in OpenShift Container Platform 4.10 to use the bootMACAddress configuration setting.

Procedure

  1. When setting the provisioningInterface setting, first identify the provisioning interface name for the cluster nodes. For example, eth0 or eno1.
  2. Enable the Preboot eXecution Environment (PXE) on the provisioning network interface of the cluster nodes.

  3. Retrieve the current state of the provisioning network and save it to a provisioning custom resource (CR) file: 

    $ oc get provisioning -o yaml > enable-provisioning-nw.yaml
    Copy to Clipboard Toggle word wrap

  4. Modify the provisioning CR file: 

    $ vim ~/enable-provisioning-nw.yaml
    Copy to Clipboard Toggle word wrap

    Scroll down to the provisioningNetwork configuration setting and change it from Disabled to Managed. Then, add the provisioningIP, provisioningNetworkCIDR, provisioningDHCPRange, provisioningInterface, and watchAllNameSpaces configuration settings after the provisioningNetwork setting. Provide appropriate values for each setting. 

    apiVersion: v1
items:
- apiVersion: metal3.io/v1alpha1
  kind: Provisioning
  metadata:
    name: provisioning-configuration
  spec:
    provisioningNetwork:
    1 
    
    provisioningIP:
    2 
    
    provisioningNetworkCIDR:
    3 
    
    provisioningDHCPRange:
    4 
    
    provisioningInterface:
    5 
    
    watchAllNameSpaces:
    6
    Copy to Clipboard Toggle word wrap
    1
    The provisioningNetwork is one of Managed, Unmanaged, or Disabled. When set to Managed, Metal3 manages the provisioning network and the CBO deploys the Metal3 pod with a configured DHCP server. When set to Unmanaged, the system administrator configures the DHCP server manually.
    2
    The provisioningIP is the static IP address that the DHCP server and ironic use to provision the network. This static IP address must be within the provisioning subnet, and outside of the DHCP range. If you configure this setting, it must have a valid IP address even if the provisioning network is Disabled. The static IP address is bound to the metal3 pod. If the metal3 pod fails and moves to another server, the static IP address also moves to the new server.
    3
    The Classless Inter-Domain Routing (CIDR) address. If you configure this setting, it must have a valid CIDR address even if the provisioning network is Disabled. For example: 192.168.0.1/24.
    4
    The DHCP range. This setting is only applicable to a Managed provisioning network. Omit this configuration setting if the provisioning network is Disabled. For example: 192.168.0.64, 192.168.0.253.
    5
    The NIC name for the provisioning interface on cluster nodes. The provisioningInterface setting is only applicable to Managed and Unmanaged provisioning networks. Omit the provisioningInterface configuration setting if the provisioning network is Disabled. Omit the provisioningInterface configuration setting to use the bootMACAddress configuration setting instead.
    6
    Set this setting to true if you want metal3 to watch namespaces other than the default openshift-machine-api namespace. The default value is false.
  5. Save the changes to the provisioning CR file.

  6. Apply the provisioning CR file to the cluster: 

    $ oc apply -f enable-provisioning-nw.yaml
    Copy to Clipboard Toggle word wrap

6.3. Creating a manifest object that includes a customized br-ex bridge
Link kopieren

As an alternative to using the configure-ovs.sh shell script to set a br-ex bridge on a bare-metal platform, you can create a NodeNetworkConfigurationPolicy (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized br-ex bridge network configuration on each node in your cluster.

Important

After creating the NodeNetworkConfigurationPolicy CR, copy content from the NMState configuration file that was created during cluster installation into the NNCP CR. An incomplete NNCP CR file means that the the network policy described in the file cannot get applied to nodes in the cluster.

This feature supports the following tasks:

  • Modifying the maximum transmission unit (MTU) for your cluster.
  • Modifying attributes of a different bond interface, such as MIImon (Media Independent Interface Monitor), bonding mode, or Quality of Service (QoS).
  • Updating DNS values.

Consider the following use cases for creating a manifest object that includes a customized br-ex bridge:

  • You want to make postinstallation changes to the bridge, such as changing the Open vSwitch (OVS) or OVN-Kubernetes br-ex bridge network. The configure-ovs.sh shell script does not support making postinstallation changes to the bridge.
  • You want to deploy the bridge on a different interface than the interface available on a host or server IP address.
  • You want to make advanced configurations to the bridge that are not possible with the configure-ovs.sh shell script. Using the script for these configurations might result in the bridge failing to connect multiple network interfaces and facilitating data forwarding between the interfaces.

Prerequisites

  • You set a customized br-ex by using the alternative method to configure-ovs.
  • You installed the Kubernetes NMState Operator.

Procedure

  • Create a NodeNetworkConfigurationPolicy (NNCP) CR and define a customized br-ex bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the ipv4.address.ip, ipv6.address.ip, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block.

    Important

    As a post-installation task, you can configure most parameters for a customized br-ex bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized br-ex bridge.

    If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.

    Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses

    apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: worker-0-br-ex
    1 
    
spec:
  nodeSelector:
    kubernetes.io/hostname: worker-0
    desiredState:
    interfaces:
    - name: enp2s0
    2 
    
      type: ethernet
    3 
    
      state: up
    4 
    
      ipv4:
        enabled: false
    5 
    
      ipv6:
        enabled: false
    - name: br-ex
      type: ovs-bridge
      state: up
      ipv4:
        enabled: false
        dhcp: false
      ipv6:
        enabled: false
        dhcp: false
      bridge:
        options:
          mcast-snooping-enable: true
        port:
        - name: enp2s0
    6 
    
        - name: br-ex
    - name: br-ex
      type: ovs-interface
      state: up
      copy-mac-from: enp2s0
      ipv4:
        enabled: true
        dhcp: true
        auto-route-metric: 48
    7 
    
        address:
        - ip: "169.254.169.2"
          prefix-length: 29
      ipv6:
        enabled: true
        dhcp: true
        auto-route-metric: 48
        address:
        - ip: "fd69::2"
        prefix-length: 125
# ...
    Copy to Clipboard Toggle word wrap

    1
    Name of the policy.
    2
    Name of the interface.
    3
    The type of ethernet.
    4
    The requested state for the interface after creation.
    5
    Disables IPv4 and IPv6 in this example.
    6
    The node NIC to which the bridge is attached.
    7
    Set the parameter to 48 to ensure the br-ex default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the NetworkManager service.

Next steps

  • Scaling compute nodes to apply the manifest object that includes a customized br-ex bridge to each compute node that exists in your cluster. For more information, see "Expanding the cluster" in the Additional resources section.

6.4. Making disruptive changes to a customized br-ex bridge
Link kopieren

For certain situations, you might need to make disruptive changes to a br-ex bridge for planned maintenance or network configuration updates. A br-ex bridge is a gateway for all external network traffic from your workloads, so any change to the bridge might temporarily disconnect pods and virtual machines (VMs) from an external network.

The following procedure uses an example to show making disruptive changes to a br-ex bridge that minimizes any impact to running cluster workloads.

For all the nodes in your cluster to receive the br-ex bridge changes, you must reboot your cluster. Editing the existing MachineConfig object does not force a reboot operation, so you must create an additional MachineConfig object to force a reboot operation for the cluster.

Important

Red Hat does not support changing IP addresses for nodes as a postintallation task.

Prerequisites

  • You created a manifest object that includes a br-ex bridge.
  • You deployed your cluster that has the configured br-ex bridge.

Procedure

  1. Make changes to the NMState configuration file that you created during cluster installation for customizing your br-ex bridge network interface.

    Important

    Before you save the MachineConfig object, check the changed parameter values. If you enter wrong values and save the file, you cannot recover the file to its original state and this impacts networking functionality for your cluster.

  2. Use the base64 command to re-encode the contents of the NMState configuration by entering the following command: 

    $ base64 -w0 <nmstate_configuration>.yml
    1
    Copy to Clipboard Toggle word wrap
    1
    Replace <nmstate_configuration> with the name of your NMState resource YAML file.
  3. Update the MachineConfig manifest file that you created during cluster installation and re-define the customized br-ex bridge network interface.

  4. Apply the updates from the MachineConfig object to your cluster by entering the following command: 

    $ oc apply -f <machine_config>.yml
    Copy to Clipboard Toggle word wrap

  5. Create a bare MachineConfig object but do not make any configuration changes to the file. 

    apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: 10-force-reboot-master
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,
        mode: 0644
        overwrite: true
        path: /etc/force-reboot
---
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 10-force-reboot-worker
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,
        mode: 0644
        overwrite: true
        path: /etc/force-reboot
# ...
    Copy to Clipboard Toggle word wrap

  6. Start a reboot operation by applying the bare MachineConfig object configuration to your cluster by entering the following command: 

    $ oc apply -f <bare_machine_config>.yml
    Copy to Clipboard Toggle word wrap

  7. Check that each node in your cluster has the Ready status to indicate that they have finished rebooting by entering the following command: 

    $ oc get nodes
    Copy to Clipboard Toggle word wrap

  8. Delete the bare MachineConfig object by entering the following command: 

    $ oc delete machineconfig <machine_config_name>
    Copy to Clipboard Toggle word wrap

Verification

  • Use the nmstatectl tool to check the configuration for the br-ex bridge interface by running the following command. The tool checks a node that runs the br-ex bridge interface and not the location where you deployed the MachineConfig objects. 

    $ sudo nmstatectl show br-ex
    Copy to Clipboard Toggle word wrap
Nach oben
Red Hat logoGithubredditYoutubeTwitter

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Wir helfen Red Hat Benutzern, mit unseren Produkten und Diensten innovativ zu sein und ihre Ziele zu erreichen – mit Inhalten, denen sie vertrauen können. Entdecken Sie unsere neuesten Updates.

Mehr Inklusion in Open Source

Red Hat hat sich verpflichtet, problematische Sprache in unserem Code, unserer Dokumentation und unseren Web-Eigenschaften zu ersetzen. Weitere Einzelheiten finden Sie in Red Hat Blog.

Über Red Hat

Wir liefern gehärtete Lösungen, die es Unternehmen leichter machen, plattform- und umgebungsübergreifend zu arbeiten, vom zentralen Rechenzentrum bis zum Netzwerkrand.

Theme

© 2025 Red Hat