Chapter 3. Recommended Host Practices

In addition to pod traffic, the most-used data-path in an OpenShift Container Platform infrastructure is between the OpenShift Container Platform master hosts and etcd. The OpenShift Container Platform API server (part of the master binary) consults etcd for node status, network configuration, secrets, and more.

Optimize this traffic path by:

The OpenShift Container Platform node configuration file at /etc/origin/node/node-config.yaml contains important options, such as the iptables synchronization period, the Maximum Transmission Unit (MTU) of the SDN network, and the proxy-mode.

The node configuration file allows you to pass arguments to the kubelet (node) process. You can view a list of possible options by running kubelet --help.

In the /etc/origin/node/node-config.yaml file, two parameters control the maximum number of pods that can be scheduled to a node: pods-per-core and max-pods. When both options are in use, the lower of the two limits the number of pods on a node. Exceeding these values can result in:

pods-per-core sets the number of pods the node can run based on the number of processor cores on the node. For example, if pods-per-core is set to 10 on a node with 4 processor cores, the maximum number of pods allowed on the node will be 40.

kubeletArguments:
  pods-per-core:
    - "10"

kubeletArguments:
  pods-per-core:
    - "10"

max-pods sets the number of pods the node can run to a fixed value, regardless of the properties of the node.

kubeletArguments:
  max-pods:
    - "250"

kubeletArguments:
  max-pods:
    - "250"

Using the above example, the default value for pods-per-core is 10 and the default value for max-pods is 250. This means that unless the node has 25 cores or more, by default, pods-per-core will be the limiting factor.

See the Sizing Considerations section in the installation documentation for the recommended limits for an OpenShift Container Platform cluster. The recommended sizing accounts for OpenShift Container Platform and Docker coordination for container status updates. This coordination puts CPU pressure on the master and docker processes, which can include writing a large amount of log data.

etcd is a distributed key-value store that OpenShift Container Platform uses for configuration.

Optimize communication between etcd and master hosts either by co-locating them on the same host, or providing a dedicated network.

Tuned is a tuning profile delivery mechanism enabled by default in Red Hat Enterprise Linux and other Red Hat products. Tuned customizes Linux settings, such as sysctls, power management, and kernel command line options, to optimize the operating system for different workload performance and scalability requirements.

OpenShift Container Platform leverages the tuned daemon and includes Tuned profiles called atomic-openshift-host and atomic-openshift-guest. These profiles safely increase some of the commonly encountered vertical scaling limits present in the kernel, and are automatically applied to your system during installation.

The Tuned profiles support inheritance between profiles. On an OpenShift Container Platform system, the findings delivered by Tuned will be the union of throughput-performance (the default for RHEL) and atomic-openshift-guest. Tuned will determine if you are running OpenShift Container Platform on a virtual machine, and, if so, automatically apply virtual-guest tuning as well.

To see which Tuned profile is enabled on your system, run:

tuned-adm active

# tuned-adm active
Current active profile: atomic-openshift-node-guest

See the Red Hat Enterprise Linux Performance Tuning Guide for more information about Tuned.