Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 7. Using Topology Manager

Topology Manager is a Kubelet component that collects hints from CPU Manager and Device Manager to align pod CPU and device resources on the same non-uniform memory access (NUMA) node.

Topology Manager uses topology information from collected hints to decide if the pod can be accepted or rejected from the node, based on the configured Topology Manager policy and Pod resources requested.

Topology Manager is useful for workloads that desire to use hardware accelerators to support latency-critical execution and high throughput parallel computation.

Note

Topology Manager is an alpha feature in OpenShift Container Platform.

7.1. Setting up Topology Manager
Copy link

Prequisites

  • Configure the CPU Manager policy to be static. Refer to Using CPU Manager in the Scalability and Performance section.

Procedure

  1. Enable the LatencySensitive FeatureGate 

    # oc edit featuregate/cluster
    Copy to Clipboard Toggle word wrap

  2. Add Feature Set: LatencySensitive to the spec: 

    # oc describe featuregate/cluster

Name:         cluster
Namespace:
Labels:       <none>
Annotations:  release.openshift.io/create-only: true
API Version:  config.openshift.io/v1
Kind:         FeatureGate
Metadata:
  Creation Timestamp:  2019-10-30T15:06:41Z
  Generation:          2
  Resource Version:    7773803
  Self Link:           /apis/config.openshift.io/v1/featuregates/cluster
  UID:                 b00204ab-cc5e-4ca5-ad93-b9bdd738c1de
Spec:
  Feature Set:  LatencySensitive
Events:         <none>
    Copy to Clipboard Toggle word wrap

  3. Configure the Topology Manager policy with KubeletConfig.

    The example YAML file below shows a single-numa-node policy specified. 

    apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: cpumanager-enabled
spec:
  machineConfigPoolSelector:
    matchLabels:
      custom-kubelet: cpumanager-enabled
  kubeletConfig:
     cpuManagerPolicy: static
     cpuManagerReconcilePeriod: 5s
     topologyManagerPolicy: single-numa-node
    1
    Copy to Clipboard Toggle word wrap
    1
    Specify your selected Topology Manager policy. 
    # oc create -f topologymanager-kubeletconfig.yaml
    Copy to Clipboard Toggle word wrap

7.2. Topology Manager policies
Copy link

Topology Manager works on Nodes and pods that meet the following conditions:

  • The Node’s CPU Manager Policy is configured as static.
  • The pods are in the Guaranteed QoS class.

When the above conditions are met, Topology Manager will align CPU and device requests for the Pod.

Topology Manager supports 4 allocation policies. These policies are set via a Kubelet flag, --topology-manager-policy. The policies are:

  • none (default)
  • best-effort
  • restricted
  • single-numa-node

7.2.1. none policy
Copy link

This is the default policy and does not perform any topology alignment.

7.2.2. best-effort policy
Copy link

For each container in a pod with the best-effort topology management policy, kubelet calls each Hint Provider to discover their resource availability. Using this information, the Topology Manager stores the preferred NUMA Node affinity for that container. If the affinity is not preferred, Topology Manager stores this and admits the pod to the node.

7.2.3. restricted policy
Copy link

For each container in a pod with the restricted topology management policy, kubelet calls each Hint Provider to discover their resource availability. Using this information, the Topology Manager stores the preferred NUMA Node affinity for that container. If the affinity is not preferred, Topology Manager rejects this pod from the node, resulting in a pod in a Terminated state with a pod admission failure.

7.2.4. single-numa-node
Copy link

For each container in a pod with the single-numa-node topology management policy, kubelet calls each Hint Provider to discover their resource availability. Using this information, the Topology Manager determines if a single NUMA Node affinity is possible. If it is, the pod is admitted to the node. If a single NUMA Node affinity is not possible, the Topology Manager rejects the pod from the node. This results in a pod in a Terminated state with a pod admission failure.

The example Pod specs below help illustrate pod interactions with Topology Manager.

The following pod runs in the BestEffort QoS class because no resource requests or limits are specified. 

spec:
  containers:
  - name: nginx
    image: nginx
Copy to Clipboard Toggle word wrap

The next pod runs in the Burstable QoS class because requests are less than limits. 

spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      limits:
        memory: "200Mi"
      requests:
        memory: "100Mi"
Copy to Clipboard Toggle word wrap

If the selected policy is anything other than none, Topology Manager would not consider either of these Pod specifications.

The last example pod below runs in the Guaranteed QoS class because requests are equal to limits. 

spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      limits:
        memory: "200Mi"
        cpu: "2"
        example.com/device: "1"
      requests:
        memory: "200Mi"
        cpu: "2"
        example.com/device: "1"
Copy to Clipboard Toggle word wrap

Topology Manager would consider this pod. The Topology Manager consults the CPU Manager static policy, which returns the topology of available CPUs. Topology Manager also consults Device Manager to discover the topology of available devices for example.com/device.

Topology Manager will use this information to store the best Topology for this container. In the case of this pod, CPU Manager and Device Manager will use this stored information at the resource allocation stage.

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat