Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.
Chapter 14. Scheduling resources
14.1. Using node selectors to move logging resources Link kopierenLink in die Zwischenablage kopiert!
A node selector specifies a map of key/value pairs that are defined using custom labels on nodes and selectors specified in pods.
For the pod to be eligible to run on a node, the pod must have the same key/value node selector as the label on the node.
14.1.1. About node selectors Link kopierenLink in die Zwischenablage kopiert!
You can use node selectors on pods and labels on nodes to control where the pod is scheduled. With node selectors, OpenShift Container Platform schedules the pods on nodes that contain matching labels.
You can use a node selector to place specific pods on specific nodes, cluster-wide node selectors to place new pods on specific nodes anywhere in the cluster, and project node selectors to place new pods in a project on specific nodes.
For example, as a cluster administrator, you can create an infrastructure where application developers can deploy pods only onto the nodes closest to their geographical location by including a node selector in every pod they create. In this example, the cluster consists of five data centers spread across two regions. In the U.S., label the nodes as
us-east
us-central
us-west
apac-east
apac-west
A pod is not scheduled if the
Pod
If you are using node selectors and node affinity in the same pod configuration, the following rules control pod placement onto nodes:
-
If you configure both and
nodeSelector, both conditions must be satisfied for the pod to be scheduled onto a candidate node.nodeAffinity -
If you specify multiple associated with
nodeSelectorTermstypes, then the pod can be scheduled onto a node if one of thenodeAffinityis satisfied.nodeSelectorTerms -
If you specify multiple associated with
matchExpressions, then the pod can be scheduled onto a node only if allnodeSelectorTermsare satisfied.matchExpressions
- Node selectors on specific pods and nodes
You can control which node a specific pod is scheduled on by using node selectors and labels.
To use node selectors and labels, first label the node to avoid pods being descheduled, then add the node selector to the pod.
NoteYou cannot add a node selector directly to an existing scheduled pod. You must label the object that controls the pod, such as deployment config.
For example, the following
object has theNodelabel:region: eastSample
Nodeobject with a labelkind: Node apiVersion: v1 metadata: name: ip-10-0-131-14.ec2.internal selfLink: /api/v1/nodes/ip-10-0-131-14.ec2.internal uid: 7bc2580a-8b8e-11e9-8e01-021ab4174c74 resourceVersion: '478704' creationTimestamp: '2019-06-10T14:46:08Z' labels: kubernetes.io/os: linux topology.kubernetes.io/zone: us-east-1a node.openshift.io/os_version: '4.5' node-role.kubernetes.io/worker: '' topology.kubernetes.io/region: us-east-1 node.openshift.io/os_id: rhcos node.kubernetes.io/instance-type: m4.large kubernetes.io/hostname: ip-10-0-131-14 kubernetes.io/arch: amd64 region: east1 type: user-node #...- 1
- Labels to match the pod node selector.
A pod has the
node selector:type: user-node,region: eastSample
Podobject with node selectorsapiVersion: v1 kind: Pod metadata: name: s1 #... spec: nodeSelector:1 region: east type: user-node #...- 1
- Node selectors to match the node label. The node must have a label for each node selector.
When you create the pod using the example pod spec, it can be scheduled on the example node.
- Default cluster-wide node selectors
With default cluster-wide node selectors, when you create a pod in that cluster, OpenShift Container Platform adds the default node selectors to the pod and schedules the pod on nodes with matching labels.
For example, the following
object has the default cluster-wideSchedulerandregion=eastnode selectors:type=user-nodeExample Scheduler Operator Custom Resource
apiVersion: config.openshift.io/v1 kind: Scheduler metadata: name: cluster #... spec: defaultNodeSelector: type=user-node,region=east #...A node in that cluster has the
labels:type=user-node,region=eastExample
NodeobjectapiVersion: v1 kind: Node metadata: name: ci-ln-qg1il3k-f76d1-hlmhl-worker-b-df2s4 #... labels: region: east type: user-node #...Example
Podobject with a node selectorapiVersion: v1 kind: Pod metadata: name: s1 #... spec: nodeSelector: region: east #...When you create the pod using the example pod spec in the example cluster, the pod is created with the cluster-wide node selector and is scheduled on the labeled node:
Example pod list with the pod on the labeled node
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-s1 1/1 Running 0 20s 10.131.2.6 ci-ln-qg1il3k-f76d1-hlmhl-worker-b-df2s4 <none> <none>NoteIf the project where you create the pod has a project node selector, that selector takes preference over a cluster-wide node selector. Your pod is not created or scheduled if the pod does not have the project node selector.
- Project node selectors
With project node selectors, when you create a pod in this project, OpenShift Container Platform adds the node selectors to the pod and schedules the pods on a node with matching labels. If there is a cluster-wide default node selector, a project node selector takes preference.
For example, the following project has the
node selector:region=eastExample
NamespaceobjectapiVersion: v1 kind: Namespace metadata: name: east-region annotations: openshift.io/node-selector: "region=east" #...The following node has the
labels:type=user-node,region=eastExample
NodeobjectapiVersion: v1 kind: Node metadata: name: ci-ln-qg1il3k-f76d1-hlmhl-worker-b-df2s4 #... labels: region: east type: user-node #...When you create the pod using the example pod spec in this example project, the pod is created with the project node selectors and is scheduled on the labeled node:
Example
PodobjectapiVersion: v1 kind: Pod metadata: namespace: east-region #... spec: nodeSelector: region: east type: user-node #...Example pod list with the pod on the labeled node
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod-s1 1/1 Running 0 20s 10.131.2.6 ci-ln-qg1il3k-f76d1-hlmhl-worker-b-df2s4 <none> <none>A pod in the project is not created or scheduled if the pod contains different node selectors. For example, if you deploy the following pod into the example project, it is not be created:
Example
Podobject with an invalid node selectorapiVersion: v1 kind: Pod metadata: name: west-region #... spec: nodeSelector: region: west #...
14.1.2. Loki pod placement Link kopierenLink in die Zwischenablage kopiert!
You can control which nodes the Loki pods run on, and prevent other workloads from using those nodes, by using tolerations or node selectors on the pods.
You can apply tolerations to the log store pods with the LokiStack custom resource (CR) and apply taints to a node with the node specification. A taint on a node is a
key:value
key:value
Example LokiStack with node selectors
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
name: logging-loki
namespace: openshift-logging
spec:
# ...
template:
compactor:
nodeSelector:
node-role.kubernetes.io/infra: ""
distributor:
nodeSelector:
node-role.kubernetes.io/infra: ""
gateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
indexGateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
ingester:
nodeSelector:
node-role.kubernetes.io/infra: ""
querier:
nodeSelector:
node-role.kubernetes.io/infra: ""
queryFrontend:
nodeSelector:
node-role.kubernetes.io/infra: ""
ruler:
nodeSelector:
node-role.kubernetes.io/infra: ""
# ...
In the previous example configuration, all Loki pods are moved to nodes containing the
node-role.kubernetes.io/infra: ""
Example LokiStack CR with node selectors and tolerations
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
name: logging-loki
namespace: openshift-logging
spec:
# ...
template:
compactor:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
distributor:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
indexGateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
ingester:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
querier:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
queryFrontend:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
ruler:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
gateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
# ...
To configure the
nodeSelector
tolerations
oc explain
$ oc explain lokistack.spec.template
Example output
KIND: LokiStack
VERSION: loki.grafana.com/v1
RESOURCE: template <Object>
DESCRIPTION:
Template defines the resource/limits/tolerations/nodeselectors per
component
FIELDS:
compactor <Object>
Compactor defines the compaction component spec.
distributor <Object>
Distributor defines the distributor component spec.
...
For more detailed information, you can add a specific field:
$ oc explain lokistack.spec.template.compactor
Example output
KIND: LokiStack
VERSION: loki.grafana.com/v1
RESOURCE: compactor <Object>
DESCRIPTION:
Compactor defines the compaction component spec.
FIELDS:
nodeSelector <map[string]string>
NodeSelector defines the labels required by a node to schedule the
component onto it.
...
14.1.3. Configuring resources and scheduling for logging collectors Link kopierenLink in die Zwischenablage kopiert!
Administrators can modify the resources or scheduling of the collector by creating a
ClusterLogging
ClusterLogForwarder
The applicable stanzas for the
ClusterLogging
managementState
collection
Prerequisites
- You have administrator permissions.
- You have installed the Red Hat OpenShift Logging Operator version 5.8 or newer.
-
You have created a CR.
ClusterLogForwarder
Procedure
Create a
CR that supports your existingClusterLoggingCR:ClusterLogForwarderExample
ClusterLoggingCR YAMLapiVersion: logging.openshift.io/v1 kind: ClusterLogging metadata: name: <name>1 namespace: <namespace>2 spec: managementState: "Managed" collection: type: "vector" tolerations: - key: "logging" operator: "Exists" effect: "NoExecute" tolerationSeconds: 6000 resources: limits: memory: 1Gi requests: cpu: 100m memory: 1Gi nodeSelector: collector: needed # ...Apply the
CR by running the following command:ClusterLogging$ oc apply -f <filename>.yaml
14.1.4. Viewing logging collector pods Link kopierenLink in die Zwischenablage kopiert!
You can view the logging collector pods and the corresponding nodes that they are running on.
Procedure
Run the following command in a project to view the logging collector pods and their details:
$ oc get pods --selector component=collector -o wide -n <project_name>Example output
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES collector-8d69v 1/1 Running 0 134m 10.130.2.30 master1.example.com <none> <none> collector-bd225 1/1 Running 0 134m 10.131.1.11 master2.example.com <none> <none> collector-cvrzs 1/1 Running 0 134m 10.130.0.21 master3.example.com <none> <none> collector-gpqg2 1/1 Running 0 134m 10.128.2.27 worker1.example.com <none> <none> collector-l9j7j 1/1 Running 0 134m 10.129.2.31 worker2.example.com <none> <none>
14.2. Using taints and tolerations to control logging pod placement Link kopierenLink in die Zwischenablage kopiert!
Taints and tolerations allow the node to control which pods should (or should not) be scheduled on them.
14.2.1. Understanding taints and tolerations Link kopierenLink in die Zwischenablage kopiert!
A taint allows a node to refuse a pod to be scheduled unless that pod has a matching toleration.
You apply taints to a node through the
Node
NodeSpec
Pod
PodSpec
Example taint in a node specification
apiVersion: v1
kind: Node
metadata:
name: my-node
#...
spec:
taints:
- effect: NoExecute
key: key1
value: value1
#...
Example toleration in a Pod spec
apiVersion: v1
kind: Pod
metadata:
name: my-pod
#...
spec:
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
tolerationSeconds: 3600
#...
Taints and tolerations consist of a key, value, and effect.
| Parameter | Description | ||||||
|---|---|---|---|---|---|---|---|
|
| The
| ||||||
|
| The
| ||||||
|
| The effect is one of the following:
| ||||||
|
|
|
If you add a
taint to a control plane node, the node must have theNoScheduletaint, which is added by default.node-role.kubernetes.io/master=:NoScheduleFor example:
apiVersion: v1 kind: Node metadata: annotations: machine.openshift.io/machine: openshift-machine-api/ci-ln-62s7gtb-f76d1-v8jxv-master-0 machineconfiguration.openshift.io/currentConfig: rendered-master-cdc1ab7da414629332cc4c3926e6e59c name: my-node #... spec: taints: - effect: NoSchedule key: node-role.kubernetes.io/master #...
A toleration matches a taint:
If the
parameter is set tooperator:Equal-
the parameters are the same;
key -
the parameters are the same;
value -
the parameters are the same.
effect
-
the
If the
parameter is set tooperator:Exists-
the parameters are the same;
key -
the parameters are the same.
effect
-
the
The following taints are built into OpenShift Container Platform:
-
: The node is not ready. This corresponds to the node condition
node.kubernetes.io/not-ready.Ready=False -
: The node is unreachable from the node controller. This corresponds to the node condition
node.kubernetes.io/unreachable.Ready=Unknown -
: The node has memory pressure issues. This corresponds to the node condition
node.kubernetes.io/memory-pressure.MemoryPressure=True -
: The node has disk pressure issues. This corresponds to the node condition
node.kubernetes.io/disk-pressure.DiskPressure=True -
: The node network is unavailable.
node.kubernetes.io/network-unavailable -
: The node is unschedulable.
node.kubernetes.io/unschedulable -
: When the node controller is started with an external cloud provider, this taint is set on a node to mark it as unusable. After a controller from the cloud-controller-manager initializes this node, the kubelet removes this taint.
node.cloudprovider.kubernetes.io/uninitialized - : The node has pid pressure. This corresponds to the node condition
node.kubernetes.io/pid-pressure.PIDPressure=TrueImportantOpenShift Container Platform does not set a default pid.available
.evictionHard
14.2.2. Loki pod placement Link kopierenLink in die Zwischenablage kopiert!
You can control which nodes the Loki pods run on, and prevent other workloads from using those nodes, by using tolerations or node selectors on the pods.
You can apply tolerations to the log store pods with the LokiStack custom resource (CR) and apply taints to a node with the node specification. A taint on a node is a
key:value
key:value
Example LokiStack with node selectors
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
name: logging-loki
namespace: openshift-logging
spec:
# ...
template:
compactor:
nodeSelector:
node-role.kubernetes.io/infra: ""
distributor:
nodeSelector:
node-role.kubernetes.io/infra: ""
gateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
indexGateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
ingester:
nodeSelector:
node-role.kubernetes.io/infra: ""
querier:
nodeSelector:
node-role.kubernetes.io/infra: ""
queryFrontend:
nodeSelector:
node-role.kubernetes.io/infra: ""
ruler:
nodeSelector:
node-role.kubernetes.io/infra: ""
# ...
In the previous example configuration, all Loki pods are moved to nodes containing the
node-role.kubernetes.io/infra: ""
Example LokiStack CR with node selectors and tolerations
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
name: logging-loki
namespace: openshift-logging
spec:
# ...
template:
compactor:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
distributor:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
indexGateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
ingester:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
querier:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
queryFrontend:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
ruler:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
gateway:
nodeSelector:
node-role.kubernetes.io/infra: ""
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/infra
value: reserved
- effect: NoExecute
key: node-role.kubernetes.io/infra
value: reserved
# ...
To configure the
nodeSelector
tolerations
oc explain
$ oc explain lokistack.spec.template
Example output
KIND: LokiStack
VERSION: loki.grafana.com/v1
RESOURCE: template <Object>
DESCRIPTION:
Template defines the resource/limits/tolerations/nodeselectors per
component
FIELDS:
compactor <Object>
Compactor defines the compaction component spec.
distributor <Object>
Distributor defines the distributor component spec.
...
For more detailed information, you can add a specific field:
$ oc explain lokistack.spec.template.compactor
Example output
KIND: LokiStack
VERSION: loki.grafana.com/v1
RESOURCE: compactor <Object>
DESCRIPTION:
Compactor defines the compaction component spec.
FIELDS:
nodeSelector <map[string]string>
NodeSelector defines the labels required by a node to schedule the
component onto it.
...
14.2.3. Using tolerations to control log collector pod placement Link kopierenLink in die Zwischenablage kopiert!
By default, log collector pods have the following
tolerations
apiVersion: v1
kind: Pod
metadata:
name: collector-example
namespace: openshift-logging
spec:
# ...
collection:
type: vector
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- effect: NoSchedule
key: node.kubernetes.io/disk-pressure
operator: Exists
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
- effect: NoSchedule
key: node.kubernetes.io/memory-pressure
operator: Exists
- effect: NoSchedule
key: node.kubernetes.io/pid-pressure
operator: Exists
- effect: NoSchedule
key: node.kubernetes.io/unschedulable
operator: Exists
# ...
Prerequisites
-
You have installed the Red Hat OpenShift Logging Operator and OpenShift CLI ().
oc
Procedure
Add a taint to a node where you want logging collector pods to schedule logging collector pods by running the following command:
$ oc adm taint nodes <node_name> <key>=<value>:<effect>Example command
$ oc adm taint nodes node1 collector=node:NoExecuteThis example places a taint on
that has keynode1, valuecollector, and taint effectnode. You must use theNoExecutetaint effect.NoExecuteschedules only pods that match the taint and removes existing pods that do not match.NoExecuteEdit the
stanza of thecollectioncustom resource (CR) to configure a toleration for the logging collector pods:ClusterLoggingapiVersion: logging.openshift.io/v1 kind: ClusterLogging metadata: # ... spec: # ... collection: type: vector tolerations: - key: collector1 operator: Exists2 effect: NoExecute3 tolerationSeconds: 60004 resources: limits: memory: 2Gi requests: cpu: 100m memory: 1Gi # ...
This toleration matches the taint created by the
oc adm taint
node1
14.2.4. Configuring resources and scheduling for logging collectors Link kopierenLink in die Zwischenablage kopiert!
Administrators can modify the resources or scheduling of the collector by creating a
ClusterLogging
ClusterLogForwarder
The applicable stanzas for the
ClusterLogging
managementState
collection
Prerequisites
- You have administrator permissions.
- You have installed the Red Hat OpenShift Logging Operator version 5.8 or newer.
-
You have created a CR.
ClusterLogForwarder
Procedure
Create a
CR that supports your existingClusterLoggingCR:ClusterLogForwarderExample
ClusterLoggingCR YAMLapiVersion: logging.openshift.io/v1 kind: ClusterLogging metadata: name: <name>1 namespace: <namespace>2 spec: managementState: "Managed" collection: type: "vector" tolerations: - key: "logging" operator: "Exists" effect: "NoExecute" tolerationSeconds: 6000 resources: limits: memory: 1Gi requests: cpu: 100m memory: 1Gi nodeSelector: collector: needed # ...Apply the
CR by running the following command:ClusterLogging$ oc apply -f <filename>.yaml
14.2.5. Viewing logging collector pods Link kopierenLink in die Zwischenablage kopiert!
You can view the logging collector pods and the corresponding nodes that they are running on.
Procedure
Run the following command in a project to view the logging collector pods and their details:
$ oc get pods --selector component=collector -o wide -n <project_name>Example output
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES collector-8d69v 1/1 Running 0 134m 10.130.2.30 master1.example.com <none> <none> collector-bd225 1/1 Running 0 134m 10.131.1.11 master2.example.com <none> <none> collector-cvrzs 1/1 Running 0 134m 10.130.0.21 master3.example.com <none> <none> collector-gpqg2 1/1 Running 0 134m 10.128.2.27 worker1.example.com <none> <none> collector-l9j7j 1/1 Running 0 134m 10.129.2.31 worker2.example.com <none> <none>