Chapter 20. Opaque Integer Resources
20.1. Overview
Opaque integer resources allow cluster operators to provide new node-level resources that would be otherwise unknown to the system. Users can consume these resources in pod specifications, similar to CPU and memory. The scheduler performs resource accounting so that no more than the available amount is simultaneously allocated to pods.
Opaque integer resources are Alpha currently, and only resource accounting is implemented. There is no resource quota or limit range support for these resources, and they have no impact on QoS.
Opaque integer resources are called opaque because OpenShift Container Platform does not know what the resource is, but will schedule a pod on a node only if enough of that resource is available. They are called integer resources because they must be available, or advertised, in integer amounts. The API server restricts quantities of these resources to whole numbers. Examples of valid quantities are 3, 3000m, and 3Ki.
Opaque integer resources can be used to allocate:
- Last-level cache (LLC)
- Graphics processing unit (GPU) devices
- Field-programmable gate array (FPGA) devices
- Slots for sharing bandwidth to a parallel file system.
For example, if a node has 800 GiB of a special kind of disk storage, you could create a name for the special storage, such as opaque-int-resource-special-storage. You could advertise it in chunks of a certain size, such as 100 GiB. In that case, your node would advertise that it has eight resources of type opaque-int-resource-special-storage.
Opaque integer resource names must begin with the prefix pod.alpha.kubernetes.io/opaque-int-resource-.
20.2. Creating Opaque Integer Resources
There are two steps required to use opaque integer resources. First, the cluster operator must name and advertise a per-node opaque resource on one or more nodes. Second, application developer must request the opaque resource in pods.
To make opaque integer resources available:
-
Allocate the resource and assign a name starting with
pod.alpha.kubernetes.io/opaque-int-resource- Advertise a new opaque integer resource by submitting a PATCH HTTP request to the API server that specifies the available quantity in the
status.capacityfor a node in the cluster.For example, the following HTTP request advertises five
fooresources on theopenshift-node-1node.PATCH /api/v1/nodes/openshift-node-1/status HTTP/1.1 Accept: application/json Content-Type: application/json-patch+json Host: openshift-master:8080 [ { "op": "add", "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-foo", "value": "5" } ]NoteThe
~1in thepathis the encoding for the character/. The operation path value in the JSON-Patch is interpreted as a JSON-Pointer. For more details, refer to IETF RFC 6901, section 3.After this operation, the node
status.capacityincludes a new resource. Thestatus.allocatablefield is updated automatically with the new resource asynchronously.NoteSince the scheduler uses the node
status.allocatablevalue when evaluating pod fitness, there might be a short delay between patching the node capacity with a new resource and the first pod that requests the resource to be scheduled on that node.
The application developer can then consume the opaque resources by editing the pod config to include the name of the opaque resource as a key in the spec.containers[].resources.requests field.
For example: The following pod requests two CPUs and one foo (an opaque resource).
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: my-container
image: myimage
resources:
requests:
cpu: 2
pod.alpha.kubernetes.io/opaque-int-resource-foo: 1
The pod will be scheduled only if all of the resource requests are satisfied (including CPU, memory, and any opaque resources). The pod will remain in the PENDING state while the resource request cannot be met by any node.
Conditions: Type Status PodScheduled False ... Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 14s 0s 6 default-scheduler Warning FailedScheduling No nodes are available that match all of the following predicates:: Insufficient pod.alpha.kubernetes.io/opaque-int-resource-foo (1).
This information can also be found in the Developer Guide under Quotas and Limit Ranges.
