13.2. Installation du tableau de bord d'administration des GPU NVIDIA

Ajouter le dépôt Helm :

$ helm repo add rh-ecosystem-edge https://rh-ecosystem-edge.github.io/console-plugin-nvidia-gpu

$ helm repo update

Installez la carte Helm dans l'espace de noms par défaut de l'opérateur NVIDIA GPU :

$ helm install -n nvidia-gpu-operator console-plugin-nvidia-gpu rh-ecosystem-edge/console-plugin-nvidia-gpu

NAME: console-plugin-nvidia-gpu
LAST DEPLOYED: Tue Aug 23 15:37:35 2022
NAMESPACE: nvidia-gpu-operator
STATUS: deployed
REVISION: 1
NOTES:
View the Console Plugin NVIDIA GPU deployed resources by running the following command:

$ oc -n {{ .Release.Namespace }} get all -l app.kubernetes.io/name=console-plugin-nvidia-gpu

Enable the plugin by running the following command:

# Check if a plugins field is specified
$ oc get consoles.operator.openshift.io cluster --output=jsonpath="{.spec.plugins}"

# if not, then run the following command to enable the plugin
$ oc patch consoles.operator.openshift.io cluster --patch '{ "spec": { "plugins": ["console-plugin-nvidia-gpu"] } }' --type=merge

# if yes, then run the following command to enable the plugin
$ oc patch consoles.operator.openshift.io cluster --patch '[{"op": "add", "path": "/spec/plugins/-", "value": "console-plugin-nvidia-gpu" }]' --type=json

# add the required DCGM Exporter metrics ConfigMap to the existing NVIDIA operator ClusterPolicy CR:
oc patch clusterpolicies.nvidia.com gpu-cluster-policy --patch '{ "spec": { "dcgmExporter": { "config": { "name": "console-plugin-nvidia-gpu" } } } }' --type=merge

Le tableau de bord s'appuie principalement sur les métriques Prometheus exposées par l'exportateur NVIDIA DCGM, mais les métriques exposées par défaut ne sont pas suffisantes pour que le tableau de bord rende les jauges nécessaires. C'est pourquoi l'exportateur DGCM est configuré pour exposer un ensemble personnalisé de mesures, comme indiqué ici.

apiVersion: v1
data:
  dcgm-metrics.csv: |
    DCGM_FI_PROF_GR_ENGINE_ACTIVE, gauge, gpu utilization.
    DCGM_FI_DEV_MEM_COPY_UTIL, gauge, mem utilization.
    DCGM_FI_DEV_ENC_UTIL, gauge, enc utilization.
    DCGM_FI_DEV_DEC_UTIL, gauge, dec utilization.
    DCGM_FI_DEV_POWER_USAGE, gauge, power usage.
    DCGM_FI_DEV_POWER_MGMT_LIMIT_MAX, gauge, power mgmt limit.
    DCGM_FI_DEV_GPU_TEMP, gauge, gpu temp.
    DCGM_FI_DEV_SM_CLOCK, gauge, sm clock.
    DCGM_FI_DEV_MAX_SM_CLOCK, gauge, max sm clock.
    DCGM_FI_DEV_MEM_CLOCK, gauge, mem clock.
    DCGM_FI_DEV_MAX_MEM_CLOCK, gauge, max mem clock.
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: console-plugin-nvidia-gpu
    meta.helm.sh/release-namespace: nvidia-gpu-operator
  creationTimestamp: "2022-10-26T19:46:41Z"
  labels:
    app.kubernetes.io/component: console-plugin-nvidia-gpu
    app.kubernetes.io/instance: console-plugin-nvidia-gpu
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: console-plugin-nvidia-gpu
    app.kubernetes.io/part-of: console-plugin-nvidia-gpu
    app.kubernetes.io/version: latest
    helm.sh/chart: console-plugin-nvidia-gpu-0.2.3
  name: console-plugin-nvidia-gpu
  namespace: nvidia-gpu-operator
  resourceVersion: "19096623"
  uid: 96cdf700-dd27-437b-897d-5cbb1c255068

Installez le ConfigMap et modifiez le CR de la politique de cluster de l'opérateur NVIDIA pour ajouter ce ConfigMap à la configuration de l'exportateur DCGM. L'installation du ConfigMap est effectuée par la nouvelle version du plugin de console NVIDIA GPU Helm Chart, mais l'édition du CR ClusterPolicy est effectuée par l'utilisateur.

Voir les ressources déployées :

$ oc -n nvidia-gpu-operator get all -l app.kubernetes.io/name=console-plugin-nvidia-gpu

NAME                                             READY   STATUS    RESTARTS   AGE
pod/console-plugin-nvidia-gpu-7dc9cfb5df-ztksx   1/1     Running   0          2m6s

NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/console-plugin-nvidia-gpu   ClusterIP   172.30.240.138   <none>        9443/TCP   2m6s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console-plugin-nvidia-gpu   1/1     1            1           2m6s

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/console-plugin-nvidia-gpu-7dc9cfb5df   1         1         1       2m6s