Chapter 5. Deploy an AutoML model for inference

Prerequisites

• You have registered a model from the AutoML leaderboard to a model registry. For more information, see Evaluate AutoML results.

• An administrator has added the AutoGluon serving runtime to your cluster by following the steps in Adding a custom model-serving runtime with the following ServingRuntime definition:

  Example ServingRuntime with AutoGluon

  apiVersion: serving.kserve.io/v1alpha1
  kind: ServingRuntime
  metadata:
    name: kserve-autogluonserver
    annotations:
      openshift.io/display-name: "AutoGluon ServingRuntime for KServe"
  spec:
    annotations:
      prometheus.kserve.io/port: "8080"
      prometheus.kserve.io/path: "/metrics"
    supportedModelFormats:
      - name: autogluon
        version: "1"
    protocolVersions:
      - v1
      - v2
    containers:
      - name: kserve-container
        image: registry.redhat.io/rhoai/odh-kserve-autogluon-server-rhel9@sha256:6d55374ef2e2ac09d772701a72d2ea370c57c34eb89c7b7a71ee204559c699a3
        args:
          - --model_name={{.Name}}
          - --model_dir=/mnt/models
          - --http_port=8080
        securityContext:
          allowPrivilegeEscalation: false
          privileged: false
          runAsNonRoot: true
          capabilities:
            drop:
              - ALL
        resources:
          requests:
            cpu: "1"
            memory: 2Gi
          limits:
            cpu: "1"
            memory: 2Gi

Verification

• On the Deployments page, verify that the model deployment shows a status of Ready.