Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 3. Configuring model servers on the NVIDIA NIM model serving platform

You configure and create a model server on the NVIDIA NIM model serving platform when you deploy an NVIDIA-optimized model. During the deployment process, you select a specific NIM from the available list and configure its properties, such as the number of replicas, server size, and the hardware profile.

As an OpenShift AI administrator, you can use the Red Hat OpenShift AI dashboard to enable the NVIDIA NIM model serving platform.

Note

If you previously enabled the NVIDIA NIM model serving platform in OpenShift AI, and then upgraded to a newer version, re-enter your NVIDIA personal API key to re-enable the NVIDIA NIM model serving platform.

Prerequisites

  • You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges.
  • You have enabled the model serving platform. You do not need to enable a preinstalled runtime. For more information about enabling the model serving platform, see Enabling the model serving platform.

  • The disableNIMModelServing dashboard configuration option is set to false.

    For more information about setting dashboard configuration options, see Customizing the dashboard.

  • You have enabled GPU support in OpenShift AI. This includes installing the Node Feature Discovery Operator and NVIDIA GPU Operator. For more information, see Installing the Node Feature Discovery Operator and Enabling NVIDIA GPUs.
  • You have an NVIDIA Cloud Account (NCA) and can access the NVIDIA GPU Cloud (NGC) portal. For more information, see NVIDIA GPU Cloud user guide.
  • Your NCA account is associated with the NVIDIA AI Enterprise Viewer role.
  • You have generated a personal API key on the NGC portal. For more information, see Generating a Personal API Key.

Procedure

  1. In the left menu of the OpenShift AI dashboard, click Applications Explore.
  2. On the Explore page, find the NVIDIA NIM tile.
  3. Click Enable on the application tile.
  4. Enter your personal API key and then click Submit.

Verification

  • The NVIDIA NIM application that you enabled is displayed on the Enabled page.
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top