Chapter 3. Technology Preview features
This section describes Technology Preview features in Red Hat OpenShift AI 2.16. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
- RStudio Server notebook image
With the RStudio Server notebook image, you can access the RStudio IDE, an integrated development environment for R. The R programming language is used for statistical computing and graphics to support data analysis and predictions.
To use the RStudio Server notebook image, you must first build it by creating a secret and triggering the
BuildConfig, and then enable it in the OpenShift AI UI by editing therstudio-rhel9image stream. For more information, see Building the RStudio Server workbench images.ImportantDisclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench.
- CUDA - RStudio Server notebook image
With the CUDA - RStudio Server notebook image, you can access the RStudio IDE and NVIDIA CUDA Toolkit. The RStudio IDE is an integrated development environment for the R programming language for statistical computing and graphics. With the NVIDIA CUDA toolkit, you can enhance your work by using GPU-accelerated libraries and optimization tools.
To use the CUDA - RStudio Server notebook image, you must first build it by creating a secret and triggering the
BuildConfig, and then enable it in the OpenShift AI UI by editing therstudio-rhel9image stream. For more information, see Building the RStudio Server workbench images.ImportantDisclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench.
The CUDA - RStudio Server notebook image contains NVIDIA CUDA technology. CUDA licensing information is available in the CUDA Toolkit documentation. You should review their licensing terms before you use this sample workbench.
- Support for AMD GPUs
- The AMD ROCm workbench image adds support for the AMD graphics processing units (GPU) Operator, significantly boosting the processing performance for compute-intensive activities. This feature provides you with access to drivers, development tools, and APIs that support AI workloads and a wide range of models. Additionally, the AMD ROCm workbench image includes machine learning libraries to support AI frameworks such as TensorFlow and PyTorch. The Technology Preview release also provides access to images that can be used to explore serving and training or tuning use cases with AMD GPUs.
- Model Registry
- OpenShift AI now supports the Model Registry Operator. The Model Registry Operator is not installed by default in Technology Preview mode. The model registry is a central repository that contains metadata related to machine learning models from inception to deployment.
- NVIDIA NIM model serving platform
With the NVIDIA NIM model serving platform, you can deploy NVIDIA optimized models using NVIDIA NIM inference services in OpenShift AI. NVIDIA NIM, part of NVIDIA AI Enterprise, is a set of easy-to-use microservices designed for secure, reliable deployment of high-performance AI model inferencing across the cloud, data center and workstations. Supporting a wide range of AI models, including open-source community and NVIDIA AI Foundation models, it ensures seamless, scalable AI inferencing, on-premises or in the cloud, leveraging industry standard APIs. You need an NVIDIA AI Enterprise license key to enable the NVIDIA NIM model serving platform in OpenShift AI.
For more information, see About the NVIDIA NIM model serving platform.
- OCI containers for model storage
You can use OCI storage as an alternative to cloud storage services for model serving. First, you create an OCI container image to contain the model. The image is uploaded to an OCI-compatible registry, such as Quay. Later, when deploying a model, the model serving platform references the repository of the containerized model.
Using an OCI container can provide the following advantages:
- Reduced startup times, because the cluster keeps a cache of downloaded images. Restarting the model pod does not download the model again.
- Lower disk space usage, because the model is not downloaded on each pod replica, assuming pods are scheduled on the same node.
- Enhanced performance when pre-fetching images or asynchronous loading.
- Compatibility and integration, because it can be easily integrated with KServe. No additional dependencies are required and the infrastructure might already be available.
- Support for multinode deployment of very large models
- Serving models over multiple graphical processing unit (GPU) nodes when using a single-model serving runtime is now available as a Technology Preview feature. Deploy your models across multiple GPU nodes to improve efficiency when deploying large models such as large language models (LLMs). For more information, see Deploying models across multiple GPU nodes.
- Support for Intel Gaudi 3 accelerators
- Support for Intel Gaudi 3 accelerators is now available as a Technology Preview feature. The vLLM ServingRuntime with Gaudi accelerators support for KServe runtime is a high-throughput and memory-efficient inference and serving runtime that supports Intel Gaudi accelerators. For more information, see Deploying models on the single-model serving platform.
