Chapter 2. New features and enhancements

NeMo Guardrails, introduced in Red Hat OpenShift AI 3.3 as a Technology Preview, is fully supported with this release. You can add guardrails and safety controls to your deployed models. NeMo Guardrails provides a framework for controlling conversations with large language models, enabling you to define a variety of rails, such as sensitive data detection, content filtering, or custom validation rules. NeMo Guardrails introduces the following capabilities:

Models-as-a-Service (MaaS) is now Generally Available in Red Hat OpenShift AI 3.4. Previously introduced as a Technology Preview feature in version 3.3, MaaS provides an enterprise platform for centralized governance, consumption tracking, and self-service access to large language models. MaaS addresses resource consumption and governance challenges by exposing models through managed API endpoints with subscription-based controls. Administrators can define subscriptions that grant groups access to specific models with configurable token limits, while users can generate and manage their own API keys for programmatic access.

Key capabilities include:

The Models-as-a-Service subscription model has been redesigned to replace the tier-based model introduced in version 3.3. The new subscription model provides administrators with flexible access control and resource management for large language models, while enabling data scientists to select from available subscriptions when accessing models.

Key enhancements include:

You can now create and manage your own API keys for programmatic access to large language models through Models-as-a-Service. This self-service capability streamlines access to large language models while maintaining centralized governance through subscriptions and authorization policies.

Key capabilities include:

You can now use the OpenShift AI dashboard to register a model from an S3-compatible source or URI, transform it into an OCI ModelCar image, and store it in an OCI registry. The ModelCar target format enables fast deployment with KServe.

The model transfer job runs as a background Kubernetes Job that you can monitor from the dashboard. This feature provides the following capabilities:

The MLServer serving runtime for KServe is now generally available in Red Hat OpenShift AI. You can use this runtime to deploy models trained on structured data, such as classical machine learning models. You can deploy models directly in their native format, simplifying the deployment process.

Supported model frameworks include: * Scikit-learn * XGBoost * LightGBM * ONNX For models with well-known file names, MLServer automatically configures all required environment variables during deployment through the Deploy a model wizard.

The model catalog includes support for IBM Power (ppc64le) architecture. With this enhancement, you can discover and deploy models directly from the dashboard. Support is available for the following validated models:

The model catalog now includes support for the IBM Z architecture. This enhancement enables users on IBM Z platforms to discover and deploy models directly from the OpenShift AI dashboard. Currently, support is available for the following model:

Kubeflow Trainer now provides Just-In-Time (JIT) and periodic checkpointing for distributed training jobs on OpenShift AI. This enhancement automatically saves the training state, including model weights, optimizer state, and training step at regular intervals and immediately before interruptions such as preemption, eviction, or maintenance. Interrupted jobs automatically resume from the latest valid checkpoint, significantly reducing wasted GPU compute and improving overall training efficiency.

Checkpoints can be stored on PersistentVolumeClaims (PVCs) or S3-compatible object storage. With S3, checkpoints are uploaded in the background without pausing training, enabling low-overhead, continuous protection of progress. S3-backed storage also provides a cost-efficient, portable alternative to PVCs, allowing checkpoints to be retained, shared, and reused across clusters.

Model Registry includes a default database solution for testing. Use this solution to start using Model Registry without configuring an external database.