Chapter 3. New features and enhancements

You can now use PyTorch v2.8.0 training images for distributed workloads in OpenShift AI.

The following new images are available:

A new Hardware Profiles feature replaces the previous Accelerator Profiles and legacy Container Size selector for workbenches.

Hardware profiles provide a more flexible and consistent way to define compute configurations for AI workloads, simplifying resource management across different hardware types.

The Connections API is now available as a general availability (GA) feature in OpenShift AI.

This API enables you to create and manage connections to external data sources and services directly within OpenShift AI. Connections are stored as Kubernetes Secrets with standardized annotations, allowing protocol-based validation and routing across integrated components.

OpenShift AI now supports the IBM Power (ppc64le) and IBM Z (s390x) architectures.

This expanded platform support enables deployment of AI and machine learning workloads on IBM enterprise hardware, providing greater flexibility and scalability across heterogeneous environments.

For more information about supported software platforms, components, and dependencies, see the Knowledgebase article: Red Hat OpenShift AI: Supported Configurations.

TrustyAI is now available as a general availability (GA) feature for the IBM Power (ppc64le) and IBM Z (s390x) architectures.

TrustyAI is an open-source Responsible AI toolkit that provides a suite of tools to support responsible and transparent AI workflows. It offers capabilities such as fairness and data drift metrics, local and global model explanations, text detoxification, language model benchmarking, and language model guardrails.

These capabilities help ensure transparency, accountability, and the ethical use of AI systems within OpenShift AI environments on IBM Power and IBM Z systems.

Model Registry is now available for the IBM Power (ppc64le) and IBM Z (s390x) architectures.

Model Registry is an open-source component that simplifies and standardizes the management of AI and machine learning (AI/ML) model lifecycles. It provides a centralized platform for storing, versioning, and governing models, enabling seamless collaboration across data science and MLOps teams.

Model Registry supports capabilities such as model versioning and lineage tracking, metadata management and model discovery, model approval and promotion workflows, integration with CI/CD and deployment pipelines, and governance, auditability, and compliance features on IBM Power and IBM Z systems.

Notebooks are now available for the IBM Power (ppc64le) and IBM Z (s390x) architectures.

Notebooks provide containerized, browser-based development environments for data science, machine learning, research, and coding within the OpenShift AI ecosystem. These environments can be launched through Workbenches and include the following options:

Feature Store is now supported on the IBM Power (ppc64le) architecture.

This support enables users to build, register, and manage features for machine learning models directly on IBM Power-based environments, with full integration with OpenShift AI.

AI Pipelines are now supported on the IBM Power (ppc64le) architecture.

This capability enables users to define, run, and monitor AI pipelines natively within OpenShift AI, leveraging the performance and scalability of IBM Power systems for AI workloads.

AI Pipelines executed on IBM Power systems maintain functional parity with x86 deployments.

You can now enable Power architecture support for Triton inference server (CPU only) with FIL, PyTorch, Python and ONNX backend. You can deploy Triton inference server as a custom model serving runtime on IBM Power architecture in Red Hat OpenShift AI.

For details, see Triton Inference Server image.

You can now enable Z architecture support for the Triton Inference Server (Telum I/Telum II) with multiple backend options, including ONNX-MLIR, Snap ML (C++), and PyTorch. The Triton Inference Server can be deployed as a custom model serving runtime on IBM Z architecture as a Technology Preview feature in Red Hat OpenShift AI.

For details, see IBM Z accelerated Triton Inference Server.

Model serving with the IBM Spyre AI Accelerator is now available as a general availability (GA) feature for IBM Z platforms.

The IBM Spyre Operator automates installation and integrates key components such as the device plugin, secondary scheduler, and monitoring.

For more information, see the IBM Spyre Operator catalog entry: IBM Spyre Operator — Red Hat Ecosystem Catalog.

OpenShift AI 3.0 introduces a suite of model customization components that streamline and enhance the process of preparing, fine-tuning, and deploying AI models.

The following components are now available:

You can now enable hybrid search on remote vector databases in Llama Stack in OpenShift AI.

This enhancement allows enterprises to use their existing managed vector database infrastructure while maintaining high retrieval performance and flexibility across different database types.

You can now serve models with IBM Spyre AI accelerators on IBM Z (s390x architecture) by using the vLLM Spyre s390x ServingRuntime for KServe with the Caikit-TGIS gRPC adapter.

This integration enables high-performance model serving and inference for generative AI workloads on IBM Z systems within OpenShift AI.

OpenShift AI now uses the term "AI Pipelines" instead of "Data Science Pipelines" to better reflect the broader range of AI and generative AI use cases supported by the platform.

In the default DataScienceCluster (default-dsc), the datasciencepipelines component has been renamed to aipipelines to align with this terminology update.

This is a naming change only. The AI pipelines functionality remains the same.

The Model Details page in the OpenShift AI Model Catalog now includes comprehensive model validation data, such as performance benchmarks, hardware compatibility, and other key metrics.

This enhancement provides a unified and detailed view consistent with the Jounce UI model details layout, enabling users to evaluate models more effectively from a single interface.

The Model Catalog now includes detailed performance and validation data for Red Hat-validated third-party models, such as benchmarks and hardware compatibility metrics.

Enhanced search and filtering capabilities, such as filtering by latency or hardware profile, help users quickly identify models optimized for their specific use cases and available resources, providing a unified discovery experience within the Red Hat AI Hub.

Distributed Inference with llm-d supports multi-model serving, intelligent inference scheduling, and disaggregated serving for improved GPU utilization on generative AI models.

OpenShift AI now includes a user interface (UI) for configuring and deploying large language models (LLMs) on the llm-d Serving Runtime.

This streamlined interface simplifies common deployment scenarios by providing essential configuration options with sensible defaults while still allowing explicit selection of the llm-d runtime.

The new UI reduces setup complexity and helps users deploy distributed inference workloads more efficiently.

OpenShift AI 3.0 introduces a redesigned, streamlined navigation system that improves usability and workflow efficiency.

The new layout enables users to move seamlessly between features, simplifying access to key capabilities and supporting a smoother end-to-end experience.

OpenShift AI 3.0 replaces oauth-proxy with kube-rbac-proxy for AI Pipelines as part of the platform-wide authentication transition.

This update improves security and compatibility, particularly for environments without an internal OAuth server, such as Red Hat OpenShift Service on AWS.

When migrating to kube-rbac-proxy, SubjectAccessReview (SAR) requirements and RBAC permissions change accordingly. Users who rely on the built-in ds-pipeline-user-access-<dspa-name> role are updated automatically, while others must ensure their roles include access to the datasciencepipelinesapplications/api subresource with the following verbs: create, update, patch, delete, get, list, and watch.

In OpenShift AI 3.0, platform administrators can connect observability components to Distributed Inference with llm-d deployments and integrate with self-hosted Grafana instances to monitor inference workloads.

This capability allows teams to collect and visualize Prometheus metrics from Distributed Inference with llm-d for performance analysis and custom dashboard creation.