Release notes


Red Hat OpenShift AI Self-Managed 2.15

Features, enhancements, resolved issues, and known issues associated with this release

Abstract

These release notes provide an overview of new features, enhancements, resolved issues, and known issues in version 2.15 of Red Hat OpenShift AI.

Chapter 1. Overview of OpenShift AI

Red Hat OpenShift AI is a platform for data scientists and developers of artificial intelligence and machine learning (AI/ML) applications.

OpenShift AI provides an environment to develop, train, serve, test, and monitor AI/ML models and applications on-premise or in the cloud.

For data scientists, OpenShift AI includes Jupyter and a collection of default notebook images optimized with the tools and libraries required for model development, and the TensorFlow and PyTorch frameworks. Deploy and host your models, integrate models into external applications, and export models to host them in any hybrid cloud environment. You can enhance your data science projects on OpenShift AI by building portable machine learning (ML) workflows with data science pipelines, using Docker containers. You can also accelerate your data science experiments through the use of graphics processing units (GPUs) and Intel Gaudi AI accelerators.

For administrators, OpenShift AI enables data science workloads in an existing Red Hat OpenShift or ROSA environment. Manage users with your existing OpenShift identity provider, and manage the resources available to notebook servers to ensure data scientists have what they require to create, train, and host models. Use accelerators to reduce costs and allow your data scientists to enhance the performance of their end-to-end data science workflows using graphics processing units (GPUs) and Intel Gaudi AI accelerators.

OpenShift AI has two deployment options:

  • Self-managed software that you can install on-premise or in the cloud. You can install OpenShift AI Self-Managed in a self-managed environment such as OpenShift Container Platform, or in Red Hat-managed cloud environments such as Red Hat OpenShift Dedicated (with a Customer Cloud Subscription for AWS or GCP), Red Hat OpenShift Service on Amazon Web Services (ROSA Classic or ROSA HCP), or Microsoft Azure Red Hat OpenShift.

    For information about OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see Product Documentation for Red Hat OpenShift AI Self-Managed.

  • A managed cloud service, installed as an add-on in Red Hat OpenShift Dedicated (with a Customer Cloud Subscription for AWS or GCP) or in Red Hat OpenShift Service on Amazon Web Services (ROSA Classic).

    For information about OpenShift AI Cloud Service, see Product Documentation for Red Hat OpenShift AI.

For information about OpenShift AI supported software platforms, components, and dependencies, see Supported configurations.

For a detailed view of the 2.15 release lifecycle, including the full support phase window, see Red Hat OpenShift AI Self-Managed Life Cycle.

Chapter 2. New features and enhancements

This section describes new features and enhancements in Red Hat OpenShift AI 2.15.

Important

This version of OpenShift AI supports using data science pipelines version 2.0. If you are using OpenShift AI 2.8 and want to continue using data science pipelines version 1.0, Red Hat recommends that you stay on OpenShift AI 2.8. For more information, see Support Removals.

2.1. New Features

Tabular data drift detection with KServe

As part of ongoing advancements in the Responsible AI space, backend metrics support for drift monitoring is now available in the TrustyAI project. These capabilities align with the TrustyAI framework to ensure secure and reliable AI operations.

Drift monitoring is crucial for maintaining AI model performance and fairness over time. By implementing backend metrics first, this release lays the groundwork for comprehensive drift detection and monitoring, enhancing the trustworthiness and compliance of AI deployments. Future updates will extend these capabilities with a user interface for streamlined interaction.

Bias and fairness monitoring
Using TrustyAI, data scientists can now evaluate deployed models for bias and fairness. The update includes both backend functionality and dashboard visualizations.
Support heterogeneous clusters in distributed workloads

Data scientists can now select specific queues based on their workload requirements, improving efficiency in resource-constrained environments. Administrators can configure these workload queues per cluster and data science project.

Additionally, the dashboard now supports visibility across all queues in heterogeneous clusters. You can view both the assigned queue and resource details for each workload, providing clear insights into workload distribution.

2.2. Enhancements

Data science pipelines updates

The data science pipelines components are updated to Kubeflow Pipelines 2.2.0. For more information, see the version 2.2.0 changelog.

Additionally, the internal Argo workflows engine was updated to version 3.4.17 and includes security fixes.

Chapter 3. Technology Preview features

Important

This section describes Technology Preview features in Red Hat OpenShift AI 2.15. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

RStudio Server notebook image

With the RStudio Server notebook image, you can access the RStudio IDE, an integrated development environment for R. The R programming language is used for statistical computing and graphics to support data analysis and predictions.

To use the RStudio Server notebook image, you must first build it by creating a secret and triggering the BuildConfig, and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images.

Important

Disclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench.

CUDA - RStudio Server notebook image

With the CUDA - RStudio Server notebook image, you can access the RStudio IDE and NVIDIA CUDA Toolkit. The RStudio IDE is an integrated development environment for the R programming language for statistical computing and graphics. With the NVIDIA CUDA toolkit, you can enhance your work by using GPU-accelerated libraries and optimization tools.

To use the CUDA - RStudio Server notebook image, you must first build it by creating a secret and triggering the BuildConfig, and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images.

Important

Disclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench.

The CUDA - RStudio Server notebook image contains NVIDIA CUDA technology. CUDA licensing information is available in the CUDA Toolkit documentation. You should review their licensing terms before you use this sample workbench.

code-server workbench image

Red Hat OpenShift AI now includes the code-server workbench image. See code-server in GitHub for more information.

With the code-server workbench image, you can customize your workbench environment by using a variety of extensions to add new languages, themes, debuggers, and connect to additional services. You can also enhance the efficiency of your data science work with syntax highlighting, auto-indentation, and bracket matching.

Note

Elyra-based pipelines are not available with the code-server workbench image.

The code-server workbench image is currently available in Red Hat OpenShift AI 2.15 as a Technology Preview feature. This feature was first introduced in OpenShift AI 2.6.

Support for AMD GPUs
The AMD ROCm workbench image adds support for the AMD graphics processing units (GPU) Operator, significantly boosting the processing performance for compute-intensive activities. This provides you with access to drivers, development tools, and APIs that support AI workloads and a wide range of models. Additionally, the AMD ROCm workbench image includes machine learning libraries to support AI frameworks such as TensorFlow and PyTorch. The Technology Preview release also enables access to images that can be used to explore serving and training/tuning use cases with AMD GPUs.
Model Registry
OpenShift AI now supports the Model Registry Operator. The Model Registry Operator is not installed by default in Technology Preview mode. The model registry is a central repository that contains metadata related to machine learning models from inception to deployment.
NVIDIA NIM model serving platform
With the NVIDIA NIM model serving platform, you can deploy NVIDIA optimized models using NVIDIA NIM inference services in OpenShift AI. NVIDIA NIM, part of NVIDIA AI Enterprise, is a set of easy-to-use microservices designed for secure, reliable deployment of high-performance AI model inferencing across the cloud, data center and workstations. Supporting a wide range of AI models, including open-source community and NVIDIA AI Foundation models, it ensures seamless, scalable AI inferencing, on-premises or in the cloud, leveraging industry standard APIs. You need an NVIDIA AI Enterprise license key to enable the NVIDIA NIM model serving platform in OpenShift AI.

For more information, see About the NVIDIA NIM model serving platform.

OCI containers for model storage

You can use OCI storage as an alternative to cloud storage services for model serving. First, you create an OCI container image to contain the model. The image is uploaded to an OCI-compatible registry, such as Quay. Later, when deploying a model, the model serving platform references the repository of the containerized model.

Using an OCI container can provide the following advantages:

  • Reduced startup times, because the cluster keeps a cache of downloaded images. Restarting the model pod does not download the model again.
  • Lower disk space usage, because the model is not downloaded on each pod replica, assuming pods are scheduled on the same node.
  • Enhanced performance when pre-fetching images or asynchronous loading.
  • Compatibility and integration, because it can be easily integrated with KServe. No additional dependencies are required and the infrastructure might already be available.

Chapter 4. Developer Preview features

Important

This section describes Developer Preview features in Red Hat OpenShift AI 2.15. Developer Preview features are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Developer Preview features for production or business-critical workloads. Developer Preview features provide early access to functionality in advance of possible inclusion in a Red Hat product offering. Customers can use these features to test functionality and provide feedback during the development process. Developer Preview features might not have any documentation, are subject to change or removal at any time, and have received limited testing. Red Hat might provide ways to submit feedback on Developer Preview features without an associated SLA.

For more information about the support scope of Red Hat Developer Preview features, see Developer Preview Support Scope.

Support for AppWrapper in Kueue
AppWrapper support in Kueue is available as a Developer Preview feature. The experimental API enables the use of AppWrapper-based workloads with the distributed workloads feature.
ROCm-compatible Ray cluster image
An additional Ray cluster image quay.io/modh/ray:2.35.0-py39-rocm61 is available as Developer Preview software. This image is compatible with AMD accelerators that are supported by ROCm 6.1. This image is an AMD64 image, which might not work on other architectures.
Distributed InstructLab training

Distributed InstructLab training is available as a Developer Preview feature, enabling enhanced performance for training tasks on distributed environments compared to single-node setups. This feature improves the training efficiency and scalability of InstructLab, allowing users to leverage distributed resources for more effective AI model development.

Key features:

  • Data transfer support: Facilitates the movement of synthetically generated data from InstructLab on Red Hat Enterprise Linux AI to S3-compatible storage for efficient access by distributed training tasks.
  • Distributed training execution: Enables orchestration of multi-node InstructLab training jobs, leveraging the performance benefits of distributed infrastructure.
  • End-to-end documentation: Comprehensive guidance for users to implement the full InstructLab training flow, including data preparation, transfer, and distributed execution. To access the documentation, see Distributed InstructLab Training on RHOAI.

Chapter 5. Limited Availability features

Important

This section describes Limited Availability features in Red Hat OpenShift AI 2.15. Limited Availability means that you can install and receive support for the feature only with specific approval from Red Hat. Without such approval, the feature is unsupported. This applies to all features described in this section.

Tuning in OpenShift AI
Tuning in OpenShift AI is available as a Limited Availability feature. The Kubeflow Training Operator and the Hugging Face Supervised Fine-tuning Trainer (SFT Trainer) enable users to fine-tune and train their models easily in a distributed environment. In this release, you can use this feature for models that are based on the PyTorch machine-learning framework.

Chapter 6. Support removals

This section describes major changes in support for user-facing features in Red Hat OpenShift AI. For information about OpenShift AI supported software platforms, components, and dependencies, see Supported configurations.

6.1. Removed functionality

6.1.1. Pipeline logs for Python scripts running in Elyra pipelines are no longer stored in S3

Logs are no longer stored in S3-compatible storage for Python scripts which are running in Elyra pipelines. From OpenShift AI version 2.11, you can view these logs in the pipeline log viewer in the OpenShift AI dashboard.

Note

For this change to take effect, you must use the Elyra runtime images provided in the 2024.1 or 2024.2 workbench images.

If you have an older workbench image version, update the Version selection field to 2024.1 or 2024.2, as described in Updating a project workbench.

Updating your workbench image version will clear any existing runtime image selections for your pipeline. After you have updated your workbench version, open your workbench IDE and update the properties of your pipeline to select a runtime image.

6.1.2. Data science pipelines v1 upgraded to v2

Previously, data science pipelines in OpenShift AI were based on KubeFlow Pipelines v1. Starting with OpenShift AI 2.9, data science pipelines are based on KubeFlow Pipelines v2, which uses a different workflow engine. Data science pipelines 2.0 is enabled and deployed by default in OpenShift AI. For more information, see Enabling data science pipelines 2.0.

Important

Data science pipelines 2.0 contains an installation of Argo Workflows. OpenShift AI does not support direct customer usage of this installation of Argo Workflows. To install or upgrade to OpenShift AI 2.9 or later with data science pipelines 2.0, ensure that there is no existing installation of Argo Workflows on your cluster.

It is no longer possible to deploy, view, or edit the details of pipelines that are based on data science pipelines 1.0 from the dashboard in OpenShift AI 2.15. If you already use data science pipelines, Red Hat recommends that you stay on OpenShift AI 2.8 until full feature parity in data science pipelines 2.0 has been delivered in a stable OpenShift AI release and you are ready to migrate to the new pipeline solution. For a detailed view of the release lifecycle, including the full support phase window, see Red Hat OpenShift AI Self-Managed Life Cycle.

Important

If you want to use existing pipelines and workbenches with data science pipelines 2.0 after upgrading to OpenShift AI 2.15, you must update your workbenches to use the 2024.1 notebook image version or later and then manually migrate your pipelines from data science pipelines 1.0 to 2.0. For more information, see Upgrading to data science pipelines 2.0.

6.1.3. Embedded subscription channel no longer used

Starting with OpenShift AI 2.8, the embedded subscription channel is no longer used. You can no longer select the embedded channel for a new installation of the Operator. For more information about subscription channels, see Installing the Red Hat OpenShift AI Operator.

6.1.4. Version 1.2 notebook container images for workbenches are no longer supported

When you create a workbench, you specify a notebook container image to use with the workbench. Starting with OpenShift AI 2.5, when you create a new workbench, version 1.2 notebook container images are not available to select. Workbenches that are already running with a version 1.2 notebook image continue to work normally. However, Red Hat recommends that you update your workbench to use the latest notebook container image.

6.1.5. Beta subscription channel no longer used

Starting with OpenShift AI 2.5, the beta subscription channel is no longer used. You can no longer select the beta channel for a new installation of the Operator. For more information about subscription channels, see Installing the Red Hat OpenShift AI Operator.

6.1.6. HabanaAI workbench image removal

Support for the HabanaAI 1.10 workbench image has been removed. New installations of OpenShift AI from version 2.14 do not include the HabanaAI workbench image. However, if you upgrade OpenShift AI from a previous version, the HabanaAI workbench image remains available, and existing HabanaAI workbench images continue to function.

6.2. Deprecated functionality

6.2.1. Deprecated cluster configuration parameters

When using the CodeFlare SDK to run distributed workloads in Red Hat OpenShift AI, the following parameters in the Ray cluster configuration are now deprecated and should be replaced with the new parameters as indicated.

Deprecated parameterReplaced by

head_cpus

head_cpu_requests, head_cpu_limits

head_memory

head_memory_requests, head_memory_limits

min_cpus

worker_cpu_requests

max_cpus

worker_cpu_limits

min_memory

worker_memory_requests

max_memory

worker_memory_limits

head_gpus

head_extended_resource_requests

num_gpus

worker_extended_resource_requests

You can also use the new extended_resource_mapping and overwrite_default_resource_mapping parameters, as appropriate. For more information about these new parameters, see the CodeFlare SDK documentation (external).

Chapter 7. Resolved issues

The following notable issues are resolved in Red Hat OpenShift AI 2.15. Security updates, bug fixes, and enhancements for Red Hat OpenShift AI 2.15 are released as asynchronous errata. All OpenShift AI errata advisories are published on the Red Hat Customer Portal.

7.1. Issues resolved in Red Hat OpenShift AI 2.15

RHOAIENG-14571 - Data Science Pipelines API Server unreachable in managed IBM Cloud OpenShift OpenShift AI installation

Previously, when configuring a data science pipeline server, communication errors that prevented successful interaction with the pipeline server occurred. This issue is now resolved.

RHOAIENG-14195 - Ray cluster creation fails when deprecated head_memory parameter is used

Previously, if you included the deprecated head_memory parameter in your Ray cluster configuration, the Ray cluster creation failed. This issue is now resolved.

RHOAIENG-11895 - Unable to clone a GitHub repo in JupyterLab when configuring a custom CA bundle using |-

Previously, if you configured a custom Certificate Authority (CA) bundle in the DSCInitialization (DSCI) object using |-, cloning a repo from JupyterLab failed. This issue is now resolved.

RHOAIENG-1132 (previously documented as RHODS-6383) - An ImagePullBackOff error message is not displayed when required during the workbench creation process

Previously, pods experienced issues pulling container images from the container registry. When an error occurred, the relevant pod entered into an ImagePullBackOff state. During the workbench creation process, if an ImagePullBackOff error occurred, an appropriate message was not displayed. This issue is now resolved.

RHOAIENG-13327 - Importer component (dsl.importer) prevents pipelines from running

Pipelines could not run when using the data science pipelines importer component, dsl.importer. This issue is now resolved.

RHOAIENG-14652 - kfp-client unable to connect to the pipeline server on OCP 4.16 and later

In OpenShift 4.16 and later FIPS clusters, data science pipelines were accessible through the OpenShift AI Dashboard. However, connections to the pipelines API server from the KFP SDK failed due to a TLS handshake error. This issue is now resolved.

Chapter 8. Known issues

This section describes known issues in Red Hat OpenShift AI 2.15 and any known methods of working around these issues.

RHOAIENG-15123 (also documented as RHOAIENG-10790 and RHOAIENG-14265) - Pipelines schedule fail after upgrading

When you upgrade to OpenShift AI 2.15, data science pipeline scheduled runs that existed before the upgrade fail to execute. An error message appears in the task pod.

RHOAIENG-15033 - Model registry instances do not restart or update after upgrade from OpenShift AI 2.14 to 2.15

When you upgrade from OpenShift AI 2.14.z to 2.15, existing instances of the model registry component are not updated, which causes the instance pods to use older images than the ones referenced by the operator pod.

Workaround
After upgrading to OpenShift AI 2.15, for each existing model registry, create a new model registry instance that uses the same database configuration, and delete the old model registry instance. Your new model registry instance will contain all existing registered models and their metadata. You can perform these steps from the OpenShift AI dashboard or from the OpenShift command-line interface (CLI).

From the OpenShift AI dashboard:

  1. For each existing model registry, create a new instance that uses the same database configuration.

    1. From the OpenShift AI dashboard, click SettingsModel registry settings.
    2. Check the database connection details for an existing registry by clicking the action menu () beside the model registry on the Model registry settings page, and then clicking View database configuration.
    3. Create a new model registry with the same database connection details as the existing model registry instance.
    4. In OpenShift, verify that the REST and GRPC images defined in your model registry instance pod match the ones in the model registry operator pod.
  2. After creating the new model registry, delete the original registry:

    1. On the Model registry settings page, click the action menu () beside the model registry, and then click Delete model registry.
    2. In the Delete model registry? dialog that appears, enter the name of the model registry in the text field to confirm that you intend to delete it.
    3. Click Delete model registry.

Alternatively, from the OpenShift command-line interface (CLI):

  1. Run the following command, replacing <model_registry_name> with the name of your model registry:

    oc patch -n rhoai-model-registries mr <model_registry_name> --type=merge -p='{"spec":{"grpc":{"image":""},"rest":{"image":""}}}'
  2. In OpenShift, verify that the REST and GRPC images defined in your model registry instance pod match the ones in the model registry operator pod.

RHOAIENG-15008 - Error when creating a bias metric from the CLI without a request name

The user interface might display an error message when viewing bias metrics if the requestName parameter is not set. If you use the user interface to view bias metrics, but want to configure them through the CLI, you must specify a requestName parameter within your payload:

curl -sk -H "Authorization: Bearer ${TOKEN}" -X POST
      --location $TRUSTY_ROUTE/metrics/group/fairness/spd/request
      --header 'Content-Type: application/json'
      --data "{
                 \"modelId\": \"demo-loan-nn-onnx-alpha\",
                 \"protectedAttribute\": \"Is Male-Identifying?\",
                 \"privilegedAttribute\": 1.0,
                 \"unprivilegedAttribute\": 0.0,
                 \"outcomeName\": \"Will Default?\",
                 \"favorableOutcome\": 0,
                 \"batchSize\": 5000,
                 \"requestName\": \"My Request\"
Workaround
Use the CLI to delete metric requests that have no specified requestName and then refresh the UI.

RHOAIENG-14986 - Incorrect package path causes copy_demo_nbs to fail

In OpenShift AI SDK version 0.22.0, the copy_demo_nbs() function of the CodeFlare SDK fails because of an incorrect path to the SDK package. Running this function results in a FileNotFound error.

Workaround

There are two options to work around the issue:

  1. Manually create a demo-notebooks directory.
  2. From a Python shell or a blank notebook, run the following Python script to create the demo-notebooks directory:
import codeflare_sdk
import os
import pathlib
import shutil

nb_dir = os.path.join(os.path.dirname(codeflare_sdk.__file__), "demo-notebooks")
def copy_nbs(dir: str = "./demo-notebooks", overwrite: bool = False):
    # does dir exist already?
    if overwrite is False and pathlib.Path(dir).exists():
        raise FileExistsError(
            f"Directory {dir} already exists. Please remove it or provide a different location."
        )

    shutil.copytree(nb_dir, dir, dirs_exist_ok=True)

copy_nbs()

RHOAIENG-14552 - Workbench or notebook OAuth proxy fails with FIPS on OCP 4.16

When using OpenShift 4.16 or newer in a FIPS-enabled cluster, connecting to a running workbench fails because the connection between the internal component oauth-proxy and the OpenShift ingress fails with a TLS handshake error. When opening a workbench, the browser shows an "Application is not available" screen without any additional diagnostics.

Workaround
None

RHOAIENG-14271 - Compatibility errors occur when using different Python versions in Ray clusters with Jupyter notebooks

When you use Python version 3.11 in a Jupyter notebook, and then create a Ray cluster, the cluster defaults to a workbench image that contains Ray version 2.35 and Python version 3.9. This mismatch can cause compatibility errors, as the Ray Python client requires a Python version that matches your workbench configuration.

Workaround
Use a workbench image with your Ray cluster that contains a matching Python version with the ClusterConfiguration argument.

RHOAIENG-14197 - Tooltip text for CPU and Memory graphs is clipped and therefore unreadable

When you hover the cursor over the CPU and Memory graphs in the Top resource-consuming distributed workloads section on the Project metrics tab of the Distributed Workloads Metrics page, the tooltip text is clipped, and therefore unreadable.

Workaround
View the project metrics for your distributed workloads within the CPU and Memory graphs themselves.

RHOAIENG-14095 - The dashboard is temporarily unavailable after the installing OpenShift AI Operator

After you install the OpenShift AI Operator, the OpenShift AI dashboard is unavailable for approximately three minutes. As a result, a Cannot read properties of undefined page might appear.

Workaround
After you first access the OpenShift AI dashboard, wait for a few minutes and then refresh the page in your browser.

RHOAIENG-13633 - Cannot set a serving platform for a project without first deploying a model from outside of the model registry

You cannot set a serving platform for a project without first deploying a model from outside of the model registry.

Currently, you cannot deploy a model from a model registry to a project unless the project already has single-model or multi-model serving selected. The only way to select single-model or multi-model serving from the OpenShift AI UI is to first deploy a model or model server from outside the registry.

Workaround

Before deploying a model from a model registry to a new project, select a serving platform for the project from the OpenShift AI dashboard, or from the OpenShift console.

  • From the OpenShift AI dashboard:

    1. From the OpenShift AI dashboard, click Data Science Projects.
    2. Click the Models tab.
    3. Deploy a model or create a model server, and then delete it. For more information about models and model servers in OpenShift AI, see Serving models.
  • From the OpenShift console:

    1. In the OpenShift web console, select HomeProjects.
    2. Click the name of the project.
    3. Click the YAML tab.
    4. In the metadata.labels section, add the modelmesh-enabled label.

      1. To choose multi-model serving, set the modelmesh-enabled value to true.
      2. To choose single-model serving, set the modelmesh-enabled value to false.
    5. Click Save.

After you select a serving platform for a project, you can deploy models from a model registry to that project.

RHOAIENG-12516 - fast releases are available in unintended release channels

Due to a known issue with the stream image delivery process, fast releases are currently available on unintended streaming channels, for example, stable, and stable-x.y. For accurate release type, channel, and support lifecycle information, refer to the Life-cycle Dates table on the Red Hat OpenShift AI Self-Managed Life Cycle page.

Workaround
None

RHOAIENG-12233 - The chat_template parameter is required when using the /v1/chat/completions endpoint

When configuring the vLLM ServingRuntime for KServe runtime and querying a model using the /v1/chat/completions endpoint, the model fails with the following 400 Bad Request error:

As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one

Transformers v4.44.2 is shipped with vLLM v0.5.5. As of vLLM v0.5.5, you must provide a chat template while querying a model using the /v1/chat/completions endpoint.

Workaround

If your model does not include a predefined chat template, you can use the chat-template command-line parameter to specify a chat template in your custom vLLM runtime, as shown in the example. Replace <CHAT_TEMPLATE> with the path to your template.

containers:
  - args:
      - --chat-template=<CHAT_TEMPLATE>

You can use the chat templates that are available as .jinja files in https://github.com/opendatahub-io/vllm/tree/main/examples or with the vLLM image under /apps/data/template. For more information, see Chat templates.

RHOAIENG-11024 - Resources entries get wiped out after removing opendatahub.io/managed annotation

Manually removing the opendatahub.io/managed annotation from any component deployment YAML file might cause resource entry values in the file to be erased.

Workaround

To remove the annotation from a deployment, use the following steps to delete the deployment.

The controller pod for the component will redeploy automatically with default values.

  1. Log in to the OpenShift console as a cluster administrator.
  2. In the Administrator perspective, click Workloads > Deployments.
  3. From the Project drop-down list, select redhat-ods-applications.
  4. Click the options menu Options menu beside the deployment for which you want to remove the annotation.
  5. Click Delete Deployment.

RHOAIENG-8294 - CodeFlare error when upgrading OpenShift AI 2.8 to version 2.10 or later

If you try to upgrade OpenShift AI 2.8 to version 2.10 or later, the following error message is shown for the CodeFlare component, due to a mismatch with the AppWrapper custom resource definition (CRD) version.

ReconcileCompletedWithComponentErrors DataScienceCluster resource reconciled with component errors: 1 error occurred: * CustomResourceDefinition.apiextensions.k8s.io "appwrappers.workload.codeflare.dev" is invalid: status.storedVersions[0]: Invalid value: "v1beta1": must appear in spec.versions
Workaround
  1. Delete the existing AppWrapper CRD.

    $ oc delete crd appwrappers.workload.codeflare.dev
  2. Install the latest version of the AppWrapper CRD.

    $ oc apply -f https://raw.githubusercontent.com/project-codeflare/codeflare-operator/main/config/crd/crd-appwrapper.yml

RHOAIENG-7947 - Model serving fails during query in KServe

If you initially install the ModelMesh component and enable the multi-model serving platform, but later install the KServe component and enable the single-model serving platform, inference requests to models deployed on the single-model serving platform might fail. In these cases, inference requests return a 404 - Not Found error and the logs for the odh-model-controller deployment object show a Reconciler error message.

Workaround
In OpenShift, restart the odh-model-controller deployment object.

RHOAIENG-7716 - Pipeline condition group status does not update

When you run a pipeline that has condition groups, for example, dsl.lf, the UI displays a Running status for the groups, even after the pipeline execution is complete.

Workaround

You can confirm if a pipeline is still running by checking that no child tasks remain active.

  1. From the OpenShift AI dashboard, click Data Science PipelinesRuns.
  2. From the Project drop-down menu, click your data science project.
  3. From the Runs tab, click the pipeline run that you want to check the status of.
  4. Expand the condition group and click a child task.

    A panel that contains information about the child task is displayed

  5. On the panel, click the Task details tab.

    The Status field displays the correct status for the child task.

RHOAIENG-6486 - Pod labels, annotations, and tolerations cannot be configured when using the Elyra JupyterLab extension with the TensorFlow 2024.1 notebook image

When using the Elyra JupyterLab extension with the TensorFlow 2024.1 notebook image, you cannot configure pod labels, annotations, or tolerations from an executed pipeline. This is due to a dependency conflict with the kfp and tf2onnx packages.

Workaround

If you are working with the TensorFlow 2024.1 notebook image, after you have completed your work, change the assigned workbench notebook image to the Standard Data Science 2024.1 notebook image.

In the Pipeline properties tab in the Elyra JupyterLab extension, set the Tensorflow runtime image as the default runtime image for the pipeline node individually, along with the relevant pod label, annotation or toleration, for each pipeline node.

RHOAIENG-6435 - Distributed workloads resources are not included in Project metrics

When you click Distributed Workloads Metrics > Project metrics and view the Requested resources section, the Requested by all projects value currently excludes the resources for distributed workloads that have not yet been admitted to the queue.

Workaround
None.

RHOAIENG-6409 - Cannot save parameter errors appear in pipeline logs for successful runs

When you run a pipeline more than once with data science pipelines 2.0, Cannot save parameter errors appear in the pipeline logs for successful pipeline runs. You can safely ignore these errors.

Workaround
None.

RHOAIENG-12294 (previously documented as RHOAIENG-4812) - Distributed workload metrics exclude GPU metrics

In this release of OpenShift AI, the distributed workload metrics exclude GPU metrics.

Workaround
None.

RHOAIENG-4570 - Existing Argo Workflows installation conflicts with install or upgrade

Data science pipelines 2.0 contains an installation of Argo Workflows. OpenShift AI does not support direct customer usage of this installation of Argo Workflows. To install or upgrade OpenShift AI with data science pipelines 2.0, ensure that there is no existing installation of Argo Workflows on your cluster. For more information, see Enabling data science pipelines 2.0.

Workaround
Remove the existing Argo Workflows installation or set datasciencepipelines to Removed, and then proceed with the installation or upgrade.

RHOAIENG-3913 - Red Hat OpenShift AI Operator incorrectly shows Degraded condition of False with an error

If you have enabled the KServe component in the DataScienceCluster (DSC) object used by the OpenShift AI Operator, but have not installed the dependent Red Hat OpenShift Service Mesh and Red Hat OpenShift Serverless Operators, the kserveReady condition in the DSC object correctly shows that KServe is not ready. However, the Degraded condition incorrectly shows a value of False.

Workaround
Install the Red Hat OpenShift Serverless and Red Hat OpenShift Service Mesh Operators, and then recreate the DSC.

RHOAIENG-3025 - OVMS expected directory layout conflicts with the KServe StoragePuller layout

When you use the OpenVINO Model Server (OVMS) runtime to deploy a model on the single-model serving platform (which uses KServe), there is a mismatch between the directory layout expected by OVMS and that of the model-pulling logic used by KServe. Specifically, OVMS requires the model files to be in the /<mnt>/models/1/ directory, while KServe places them in the /<mnt>/models/ directory.

Workaround

Perform the following actions:

  1. In your S3-compatible storage bucket, place your model files in a directory called 1/, for example, /<s3_storage_bucket>/models/1/<model_files>.
  2. To use the OVMS runtime to deploy a model on the single-model serving platform, choose one of the following options to specify the path to your model files:

    • If you are using the OpenShift AI dashboard to deploy your model, in the Path field for your data connection, use the /<s3_storage_bucket>/models/ format to specify the path to your model files. Do not specify the 1/ directory as part of the path.
    • If you are creating your own InferenceService custom resource to deploy your model, configure the value of the storageURI field as /<s3_storage_bucket>/models/. Do not specify the 1/ directory as part of the path.

KServe pulls model files from the subdirectory in the path that you specified. In this case, KServe correctly pulls model files from the /<s3_storage_bucket>/models/1/ directory in your S3-compatible storage.

RHOAIENG-3018 - OVMS on KServe does not expose the correct endpoint in the dashboard

When you use the OpenVINO Model Server (OVMS) runtime to deploy a model on the single-model serving platform, the URL shown in the Inference endpoint field for the deployed model is not complete.

Workaround
To send queries to the model, you must add the /v2/models/_<model-name>_/infer string to the end of the URL. Replace _<model-name>_ with the name of your deployed model.

RHOAIENG-2759 - Model deployment fails when both secured and regular model servers are present in a project

When you create a second model server in a project where one server is using token authentication, and the other server does not use authentication, the deployment of the second model might fail to start.

Workaround
None.

RHOAIENG-2602 - “Average response time" server metric graph shows multiple lines due to ModelMesh pod restart

The Average response time server metric graph shows multiple lines if the ModelMesh pod is restarted.

Workaround
None.

RHOAIENG-2585 - UI does not display an error/warning when UWM is not enabled in the cluster

Red Hat OpenShift AI does not correctly warn users if User Workload Monitoring (UWM) is disabled in the cluster. UWM is necessary for the correct functionality of model metrics.

Workaround
Manually ensure that UWM is enabled in your cluster, as described in Enabling monitoring for user-defined projects.

RHOAIENG-2555 - Model framework selector does not reset when changing Serving Runtime in form

When you use the Deploy model dialog to deploy a model on the single-model serving platform, if you select a runtime and a supported framework, but then switch to a different runtime, the existing framework selection is not reset. This means that it is possible to deploy the model with a framework that is not supported for the selected runtime.

Workaround
While deploying a model, if you change your selected runtime, click the Select a framework list again and select a supported framework.

RHOAIENG-2468 - Services in the same project as KServe might become inaccessible in OpenShift

If you deploy a non-OpenShift AI service in a data science project that contains models deployed on the single-model serving platform (which uses KServe), the accessibility of the service might be affected by the network configuration of your OpenShift cluster. This is particularly likely if you are using the OVN-Kubernetes network plugin in combination with host network namespaces.

Workaround

Perform one of the following actions:

  • Deploy the service in another data science project that does not contain models deployed on the single-model serving platform. Or, deploy the service in another OpenShift project.
  • In the data science project where the service is, add a network policy to accept ingress traffic to your application pods, as shown in the following example:

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: allow-ingress-to-myapp
    spec:
      podSelector:
        matchLabels:
          app: myapp
      ingress:
         - {}

RHOAIENG-2228 - The performance metrics graph changes constantly when the interval is set to 15 seconds

On the Endpoint performance tab of the model metrics screen, if you set the Refresh interval to 15 seconds and the Time range to 1 hour, the graph results change continuously.

Workaround
None.

RHOAIENG-2183 - Endpoint performance graphs might show incorrect labels

In the Endpoint performance tab of the model metrics screen, the graph tooltip might show incorrect labels.

Workaround
None.

RHOAIENG-1919 - Model Serving page fails to fetch or report the model route URL soon after its deployment

When deploying a model from the OpenShift AI dashboard, the system displays the following warning message while the Status column of your model indicates success with an OK/green checkmark.

Failed to get endpoint for this deployed model. routes.rout.openshift.io"<model_name>" not found
Workaround
Refresh your browser page.

RHOAIENG-404 - No Components Found page randomly appears instead of Enabled page in OpenShift AI dashboard

A No Components Found page might appear when you access the Red Hat OpenShift AI dashboard.

Workaround
Refresh the browser page.

RHOAIENG-234 - Unable to view .ipynb files in VSCode in Insecured cluster

When you use the code-server notebook image on Google Chrome in an insecure cluster, you cannot view .ipynb files.

Workaround
Use a different browser.

RHOAIENG-1128 - Unclear error message displays when attempting to increase the size of a Persistent Volume (PV) that is not connected to a workbench

When attempting to increase the size of a Persistent Volume (PV) that is not connected to a workbench, an unclear error message is displayed.

Workaround
Verify that your PV is connected to a workbench before attempting to increase the size.

RHOAIENG-545 - Cannot specify a generic default node runtime image in JupyterLab pipeline editor

When you edit an Elyra pipeline in the JupyterLab IDE pipeline editor, and you click the PIPELINE PROPERTIES tab, and scroll to the Generic Node Defaults section and edit the Runtime Image field, your changes are not saved.

Workaround
Define the required runtime image explicitly for each node. Click the NODE PROPERTIES tab, and specify the required image in the Runtime Image field.

RHOAIENG-497 - Removing DSCI Results In OpenShift Service Mesh CR Being Deleted Without User Notification

If you delete the DSCInitialization resource, the OpenShift Service Mesh CR is also deleted. A warning message is not shown.

Workaround
None.

RHOAIENG-282 - Workload should not be dispatched if required resources are not available

Sometimes a workload is dispatched even though a single machine instance does not have sufficient resources to provision the RayCluster successfully. The AppWrapper CRD remains in a Running state and related pods are stuck in a Pending state indefinitely.

Workaround
Add extra resources to the cluster.

RHOAIENG-131 - gRPC endpoint not responding properly after the InferenceService reports as Loaded

When numerous InferenceService instances are generated and directed requests, Service Mesh Control Plane (SMCP) becomes unresponsive. The status of the InferenceService instance is Loaded, but the call to the gRPC endpoint returns with errors.

Workaround
Edit the ServiceMeshControlPlane custom resource (CR) to increase the memory limit of the Istio egress and ingress pods.

RHOAIENG-130 - Synchronization issue when the model is just launched

When the status of the KServe container is Ready, a request is accepted even though the TGIS container is not ready.

Workaround
Wait a few seconds to ensure that all initialization has completed and the TGIS container is actually ready, and then review the request output.

RHOAIENG-3115 - Model cannot be queried for a few seconds after it is shown as ready

Models deployed using the multi-model serving platform might be unresponsive to queries despite appearing as Ready in the dashboard. You might see an “Application is not available" response when querying the model endpoint.

Workaround
Wait 30-40 seconds and then refresh the page in your browser.

RHOAIENG-1619 (previously documented as DATA-SCIENCE-PIPELINES-165) - Poor error message when S3 bucket is not writable

When you set up a data connection and the S3 bucket is not writable, and you try to upload a pipeline, the error message Failed to store pipelines is not helpful.

Workaround
Verify that your data connection credentials are correct and that you have write access to the bucket you specified.

RHOAIENG-1207 (previously documented as ODH-DASHBOARD-1758) - Error duplicating OOTB custom serving runtimes several times

If you duplicate a model-serving runtime several times, the duplication fails with the Serving runtime name "<name>" already exists error message.

Workaround
Change the metadata.name field to a unique value.

RHOAIENG-1201 (previously documented as ODH-DASHBOARD-1908) - Cannot create workbench with an empty environment variable

When creating a workbench, if you click Add variable but do not select an environment variable type from the list, you cannot create the workbench. The field is not marked as required, and no error message is shown.

Workaround
None.

RHOAIENG-432 (previously documented as RHODS-12928) - Using unsupported characters can generate Kubernetes resource names with multiple dashes

When you create a resource and you specify unsupported characters in the name, then each space is replaced with a dash and other unsupported characters are removed, which can result in an invalid resource name.

Workaround
None.

RHOAIENG-226 (previously documented as RHODS-12432) - Deletion of the notebook-culler ConfigMap causes Permission Denied on dashboard

If you delete the notebook-controller-culler-config ConfigMap in the redhat-ods-applications namespace, you can no longer save changes to the Cluster Settings page on the OpenShift AI dashboard. The save operation fails with an HTTP request has failed error.

Workaround

Complete the following steps as a user with cluster-admin permissions:

  1. Log in to your cluster by using the oc client.
  2. Enter the following command to update the OdhDashboardConfig custom resource in the redhat-ods-applications application namespace:

    $ oc patch OdhDashboardConfig odh-dashboard-config -n redhat-ods-applications --type=merge -p '{"spec": {"dashboardConfig": {"notebookController.enabled": true}}}'

RHOAIENG-133 - Existing workbench cannot run Elyra pipeline after notebook restart

If you use the Elyra JupyterLab extension to create and run data science pipelines within JupyterLab, and you configure the pipeline server after you created a workbench and specified a notebook image within the workbench, you cannot execute the pipeline, even after restarting the notebook.

Workaround
  1. Stop the running notebook.
  2. Edit the workbench to make a small modification. For example, add a new dummy environment variable, or delete an existing unnecessary environment variable. Save your changes.
  3. Restart the notebook.
  4. In the left sidebar of JupyterLab, click Runtimes.
  5. Confirm that the default runtime is selected.

RHOAIENG-11 - Separately installed instance of CodeFlare Operator not supported

In Red Hat OpenShift AI, the CodeFlare Operator is included in the base product and not in a separate Operator. Separately installed instances of the CodeFlare Operator from Red Hat or the community are not supported.

Workaround
Delete any installed CodeFlare Operators, and install and configure Red Hat OpenShift AI, as described in the Red Hat Knowledgebase solution How to migrate from a separately installed CodeFlare Operator in your data science cluster.

RHODS-12798 - Pods fail with "unable to init seccomp" error

Pods fail with CreateContainerError status or Pending status instead of Running status, because of a known kernel bug that introduced a seccomp memory leak. When you check the events on the namespace where the pod is failing, or run the oc describe pod command, the following error appears:

runc create failed: unable to start container process: unable to init seccomp: error loading seccomp filter into kernel: error loading seccomp filter: errno 524
Workaround
Increase the value of net.core.bpf_jit_limit as described in the Red Hat Knowledgebase solution Pods failing with error loading seccomp filter into kernel: errno 524 in OpenShift 4.

KUBEFLOW-177 - Bearer token from application not forwarded by OAuth-proxy

You cannot use an application as a custom workbench image if its internal authentication mechanism is based on a bearer token. The OAuth-proxy configuration removes the bearer token from the headers, and the application cannot work properly.

Workaround
None.

RHOAIENG-5646 (previously documented as NOTEBOOKS-218) - Data science pipelines saved from the Elyra pipeline editor reference an incompatible runtime

When you save a pipeline in the Elyra pipeline editor with the format .pipeline in OpenShift AI version 1.31 or earlier, the pipeline references a runtime that is incompatible with OpenShift AI version 1.32 or later.

As a result, the pipeline fails to run after you upgrade OpenShift AI to version 1.32 or later.

Workaround
After you upgrade to OpenShift AI to version 1.32 or later, select the relevant runtime images again.

NOTEBOOKS-210 - A notebook fails to export as a PDF file in Jupyter

When you export a notebook as a PDF file in Jupyter, the export process fails with an error.

Workaround
None.

RHOAIENG-1210 (previously documented as ODH-DASHBOARD-1699) - Workbench does not automatically restart for all configuration changes

When you edit the configuration settings of a workbench, a warning message appears stating that the workbench will restart if you make any changes to its configuration settings. This warning is misleading because in the following cases, the workbench does not automatically restart:

  • Edit name
  • Edit description
  • Edit, add, or remove keys and values of existing environment variables
Workaround
Manually restart the workbench.

RHOAIENG-1208 (previously documented as ODH-DASHBOARD-1741) - Cannot create a workbench whose name begins with a number

If you try to create a workbench whose name begins with a number, the workbench does not start.

Workaround
Delete the workbench and create a new one with a name that begins with a letter.

KUBEFLOW-157 - Logging out of JupyterLab does not work if you are already logged out of the OpenShift AI dashboard

If you log out of the OpenShift AI dashboard before you log out of JupyterLab, then logging out of JupyterLab is not successful. For example, if you know the URL for a Jupyter notebook, you are able to open this again in your browser.

Workaround
Log out of JupyterLab before you log out of the OpenShift AI dashboard.

RHODS-9789 - Pipeline servers fail to start if they contain a custom database that includes a dash in its database name or username field

When you create a pipeline server that uses a custom database, if the value that you set for the dbname field or username field includes a dash, the pipeline server fails to start.

Workaround
Edit the pipeline server to omit the dash from the affected fields.

RHOAIENG-580 (previously documented as RHODS-9412) - Elyra pipeline fails to run if workbench is created by a user with edit permissions

If a user who has been granted edit permissions for a project creates a project workbench, that user sees the following behavior:

  • During the workbench creation process, the user sees an Error creating workbench message related to the creation of Kubernetes role bindings.
  • Despite the preceding error message, OpenShift AI still creates the workbench. However, the error message means that the user will not be able to use the workbench to run Elyra data science pipelines.
  • If the user tries to use the workbench to run an Elyra pipeline, Jupyter shows an Error making request message that describes failed initialization.

    Workaround
    A user with administrator permissions (for example, the project owner) must create the workbench on behalf of the user with edit permissions. That user can then use the workbench to run Elyra pipelines.

RHODS-7718 - User without dashboard permissions is able to continue using their running notebooks and workbenches indefinitely

When a Red Hat OpenShift AI administrator revokes a user’s permissions, the user can continue to use their running notebooks and workbenches indefinitely.

Workaround
When the OpenShift AI administrator revokes a user’s permissions, the administrator should also stop any running notebooks and workbenches for that user.

RHOAIENG-1157 (previously documented as RHODS-6955) - An error can occur when trying to edit a workbench

When editing a workbench, an error similar to the following can occur:

Error creating workbench
Operation cannot be fulfilled on notebooks.kubeflow.org "workbench-name": the object has been modified; please apply your changes to the latest version and try again
Workaround
None.

RHOAIENG-1152 (previously documented as RHODS-6356) - The notebook creation process fails for users who have never logged in to the dashboard

The dashboard’s notebook Administration page displays users belonging to the user group and admin group in OpenShift. However, if an administrator attempts to start a notebook server on behalf of a user who has never logged in to the dashboard, the server creation process fails and displays the following error message:

Request invalid against a username that does not exist.
Workaround
Request that the relevant user logs into the dashboard.

RHODS-5763 - Incorrect package version displayed during notebook selection

The Start a notebook server page displays an incorrect version number for the Anaconda notebook image.

Workaround
None.

RHODS-5543 - When using the NVIDIA GPU Operator, more nodes than needed are created by the Node Autoscaler

When a pod cannot be scheduled due to insufficient available resources, the Node Autoscaler creates a new node. There is a delay until the newly created node receives the relevant GPU workload. Consequently, the pod cannot be scheduled and the Node Autoscaler’s continuously creates additional new nodes until one of the nodes is ready to receive the GPU workload. For more information about this issue, see the Red Hat Knowledgebase solution When using the NVIDIA GPU Operator, more nodes than needed are created by the Node Autoscaler.

Workaround
Apply the cluster-api/accelerator label in machineset.spec.template.spec.metadata. This causes the autoscaler to consider those nodes as unready until the GPU driver has been deployed.

RHOAIENG-1149 (previously documented RHODS-5216) - The application launcher menu incorrectly displays a link to OpenShift Cluster Manager

Red Hat OpenShift AI incorrectly displays a link to the OpenShift Cluster Manager from the application launcher menu. Clicking this link results in a "Page Not Found" error because the URL is not valid.

Workaround
None.

RHOAIENG-1137 (previously documented as RHODS-5251) - Notebook server administration page shows users who have lost permission access

If a user who previously started a notebook server in Jupyter loses their permissions to do so (for example, if an OpenShift AI administrator changes the user’s group settings or removes the user from a permitted group), administrators continue to see the user’s notebook servers on the server Administration page. As a consequence, an administrator is able to restart notebook servers that belong to the user whose permissions were revoked.

Workaround
None.

RHODS-4799 - Tensorboard requires manual steps to view

When a user has TensorFlow or PyTorch notebook images and wants to use TensorBoard to display data, manual steps are necessary to include environment variables in the notebook environment, and to import those variables for use in your code.

Workaround

When you start your notebook server, use the following code to set the value for the TENSORBOARD_PROXY_URL environment variable to use your OpenShift AI user ID.

import os
os.environ["TENSORBOARD_PROXY_URL"]= os.environ["NB_PREFIX"]+"/proxy/6006/"

RHODS-4718 - The Intel® oneAPI AI Analytics Toolkits quick start references nonexistent sample notebooks

The Intel® oneAPI AI Analytics Toolkits quick start, located on the Resources page on the dashboard, requires the user to load sample notebooks as part of the instruction steps, but refers to notebooks that do not exist in the associated repository.

Workaround
None.

RHODS-4627 - The CronJob responsible for validating Anaconda Professional Edition’s license is suspended and does not run daily

The CronJob responsible for validating Anaconda Professional Edition’s license is automatically suspended by the OpenShift AI operator. As a result, the CronJob does not run daily as scheduled. In addition, when Anaconda Professional Edition’s license expires, Anaconda Professional Edition is not indicated as disabled on the OpenShift AI dashboard.

Workaround
None.

RHOAIENG-1141 (previously documented as RHODS-4502) - The NVIDIA GPU Operator tile on the dashboard displays button unnecessarily

GPUs are automatically available in Jupyter after the NVIDIA GPU Operator is installed. The Enable button, located on the NVIDIA GPU Operator tile on the Explore page, is therefore redundant. In addition, clicking the Enable button moves the NVIDIA GPU Operator tile to the Enabled page, even if the Operator is not installed.

Workaround
None.

RHOAIENG-1135 (previously documented as RHODS-3985) - Dashboard does not display Enabled page content after ISV operator uninstall

After an ISV operator is uninstalled, no content is displayed on the Enabled page on the dashboard. Instead, the following error is displayed:

Error loading components
HTTP request failed
Workaround
Wait 30-40 seconds and then refresh the page in your browser.

RHODS-3984 - Incorrect package versions displayed during notebook selection

In the OpenShift AI interface, the Start a notebook server page displays incorrect version numbers for the JupyterLab and Notebook packages included in the oneAPI AI Analytics Toolkit notebook image. The page might also show an incorrect value for the Python version used by this image.

Workaround
When you start your oneAPI AI Analytics Toolkit notebook server, you can check which Python packages are installed on your notebook server and which version of the package you have by running the !pip list command in a notebook cell.

RHODS-2956 - Error can occur when creating a notebook instance

When creating a notebook instance in Jupyter, a Directory not found error appears intermittently. This error message can be ignored by clicking Dismiss.

Workaround
None.

RHOAING-1147 (previously documented as RHODS-2881) - Actions on dashboard not clearly visible

The dashboard actions to revalidate a disabled application license and to remove a disabled application tile are not clearly visible to the user. These actions appear when the user clicks on the application tile’s Disabled label. As a result, the intended workflows might not be clear to the user.

Workaround
None.

RHOAIENG-1134 (previously documented as RHODS-2879) - License revalidation action appears unnecessarily

The dashboard action to revalidate a disabled application license appears unnecessarily for applications that do not have a license validation or activation system. In addition, when a user attempts to revalidate a license that cannot be revalidated, feedback is not displayed to state why the action cannot be completed.

Workaround
None.

RHOAIENG-2305 (previously documented as RHODS-2650) - Error can occur during Pachyderm deployment

When creating an instance of the Pachyderm operator, a webhook error appears intermittently, preventing the creation process from starting successfully. The webhook error is indicative that, either the Pachyderm operator failed a health check, causing it to restart, or that the operator process exceeded its container’s allocated memory limit, triggering an Out of Memory (OOM) kill.

Workaround
Repeat the Pachyderm instance creation process until the error no longer appears.

RHODS-2096 - IBM Watson Studio not available in OpenShift AI

IBM Watson Studio is not available when OpenShift AI is installed on OpenShift Dedicated 4.9 or higher, because it is not compatible with these versions of OpenShift Dedicated.

Workaround
Contact Marketplace support for assistance manually configuring Watson Studio on OpenShift Dedicated 4.9 and higher.

Chapter 9. Product features

Red Hat OpenShift AI provides a rich set of features for data scientists and IT operations administrators. To learn more, see Introduction to Red Hat OpenShift AI.

Legal Notice

Copyright © 2024 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.