SupportConsoleDevelopersStart a trialContact

Chapter 3. Managing pipeline experiments

3.1. Overview of pipeline experiments

A pipeline experiment is a workspace where you can try different configurations of your pipelines. You can use experiments to organize your runs into logical groups. As a data scientist, you can use OpenShift AI to define, manage, and track pipeline experiments. You can view a record of previously created and archived experiments from the Experiments page in the OpenShift AI user interface. Pipeline experiments contain pipeline runs, including recurring runs. This allows you to try different configurations of your pipelines.

When you work with data science pipelines, it is important to monitor and record your pipeline experiments to track the performance of your data science pipelines. You can compare the results of up to 10 pipeline runs at one time, and view available parameter, scalar metric, confusion matrix, and receiver operating characteristic (ROC) curve data for all selected runs.

You can view artifacts for an executed pipeline run from the OpenShift AI dashboard. Pipeline artifacts can help you to evaluate the performance of your pipeline runs and make it easier to understand your pipeline components. Pipeline artifacts can range from plain text data to detailed, interactive data visualizations.

3.2. Creating a pipeline experiment

Pipeline experiments are workspaces where you can try different configurations of your pipelines. You can also use experiments to organize your pipeline runs into logical groups. Pipeline experiments contain pipeline runs, including recurring runs.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins) in OpenShift.
  • You have previously created a data science project that is available and contains a configured pipeline server.
  • You have imported a pipeline to an active pipeline server.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Experiments and runs.
  2. On the Experiments page, from the Project drop-down list, select the project to create the pipeline experiment in.
  3. Click Create experiment.

  4. In the Create experiment dialog, configure the pipeline experiment:

    1. In the Experiment name field, enter a name for the pipeline experiment.
    2. In the Description field, enter a description for the pipeline experiment.
    3. Click Create experiment.

Verification

  • The pipeline experiment that you created appears on the Experiments tab.

3.3. Archiving a pipeline experiment

You can retain records of your pipeline experiments by archiving them. If required, you can restore pipeline experiments from your archive to reuse, or delete pipeline experiments that are no longer required.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins ) in OpenShift.
  • You have previously created a data science project that is available and has a pipeline server.
  • You have imported a pipeline to an active pipeline server.
  • A pipeline experiment is available to archive.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Experiments and runs.
  2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment that you want to archive.
  3. Click the action menu (⋮) beside the pipeline experiment that you want to archive, and then click Archive.
  4. In the Archiving experiment dialog, enter the pipeline experiment name in the text field to confirm that you intend to archive it.
  5. Click Archive.

Verification

  • The archived pipeline experiment does not appear in the Runs tab, and instead appears in the Archive tab on the Experiments page for the pipeline experiment.

3.4. Deleting an archived pipeline experiment

You can delete pipeline experiments from the OpenShift AI experiment archive.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins) in OpenShift.
  • You have previously created a data science project that is available and contains a configured pipeline server.
  • You have imported a pipeline to an active pipeline server.
  • A pipeline experiment is available in the pipeline archive.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Experiments and runs.
  2. On the Experiments page, from the Project drop-down list, select the project that contains the archived pipeline experiment that you want to delete.
  3. Click the Archive tab.
  4. Click the action menu (⋮) beside the pipeline experiment that you want to delete, and then click Delete.
  5. In the Delete experiment dialog, enter the pipeline experiment name in the text field to confirm that you intend to delete it.
  6. Click Delete.

Verification

  • The pipeline experiment that you deleted no longer appears on the Archive tab on the Experiments page.

3.5. Restoring an archived pipeline experiment

You can restore an archived pipeline experiment to the active state.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins ) in OpenShift.
  • You have previously created a data science project that is available and has a pipeline server.
  • An archived pipeline experiment exists in your project.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Experiments and runs.
  2. On the Experiments page, from the Project drop-down list, select the project that contains the archived pipeline experiment that you want to restore.
  3. Click the Archive tab.
  4. Click the action menu (⋮) beside the pipeline experiment that you want to restore, and then click Restore.
  5. In the Restore experiment dialog, click Restore.

Verification

  • The restored pipeline experiment appears in the Experiments tab on the Experiments page.

3.6. Viewing pipeline task executions

When a pipeline run executes, you can view details of executed tasks in each step in a pipeline run from the OpenShift AI dashboard. A step forms part of a task in a pipeline.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins) in OpenShift.
  • You have previously created a data science project that is available and contains a pipeline server.
  • You have imported a pipeline to an active pipeline server.
  • You have previously triggered a pipeline run.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Executions.
  2. On the Executions page, from the Project drop-down list, select the project that contains the experiment for the pipeline task executions that you want to view.

Verification

  • On the Executions page, you can view the execution details of each pipeline task execution, such as its name, status, unique ID, and execution type. The execution status indicates whether the pipeline task has successfully executed. For further information about the details of the task execution, click the execution name.

3.7. Viewing pipeline artifacts

After a pipeline run executes, you can view its pipeline artifacts from the OpenShift AI dashboard. Pipeline artifacts can help you to evaluate the performance of your pipeline runs and make it easier to understand your pipeline components. Pipeline artifacts can range from plain text data to detailed, interactive data visualizations.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins) in OpenShift.
  • You have previously created a data science project that is available and contains a pipeline server.
  • You have imported a pipeline to an active pipeline server.
  • You have previously triggered a pipeline run.

Procedure

  1. From the OpenShift AI dashboard, click Experiments Artifacts.
  2. On the Artifacts page, from the Project drop-down list, select the project that contains the pipeline experiment for the pipeline artifacts that you want to view.

Verification

  • On the Artifacts page, you can view the details of each pipeline artifact, such as its name, unique ID, type, and URI.

3.8. Comparing runs

You can compare up to 10 pipeline runs at one time, and view available parameter, scalar metric, confusion matrix, and receiver operating characteristic (ROC) curve data for all selected runs.

Prerequisites

  • You have logged in to Red Hat OpenShift AI.
  • If you are using specialized OpenShift AI groups, you are part of the user group or admin group (for example, rhoai-users or rhoai-admins ) in OpenShift.
  • You have previously created a data science project that is available and has a pipeline server.
  • You have imported a pipeline to an active pipeline server.
  • You have created at least 2 pipeline runs.

Procedure

  1. In the OpenShift AI dashboard, select Experiments > Experiments and runs.

    The Experiments page opens.

  2. From the Project drop-down list, select the project that contains the runs that you want to compare.

  3. In the Experiments column, click the experiment that you want to compare runs for. To select runs that are not in an experiment, click Default. All runs that are created without specifying an experiment will appear in the Default group.

    The Runs page opens.

  4. Select the checkbox next to each run that you want to compare, and then click Compare runs. You can compare a maximum of 10 runs at one time.

    The Compare runs page opens and displays available parameter, scalar metric, confusion matrix, and receiver operating characteristic (ROC) curve data for the runs that you selected.

    1. The Run list section displays a list of selected runs. You can filter the list by run name, experiment, pipeline version start date, duration, and status.
    2. The Parameters section displays parameter information for each selected run. Set the Hide parameters with no differences switch to On to hide parameters that have the same values.

    3. The Metrics section displays scalar metric, confusion matrix, and ROC curve data for all selected runs.

      1. On the Scalar metrics tab, set the Hide parameters with no differences switch to On to hide parameters that have the same values.
      2. On the ROC curve tab, in the artifacts list, adjust the ROC curve chart by deselecting the checkbox next to artifacts that you want to remove from the chart.

  5. To select different runs for comparison, click Manage runs.

    The Manage runs dialog opens.

    1. From the Search filter drop-down list, select Run, Experiment, Pipeline version, Start date, or Status to filter the run list by each value.
    2. Deselect the checkbox next to each run that you want to remove from your comparison.
    3. Select the checkbox next to each run that you want to add to your comparison.
  6. Click Update.

Verification

  • The Compare runs page opens and displays data for the runs that you selected.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.