Working with data science pipelines

Procedure

1. From the OpenShift AI dashboard, click Data Science Projects.

   The Data Science Projects page opens.

2. Click the name of the project that you want to configure a pipeline server for.

   A project details page opens.

3. Click the Pipelines tab.

4. Click Configure pipeline server.

   The Configure pipeline server dialog appears.

5. In the Object storage connection section, provide values for the mandatory fields:

   1. In the Access key field, enter the access key ID for the S3-compatible object storage provider.
   2. In the Secret key field, enter the secret access key for the S3-compatible object storage account that you specified.
   3. In the Endpoint field, enter the endpoint of your S3-compatible object storage bucket.
   4. In the Region field, enter the default region of your S3-compatible object storage account.

   5. In the Bucket field, enter the name of your S3-compatible object storage bucket.

      Important

      If you specify incorrect connection settings, you cannot update these settings on the same pipeline server. Therefore, you must delete the pipeline server and configure another one.

      If you want to use an existing artifact that was not generated by a task in a pipeline, you can use the kfp.dsl.importer component to import the artifact from its URI. You can only import these artifacts to the S3-compatible object storage bucket that you define in the Bucket field in your pipeline server configuration. For more information about the kfp.dsl.importer component, see Special Case: Importer Components.

6. In the Database section, click Show advanced database options to specify the database to store your pipeline data and select one of the following sets of actions:

   • Select Use default database stored on your cluster to deploy a MariaDB database in your project.

     Important

     The Use default database stored on your cluster option is intended for development and testing purposes only. For production pipeline workloads, select the Connect to external MySQL database option to use an external MySQL or MariaDB database.

   • Select Connect to external MySQL database to add a new connection to an external MySQL or MariaDB database that your pipeline server can access.

     1. In the Host field, enter the database’s host name.
     2. In the Port field, enter the database’s port.
     3. In the Username field, enter the default user name that is connected to the database.
     4. In the Password field, enter the password for the default user account.
     5. In the Database field, enter the database name.
7. Click Configure pipeline server.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that you want to import a pipeline to.
3. Click Import pipeline.

4. In the Import pipeline dialog, enter the details for the pipeline that you want to import.

   1. In the Pipeline name field, enter a name for the pipeline that you want to import.
   2. In the Pipeline description field, enter a description for the pipeline that want to import.

   3. Select where you want to import your pipeline from by performing one of the following actions:

      • Select Upload a file to upload your pipeline from your local machine’s file system. Import your pipeline by clicking Upload, or by dragging and dropping a file.
      • Select Import by url to upload your pipeline from a URL, and then enter the URL into the text box.
   4. Click Import pipeline.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline that you want to delete.
3. Click the action menu (⋮) beside the pipeline that you want to delete, and then click Delete pipeline.
4. In the Delete pipeline dialog, enter the pipeline name in the text field to confirm that you intend to delete it.
5. Click Delete pipeline.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline server that you want to delete.
3. From the Pipeline server actions list, select Delete pipeline server.
4. In the Delete pipeline server dialog, enter the name of the pipeline server in the text field to confirm that you intend to delete it.
5. Click Delete.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline server that you want to view.
3. From the Pipeline server actions list, select View pipeline server configuration.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipelines that you want to view.
3. Optional: Click Expand ( ) on the row of a pipeline to view its pipeline versions.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that you want to upload a pipeline version to.
3. Click the Import pipeline drop-down list, and then select Upload new version.

4. In the Upload new version dialog, enter the details for the pipeline version that you are uploading.

   1. From the Pipeline list, select the pipeline that you want to upload your pipeline version to.
   2. In the Pipeline version name field, confirm the name for the pipeline version, and change it if necessary.
   3. In the Pipeline version description field, enter a description for the pipeline version.

   4. Select where you want to upload your pipeline version from by performing one of the following actions:

      • Select Upload a file to upload your pipeline version from your local machine’s file system. Import your pipeline version by clicking Upload, or by dragging and dropping a file.
      • Select Import by url to upload your pipeline version from a URL, and then enter the URL into the text box.
   5. Click Upload.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.

   The Pipelines page opens.

2. Delete the pipeline versions that you no longer require:

   • To delete a single pipeline version:

     1. From the Project list, select the project that contains a version of a pipeline that you want to delete.
     2. On the row containing the pipeline, click Expand ( ).

     3. Click the action menu (⋮) beside the project version that you want to delete, and then click Delete pipeline version.

        The Delete pipeline version dialog opens.

     4. Enter the name of the pipeline version in the text field to confirm that you intend to delete it.
     5. Click Delete.

   • To delete multiple pipeline versions:

     1. On the row containing each pipeline version that you want to delete, select the checkbox.
     2. Click the action menu (⋮) next to the Import pipeline drop-down list, and then select Delete from the list.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.

   The Pipelines page opens.

2. From the Project drop-down list, select the project that contains the pipeline versions that you want to view details for.

3. Click the project name to view further details of its most recent version.

   The pipeline version details page opens, displaying the Graph, Summary, and Pipeline spec tabs.

   Alternatively, click Expand ( ) on the row containing the pipeline that you want to view versions for, and then click the pipeline version that you want to view the details of.

   The pipeline version details page opens, displaying the Graph, Summary, and Pipeline spec tabs.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the version that you want to download.
3. Click Expand ( ) beside the pipeline that contains the version that you want to download.

4. Click the pipeline version that you want to download.

   The pipeline version details page opens.

5. On the Pipeline spec tab, click the Download button ( ) to download the YAML file that contains the pipeline version code to your local machine.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project to create the pipeline experiment in.
3. Click Create experiment.

4. In the Create experiment dialog, configure the pipeline experiment:

   1. In the Experiment name field, enter a name for the pipeline experiment.
   2. In the Description field, enter a description for the pipeline experiment.
   3. Click Create experiment.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment that you want to archive.
3. Click the action menu (⋮) beside the pipeline experiment that you want to archive, and then click Archive.
4. In the Archiving experiment dialog, enter the pipeline experiment name in the text field to confirm that you intend to archive it.
5. Click Archive.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the archived pipeline experiment that you want to delete.
3. Click the Archive tab.
4. Click the action menu (⋮) beside the pipeline experiment that you want to delete, and then click Delete.
5. In the Delete experiment? dialog, enter the pipeline experiment name in the text field to confirm that you intend to delete it.
6. Click Delete.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the archived pipeline experiment that you want to restore.
3. Click the Archive tab.
4. Click the action menu (⋮) beside the pipeline experiment that you want to restore, and then click Restore.
5. In the Restore experiment dialog, click Restore.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Executions.
2. On the Executions page, from the Project drop-down list, select the project that contains the experiment for the pipeline task executions that you want to view.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Artifacts.
2. On the Artifacts page, from the Project drop-down list, select the project that contains the pipeline experiment for the pipeline artifacts that you want to view.

Procedure

1. In the OpenShift AI dashboard, select Experiments > Experiments and runs.

   The Experiments page opens.

2. From the Project drop-down list, select the project that contains the runs that you want to compare.

3. On the Experiments tab, in the Experiment column, click the experiment that you want to compare runs for. To select runs that are not in an experiment, click Default. All runs that are created without specifying an experiment will appear in the Default group.

   The Runs page opens.

4. Select the checkbox next to each run that you want to compare, and then click Compare runs. You can compare a maximum of 10 runs at one time.

   The Compare runs page opens and displays available parameter, scalar metric, confusion matrix, and receiver operating characteristic (ROC) curve data for the runs that you selected.

   1. The Run list section displays a list of selected runs. You can filter the list by run name, experiment, pipeline version start date, duration, and status.
   2. The Parameters section displays parameter information for each selected run. Set the Hide parameters with no differences switch to On to hide parameters that have the same values.

   3. The Metrics section displays scalar metric, confusion matrix, and ROC curve data for all selected runs.

      1. On the Scalar metrics tab, set the Hide parameters with no differences switch to On to hide parameters that have the same values.
      2. On the ROC curve tab, in the artifacts list, adjust the ROC curve chart by deselecting the checkbox next to artifacts that you want to remove from the chart.

5. To select different runs for comparison, click Manage runs.

   The Manage runs dialog opens.

   1. From the Search filter drop-down list, select Run, Experiment, Pipeline version, Created after, or Status to filter the run list by each value.
   2. Clear the checkbox next to each run that you want to remove from your comparison.
   3. Select the checkbox next to each run that you want to add to your comparison.
6. Click Update.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the active pipeline runs that you want to view.

3. From the list of experiments, click the experiment that contains the active pipeline runs that you want to view.

   The Runs page opens.

   After a run has completed its execution, the run status appears in the Status column of the Runs tab, indicating whether the run succeeded or failed.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment that you want to create a run for.
3. From the list of pipeline experiments, click the experiment that you want to create a run for.
4. Click Create run.

5. On the Create run page, configure the run:

   1. From the Experiment list, select the pipeline experiment that you want to create a run for. Alternatively, to create a new pipeline experiment, click Create new experiment, and then complete the relevant fields in the Create experiment dialog.
   2. In the Name field, enter a name for the run, up to 255 characters.
   3. In the Description field, enter a description for the run, up to 255 characters.
   4. From the Pipeline list, select the pipeline that you want to create a run for. Alternatively, to create a new pipeline, click Create new pipeline, and then complete the relevant fields in the Import pipeline dialog.
   5. From the Pipeline version list, select the pipeline version to create a run for. Alternatively, to upload a new version, click Upload new version, and then complete the relevant fields in the Upload new version dialog.
   6. Configure the input parameters for the run by selecting the parameters from the list.

   7. Click Create run.

      The details page for the run opens.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the active run that you want to stop.
3. From the list of pipeline experiments, click the pipeline experiment that contains the run that you want to stop.

4. On the Runs tab, click the action menu (⋮) beside the active run that you want to stop, and then click Stop.

   There might be a short delay while the run stops.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the pipeline run that you want to duplicate.
3. From the list of pipeline experiments, click the experiment that contains the pipeline run that you want to duplicate.
4. Click the action menu (⋮) beside the relevant active run, and then click Duplicate.

5. On the Duplicate run page, configure the duplicate run:

   1. From the Experiment list, select a pipeline experiment to contain the duplicate run. Alternatively, to create a new pipeline experiment, click Create new experiment, and then complete the relevant fields in the Create experiment dialog.
   2. In the Name field, enter a name for the duplicate run.
   3. In the Description field, enter a description for the duplicate run.
   4. From the Pipeline list, select a pipeline to contain the duplicate run. Alternatively, to create a new pipeline, click Create new pipeline, and then complete the relevant fields in the Import pipeline dialog.
   5. From the Pipeline version list, select a pipeline version to contain the duplicate run. Alternatively, to upload a new version, click Upload new version, and then complete the relevant fields in the Upload new version dialog.
   6. In the Parameters section, configure input parameters for the duplicate run by selecting parameters from the list.

   7. Click Create run.

      The details page for the run opens.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the scheduled pipeline runs that you want to view.
3. From the list of pipeline experiments, click the experiment that contains the pipeline runs that you want to view.

4. On the Runs page, click the Schedules tab.

   After a run is scheduled, the Status column indicates whether the run is ready or unavailable for execution. To change its execution availability, set the Status switch to On or Off. Alternatively, you can change its execution availability from the details page for the scheduled run by clicking the Actions drop-down menu, and then selecting Enable or Disable.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the run that you want to schedule.
3. From the list of pipeline experiments, click the experiment that contains the run that you want to schedule.
4. Click the Schedules tab.
5. Click Create schedule.

6. On the Schedule run page, configure the run that you are scheduling:

   1. From the Experiment list, select the pipeline experiment that you want to contain the scheduled run. Alternatively, to create a new pipeline experiment, click Create new experiment, and then complete the relevant fields in the Create experiment dialog.
   2. In the Name field, enter a name for the run.
   3. In the Description field, enter a description for the run.

   4. From the Trigger type list, select one of the following options:

      • Select Periodic to specify an execution frequency. In the Run every field, enter a number and select an execution frequency from the list.
      • Select Cron to specify the execution schedule in cron format in the Cron string field. This creates a cron job to execute the run. Click the Copy button ( ) to copy the cron job schedule to the clipboard. The field furthest to the left represents seconds. For more information about scheduling tasks using the supported cron format, see Cron Expression Format.
   5. In the Maximum concurrent runs field, specify the number of runs that can execute concurrently, from a range of one to ten.
   6. For Start date, specify a start date for the run. Select a start date using the calendar, and the start time from the list of times.
   7. For End date, specify an end date for the run. Select an end date using the calendar, and the end time from the list of times.
   8. For Catch up, enable or disable catch up runs. You can use catch up runs to ensure your pipeline runs do not permanently fall behind schedule when they are paused. For example, if you re-enable a paused recurring run, the run scheduler backfills each missed run interval.
   9. From the Pipeline list, select the pipeline that you want to create a run for. Alternatively, to create a new pipeline, click Create new pipeline, and then complete the relevant fields in the Import pipeline dialog.
   10. From the Pipeline version list, select the pipeline version to create a run for. Alternatively, to upload a new version, click Upload new version, and then complete the relevant fields in the Upload new version dialog.
   11. Configure the input parameters for the run by selecting the parameters from the list.
   12. Click Schedule run.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the scheduled run that you want to duplicate.
3. From the list of pipeline experiments, click the experiment that contains the pipeline run that you want to duplicate.
4. On the Runs page, click the Schedules tab.
5. Click the action menu (⋮) beside the run that you want to duplicate, and then click Duplicate.

6. On the Duplicate schedule page, configure the duplicate run:

   1. From the Experiment list, select a pipeline experiment to contain the duplicate run. Alternatively, to create a new pipeline experiment, click Create new experiment, and then complete the relevant fields in the Create experiment dialog.
   2. In the Name field, enter a name for the duplicate run.
   3. In the Description field, enter a description for the duplicate run.

   4. From the Trigger type list, select one of the following options:

      • Select Periodic to specify an execution frequency. In the Run every field, enter a number, and select an execution frequency from the list.
      • Select Cron to specify the execution schedule in cron format in the Cron string field. This creates a cron job to execute the run. Click the Copy button ( ) to copy the cron job schedule to the clipboard. The field furthest to the left represents seconds. For more information about scheduling tasks using the supported cron format, see Cron Expression Format.
   5. For Maximum concurrent runs, specify the number of runs that can execute concurrently, from a range of one to ten.
   6. For Start date, specify a start date for the duplicate run. Select a start date using the calendar, and the start time from the list of times.
   7. For End date, specify an end date for the duplicate run. Select an end date using the calendar, and the end time from the list of times.
   8. For Catch up, enable or disable catch up runs. You can use catch up runs to ensure your pipeline runs do not permanently fall behind schedule when they are paused. For example, if you re-enable a paused recurring run, the run scheduler backfills each missed run interval.
   9. From the Pipeline list, select the pipeline that you want to create a duplicate run for. Alternatively, to create a new pipeline, click Create new pipeline, and then complete the relevant fields in the Import pipeline dialog.
   10. From the Pipeline version list, select the pipeline version to create a duplicate run for. Alternatively, to upload a new version, click Upload new version, and then complete the relevant fields in the Upload new version dialog.
   11. Configure input parameters for the run by selecting parameters from the list.
   12. Click Schedule run.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the scheduled pipeline run that you want to delete.
3. From the list of pipeline experiments, click the experiment that contains the scheduled pipeline run that you want to delete.
4. On the Runs page, click the Schedules tab.
5. Click the action menu (⋮) beside the scheduled pipeline run that you want to delete, and then click Delete.
6. In the Delete schedule dialog, enter the run name in the text field to confirm that you intend to delete it.
7. Click Delete.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline that you want to view run details for.
3. Click Expand ( ) beside the pipeline that you want to view run details for.
4. Click the action menu (⋮) for the pipeline version that you want to view run details for, and then click View runs.

5. On the Runs page, click the name of the run that you want to view the details of.

   The details page for the run opens.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the archived pipeline runs that you want to view.
3. From the list of pipeline experiments, click the experiment that contains the archived pipeline runs that you want to view.
4. On the Runs page, click the Archive tab.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment for the run that you want to archive.

3. From the list of pipeline experiments, click the experiment that contains the pipeline run that you want to archive.

   The Runs page opens.

4. On the Runs tab, click the action menu (⋮) beside the pipeline run that you want to archive, and then click Archive.
5. In the Archiving run dialog, enter the run name in the text field to confirm that you intend to archive it.
6. Click Archive.

Procedure

1. From the OpenShift AI dashboard, click Experiments → Experiments and runs.
2. On the Experiments page, from the Project drop-down list, select the project that contains the pipeline experiment that you want to restore.
3. From the list of pipeline experiments, click the experiment that contains the archived pipeline run that you want to restore.
4. On the Runs page, click the Archive tab.
5. Click the action menu (⋮) beside the pipeline run that you want to restore, and then click Restore.
6. In the Restore run? dialog, click Restore.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline run that you want to delete.
3. Click Expand ( ) beside the pipeline that contains the run that you want to delete.
4. Click the action menu (⋮) beside the pipeline version that contains the run that you want to delete, and then click View runs.
5. On the Runs page, click the Archive tab.
6. Click the action menu (⋮) beside the pipeline run that you want to delete, and then click Delete.
7. In the Delete run? dialog, enter the run name in the text field to confirm that you intend to delete it.
8. Click Delete.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that contains the pipeline run that you want to duplicate.
3. Click Expand ( ) beside the pipeline that contains the run that you want to duplicate.
4. Click the action menu (⋮) beside the pipeline version that contains the run that you want to duplicate, and then click View runs.
5. On the Runs page, click the Archive tab.
6. Click the action menu (⋮) beside the pipeline run that you want to duplicate, and then click Duplicate.

7. On the Duplicate run page, configure the duplicate run:

   1. From the Experiment list, select a pipeline experiment to contain the duplicate run. Alternatively, to create a new pipeline experiment, click Create new experiment, and then complete the relevant fields in the Create experiment dialog.
   2. In the Name field, enter a name for the duplicate run.
   3. In the Description field, enter a description for the duplicate run.
   4. From the Pipeline list, select a pipeline to contain the duplicate run. Alternatively, to create a new pipeline, click Create new pipeline, and then complete the relevant fields in the Import pipeline dialog.
   5. From the Pipeline version list, select a pipeline version to contain the duplicate run. Alternatively, to upload a new version, click Upload new version, and then complete the relevant fields in the Upload new version dialog.
   6. In the Parameters section, configure input parameters for the duplicate run by selecting parameters from the list.

   7. Click Create run.

      The details page for the run opens.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that you want to view logs for.
3. Click Expand ( ) beside the pipeline that you want to view logs for.
4. Click the action menu (⋮) on the row containing the project version that you want to view pipeline logs for, and then click View runs.
5. On the Runs page, click the name of the run that you want to view logs for.
6. On the run details page, on the Graph tab, click the pipeline step that you want to view logs for.
7. Click the Logs tab.
8. To view the logs of another pipeline step, from the Steps list, select the step that you want to view logs for.

9. Analyze the log using the log viewer.

   • To search for a specific log message, enter at least part of the message in the search bar.
   • To view the full log in a separate browser window, click the action menu (⋮) and select View raw logs. Alternatively, to expand the size of the log viewer, click the action menu (⋮) and select Expand.

Procedure

1. From the OpenShift AI dashboard, click Data Science Pipelines.
2. On the Pipelines page, from the Project drop-down list, select the project that you want to download logs for.
3. Click Expand ( ) beside the pipeline that you want to download logs for.
4. Click the action menu (⋮) on the row containing the project version that you want to download logs for, and then click View runs.
5. On the Runs page, click the name of the run that you want to download logs for.
6. On the run details page, on the Graph tab, click the pipeline step that you want to download logs for.
7. Click the Logs tab.

8. In the log viewer, click the Download button ( ).

   1. Select Download current stop log to download the log for the current pipeline step.
   2. Select Download all step logs to download the logs for all steps in your pipeline run.

Procedure

1. After you open JupyterLab, confirm that the JupyterLab launcher is automatically displayed.

2. In the Elyra section of the JupyterLab launcher, click the Pipeline Editor tile.

   The Pipeline Editor opens.

Procedure

1. In the left sidebar of JupyterLab, click Runtimes ( ).

2. Click the Create new runtime configuration button ( ).

   The Add new Data Science Pipelines runtime configuration page opens.

3. Complete the relevant fields to define your runtime configuration.

   1. In the Display Name field, enter a name for your runtime configuration.
   2. Optional: In the Description field, enter a description to define your runtime configuration.
   3. Optional: In the Tags field, click Add Tag to define a category for your pipeline instance. Enter a name for the tag and press Enter.

   4. Define the credentials of your data science pipeline:

      1. In the Data Science Pipelines API Endpoint field, enter the API endpoint of your data science pipeline. Do not specify the pipelines namespace in this field.

      2. In the Public Data Science Pipelines API Endpoint field, enter the public API endpoint of your data science pipeline.

         Important

         You can obtain the data science pipelines API endpoint from the Data Science Pipelines → Runs page in the dashboard. Copy the relevant endpoint and enter it in the Public Data Science Pipelines API Endpoint field.

      3. Optional: In the Data Science Pipelines User Namespace field, enter the relevant user namespace to run pipelines.

      4. From the Authentication Type list, select the authentication type required to authenticate your pipeline.

         Important

         If you created a notebook directly from the Jupyter tile on the dashboard, select EXISTING_BEARER_TOKEN from the Authentication Type list.

      5. In the Data Science Pipelines API Endpoint Username field, enter the user name required for the authentication type.

      6. In the Data Science Pipelines API Endpoint Password Or Token, enter the password or token required for the authentication type.

         Important

         To obtain the data science pipelines API endpoint token, in the upper-right corner of the OpenShift web console, click your user name and select Copy login command. After you have logged in, click Display token and copy the value of --token= from the Log in with this token command.

   5. Define the connectivity information of your S3-compatible storage:

      1. In the Cloud Object Storage Endpoint field, enter the endpoint of your S3-compatible storage. For more information about Amazon s3 endpoints, see Amazon Simple Storage Service endpoints and quotas.
      2. Optional: In the Public Cloud Object Storage Endpoint field, enter the URL of your S3-compatible storage.
      3. In the Cloud Object Storage Bucket Name field, enter the name of the bucket where your pipeline artifacts are stored. If the bucket name does not exist, it is created automatically.
      4. From the Cloud Object Storage Authentication Type list, select the authentication type required to access to your S3-compatible cloud storage. If you use AWS S3 buckets, select KUBERNETES_SECRET from the list.
      5. In the Cloud Object Storage Credentials Secret field, enter the secret that contains the storage user name and password. This secret is defined in the relevant user namespace, if applicable. In addition, it must be stored on the cluster that hosts your pipeline runtime.
      6. In the Cloud Object Storage Username field, enter the user name to connect to your S3-compatible cloud storage, if applicable. If you use AWS S3 buckets, enter your AWS Secret Access Key ID.
      7. In the Cloud Object Storage Password field, enter the password to connect to your S3-compatible cloud storage, if applicable. If you use AWS S3 buckets, enter your AWS Secret Access Key.
   6. Click Save & Close.

Procedure

1. In the left sidebar of JupyterLab, click Runtimes ( ).

2. Hover the cursor over the runtime configuration that you want to update and click the Edit button ( ).

   The Data Science Pipelines runtime configuration page opens.

3. Fill in the relevant fields to update your runtime configuration.

   1. In the Display Name field, update name for your runtime configuration, if applicable.
   2. Optional: In the Description field, update the description of your runtime configuration, if applicable.
   3. Optional: In the Tags field, click Add Tag to define a category for your pipeline instance. Enter a name for the tag and press Enter.

   4. Define the credentials of your data science pipeline:

      1. In the Data Science Pipelines API Endpoint field, update the API endpoint of your data science pipeline, if applicable. Do not specify the pipelines namespace in this field.
      2. In the Public Data Science Pipelines API Endpoint field, update the API endpoint of your data science pipeline, if applicable.
      3. Optional: In the Data Science Pipelines User Namespace field, update the relevant user namespace to run pipelines, if applicable.

      4. From the Authentication Type list, select a new authentication type required to authenticate your pipeline, if applicable.

         Important

         If you created a notebook directly from the Jupyter tile on the dashboard, select EXISTING_BEARER_TOKEN from the Authentication Type list.

      5. In the Data Science Pipelines API Endpoint Username field, update the user name required for the authentication type, if applicable.

      6. In the Data Science Pipelines API Endpoint Password Or Token, update the password or token required for the authentication type, if applicable.

         Important

         To obtain the data science pipelines API endpoint token, in the upper-right corner of the OpenShift web console, click your user name and select Copy login command. After you have logged in, click Display token and copy the value of --token= from the Log in with this token command.

   5. Define the connectivity information of your S3-compatible storage:

      1. In the Cloud Object Storage Endpoint field, update the endpoint of your S3-compatible storage, if applicable. For more information about Amazon s3 endpoints, see Amazon Simple Storage Service endpoints and quotas.
      2. Optional: In the Public Cloud Object Storage Endpoint field, update the URL of your S3-compatible storage, if applicable.
      3. In the Cloud Object Storage Bucket Name field, update the name of the bucket where your pipeline artifacts are stored, if applicable. If the bucket name does not exist, it is created automatically.
      4. From the Cloud Object Storage Authentication Type list, update the authentication type required to access to your S3-compatible cloud storage, if applicable. If you use AWS S3 buckets, you must select USER_CREDENTIALS from the list.
      5. Optional: In the Cloud Object Storage Credentials Secret field, update the secret that contains the storage user name and password, if applicable. This secret is defined in the relevant user namespace. You must save the secret on the cluster that hosts your pipeline runtime.
      6. Optional: In the Cloud Object Storage Username field, update the user name to connect to your S3-compatible cloud storage, if applicable. If you use AWS S3 buckets, update your AWS Secret Access Key ID.
      7. Optional: In the Cloud Object Storage Password field, update the password to connect to your S3-compatible cloud storage, if applicable. If you use AWS S3 buckets, update your AWS Secret Access Key.
   6. Click Save & Close.

Procedure

1. In the left sidebar of JupyterLab, click Runtimes ( ).

2. Hover the cursor over the runtime configuration that you want to delete and click the Delete Item button ( ).

   A dialog box appears prompting you to confirm the deletion of your runtime configuration.

3. Click OK.

Procedure

1. In the left sidebar of JupyterLab, click Runtimes ( ).
2. Hover the cursor over the runtime configuration that you want to duplicate and click the Duplicate button ( ).

Procedure

1. In the Pipeline Editor user interface, click Run Pipeline ( ).

   The Run Pipeline dialog appears. The Pipeline Name field is automatically populated with the pipeline file name.

   Note

   After you run your pipeline, a pipeline experiment containing your pipeline run is automatically created on the Experiments → Experiments and runs page in the OpenShift AI dashboard. The experiment name matches the name that you assigned to the pipeline.

2. Define the settings for your pipeline run.

   1. From the Runtime Configuration list, select the relevant runtime configuration to run your pipeline.
   2. Optional: Configure your pipeline parameters, if applicable. If your pipeline contains nodes that reference pipeline parameters, you can change the default parameter values. If a parameter is required and has no default value, you must enter a value.
3. Click OK.

Procedure

1. In the Pipeline Editor user interface, click Export Pipeline ( ).

   The Export Pipeline dialog appears. The Pipeline Name field is automatically populated with the pipeline file name.

2. Define the settings to export your pipeline.

   1. From the Runtime Configuration list, select the relevant runtime configuration to export your pipeline.
   2. From the Export Pipeline as select an appropriate file format
   3. In the Export Filename field, enter a file name for the exported pipeline.
   4. Select the Replace if file already exists check box to replace an existing file of the same name as the pipeline you are exporting.
   5. Optional: Configure your pipeline parameters, if applicable. If your pipeline contains nodes that reference pipeline parameters, you can change the default parameter values. If a parameter is required and has no default value, you must enter a value.
3. Click OK.

1. Back up your pipelines data.
2. Deploy a new cluster (or use a different existing cluster) with Red Hat OpenShift AI 2.16 to use as an intermediate cluster. You will use this intermediate cluster to upload, test, and verify your new pipelines.

3. In OpenShift AI 2.16 on the intermediate cluster, do the following tasks:

   1. Create a new data science project.

   2. Configure a new pipeline server.

      Important

      If you use an external database, you must use a different external database than the one you use for data science pipelines 1.0, as the database is migrated to data science pipelines 2.0 format.

   3. Update and recompile your data science pipelines 1.0 pipelines as described in Migrate to Kubeflow Pipelines v2.

      Note

      Data science pipelines 2.0 does not use the kfp-tekton library. In most cases, you can replace usage of kfp-tekton with the kfp library. For data science pipelines 2.0, use the latest version of the KFP SDK. For more information, see the Kubeflow Pipelines SDK API Reference.

      Tip

      You can view historical data science pipelines 1.0 pipeline run information on your primary cluster in the OpenShift Console Developer perspective under Pipelines → Project → PipelineRuns.

   4. Import your updated pipelines to the new data science project.
   5. Test and verify your new pipelines.

4. On your primary cluster, do the following tasks:

   1. Remove your data science pipelines 1.0 pipeline servers.
   2. Optional: Remove your data science pipelines 1.0 resources. For more information, see Removing data science pipelines 1.0 resources.
   3. Upgrade to Red Hat OpenShift AI 2.16. For more information, see Upgrading OpenShift AI Self-Managed, or for disconnected environments, Upgrading Red Hat OpenShift AI in a disconnected environment.

5. In the upgraded instance of Red Hat OpenShift AI 2.16 on your primary cluster, do the following tasks:

   1. Recreate the pipeline servers for each data science project where the data science pipelines 1.0 pipeline servers existed.

      Note

      If you are using GitOps to manage your DSPAs, do the following tasks in your DSPAs before performing sync operations:

      • Set spec.dspVersion to v2.
      • Verify that the apiVersion is using v1 instead of v1alpha1.

   2. Import your updated data science pipelines to the applicable pipeline servers.

      Tip

      You can perform a batch upload by creating a script that uses the KFP SDK Client and the .upload_pipeline and .get_pipeline methods.

6. For any workbenches that communicate with data science pipelines 1.0, do the following tasks in the upgraded instance of Red Hat OpenShift AI:

   1. Delete the existing workbench. For more information, see Deleting a workbench from a data science project.
   2. If you want to use the notebook image version 2024.2, upgrade to Python 3.11 before creating a new workbench.
   3. Create a new workbench that uses the existing persistent storage of the deleted workbench. For more information, see Creating a workbench.
   4. Run the pipeline so that the data science pipelines 2.0 pipeline server schedules it.

1. Identify the DataSciencePipelinesApplication (DSPA) resource that corresponds to the data science pipelines 1.0 pipeline server:

   oc get dspa -n <YOUR_DS_PROJECT>

2. Delete the cluster role binding associated with this DSPA:

   oc delete clusterrolebinding
   ds-pipeline-ui-auth-delegator-<YOUR_DS_PROJECT>-dspa

3. Delete the DSPA:

   oc delete dspa dspa -n <YOUR_DS_PROJECT>

4. If necessary, delete the DataSciencePipelinesApplication finalizer to complete the removal of the resource:

   oc patch dspa dspa -n <YOUR_DS_PROJECT> --type=merge
   -p "{\"metadata\":{\"finalizers\":null}}"

5. If you are not using OpenShift Pipelines for any purpose other than data science pipelines 1.0, you can remove the OpenShift Pipelines Operator.
6. Data science pipelines 1.0 used the kfp-tekton Python library. Data science pipelines 2.0 does not use kfp-tekton. You can uninstall kfp-tekton when there are no remaining data science pipelines 1.0 pipeline servers in use on your cluster.