Chapter 2. Install and configure MLflow

To track machine learning experiments in OpenShift AI, use the Red Hat OpenShift AI Operator to deploy a cluster-scoped mlflow instance.

Prerequisites

You have cluster administrator privileges for your OpenShift cluster.
You have installed the OpenShift Container Platform
You have permission to patch the DataScienceCluster resource and apply MLflow custom resources.
For production-oriented deployments, you have created the required secrets for database credentials and S3-compatible storage.

Procedure

The MLflow resource is cluster scoped. Therefore, you can install only one instance of mlflow in the cluster. The Red Hat OpenShift AI Operator creates the MLflow resource in the redhat-ods-applications namespace.

Enable the MLflow Operator component by patching the DataScienceCluster object:

$ oc patch datasciencecluster default-dsc \
  --type=merge \
  -p {"spec":{"components":{"mlflowoperator":{"managementState":"Managed"}}}}

Create an MLflow custom resource (CR). Choose the configuration that matches your environment:

Minimal development or test deployment

Uses SQLite for the backend store and a persistent volume claim (PVC) for file-based artifact storage.

apiVersion: mlflow.opendatahub.io/v1
kind: MLflow
metadata:
  name: mlflow
  namespace: redhat-ods-applications
spec:
  storage:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 10Gi
  backendStoreUri: "sqlite:////mlflow/mlflow.db"
  artifactsDestination: "file:///mlflow/artifacts"
  serveArtifacts: true

Production-oriented deployment

Uses PostgreSQL for metadata and S3-compatible object storage for artifacts.

apiVersion: mlflow.opendatahub.io/v1
kind: MLflow
metadata:
  name: mlflow
  namespace: redhat-ods-applications
spec:
  replicas: 2
  backendStoreUriFrom:
    name: mlflow-db-credentials
    key: backend-store-uri
  artifactsDestination: "s3://my-mlflow-bucket/artifacts"
  serveArtifacts: true
  envFrom:
    - secretRef:
        name: aws_access_key_id=<your_aws_access_key>
              aws_secret_access_key=<your_aws_secret_access_key>
              aws_session_token=<your_aws_session_token>

Expand

Table 2.1. MLflow Deployment and Artifact Configuration
Parameter or Feature	Configuration Details
`replicas`	Acts as an important scaling knob specifically for production-oriented deployments.
`registryStoreUri`	Defaults to the value of `backendStoreUri` if not specified. In practice, the same PostgreSQL database is typically utilized for both.
`serveArtifacts`	When enabled, clients log and retrieve artifacts through the MLflow server REST API via the `mlflow-artifacts:/` proxy URI scheme. If `defaultArtifactRoot` is not specified while this is enabled, it automatically defaults to `mlflow-artifacts:/`.
`defaultArtifactRoot`	Defines the directory for storing artifacts for each new experiment. Do not set this to a direct-storage URI (e.g., `s3://`) when `serveArtifacts` is enabled; doing so causes clients to bypass the proxy and attempt direct storage access.
Artifact Storage Path	If no per-project override is configured, artifacts are stored under `<default_artifact_root>/workspaces/<project_name>`. Note that file-based artifact storage requires `serveArtifacts: true`.

In the OpenShift AI dashboard navigation bar, click the Applications menu.
Confirm that the MLflow UI is displayed in the list.
Open the UI, select your workspace, and verify that you can compare runs and visualize metrics.
Alternatively, click Start Demo, which loads demo data to compare runs and visualize metrics.
Additional resources
- To view all available configuration options for the MLflow custom resource, run the oc explain mlflow.spec command.

2.1. MLflow and MLflowConfig configuration parameters
Copy link

The MLflow deployment on Red Hat OpenShift AI relies on two custom resources: MLflow and MLflowConfig. These resources configure metadata storage, artifact destinations, and project-specific overrides.

The MLflow resource is cluster-scoped and must be named mlflow. This resource defines the global state of the MLflow deployment, including the database backend, artifact storage, replica count, and TLS configuration.

Expand

Table 2.2. Key fields in the MLflow resource
Field	Description
`spec.replicas`	The number of MLflow pods. The default value is 1. High-availability deployments require remote storage because PVC-backed storage does not support multiple concurrent writers.
`spec.backendStoreUri`	The tracking metadata database URI. Supported values include `sqlite://` for development and `postgresql://` for production. For production usage, use `spec.backendStoreUriFrom` to avoid storing database credentials in the MLflow CR.
`spec.backendStoreUriFrom`	A secret reference for the backend database URI. This field is used when the URI contains credentials.
`mlflow.spec.resources`	Specifies the compute resources for the MLflow container.

The MLflowConfig resource is namespace-scoped and must be named mlflow. Project owners can use this resource to override default artifact storage settings for their specific projects.

Expand

Table 2.3. Key fields in the MLflowConfig resource
Field	Description
`spec.artifactRootSecret`	The name of a secret in the project that contains S3-compatible connection credentials and bucket information. The value must be `mlflow-artifact-connection`.
`spec.artifactRootPath`	An optional relative path appended to the bucket root from the secret. For example, if the bucket is `my-bucket` and the path is `mlflow-artifacts`, the resolved artifact root is `s3://my-bucket/mlflow-artifacts`.

To view all available configuration fields for a resource, use the oc explain <resource>.spec command.

2.2. Aggregate cluster roles
Copy link

MLflow uses aggregate cluster roles to control access to MLflow resources in Red Hat OpenShift AI. These roles determine which MLflow operations a user can perform based on their existing role bindings.

Important

Standard bindings for view, edit, and admin roles do not grant identical access to all MLflow resources. While mlflow resources are cluster-scoped, mlflowconfig resources are namespace-scoped. Effective access depends on whether permissions are granted at the cluster level or the namespace level.

The mlflow-view ClusterRole aggregates into the standard OpenShift AI view, edit, and admin roles. User access depends on the scope of the binding set at the cluster level.

This role grants read-only access to the following MLflow resources:

get, list, and watch permissions for the mlflow and mlflowconfigs Custom Resource Definitions (CRDs)
get and list permissions for MLflow pseudo-resources, such as datasets, experiments, and registeredmodels
get permissions for MLflow status

The mlflow-edit ClusterRole aggregates into the standard OpenShift AI edit and admin roles. User access depends on the scope of the binding set at the cluster level.

This role grants write access to the following MLflow resources:

create, read, update, and delete permissions for the mlflow CRD
create, delete, patch, and update permissions for the mlflowconfigs CRD
create, update, and delete permissions for MLflow pseudo-resources
update permissions for finalizers

The mlflow-integration ClusterRole grants service accounts data-plane access to MLflow without requiring full edit or delete privileges. When you bind this role with a RoleBinding, it provides namespace-scoped access. This ClusterRole does not grant delete or access to the control-plane resources.

This role is intended for integration-focused tasks and grants the following access:

get, list, create, and update permissions for the experiments, datasets, and registeredmodels authorization-plugin pseudo-resources
Read access to MLflow resources and statuses
Permissions to interact with the MLflow tracking API and model registry

2.3. RBAC model for MLflow API usage
Copy link

The MLflow server authorizes every API request by using a Kubernetes SelfSubjectAccessReview. The server uses the caller’s bearer token to determine if the token has permission to perform a specific verb on an MLflow pseudo-resource within the target project namespace.

The authorization checks use the mlflow.kubeflow.org API group. These pseudo-resources are used solely for role-based access control (RBAC) policy evaluation. Pseudo-resources are not actual Kubernetes resources or Custom Resource Definitions (CRDs), they do not exist as objects on the cluster. You cannot create, list, or inspect pseudo-resources by using the Kubernetes API or command-line tools such as kubectl.

Access is granted through standard Kubernetes Role, ClusterRole, RoleBinding and ClusterRoleBinding objects.

The workspace name serves as the Kubernetes namespace for access checks. The MLflow API access is project-scoped.

The following table outlines primary pseudo-resources:

Expand

Table 2.4. MLflow pseudo-resources
Pseudo-resource	Controls access to
`experiments`	Experiments, runs, traces, artifacts, logged models, scorers, and related tracking operations.
`registeredmodels`	Registered models, model versions, and prompts.
`datasets`	Evaluation datasets and related data set operations.

+ .Role assignments for MLflow API usage

The Red Hat OpenShift AI admin, edit, and view roles include the permissions required for MLflow API authorization checks through aggregate ClusterRoles. These roles allow users to interact with MLflow components within their assigned namespaces.

Workloads or agents that require MLflow API access without using broad human-facing roles can use built-in integration roles. For example, a dedicated mlflow-integration ClusterRole is available for service accounts that need to perform automated tracking or model registry tasks.

+ .Resource-name granularity The MLflow Kubernetes authorization plugin allows you to assign permissions with resourceName granularity. For example, you can restrict an agent so that it sends traces to only a specific experiment by defining the following attributes in the RBAC policy:

resourceName: The name of the specific experiment
resource: experiments
apiGroup: mlflow.kubeflow.org
verbs: get, list, update

Chapter 2. Install and configure MLflow

2.1. MLflow and MLflowConfig configuration parameters
Copy link

2.2. Aggregate cluster roles
Copy link

2.3. RBAC model for MLflow API usage
Copy link

Learn

Try, buy, & sell

Communities

About Red Hat

Making open source more inclusive

About Red Hat Documentation

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links

Chapter 2. Install and configure MLflow

2.1. MLflow and MLflowConfig configuration parametersCopy linkLink copied to clipboard!

2.2. Aggregate cluster rolesCopy linkLink copied to clipboard!

2.3. RBAC model for MLflow API usageCopy linkLink copied to clipboard!

Learn

Try, buy, & sell

Communities

About Red Hat

Making open source more inclusive

About Red Hat Documentation

Theme

Red Hat legal and privacy links

Red Hat legal and privacy links

2.1. MLflow and MLflowConfig configuration parameters
Copy link

2.2. Aggregate cluster roles
Copy link

2.3. RBAC model for MLflow API usage
Copy link