Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 5. Train the model by using your prepared data

To train the model, you can use the Red Hat Training Hub and the Kubeflow Training Operator (KFTO).

You can simplify and accelerate the process of fine-tuning and customizing a foundation model by using your own data. The Red Hat Training Hub is an algorithm-focused interface for common LLM training, continual learning, and reinforcement learning techniques.

5.1. Explore Training Hub examples
Copy link

The Training Hub repository hosts multiple cookbooks for using different LLM algorithms such as Supervised Fine-tuning (SFT), Orthogonal Subspace Fine-tuning (OSFT)/Continual Learning, and Low-Rank Adaptation (LoRA)/Quantized Low-Rank Adaptation (QLoRA). OSFT is a training algorithm built by the Red Hat AI Innovation team. With OSFT, you can continually post-train a fine-tuned model to expand its knowledge on new data. You can tinker with the Training Hub cookbooks from a workbench within your OpenShift AI project.

To get started with Training Hub, explore the provided examples.

Prerequisites

Procedure

  1. To access Training Hub examples, clone the Training Hub Git repository:

    • To clone the https://github.com/Red-Hat-AI-Innovation-Team/training_hub.git repository from JupyterLab, follow the steps in Clone an example Git repository.

    • To create a local clone of the repository, run the following command: 

      git clone https://github.com/Red-Hat-AI-Innovation-Team/training_hub

  2. Go to the examples directory to view Training Hub notebooks, Python scripts, and documentation.

    • For a quick overview and descriptions of the supported algorithms and features, with links to examples and getting started code, see the top-level README file.
    • For detailed parameter documentation, see the docs directory.
    • For hands-on learning with the interactive notebooks, see the notebooks directory.
    • For pre-written, configurable python scripts to run training algorithms with various language models, see the scripts directory.

To simplify tuning for enterprise customers, Training Hub supports multiple backends and exposes a unified API surface to access the latest training algorithms from different backends.

The following table lists Training Hub algorithm and model support matrix.

Expand
Table 5.1. Training Hub algorithm and model support matrix

Algorithm

Backend

Supported Model Architectures

Supervised Fine-tuning (SFT)

instructlab.training

GPTOssForCausalLM (GPT OSS 20B/120B)

LlamaForCausalLM (Llama 3 Models)

Qwen2ForCausalLM (Qwen 2.5 models)

Qwen3ForCausalLM (Qwen 3 models)

GraniteForCausalLM (Granite 3 models)

GraniteMoeHybridForCausalLM (Granite 4 models)

Phi3ForCausalLM (Phi 3 and 4 models)

MistralForCausalLM (Mistral models)

Orthogonal Subspace Fine-tuning (OSFT)

mini-trainer

Same as SFT

Low-Rank Adaptation (LoRA) /Quantized Low-Rank Adaptation (QLoRA)

Unsloth

GPTOssForCausalLM (GPT OSS 20B/120B) (QLoRA ONLY)

LlamaForCausalLM (Llama 3 Models)

Qwen2ForCausalLM (Qwen 2.5 models)

Qwen3ForCausalLM (Qwen 3 models)

GraniteForCausalLM (Granite 3 models)

GraniteMoeHybridForCausalLM (Granite 4 models)

MistralForCausalLM (Mistral models)

NOTE: If you experience an issue with the model classes listed for OSFT with use_liger=True, try setting use_liger=False. Liger kernels are supported for most model architectures, but some newer architectures might experience errors or instability if not fully supported. For up-to-date support information, see the Liger-Kernel GitHub repository.

5.3. Estimate memory usage
Copy link

To learn how to estimate the amount of memory you need for running and training a specific model, as well as whether your configured GPUs can support the model, use the memory estimator. The memory estimator currently supports only Supervised Fine-tuning (SFT) and Orthogonal Subspace Fine-tuning (OSFT) algorithms. See the following example files in the Training Hub Git repository:

  • For the Memory Estimator API, see the src/training_hub/profiling/memory_estimator.py file.
  • For an example notebook that uses the API, see notebooks/memory_estimator_example.ipynb file.

You can use the Orthogonal Subspace Fine-Tuning (OSFT), Supervised Fine-Tuning (SFT), and Low-Rank Adaptation (LoRA) algorithms in Training Hub.

Use SFT to fine-tune a model on supervised data sets with support for:

  • Single-node and multi-node distributed training
  • Configurable training parameters, for example, epochs, batch size, and learning rate.
  • InstructLab-Training backend integration

Use OSFT to fine-tune a model while controlling how much of its existing behavior to preserve, with support for:

  • Single-node and multi-node distributed training
  • Configurable training parameters (for example, epochs, batch size, learning rate)
  • RHAI Innovation Mini-Trainer backend integration

Use LoRA for parameter-efficient fine-tuning with significantly reduced memory requirements, with support for:

  • Training low-rank adaptation matrices instead of full model weights
  • Unsloth backend integration
  • QLoRA variant for further memory reduction (Float4)

The examples/docs directory contains information and examples for how to use each algorithm.

Here is a performance comparison of using OSFT, SFT, and LoRA in Training Hub.

NOTE: When scaling the usage of Liger Kernels for all methods, some amount of fixed overhead memory is added to all methods that do not use Liger Kernels.

  • Memory scaling: OSFT adds additional memory overhead to the model storage due to its unique matrices, roughly about 1.25-1.5x that of the normal model storage in SFT. However, the rest of OSFT memory scales linearly with the unfreeze rank ratio (URR). The URR is a hyperparameter for OSFT that is a value between 0 and 1. It represents the fraction of the matrix rank that is unfrozen and updated during fine-tuning.

    A rough comparison is: OSFT Memory ~ 3 x r x SFT Memory, where r is the URR unfreeze rank ratio, the fraction of the matrix being fine-tuned. At URR = 1/3, OSFT and SFT have similar memory usage.

    In most post-training setups, URR values below 1/3 are sufficient for learning new tasks, making OSFT notably lighter in memory.

    Like SFT, LoRA requires a fixed amount of overhead memory to store the base model, intermediate activations, and outputs. The rest of the memory needed for LoRA scales linearly based on the LoRA rank (lora_r) parameter. The lora_r value is an integer, ideally no more than the size of any of the model’s weight dimensions, that determines how many rows should be used in each of LoRA’s approximated matrices.

    You should keep the lora_r value as low as possible. As lora_r approaches 0, the memory that LoRA uses should approach 1/4 * SFT. While it is difficult to precisely compare SFT and LoRA, LoRA’s memory usage should begin to reach or exceed that of SFT’s if the value of lora_r is more than 3/8 of the size of the hidden dimensionality. Note that the memory used by LoRA in Training Hub is further reduced by the fact that LoRA uses Float16 as its main datatype. QLoRA uses Float4 instead. Note that when using QLoRA, you must briefly place the Float16 model onto the GPU, which can bottleneck memory usage.

  • Training time: On data sets of equal size, OSFT typically takes about twice as long per phase. However, because OSFT does not require replay buffers from past tasks, unlike SFT, the total training time across multiple phases or tasks is lower with clear benefits as the number of tasks grows. Because OSFT supports continual learning without maintaining or reusing old data, it enables lighter, single-pass end-to-end runs.

5.5. Training Hub in OpenShift AI
Copy link

On OpenShift AI, you can run Training Hub training in an interactive mode where training runs directly in your notebook environment. This single in-workbench pod approach gives you fast iteration for small experiments, immediate feedback during development, and easy debugging. You can inspect variables, logs, and intermediate artifacts in real time.

To run in-workbench training, use the Training | Jupyter | PyTorch | CUDA | Python notebook image. This image is pre-packaged with all the libraries and dependencies required by Training Hub. All included packages are curated and distributed through the Red Hat package index, ensuring they are tested, supported, and kept up to date. You can start training without installing additional packages.

For comprehensive tutorials on Fine Tuning with Training Hub, follow these guided examples:

Training Hub provides automatic experiment tracking integration with MLflow. You can track, compare, and manage machine learning experiments across different training runs and algorithms. MLflow is an open-source platform for managing the ML lifecycle, including experimentation, reproducibility, and deployment.

Training Hub automatically logs the following information to MLflow:

  • All training hyperparameters, such as learning rate, batch size, and number of epochs.
  • Training metrics, such as loss, learning rate schedule, and gradient norms.
  • Model configuration and architecture details.
  • Run metadata, such as timestamps, duration, and hardware configuration.
Note

In distributed training for multi-GPU or multi-node, to avoid duplicate entries, only rank 0 performs logging.

The integration works alongside other logging systems, such as Weights & Biases, TensorBoard, and JSONL logging.

5.6.1. Benefits of experiment tracking
Copy link

Experiment tracking eliminates the need to manually track hyperparameters, metrics, and results in spreadsheets or notebooks. With MLflow integration, Training Hub automatically logs all training parameters and metrics. You can use the logs to help you with the following tasks: * Compare performance across different model configurations with standardized logging * Maintain a complete record of all training runs with full reproducibility context * Share experimental results with team members through live dashboards instead of files or screenshots * Make data-driven decisions for hyperparameter selection based on easy comparison across runs * Accelerate experimentation velocity by eliminating manual bookkeeping overhead.

5.6.2. Enable MLflow tracking
Copy link

Training Hub supports MLflow tracking for all training algorithms: Supervised Fine-tuning (SFT), Orthogonal Subspace Fine-tuning (OSFT), and Low-Rank Adaptation (LoRA).

Prerequisites

  • You have access to an MLflow tracking server.
  • You can run MLflow locally or use an enterprise MLflow deployment.

Procedure

  1. To enable MLflow tracking, provide the MLflow tracking URI when you call a training function as shown in the following example: 

    from training_hub import sft

  sft(
      model_path="Qwen/Qwen2.5-7B-Instruct",
      data_path="./data.jsonl",
      ckpt_output_dir="./checkpoints",
      num_epochs=3,
      learning_rate=2e-5,
      mlflow_tracking_uri="http://localhost:5000",
      mlflow_experiment_name="qwen-sft-experiments",
      mlflow_run_name="baseline-run"
  )
  2. Optional. Configure MLflow tracking by using API parameters or environment variables as listed in the following table:
Expand
Configuration descriptionAPI ParameterEnvironment VariableRequired?

The URI of the MLflow tracking server.

Training Hub automatically enables MLflow logging without requiring additional configuration.

mlflow_tracking_uri

MLFLOW_TRACKING_URI

Required.

The workbench exposes the MLFLOW_TRACKING_URI environment variable.

The name of the MLflow experiment to log runs under

mlflow_experiment_name

MLFLOW_EXPERIMENT_NAME

Optional

The name for this specific training run

mlflow_run_name

MLFLOW_RUN_NAME

Optional

Additional resources

For detailed parameter documentation and advanced configuration options, see the Training Hub documentation in the Training Hub repository.

If you want to implement distributed training across multiple nodes to meet the needs of your training workloads, you can use the Kubeflow Trainer. The Kubeflow Trainer abstracts the underlying infrastructure complexity of distributed training and fine-tuning of models. The iterative process of fine-tuning significantly reduces the time and resources required compared to training models from scratch.

Learn more about the Kubeflow Trainer in the following OpenShift AI documentation:

The Kubeflow Trainer supports distributed fine-tuning by using Training Hub, abstracting the complexity of distributed training. It seamlessly manages scaling and orchestration for you, allowing you to focus on your domain-specific fine-tuning logic by using the simplified Training Hub APIs.

For a comprehensive tutorial on Fine Tuning with Training Hub leveraging distributed nodes with the Kubeflow Trainer, follow these guided examples:

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top