Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. Using a containerized command-line assistant for disconnected environments

The command-line assistant for disconnected environments is available as a Developer Preview. You can use it in air-gapped networks, remote locations, and other environments with limited internet connectivity.

The command-line assistant for disconnected environments is packaged in a UBI-based container, with key components such as the redhat-knowledge-bridge image built on Red Hat Hardened Images and includes simplified configuration, orchestration, and basic lifecycle management.

Warning

Changing the RHEL Offline Container model to unsupported models can enable the execution of arbitrary code or compromise the integrity of your Red Hat Enterprise Linux (RHEL) systems.

Important

The command-line assistant tool is Developer Preview software only. Red Hat does not support Developer Preview software in any way and is not functionally complete or production-ready. Do not use Developer Preview software for production or business-critical workloads. Developer Preview software provides early access to upcoming product software in advance of its possible inclusion in a Red Hat product offering. Customers can use this software to test functionality and provide feedback during the development process. This software might not have any documentation, is subject to change or removal at any time, and has received limited testing. Red Hat might provide ways to submit feedback on Developer Preview software without an associated SLA.

For more information about the support scope of Red Hat Developer Preview software, see Developer Preview Support Scope.

With the command-line assistant for disconnected environments, you can create a container image targeted for a single system. This container image includes the following components:

  • An installer container to pull the other required containers, install the rhel-cla command, and optionally configures a systemd service.
  • The RHEL Knowledge Bridge container to bridge offline knowledge components.
  • A postgres database container to store local application data.
  • A ramalama container to run the Large Language Model (LLM) inference service.
  • The lightspeed stack container, which includes the command-line assistant tool.

  • The Red Hat Offline Knowledge Portal container to serve local documentation.

    The container image includes the Phi-4-mini-instruct-Q4_K_M model, and serves as the default model for the ramalama container.

The container image is intended for local, disconnected use on a single system. It is not designed for multi-system deployments or large-scale environments.

Install the command-line assistant in a disconnected environment by using container images available from the Red Hat Container Registry and Quay.io. Manually transfer the installer image and its dependency images to the local storage of the offline system in the disconnected environment.

Prerequisites

  • Your RHEL system is registered to a Red Hat Satellite subscription. For additional details, see the Does the offline RHEL command-line-assistant require the system to be registered? article.
  • You have requested and obtained an API key for the RHOKP container. To request your API key, access the Red Hat Offline Knowledge Portal or contact your Red Hat account representative.
  • You have Podman installed.
  • You have the container-tools meta-package installed.
  • You have created the $HOME/.config and $HOME/.local/bin folders.

  • Software requirements:

    • Operating system: A supported operating system version, such as RHEL 9.6 or later, RHEL 10 or later, Fedora 43 or later, Windows 11 with WSL2 and Podman, or macOS 15 or later.

  • Hardware requirements:

    • Storage: At least 35 to 40 GB of available disk space across all system types, to accommodate the container images and local LLM models.

    • Memory and Processing:

      • For CPU-only systems: At least 8 GB of RAM and 2 CPU cores.
      • For GPU-capable systems: At least 4 GB of RAM and a GPU with at least 4 GB of VRAM.
      • For Mac computers: A machine equipped with an M2 chip or later.

Procedure

  1. On the connected RHEL system, complete the following steps:

    1. Log in to the Red Hat registry: 

      $ podman login registry.redhat.io

    2. Download the installer and all dependency images: 

      $ podman pull registry.redhat.io/rhel-cla/installer-rhel10:latest
$ podman pull registry.access.redhat.com/hi/postgresql:18
$ podman pull registry.redhat.io/lightspeed-core/lightspeed-stack-rhel9:v0.6-latest
$ podman pull quay.io/ramalama/ramalama:latest
$ podman pull registry.redhat.io/rhel-cla/rhel-knowledge-bridge-rhel10
$ podman pull registry.redhat.io/offline-knowledge-portal/rhokp-rhel9:latest

    3. Save all the images as a single tar file: 

      $ podman save -o rhel-cla-installer-images.tar \
    registry.redhat.io/rhel-cla/installer-rhel10:latest \
    registry.access.redhat.com/hi/postgresql:18 \
    registry.redhat.io/lightspeed-core/lightspeed-stack-rhel9:v0.6-latest \
    quay.io/ramalama/ramalama:latest \
    registry.redhat.io/rhel-cla/rhel-knowledge-bridge-rhel10 \
    registry.redhat.io/offline-knowledge-portal/rhokp-rhel9:latest
  2. Transfer the rhel-cla-installer-images.tar file to the disconnected system by using a USB drive, network share, or other transfer method.

  3. On the disconnected system, load the image: 

    $ podman load -i rhel-cla-installer-images.tar

  4. On the disconnected system, run the installer container. This command uses the loaded image and installs the offline command-line assistant on your system. 

    $ podman run -u : --rm \
-v $HOME/.config:/config:Z \
-v $HOME/.local/bin:/config/.local/bin:Z \
rhel-cla/installer-rhel10:latest

  5. Optional: To automatically start the offline command-line assistant containers each time the system boots, as a systemd service, run the following command: 

    $ rhel-cla enable-service

  6. Install the command-line-assistant client package on your offline RHEL system: 

    # sudo dnf install command-line-assistant

  7. Update the endpoint configuration option.

    1. Open the /etc/xdg/command-line-assistant/config.toml file in a text editor.

    2. Locate the endpoint variable and change its value to the address of the system hosting your offline command-line assistant containers. For example, to connect to the local system: 

      endpoint = "http://127.0.0.1:8501/v1"

    3. Restart the clad service to apply the new configuration: 

      $ sudo systemctl restart clad

      After the installation is complete, you can use the rhel-cla command to start, stop, uninstall, and check the status of the offline command-line assistant.

  8. Start the containers: 

    $ rhel-cla start

Verification

  1. Check if the offline command-line assistant is running: 

    $ rhel-cla status

  2. Verify that the container is running: 

    $ podman ps
CONTAINER ID   IMAGE         COMMAND     CREATED          STATUS
ba655e5efdcd   installer-rh… /sbin/init  30 seconds ago   Up 29 seconds

    It might take about a minute for the rlsapi and the ragdb containers to initialize completely.

  3. If the assistant is not running, start it: 

    $ rhel-cla start

  4. Check if the client is correctly configured by asking the following question: 

    $ c "What is an immutable file?"

+ Asking Red Hat Lightspeed
This feature uses AI technology. Do not include any personal information or other sensitive information in your input. Interactions may be used to improve Red Hat's products or services.

For better performance, configure the graphics processing unit (GPU) setting by editing the ~/.config/rhel-cla/.env file. This file contains information to enable you to select the correct container image and device settings for your hardware.

Prerequisites

Procedure

  1. Open the ~/.config/rhel-cla/.env file in a text editor. For example: 

    $ vi ~/.config/rhel-cla/.env

  2. Optional: Configure NVIDIA-specific variables. If you need to restrict visibility to specific GPU devices or define explicit driver capabilities on Linux, locate and uncomment the following lines by removing the # symbol, then update their values as needed: 

    # NVIDIA-specific settings (Linux only - uncomment if needed)
# CUDA_VISIBLE_DEVICES=0
# NVIDIA_DRIVER_CAPABILITIES="compute,utility"

  3. Restart the assistant to apply the changes: 

    $ rhel-cla stop
$ rhel-cla start

Troubleshooting

  • Resolve GPU errors after restarting an EC2 instance.

    When you use the offline command-line assistant on a GPU-enabled AWS EC2 instance, the rhel-cla start command might fail after the instance is stopped and restarted. This error occurs because the GPU ID changes after a restart. To work around this problem, regenerate the NVIDIA Container Device Interface (CDI) configuration file by running the following command: 

    $ nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

    This updates the reference of the system to the GPU, enabling the rhel-cla service to start correctly.

6.3. Changing the LLM to a custom model
Copy link

To use a locally installed custom model, copy the model file into the ramalama LLM container and update your configuration.

Prerequisites

  • The rhel-cla service has been started at least once by (rhel-cla start).

Procedure

  1. Identify the running container ID: 

    $ podman ps | grep ramalama
6eef0fb344b7  quay.io/ramalama/ramalama:0.16.0                                                ramalama --store ...  7 seconds ago  Up 7 seconds  0.0.0.0:8000->8000/tcp, 0.0.0.0:8888->8888/tcp  rhel-cla-llamacpp-model

  2. Copy the LLM model file into the ramalama container by replacing <path_to_local_model> with your local path and <container_id> with the container ID found in the previous step: 

    $ podman cp path_to_local_llm_file.gguf 6eef0fb344b7:/models/

    Alternatively, you can copy the LLM model file directly to the local volume path: 

    $ cp path_to_local_llm_file.gguf ~/.local/share/containers/storage/volumes/llamacpp-models/_data/

  3. Stop the rhel-cla service: 

    $ rhel-cla stop

  4. Edit the ~/.config/rhel-cla/.env and add the following configuration to the LLM variable: 

    LLM="file:///models/llm_file.gguf"

  5. Restart the rhel-cla service: 

    $ rhel-cla start
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top