Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 5. Using a containerized RHEL command-line assistant for disconnected environments

A containerized version of the Red Hat Enterprise Linux (RHEL) command-line assistant for disconnected environments is available as a Developer Preview. This version is packaged in a UBI-based container, and includes simplified configuration, orchestration, and basic lifecycle management. You can use it in air-gapped networks, remote locations, and other environments with limited internet connectivity.

Important

Red Hat Enterprise Linux (RHEL) command-line assistant is Developer Preview software only. Developer Preview software is not supported by Red Hat in any way and is not functionally complete or production-ready. Do not use Developer Preview software for production or business-critical workloads. Developer Preview software provides early access to upcoming product software in advance of its possible inclusion in a Red Hat product offering. Customers can use this software to test functionality and provide feedback during the development process. This software might not have any documentation, is subject to change or removal at any time, and has received limited testing. Red Hat might provide ways to submit feedback on Developer Preview software without an associated SLA.

For more information about the support scope of Red Hat Developer Preview software, see Developer Preview Support Scope.

With the Red Hat Enterprise Linux (RHEL) command-line assistant for disconnected environments, you can create a container image targeted for a single system. This image includes the following components:

  • An installer container to pull the other required containers, install the rhel-lightspeed command, and optionally create a systemd service.
  • An rlsapi container to provide the endpoint used by the command-line assistant client.
  • A rag-database container that contains the retrieval-augmented generation (RAG) database, which supplements the LLM’s knowledge with additional data such as the RHEL documentation.
  • A ramalama container that runs the LLM inference service.
  • The command-line assistant powered by RHEL Lightspeed.
Warning

The container image is intended for local, disconnected use on a single system. It is not designed for multi-system deployments or large-scale environments.

Install the Red Hat Enterprise Linux (RHEL) command-line assistant in a disconnected environment by using the installer container image available on the Red Hat Container Registry. This container pulls the other required containers, installs the rhel-lightspeed commands, and optionally creates a systemd service.

Prerequisites

  • Your RHEL system is registered to a Red Hat Satellite subscription.
  • You have Podman installed.
  • You have the container-tools meta-package installed.

Procedure

  1. Authenticate to the Red Hat Container Registry: 

    $ podman login registry.redhat.io
    Copy to Clipboard Toggle word wrap

  2. Run the installer container. This command downloads the necessary images and installs the offline RHEL command-line assistant on your system. 

    $ podman run -u : --rm \
-v $HOME/.config:/config:Z \
-v $HOME/.local/bin:/config/.local/bin:Z \
registry.redhat.io/rhel-cla/installer:latest
    Copy to Clipboard Toggle word wrap

  3. Optional: To automatically start the offline RHEL command-line assistant containers each time the system boots, add the install-systemd argument to the podman run command: 

    $ podman run -u : --rm \
-v $HOME/.config:/config:Z \
-v $HOME/.local/bin:/config/.local/bin:Z \
registry.redhat.io/rhel-cla/installer:latest install-systemd
    Copy to Clipboard Toggle word wrap

  4. Verify that the container is running: 

    $ podman ps
CONTAINER ID   IMAGE         COMMAND     CREATED          STATUS
ba655e5efdcd   installer-rh… /sbin/init  30 seconds ago   Up 29 seconds
    Copy to Clipboard Toggle word wrap

  5. Install the RHEL command-line-assistant client package on your RHEL system: 

    # sudo dnf install command-line-assistant
    Copy to Clipboard Toggle word wrap

  6. Update the endpoint configuration option.

    1. Open the /etc/xdg/command-line-assistant/config.toml file in a text editor.

    2. Locate the endpoint variable and change its value to the address of the system hosting your offline RHEL command-line assistant containers. For example, to connect to the local system: 

      endpoint = "http://127.0.0.1:8000/"
      Copy to Clipboard Toggle word wrap

  7. Restart the clad service to apply the new configuration: 

    $ sudo systemctl restart clad
    Copy to Clipboard Toggle word wrap

    After the installation is complete, you can use the rhel-lightspeed command to start, stop, uninstall, and check the status of the offline RHEL command-line assistant.

Verification

  1. Check if the offline RHEL command-line assistant is running: 

    $ rhel-lightspeed start
    Copy to Clipboard Toggle word wrap

  2. Check if the client is correctly configured by asking a question: 

    $ c "what is an immutable file?"

+ Asking RHEL Lightspeed
This feature uses AI technology. Do not include any personal information or other sensitive information in your input. Interactions may be used to improve Red Hat's products or services.
    Copy to Clipboard Toggle word wrap

For better performance, you can change the configuration settings of the graphic processing unit (GPU) that is attached to the host system by editing the ~/.config/rhel-lightspeed/.env file. This file contains information to help you select the correct container image and device settings for your hardware.

Prerequisites

Procedure

  1. Open the ~/.config/rhel-lightspeed/.env file in a text editor. For example: 

    $ vi ~/.config/rhel-lightspeed/.env
    Copy to Clipboard Toggle word wrap

  2. Set the container image variable:

    1. Locate the LLAMACPP_IMAGE variable and set it up to the corresponding ramalama container for your GPU hardware. For example: 

      LLAMACPP_IMAGE="registry.redhat.io/rhel-cla/ramalama-rhel10-nvidia:latest"
      Copy to Clipboard Toggle word wrap

  3. Set the host device variable:

    1. Locate the HOST_DEVICE variable for the device of your GPU. For example: 

      # For AMD GPUs
HOST_DEVICE="/dev/dri"
# For NVIDIA GPUs
HOST_DEVICE="/dev/nvidia0"
      Copy to Clipboard Toggle word wrap

  4. Optional: Configure NVIDIA-specific variables:

    1. Locate and set up the NVIDIA specific variables.

  5. Restart the assistant to apply the changes: 

    $ rhel-lightspeed stop
$ rhel-lightspeed start
    Copy to Clipboard Toggle word wrap

Troubleshooting

  • Resolve GPU errors after restarting an EC2 instance.

    When you use the offline RHEL command-line assistant on a GPU-enabled AWS EC2 instance, the rhel-lightspeed start command might fail after the instance is stopped and restarted. This error happens because the GPU ID changes after a restart. To work around this problem, regenerate the NVIDIA Container Device Interface (CDI) configuration file by running the following command: 

    $ nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
    Copy to Clipboard Toggle word wrap

    This updates the reference of the system to the GPU, enabling the rhel-lightspeed service to start correctly.

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat