Interacting with the command-line assistant powered by RHEL Lightspeed

Procedure

1. Enable the terminal capture for your current terminal session:

   $ c shell --enable-capture

2. Run at least one command before you reference previous commands.

3. Reference the output of a previously run command. For example, to reference the previous command, run:

   $ c -w 1 "what_is_this"

   • To reference the second to previous command, run:

     $ c -w 2 "what_is_this"

4. To stop terminal capture, press Ctrl + D.

Procedure

1. On the connected RHEL system, complete the following steps:

   1. Log in to the Red Hat registry:

      $ podman login registry.redhat.io

   2. Download the installer and all dependency images:

      $ podman pull registry.redhat.io/rhel-cla/installer-rhel10:latest
      $ podman pull registry.redhat.io/rhel-cla/rag-database-rhel10:latest
      $ podman pull quay.io/ramalama/ramalama:0.16.0
      $ podman pull registry.redhat.io/rhel-cla/rlsapi-rhel10:latest

   3. Save all the images as a single tar file:

      $ podman save -o rhel-cla-installer-images.tar \
          registry.redhat.io/rhel-cla/installer-rhel10:latest \
          registry.redhat.io/rhel-cla/rag-database-rhel10:latest \
          quay.io/ramalama/ramalama:0.16.0 \
          registry.redhat.io/rhel-cla/rlsapi-rhel10:latest

2. Transfer the rhel-cla-installer-images.tar file to the disconnected system by using a USB drive, network share, or other transfer method.

3. On the disconnected system, load the image:

   $ podman load -i rhel-cla-installer-images.tar

4. On the disconnected system, run the installer container. This command uses the loaded image and installs the offline RHEL command-line assistant on your system.

   $ podman run -u : --rm \
   -v $HOME/.config:/config:Z \
   -v $HOME/.local/bin:/config/.local/bin:Z \
   rhel-cla/installer-rhel10:latest

5. Optional: To automatically start the offline RHEL command-line assistant containers each time the system boots, as a systemd service, run the following command:

   $ rhel-cla enable-service

6. Install the RHEL command-line-assistant client package on your offline RHEL system:

   # sudo dnf install command-line-assistant

7. Update the endpoint configuration option.

   1. Open the /etc/xdg/command-line-assistant/config.toml file in a text editor.

   2. Locate the endpoint variable and change its value to the address of the system hosting your offline RHEL command-line assistant containers. For example, to connect to the local system:

      endpoint = "http://127.0.0.1:8000/v1"

8. Restart the clad service to apply the new configuration:

   $ sudo systemctl restart clad

   After the installation is complete, you can use the rhel-cla command to start, stop, uninstall, and check the status of the offline RHEL command-line assistant.

Verification

1. Check if the offline RHEL command-line assistant is running:

   $ rhel-cla status

2. Verify that the container is running:

   $ podman ps
   CONTAINER ID   IMAGE         COMMAND     CREATED          STATUS
   ba655e5efdcd   installer-rh… /sbin/init  30 seconds ago   Up 29 seconds

   It might take about a minute for the rlsapi and the ragdb containers to initialize completely.

3. If the assistant is not running, start it:

   $ rhel-cla start

4. Check if the client is correctly configured by asking the following question:

   $ c "What is an immutable file?"
   
   + Asking Red Hat Lightspeed
   This feature uses AI technology. Do not include any personal information or other sensitive information in your input. Interactions may be used to improve Red Hat's products or services.

Procedure

1. Open the ~/.config/rhel-cla/.env file in a text editor. For example:

   $ vi ~/.config/rhel-cla/.env

2. Set the container image variable:

   1. Locate the LLAMACPP_IMAGE variable and set it to the corresponding ramalama container for your GPU hardware. For example:

      LLAMACPP_IMAGE="quay.io/ramalama/cuda:0.16.0"

3. Set the host device variable:

   1. Locate the HOST_DEVICE variable for the device of your GPU. For example:

      # For AMD GPUs
      HOST_DEVICE="/dev/dri"
      # For NVIDIA GPUs
      HOST_DEVICE="/dev/nvidia0"

4. Optional: Configure NVIDIA-specific variables:

   1. Locate and set up the NVIDIA-specific variables.

5. Restart the assistant to apply the changes:

   $ rhel-cla stop
   $ rhel-cla start

Procedure

1. Identify the running container ID:

   $ podman ps | grep ramalama
   6eef0fb344b7  quay.io/ramalama/ramalama:0.16.0                                                ramalama --store ...  7 seconds ago  Up 7 seconds  0.0.0.0:8000->8000/tcp, 0.0.0.0:8888->8888/tcp  rhel-cla-llamacpp-model

2. Copy the LLM model file into the ramalama container by replacing <path_to_local_model> with your local path and <container_id> with the container ID found in the previous step:

   $ podman cp path_to_local_llm_file.gguf 6eef0fb344b7:/models/

   Alternatively, you can copy the LLM model file directly to the local volume path:

   $ cp path_to_local_llm_file.gguf ~/.local/share/containers/storage/volumes/llamacpp-models/_data/

3. Stop the rhel-cla service:

   $ rhel-cla stop

4. Edit the ~/.config/rhel-cla/.env and add the following configuration to the LLM variable:

   LLM="file:///models/llm_file.gguf"

5. Restart the rhel-cla service:

   $ rhel-cla start

Procedure

1. Check the SSHD status and restart it.

   $ sudo systemctl status ssh
   $ sudo systemctl restart ssh

2. Enable the optional command-line assistant terminal capture feature:

   $ c shell --enable-capture

3. Use the -w 1 "your_question" option to specify to include the output from the last command that was run.

   $ c -w 1 "what_is_this"

   • If you specify the number 2, that references the output from the second-to-last command. This is also true for the additional numbers.

   • You can also specify a prompt to run with the command and ask "help me understand the output", and reference the output with the error, so that the command-line assistant understands that you are asking for more details on what the error is.

     The command-line assistant processes the request and provides several possible solutions. In the example, you can use the suggestion to run the journalctl -xeu sshd.service command to check the log files.

4. Run that journalctl command and check the log files to identify potential issues.

   $ journalctl -xeu sshd.service

5. Ask the command-line assistant to generate a command on how to fix this typing error.

   $ c "what is the command that I can use to change 'Porrt' to 'Port' in the /etc/ssh/sshd_config file?"

6. Run the command suggested by the command-line assistant as a sudo user. For example:

   $ sudo sed -i s/Porrt/Port/g /etc/ssh/sshd_config

Procedure

1. Check the httpd package version and identify the ports on which the web server accepts incoming requests:

   $ sudo rpm -qa httpd

   httpd-2.4.62-2.fc40.x86_64

   $ cat /etc/httpd/conf/httpd.conf | grep Listen

   Listen 80

2. Restart the httpd service and check for errors:

   $ systemctl restart httpd
   $ sudo journalctl -xeu httpd.service

   Job for httpd. Service failed because the control process exited with error code.

3. Use the command-line assistant to troubleshoot the service failure and check SELinux status:

   $ c "why did httpd fail to start"
   $ c "selinux httpd port"
   $ sudo sestatus

4. Check the SELinux context for httpd and ask the command-line assistant for guidance:

   $ sudo cat /usr/share/selinux/targeted/contexts/httpd_var_run_t

   No such file or directory

   $ c "i don't have a httpd_var_run_t contexts"

5. Set the SELinux context and configure the port as suggested by the command-line assistant:

   $ sudo chcon -R -t httpd_var_run_t
   $ c "selinux won't let httpd listen on port 12345"
   $ sudo semanage port -a -t httpd_port_t -p tcp 12345

6. If you receive a ValueError, ask the command-line assistant and apply the suggested fix:

   $ c "how do I fix ValueError: Type httpd_port_t is invalid, must be a port type"
   $ sudo ls -Z /usr/sbin/httpd
   $ chcon -t httpd_exec_t /usr/sbin/httpd
   $ sudo setenforce 1

7. Restart httpd and enable network connections if the service still fails:

   $ sudo systemctl restart httpd
   $ c "how do I enable httpd to listen on port 12345 selinux"
   $ sudo setsebool -P httpd_can_network_connect=1

8. If the service fails again, check the journal and search for SELinux denials:

   $ journalctl -xeu httpd
   $ c "An ExecStart= process belonging to unit httpd.service has exited."
   $ sudo ausearch -m AVC,USER_AVC -ts recent

9. Enter the AVC denial output to the command-line assistant for analysis:

   $ c "avc: denied {name_bind} for pid=7184 comm="httpd" src=12345 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket permissive=0"

10. Add the port to the correct SELinux type to resolve the binding error:

    $ sudo semanage port -a -t http_port_t -p tcp 12345

Procedure

1. Check if the system is registered:

   # subscription-manager identity

2. If the system is not registered, use the following command to register the system:

   # subscription-manager register

Procedure

1. Restart the command-line assistant daemon (clad).

   $ systemctl restart clad

2. Rerun the c command with your question:

   $ c "<Your_question>"

Procedure

1. Change the key.pem permission to root.

   $ sudo ls -l /etc/pki/consumer/
   $ sudo chown $(whoami):$(id -gn) /etc/pki/consumer/${<key_name>}

2. Run clad commands as root.

   $ sudo chown $(whoami):$(id -gn) /etc/pki/consumer/${<key_name>}

Procedure

1. Access the proxy configuration by opening the /etc/xdg/command-line-assistant/config.toml configuration file.

2. Locate and change the following block in the config.toml file :

   # Backend settings for communicating with the external API.
   [backend]
   ...
   # proxies = { http = "http://example-host:8002", https = "https://example-host:8002" }

3. Uncomment the proxies key and define your http or https proxy host:

   [backend]
   …
   # For a https proxy host
   proxies = { https = "https://<your_https_proxy_host:1234>"}

4. After making the changes, restart clad for the changes to be effective:

   $ sudo systemctl restart clad

   Note

   You can use the http value and https key control whether clad routes http or https traffic to the specified proxy. However, the protocol does not influence proxy type selection, and you can configure an http proxy for https traffic. For example:

   +

   https = "http://<your_https_proxy_host:1234>"

Procedure

1. Install the database of your choice:

   • To install MySQL, enter:

     # dnf install python3-PyMySQL

   • To install PostgreSQL, enter:

     # dnf install python3-psycopg2

2. Access your database configuration file at /etc/xdg/command-line-assistant/config.toml.

3. Locate and comment out the default configuration. For example:

    [database]
       # type = "sqlite"
       # connection_string = "/var/lib/command-line-assistant/history.db"

4. Configure the database of your choice. The following information is also available in /etc/xdg/command-line-assistant/config.toml.

   1. Set the database type, where <db_type> can be mysql or postgresql.

   2. Set the database details.

       type = <db_type>
       host = "<hostname_or_ip_address>"
       port = "5432"
       username = "<database_user_name>"
       password = "<password>"
       database = "<database_name>"

5. After changing the database type, restart the clad daemon to apply the changes:

   $ sudo systemctl restart clad

Procedure

1. Access your database configuration file at /etc/xdg/command-line-assistant/config.toml.

2. Remove the username and password parameters from the [database] section, for example:

   [database]
   type = "postgresql"
   host = "localhost"
   port = "5432"
   database = "history"

   Note

   If you leave the username and password in the configuration file, these credentials take precedence over the systemd-creds tool.

3. Generate encrypted credentials for your username or password. The following example uses systemd-ask-password commands. The name must follow the schema of database-username and database-password, otherwise, clad does not load the credentials properly.

   1. To generate an encrypted username, run the following command:

      $ systemd-ask-password -n | ( echo "[Service]" && systemd-creds encrypt --name=<database_username> -p - - ) >/etc/systemd/system/clad.service.d/<username>.conf

   2. To generate an encrypted password, run the following command:

      $ systemd-ask-password -n | ( echo "[Service]" && systemd-creds encrypt --name=<database_password> -p - - ) >/etc/systemd/system/clad.service.d/<password>.conf

4. After updating the database credentials, reload systemd and restart the clad daemon to apply the changes:

   $ sudo systemctl daemon-reload
   $ sudo systemctl restart clad

Procedure

1. On the client machine, generate an SSH keypair:

   $ ssh-keygen -t ed25519 -C "<your-email@example.com>" -f id_ed25519_mcp

2. On the RHEL target machine, create a non-root user:

   1. Access the VM by SSH and run the following commands:

      $ sudo useradd mcp
      $ sudo passwd mcp

3. On the client machine, add the target machine to your SSH configuration file:

   1. Edit the ~/.ssh/config configuration file and add the following information:

      Host <rhel-10-0>
      HostName <target-machine-ip-address>
      User <user-test>
      Port 22
      IdentityFile <path-to-your-ssh-pub-key>
      StrictHostKeyChecking no

4. Copy your public key from your client machine to the target machine.

   $ ssh-copy-id -i ~/.ssh/id_ed25519_mcp.pub mcp@<vm_ip_address>

5. Restart the SSH service:

   $ sudo systemctl restart sshd

Procedure

1. Optional: Configure environment variables to select which log files the MCP server for RHEL can access, and the logging level required.

   export LINUX_MCP_ALLOWED_LOG_PATHS="/var/log/messages,/var/log/secure,/var/log/audit/audit.log"
   export LINUX_MCP_LOG_LEVEL="INFO"

2. Run the RHEL MCP server.

   $ linux-mcp-server

3. Ask questions, for example:

   "What version of RHEL is this system running?"
   "Show me the current CPU load and usage per core."
   "How much memory is being used on this system?"
   "Which filesystems are running out of space?"
   "What hardware is installed in this system?"
   "Show me all critical and error logs since yesterday."

   Warning

   Current known issues and limitations:

   You might experience issues getting the SSH authentication working in the container, especially with Docker, because it does not support user namespace mapping. To workaround this issue, the SSH key file must be owned by UID 1001 on the container host.

   When you run the MCP server for RHEL in a container, it cannot inspect the local system because of container isolation.

Procedure

1. Pull your container image:

   $ podman pull quay.io/redhat-services-prod/rhel-lightspeed-tenant/linux-mcp-server:latest

2. Save the image to a tar file:

   $ podman save -o linux-mcp-server-latest.tar quay.io/redhat-services-prod/rhel-lightspeed-tenant/linux-mcp-server:latest

3. Transfer the tar file to a physical storage device:

   1. Identify your drive:

      $ lsblk

   2. Mount the drive and copy the tar file:

      $ mount /dev/sdb1 /mnt/usb
      cp /<path-to-file>/<file>.tar.gz /mnt/usb/
      umount /mnt/usb

4. Move the tar file to the air-gapped environment and extract the tar file to the local staging directory:

   $ mkdir -p /opt/app/staging
   tar -xzvf /media/usb/<file>.tar.gz -C /opt/app/staging

Procedure

1. On the MCP server client machine, generate an SSH keypair:

   $ ssh-keygen -t ed25519 -C "mcp-server-key" -f ~/.ssh/id_ed25519_mcp

2. Log in to your RHEL target machine and create a dedicated non-root user service account for the MCP server:

   $ sudo useradd mcp
   $ sudo passwd mcp

3. Copy your public key from your client machine to the target machine:

   $ ssh-copy-id -i ~/.ssh/id_ed25519_mcp.pub mcp@<target-ip-address>

   <target-ip-address>

   Specifies the IP address of the target machine.

   If firewall policies restrict the ssh-copy-id command, manually append the public key to the /home/mcp/.ssh/authorized_keys file on the target system.

Procedure

1. Transfer the linux-mcp-server-latest.tar file to the host.

2. Load the image:

   $ podman load -i linux-mcp-server-latest.tar

3. Run the container:

   $ podman run --rm --interactive \
     --userns keep-id:uid=1001,gid=0 \
     -e LINUX_MCP_USER \
     -v $HOME/.ssh/id_ed25519_mcp:/var/lib/mcp/.ssh/id_ed25519:ro,Z \
     linux-mcp-server:latest