6.2. Configuring the GPU that you want to use with the command-line assistant

Procedure

1. Open the ~/.config/rhel-lightspeed/.env file in a text editor. For example:

   $ vi ~/.config/rhel-lightspeed/.env

2. Set the container image variable:

   1. Locate the LLAMACPP_IMAGE variable and set it up to the corresponding ramalama container for your GPU hardware. For example:

      LLAMACPP_IMAGE="registry.redhat.io/rhel-cla/ramalama-rhel10-nvidia:latest"

3. Set the host device variable:

   1. Locate the HOST_DEVICE variable for the device of your GPU. For example:

      # For AMD GPUs
      HOST_DEVICE="/dev/dri"
      # For NVIDIA GPUs
      HOST_DEVICE="/dev/nvidia0"

4. Optional: Configure NVIDIA-specific variables:

   1. Locate and set up the NVIDIA-specific variables.

5. Restart the assistant to apply the changes:

   $ rhel-lightspeed stop
   $ rhel-lightspeed start