Chapter 6. Enable vLLM uvicorn access logs

Procedure

1. Edit your LLMInferenceService custom resource to override the container command.

   To enable uvicorn access logs, override the container startup command without the --disable-uvicorn-access-log flag:

   apiVersion: serving.kserve.io/v1alpha2
   kind: LLMInferenceService
   metadata:
     name: my-vllm-service
     namespace: <namespace>
   spec:
     replicas: 2
     model:
       uri: hf://RedHatAI/Qwen3-8B-FP8-dynamic
       name: RedHatAI/Qwen3-8B-FP8-dynamic
     router:
       route: {}
       gateway: {}
       scheduler: {}
     template:
       containers:
       - name: main
         command:
         - /bin/bash
         - -c
         - |
           vllm serve "$@"
         - --
         args:
         - /mnt/models
         - --served-model-name=RedHatAI/Qwen3-8B-FP8-dynamic
         - --port=8000
         - --max-model-len=10000
         - --gpu-memory-utilization=0.9
         # Remove the following three lines if your deployment does not use TLS
         - --enable-ssl-refresh
         - --ssl-certfile=/var/run/kserve/tls/tls.crt
         - --ssl-keyfile=/var/run/kserve/tls/tls.key
         resources:
           limits:
             cpu: '4'
             memory: 32Gi
             nvidia.com/gpu: "1"
           requests:
             cpu: '2'
             memory: 16Gi
             nvidia.com/gpu: "1"

   • command specifies a minimal bash wrapper that calls vllm serve and forwards the contents of args to it as positional arguments via "$@". The -- separator after the script body is required for bash to treat subsequent entries as positional parameters rather than bash options.

   • args specifies the vLLM server arguments. The first entry, /mnt/models, is the required model path that the storage initializer mounts into the container. The remaining entries are vLLM CLI flags. Replace the --served-model-name value with the value of spec.model.name from your LLMInferenceService.

     Note

     When you override the command field, you replace all default command-level behavior, including the --disable-uvicorn-access-log flag that is normally applied. By omitting this flag in your custom command, uvicorn access logs are enabled.

2. Apply the updated custom resource.

Verification

1. Verify that the LLMInferenceService is running:

   $ oc get llminferenceservice <SERVICE_NAME> -n <NAMESPACE>

2. View the pod logs to confirm that access log entries appear:

   $ oc logs <POD_NAME> -n <NAMESPACE> | grep "HTTP/1.1"

   where <POD_NAME> is the name of one of the vLLM pods.

   The output shows HTTP access log entries:

   INFO:     172.30.45.2:54321 - "GET /metrics HTTP/1.1" 200 OK
   INFO:     172.30.45.2:54322 - "GET /health HTTP/1.1" 200 OK
   INFO:     172.30.45.2:54323 - "POST /v1/completions HTTP/1.1" 200 OK

   Verify the log format includes the HTTP method, endpoint path, and status code.