Chapter 8. About vLLM uvicorn access logs in distributed inference

Procedure

1. Edit your LLMInferenceService custom resource:

   $ oc edit llminferenceservice <service_name> -n <namespace>

   Replace <service_name> with the name of your LLMInferenceService and <namespace> with the namespace where the LLMInferenceService is deployed.

2. In the editor, navigate to the spec.template.containers section.
3. Locate the container named main.

4. Add the --disable-access-log-for-endpoints flag to the args list with your custom endpoint list:

   Note

   The default filtering behavior is applied automatically by the platform template and does not appear in the LLMInferenceService custom resource. When you add the flag to your args list, your custom list overrides the platform default.

   To use the same endpoint list as the platform default (filtering health checks and metrics):

   apiVersion: serving.kserve.io/v1alpha2
   kind: LLMInferenceService
   metadata:
     name: my-vllm-service
     namespace: <namespace>
   spec:
     replicas: 2
     model:
       uri: hf://RedHatAI/Qwen3-8B-FP8-dynamic
       name: RedHatAI/Qwen3-8B-FP8-dynamic
     router:
       route: {}
       gateway: {}
       scheduler: {}
     template:
       containers:
       - name: main
         args:
         - --disable-access-log-for-endpoints=/health,/metrics,/ping
         - --max-model-len=10000
         - --gpu-memory-utilization=0.9

   To filter additional endpoints such as /readyz and /livez, include them in your custom list along with the default endpoints:

   ...
       containers:
       - name: main
         args:
         - --disable-access-log-for-endpoints=/health,/metrics,/ping,/readyz,/livez
         - --max-model-len=10000
         - --gpu-memory-utilization=0.9
   ...

   To reduce filtering and see logs for the /ping endpoint, specify only the endpoints you want filtered:

   ...
       containers:
       - name: main
         args:
         - --disable-access-log-for-endpoints=/health,/metrics
         - --max-model-len=10000
         - --gpu-memory-utilization=0.9
   ...

5. Save and close the editor.

   The LLMInferenceService controller automatically applies the updated configuration and restarts the affected pods.

Verification

1. Verify that the LLMInferenceService is running:

   $ oc get llminferenceservice <service_name> -n <namespace>

   The output shows the service status:

   NAME              READY   AGE
   my-vllm-service   True    5m

2. View the pod logs to confirm that health check and metrics requests are not logged:

   $ oc logs <pod_name> -n <namespace> | grep "/metrics"

   Replace <pod_name> with the name of one of the vLLM pods.

   The output should be empty, confirming that /metrics requests are filtered.

3. Make a test inference request to the /v1/completions endpoint.

4. View the pod logs again to verify that the inference request was logged:

   $ oc logs <pod-name> -n <namespace> | grep "/v1/completions"

   The output shows the inference request:

   INFO:     172.30.45.2:54325 - "POST /v1/completions HTTP/1.1" 200 OK

   The IP addresses shown are internal pod network addresses and will vary by deployment.

5. Confirm that only actual API usage is logged while health checks are filtered.

Procedure

1. Determine which vLLM version your deployment uses.

   Tip

   The vLLM version is displayed in the pod startup logs as follows:

   [access-log-detect] vllm version='<version>'
   [access-log-detect] selected ACCESS_LOG_ARGS='<flags>'

   The version is also shown as a label in the OpenShift AI under the Deployment resource.

2. For vLLM 0.16 or later, add the --disable-access-log-for-endpoints flag with an empty value:

   1. Edit the LLMInferenceService custom resource:

      $ oc edit llminferenceservice <service_name> -n <namespace>

   2. In the editor, navigate to spec.template.containers and find the container named main.

   3. Add the --disable-access-log-for-endpoints= flag with an empty value to the args list:

      spec:
        template:
          containers:
          - name: main
            args:
            - --disable-access-log-for-endpoints=
            - --max-model-len=10000
            - --gpu-memory-utilization=0.9

      The empty value disables endpoint filtering entirely, causing all endpoints to be logged.

   4. Save and close the editor.

      The LLMInferenceService controller automatically applies the updated configuration and restarts the affected pods.

3. For vLLM versions below 0.16, add the --no-disable-uvicorn-access-log flag:

   Note

   This applies only to deployments using custom vLLM images below version 0.16. All shipped Red Hat OpenShift AI 3.5 images use vLLM 0.19.1 or later.

   1. Edit the LLMInferenceService custom resource:

      $ oc edit llminferenceservice <service_name> -n <namespace>

   2. In the editor, navigate to spec.template.containers and find the container named main.

   3. Add the --no-disable-uvicorn-access-log flag to the args list:

      spec:
        template:
          containers:
          - name: main
            args:
            - --no-disable-uvicorn-access-log
            - --max-model-len=10000
            - --gpu-memory-utilization=0.9

   4. Save and close the editor.

      The LLMInferenceService controller automatically applies the updated configuration and restarts the affected pods.

Verification

1. Verify that the LLMInferenceService is running:

   $ oc get llminferenceservice <service_name> -n <namespace>

   The output shows the service status:

   NAME              READY   AGE
   my-vllm-service   True    5m

2. View the pod logs to confirm that health check and metrics access log entries appear:

   $ oc logs <pod_name> -n <namespace> | grep -E "/(health|metrics|ping)"

   Replace <pod_name> with the name of one of the vLLM pods.

   The output shows HTTP access log entries for health checks and metrics:

   INFO:     172.30.45.2:54321 - "GET /metrics HTTP/1.1" 200 OK
   INFO:     172.30.45.2:54322 - "GET /health HTTP/1.1" 200 OK
   INFO:     172.30.45.2:54323 - "GET /ping HTTP/1.1" 200 OK

   The IP addresses shown are internal pod network addresses and will vary by deployment.

3. Verify that inference request logs also appear:

   $ oc logs <pod_name> -n <namespace> | grep "/v1/completions"

   The output shows inference request access log entries:

   INFO:     172.30.45.2:54324 - "POST /v1/completions HTTP/1.1" 200 OK

4. Confirm that all endpoint types are now logged.