2.6. vllm run-batch arguments

Disable logging requests.

Default: False

Disable logging statistics.

Default: False

Enables Prometheus metrics.

Default: False

Enables prompt_tokens_details in usage when set to True.

Default: False

Maximum number of prompt characters or prompt ID numbers printed in the log.

Default: Unlimited

The directory to store the output file before uploading it to the output URL.

Default: None

Port number for the Prometheus metrics server. Only needed if enable-metrics is set.

Default: 8000

The role name to return if request.add_generation_prompt=True.

Default: assistant

Prometheus metrics server URL. Only required if enable-metrics is set).

Default: 0.0.0.0

DEPRECATED. Block manager v1 has been removed. SelfAttnBlockSpaceManager (block manager v2) is now the default. Setting --use-v2-block-manager flag to True or False has no effect on vLLM behavior.

Default: True

The path or URL to a single input file. Supports local file paths and HTTP or HTTPS. If a URL is specified, the file should be available using HTTP GET.

Default: None

The path or URL to a single output file. Supports local file paths and HTTP or HTTPS. If a URL is specified, the file should be available using HTTP PUT.

Default: None