Chapter 1. About the AI Inference Server API server

Red Hat AI Inference Server provides an OpenAI-compatible API server for inference serving. You can control the behavior of the server with arguments.

The AI Inference Server command-line interface includes commands for serving models, running chat completions, generating text completions, benchmarking performance, and collecting environment information for troubleshooting. Each command accepts specific arguments that configure resource allocation, model loading behavior, API compatibility options, and performance tuning parameters.

You can also configure AI Inference Server behavior through environment variables, which is useful for containerized deployments where command-line arguments are less practical. Built-in metrics endpoints provide observability into server performance, request latency, token throughput, and resource utilization.