Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 3. Tool call parser type reference

Tool call parsers determine how AI Inference Server interprets and extracts tool calling information from the model output. Each model has specific output formats for tool calls. You must configure the appropriate parser that matches the expected format for the model. The tool call parser is specified using the --tool-call-parser flag when launching AI Inference Server. AI Inference Server supports multiple tool call parsers for different model families and output formats.

Expand
Table 3.1. Tool call parsers
ParserDescriptionExample models

hermes

For Nous-Hermes models that use the Hermes tool calling format.

  • NousResearch/Hermes-2-Pro-Llama-3-8B
  • NousResearch/Hermes-3-Llama-3.1-8B

mistral

For Mistral models using Mistral’s native tool calling format.

  • mistralai/Mistral-7B-Instruct-v0.3
  • mistralai/Mixtral-8x7B-Instruct-v0.1

llama3_json

For Llama 3.x models configured to output tool calls in JSON format. When using llama3_json, you typically pair it with a JSON-formatted chat template.

  • meta-llama/Llama-3.2-1B-Instruct
  • meta-llama/Llama-3.1-8B-Instruct

internlm2

For InternLM2 models using InternLM’s tool calling format.

  • internlm/internlm2-chat-7b
  • internlm/internlm2-chat-20b

granite-20b-fc

For IBM Granite function-calling models.

ibm-granite/granite-20b-functioncalling

fuyu

For Adept Fuyu models.

adept/fuyu-8b

phi3_json

For Microsoft Phi-3 models configured for JSON output.

  • microsoft/Phi-3-mini-4k-instruct
  • microsoft/Phi-3-medium-128k-instruct

jamba

For AI21 Labs Jamba models.

ai21labs/Jamba-v0.1

Important

Using an incorrect parser can result in runtime errors such as the following:

  • Failed tool call extraction
  • Malformed tool call requests
  • Errors during inference
  • Unexpected model behavior

Always verify that the parser matches the expected tool calling format for the model.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top