Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Provider behavior matrix

Every adapter accepts the same OpenAI-shaped engine request, but it targets a different native API and translates only the fields that API understands.

ProviderNative APIBare-model inferenceNative namespace
OpenAIResponses APIgpt*, o1*, o3*, o4*x-openai
ChatGPT subscriptionResponses API over OAuthnone — use chatgpt/<model>x-chatgpt
AnthropicMessages APIclaude*x-anthropic
Google GeminigenerateContent / streamGenerateContentgemini*x-gemini
xAIResponses APIgrok*none
OpenRouterChat Completions (aggregator)none — address as openrouter/<vendor>/<model>x-openrouter
vLLMChat Completions (self-hosted, VLLM_BASE_URL); Responses API with VLLM_WIRE=responsesnone — address as vllm/<served-model>x-vllm
SGLangChat Completions (self-hosted, SGLANG_BASE_URL); Responses API with SGLANG_WIRE=responsesnone — address as sglang/<served-model>x-sglang

An explicit address such as anthropic/claude-sonnet-5-5 avoids inference. The named provider must be registered in the Router—that normally means its API key is configured.

ChatGPT uses a saved device-code login instead: llmshim login chatgpt. Both normal and streaming calls use upstream SSE. Normal calls aggregate a terminal response into Chat Completions JSON; a failed or truncated stream returns an error. Tools, image inputs, and reasoning use the Responses translator. ChatGPT forces store: false and stream: true and strips token limits, metadata, and sampling fields even from native overrides, following LiteLLM's backend contract. It uses a short default instruction when no instructions are supplied. Only chatgpt/gpt-6-astra, chatgpt/gpt-6-sol, chatgpt/gpt-6.1-sol, and chatgpt/gpt-6-luna are accepted; older and unlisted models fail locally before authentication or network calls. Access to those models and limits depend on the signed-in account. ChatGPT responses preserve the full chatgpt/<model> ID, including streaming chunks, so server responses identify the subscription provider correctly when API-key OpenAI is also configured. Streamed function calls are emitted once their arguments are complete. Each call includes its ID, name, and full JSON arguments, including in the proxy's tool_call event; text and reasoning still arrive incrementally.

Observable differences

ProviderMessages and toolsNotable behavior
OpenAISystem/developer text becomes Responses instructions; Chat Completions tool definitions are flattened for Responsesstore defaults to false; unified reasoning becomes the native reasoning object; x-openai.reasoning overrides that mapping
AnthropicMessages become Anthropic content blocks; tools use input_schema, tool_use, and tool_resultmax_tokens defaults to 8192 when absent; supported models receive the 1M-context beta by default; x-anthropic.extra_betas appends beta headers and disable_1m_context suppresses that automatic header
GeminiMessages become contents; tools use functionDeclarations, functionCall, and functionResponseBase64 images become inline_data, but a remote image URL becomes a text placeholder because Gemini cannot consume it directly; x-gemini.thinkingConfig replaces mapped thinking configuration
xAISystem/developer text becomes Responses instructions; tools are flattened like OpenAI ResponsesUnified reasoning becomes reasoning: {effort} where the model accepts it; Grok 4.7 clamps none to low and max to xhigh; there is no x-xai namespace
OpenRouterPassthrough — messages, tools, image_url vision, and response_format are already Chat Completions and forwarded unchangedreasoning_effort maps 1:1 to OpenRouter's reasoning:{effort} (superset vocabulary, no clamping); reasoning is normalized to provenance-bearing blocks; the middle-out transform is disabled by default; x-openrouter carries provider/models/transforms/route/native reasoning (and http_referer/x_title headers)
vLLM / SGLangPassthrough to a self-hosted server — configured by base URL (local or remote), auth optionalreasoning normalized to provenance-bearing blocks; reasoning_effort forwarded (honored per-model); server-specific params (chat_template_kwargs, guided_json, top_k, separate_reasoning, …) go under x-vllm/x-sglang. Reasoning/tool parsing depend on the server's launch flags (--reasoning-parser, --tool-call-parser)

OpenAI and xAI do not receive temperature, top_p, top_k, or stop from the portable top-level request. Anthropic and Gemini do. This is the portable-core rule in practice: unsupported fields are omitted rather than forwarded blindly.

Tools and multimodal round trips

Claude Fable 5.1

Fable 5.1 uses always-on adaptive thinking, with low, medium, high, xhigh, and max effort. Unified none maps to low; explicit native disabled/manual thinking and assistant prefill return local errors. Sampling parameters are omitted for Fable 5.1, Opus 5, and Opus 5.5 because those models reject them.

Fable 5.1 and Opus 5.5 accept only auto and none: required or a named forced tool returns a local 400 rather than changing the request's meaning. Describe the desired tool in the prompt, or use structured output when the requirement is a JSON schema. Native stop_reason: "refusal" maps to finish_reason: "content_filter" for Fable and Opus 5/5.5.

Fable 5.1 binds thinking blocks to the preceding system prompt, tools, and conversation. Keep that prefix unchanged when replaying signed reasoning. Appending turns is supported; llmshim preserves appended system/developer instructions as system turns for Fable 5.1. If your application edits or compacts prior history, remove stale thinking or use Anthropic's explicit binding controls under x-anthropic. A binding failure remains an error. See Anthropic's migration and history rules.

Opus 5.5 also requires adaptive thinking, rejects assistant prefill, and binds thinking to the conversation prefix. It accepts thinking from older Opus, Sonnet, and Haiku models, but not Fable or Mythos. Its thinking blocks replay only to Opus 5.5, Fable 5.1, or Mythos 5.1. See the Opus 5.5 migration guide.

Sonnet 5.5 requires adaptive thinking and binds thinking to the conversation prefix, and it rejects forced tool choice. Assistant prefill is not documented as rejected, so it stays allowed. Sonnet 5.5 accepts thinking from Sonnet 5, Opus 4.8, Haiku 4.5, and earlier — but not from Opus 5, Opus 5.5, or a Fable/Mythos model — and its own thinking blocks replay only to Sonnet 5.5. See the Sonnet 5.5 model page.

The raw Rust response exposes normalized thinking signatures for replay. The compact proxy response does not expose every native thinking field; applications needing native history controls can supply exact messages under provider_config.messages.

Shared message formats

Tool calls are normalized back to the OpenAI tool_calls shape. Preserve the complete assistant tool call when sending the next turn; provider adapters may need fields beyond function name and arguments.

Gemini tool signatures carry data and origin; all typed clients expose this object. Reasoning is replayed only to a compatible family and wire, and encrypted blocks also require a matching account binding.

Image input accepts OpenAI image_url, Anthropic image, and Gemini inline_data blocks. Base64 data URIs can be translated among providers. Gemini's remote-URL fallback is literal text such as [Image: URL]; llmshim does not download the URL.

Embeddings

Embeddings are a separate operation rather than a chat translation. OpenAI's /v1/embeddings, Gemini's batchEmbedContents, and an OpenAI-compatible server's /v1/embeddings are reached through llmshim::embeddings; Anthropic, xAI, OpenRouter, and the ChatGPT subscription backend have no embeddings route and refuse by name. See the embeddings guide.

Reasoning and native controls

Unified reasoning_effort and reasoning_mode are mapped by model family, not merely by provider. Floors, ceilings, and name-locked models are listed in the reasoning tables.

Use native provider controls when exact native semantics matter. Native objects are not rewritten or made portable. Changing the provider address means reconsidering the namespace as well.