Skip to main content
Watch: I Built an AI App Without Leaving Databricks

API compatibility - 5 documented limits

vLLM API compatibility

vLLM documents an OpenAI-compatible API at http://localhost:8000/v1 (completions, chat/completions, chat/completions/batch, responses, embeddings, audio/transcriptions, audio/translations and models). Its Anthropic-compatible base URL, the value for ANTHROPIC_BASE_URL, is http://localhost:8000. The docs list 5 limitations: 4 for the OpenAI-compatible layer and 1 for the Anthropic-compatible layer.

Verified against vLLM's own documentation. Not documented means the docs read that day do not say; it never means No.

Compare every provider in the API compatibility matrix.

OpenAI-compatible API

Supported
Base URL
http://localhost:8000/v1
Endpoints
  • completions
  • chat/completions
  • chat/completions/batch
  • responses
  • embeddings
  • audio/transcriptions
  • audio/translations
  • models
Auth
Optional: --api-key (or VLLM_API_KEY) on the server; pass the same value as the OpenAI client's api_key
Notes
Self-hosted server (vllm serve), default port 8000. --api-key only authenticates /v1, /v2 and /inference prefixes; other endpoints such as /invocations are not authenticated. Non-OpenAI sampling params (top_k, min_p, etc.) go through extra_body. By default the server applies the model repo's generation_config.json.

From the docs

vLLM provides an HTTP server that implements OpenAI's Completions API, Chat API, and more!
docs.vllm.ai/en/latest/serving/online_serving/openai_compatible_server/

Anthropic-compatible API

Supported
ANTHROPIC_BASE_URL
http://localhost:8000
Auth
ANTHROPIC_AUTH_TOKEN is required by Claude Code but can be any value; ANTHROPIC_API_KEY can be any value since vLLM doesn't require authentication by default
Notes
Endpoints /v1/messages and /v1/messages/count_tokens. Claude Code needs a model with tool calling enabled (--enable-auto-tool-choice plus the right --tool-call-parser). Served model names containing '/' cannot be used directly from Claude Code.

From the docs

vLLM implements the Anthropic Messages API, which is the same API that Claude Code uses to communicate with Anthropic's servers. By setting ANTHROPIC_BASE_URL to point at your vLLM server, Claude Code sends its requests to vLLM instead of Anthropic.
docs.vllm.ai/en/stable/serving/integrations/claude_code/

Documented limitations

Parameters, content types and features vLLM's docs say its compatibility layers ignore, reject or change, each with the line it comes from. A layer with nothing listed may still differ in ways the docs do not spell out.

OpenAI-compatible layer4

Anthropic-compatible layer1

Example from vLLM's docs

Copied verbatim from the provider's documentation, placeholders included.

Provider example
from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="token-abc123",
)

completion = client.chat.completions.create(
    model="NousResearch/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "user", "content": "Hello!"},
    ],
)

Newsletter

Know when a provider changes its API

One email per week. New models, new endpoints and the compatibility changes that break a base URL, and what they mean for your stack.