What is the difference between an LLM router and an LLM gateway?
The terms overlap. In this directory, hosted routers and marketplaces put many providers' models behind one hosted API (OpenRouter, Vercel AI Gateway, Requesty and Martian Gateway). A gateway sits in front of provider accounts you already have and adds routing, caching, limits and logs, either hosted (Cloudflare AI Gateway, Portkey and Helicone AI Gateway), inside a cloud API management platform (Kong AI Gateway, Azure API Management AI gateway and Google Apigee) or run yourself (Portkey AI Gateway, Helicone, Kong Gateway, LiteLLM Proxy, Agent Router, TensorZero and Bifrost). Not Diamond is a routing service that says it is not a gateway at all.
Do LLM gateways add a markup on tokens?
It depends on the product and on whose keys you use, and the fees are in different units, so they do not rank. In the vendors' own words: Vercel AI Gateway: "AI Gateway charges no markup and no platform fee on tokens." OpenRouter: "OpenRouter's fee is charged when you buy credits, 5.5% on Standard and 8% on Business, never on individual requests." Requesty: "You pay for what your application spends, plus 5%." Helicone AI Gateway: "use credits with 0% markup" Cloudflare AI Gateway: "A 5% fee is applied to all credits purchased through Unified Billing." The self-hosted entries list an open-source license as their pricing, with enterprise editions priced separately where stated.
Is there a fee for bringing your own provider keys?
OpenRouter: no OpenRouter fee on the first $25,000/month of list-price inference (Standard and Business); 5% of list price above that allowance. Vercel AI Gateway: no markup or fee. Helicone AI Gateway: managed-key (credits) fallback at 0% markup; no BYOK fee stated. For the rest, no BYOK-specific fee is stated in the pages we checked, which is not the same as no fee.
Which LLM gateways accept Anthropic API requests?
7 products state an Anthropic-compatible endpoint: OpenRouter, Vercel AI Gateway, Requesty, Martian Gateway, Azure API Management AI gateway, LiteLLM Proxy and Bifrost.
Which LLM gateways can I self-host?
7 products publish an open-source license: Portkey AI Gateway (MIT), Helicone (Apache 2.0), Kong Gateway (Apache 2.0), LiteLLM Proxy (MIT), Agent Router (Apache 2.0), TensorZero (Apache 2.0) and Bifrost (Apache 2.0). TensorZero is archived and no longer maintained by its own statement. Portkey, Helicone and Kong also offer hosted or commercial editions beside the open-source project, listed as separate entries.
Is TensorZero still maintained?
No, by its own statement: "TensorZero remains available on GitHub but is no longer maintained." GitHub shows: "This repository was archived by the owner on Jun 12, 2026. It is now read-only."
What happened to Envoy AI Gateway?
It was renamed. The docs say: "Agent Router is the new name for Envoy AI Gateway, now an Agentic AI Foundation project." They also say CRD, API and CLI names are unchanged.
Is Not Diamond an LLM gateway?
No, by its own description: "Not Diamond is not a gateway. Rather, we integrate with your existing gateway infrastructure." It is listed here as a routing product. Its stated fee is $0.05 per million tokens routed.
Is this a ranking of LLM gateways?
No. Every value is a first-party statement copied from the vendor's own pages, with the quote and the link. Nothing here measures latency, uptime or quality, and fees in different units are never converted or compared.
What do Not stated and inferred mean?
Not stated means the pages we checked did not say, so the cell is left blank with the reason rather than guessed; it is not evidence that the feature is missing. Inferred marks our reading of a quote, such as a circuit breaker read as a fallback, or a status the page implies without labelling it.