AI FrameworksOpen source
L
LiteLLM
Open-source AI gateway: call 100+ LLM providers in the OpenAI format via a Python SDK or proxy. Spend tracking, budgets, rate limiting, load balancing, and fallbacks built in.
Try LiteLLMVisit litellm.ailitellm.ai
Share
LiteLLM is the open-source standard for putting one interface in front of every LLM provider. It ships as two pieces: a Python SDK for application code and a gateway/proxy you deploy in front of your infrastructure, both speaking the OpenAI format across 100+ provider integrations including Azure, Bedrock, Gemini, OpenAI, and Anthropic. The gateway layer is what production teams come for - per-key spend tracking, budgets, rate limiting, load balancing, and automatic fallbacks when a provider degrades. The project reports 240M+ Docker pulls and over 1,000 GitHub contributors. Where OpenRouter is a hosted multi-model API, LiteLLM is the self-hosted equivalent you control end to end, and the two pair well in a fallback chain.
LiteLLM at a glance
Last verified
- Pricing
- Open source - free under the MIT license, except the enterprise/ directory, which is under the LiteLLM Commercial License
- Enterprise - custom pricing through sales, with a 30-day trial key: SSO and SCIM, RBAC, audit logs, air-gapped and multi-region deployment, 24/7 support with SLAs
- Install
uv add litellmuv tool install 'litellm[proxy]'- Models
- Routes to 100+ providers in the OpenAI format, including OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex AI, vLLM and Nvidia NIM. You pick a model with a provider-prefixed string such as openai/gpt-4o in completion() or in the proxy config. v1.105.0 added Claude Sonnet 5.5 and GPT-6.1 Sol.
- Recent changes
- v1.105.0 adds Claude Sonnet 5.5 and GPT-6.1 Sol, a Decisions API on /v1/decisions across eight providers, litellm.agent() for running coding agents through the gateway, and Lens with Agent Traces.
- v1.104.0 adds Claude Opus 5.5 and GPT-6, enforces the master key, and turns stdio MCP off by default.
- v1.103.0 adds config file ownership, Fuse and Capability routing, and gateway hardening.
Sources: BerriAI/litellm README, LiteLLM LICENSE, LiteLLM Enterprise, Proxy quickstart, LiteLLM release notes
Similar Tools
AI Frameworks
A
Agency Swarm
Multi-agent orchestration framework built on the OpenAI Agents SDK. Define agent roles, typed tools, and directional communication flows. Production-focused, open-source.
AI Frameworks
Pydantic AI
Type-safe Python agent framework from the Pydantic team. Brings the FastAPI feeling to AI development. Composable tools, durable execution, and full IDE autocomplete.
AI Frameworks
O
Outlines
Constrained generation library for LLMs. Uses finite state machines to mask invalid tokens during generation. Guarantees schema-compliant output with zero retries.
AI Frameworks
H
Haystack
Open-source AI orchestration framework by deepset. Modular pipelines for RAG, agents, semantic search, and multimodal apps. Pipeline-as-graph architecture with explicit control.
Get started with LiteLLM
Open-source AI gateway: call 100+ LLM providers in the OpenAI format via a Python SDK or proxy. Spend tracking, budgets, rate limiting, load balancing, and fallbacks built in.
Try LiteLLMGet weekly tool reviews
Honest takes on AI dev tools, frameworks, and infrastructure - delivered to your inbox.
Subscribe FreeMore AI Frameworks Tools
Vercel AI SDK
The TypeScript toolkit for building AI apps. Unified API across OpenAI, Anthropic, Google. Streaming, tool calling, structured output, multi-step agents. 27K+ GitHub stars.
CopilotKit
Frontend stack for agent-native apps. React hooks, prebuilt copilot UI, AG-UI runtime, frontend tools, shared state, and human-in-the-loop flows.
Claude Agent SDK
Anthropic's SDK for building agents on the same tools, agent loop, and context management that power Claude Code. Python and TypeScript. Subagents, hooks, MCP, and permissions.
Related Guides
Guide
AI Agent Frameworks Compared: LangGraph vs CrewAI vs Mastra vs CopilotKit
Deep comparison of the top AI agent frameworks - LangGraph, CrewAI, Mastra, CopilotKit, AutoGen, and Claude Code.
AI AgentsGuide
Getting Started with DevDigest CLI
Install the dd CLI and scaffold your first AI-powered app in under a minute: Next.js 16, Convex, Clerk, Autumn, Tailwind v4 and an agent-ready CLAUDE.md.
Getting StartedGuide
Claude Code Setup Guide
Configure Claude Code for maximum productivity -- CLAUDE.md, sub-agents, MCP servers, and autonomous workflows.
AI AgentsRelated Posts
8 min read
ai
DiffusionGemma: Google Bets Diffusion Can Make Text Generation 4x Faster
DiffusionGemma is Google's 26B MoE open model that generates 256-token blocks in parallel. What diffusion generation mea...
9 min read
Anthropic
Claude API 2026-10-21: Breaking Changes and SDK Migration
Claude API version 2026-10-21 removes temperature, top_p and top_k, and new major SDKs (TypeScript 1.0, Python 2.0) send...
13 min read
Embeddings
Which Embedding Model Should You Use in 2026? Specs Compared
Which embedding model to use in 2026: 20+ API and open-weight models compared on price per 1M tokens, dimensions, max in...
11 min read
Anthropic
Anthropic Python SDK 1.0: How to Migrate to httpx2
Anthropic Python SDK 1.0 swaps httpx for httpx2, requires Python 3.10+, and drops temperature and Text Completions. Veri...
14 min read
AI Agents
Choosing an Agent Framework in Late 2026: 10 Frameworks Compared by Use Case
Ten agent frameworks, one decision guide: a capability matrix built from official docs and a flowchart that picks by lan...
15 min read
AI Agents
LlamaIndex in 2026: Workflows, Agents, and What Changed
LlamaIndex in late 2026: Python Workflows 2.26 with typed events, human-in-the-loop and DBOS durability, plus LiteParse....
Try the member tools free
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever