AI FrameworksOpen source

LiteLLM

Open-source AI gateway: call 100+ LLM providers in the OpenAI format via a Python SDK or proxy. Spend tracking, budgets, rate limiting, load balancing, and fallbacks built in.

Try LiteLLMlitellm.ai

Save

LiteLLM is the open-source standard for putting one interface in front of every LLM provider. It ships as two pieces: a Python SDK for application code and a gateway/proxy you deploy in front of your infrastructure, both speaking the OpenAI format across 100+ provider integrations including Azure, Bedrock, Gemini, OpenAI, and Anthropic. The gateway layer is what production teams come for - per-key spend tracking, budgets, rate limiting, load balancing, and automatic fallbacks when a provider degrades. The project reports 240M+ Docker pulls and over 1,000 GitHub contributors. Where OpenRouter is a hosted multi-model API, LiteLLM is the self-hosted equivalent you control end to end, and the two pair well in a fallback chain.

ai framework gateway proxy open-source multi-provider python

Similar Tools

AI Frameworks

Agency Swarm

Multi-agent orchestration framework built on the OpenAI Agents SDK. Define agent roles, typed tools, and directional communication flows. Production-focused, open-source.

AI Frameworks

Pydantic AI

Type-safe Python agent framework from the Pydantic team. Brings the FastAPI feeling to AI development. Composable tools, durable execution, and full IDE autocomplete.

AI Frameworks

Outlines

Constrained generation library for LLMs. Uses finite state machines to mask invalid tokens during generation. Guarantees schema-compliant output with zero retries.

AI Frameworks

Haystack

Open-source AI orchestration framework by deepset. Modular pipelines for RAG, agents, semantic search, and multimodal apps. Pipeline-as-graph architecture with explicit control.

Get started with LiteLLM

Open-source AI gateway: call 100+ LLM providers in the OpenAI format via a Python SDK or proxy. Spend tracking, budgets, rate limiting, load balancing, and fallbacks built in.

Try LiteLLM

Get weekly tool reviews

Honest takes on AI dev tools, frameworks, and infrastructure - delivered to your inbox.

Subscribe Free

Compare all pricing Compare side by side

More AI Frameworks Tools

Vercel AI SDK

The TypeScript toolkit for building AI apps. Unified API across OpenAI, Anthropic, Google. Streaming, tool calling, structured output, multi-step agents. 50K+ GitHub stars.