LocalAI

LocalAI is the open-source AI engine that acts as a drop-in replacement for the OpenAI API, compatible with existing applications and libraries. It runs any model type (LLMs, vision, voice, image, video) on any hardware with no GPU required, though GPU acceleration is supported when available. It backs 35+ inference backends including llama.cpp, vLLM, transformers, and whisper, and supports every model format (GGUF, GPTQ, AWQ). Beyond inference, LocalAI includes a built-in agent platform with MCP support where you can create agents that use tools, browse the web, execute code, and interact with external services. For production deployments, distributed mode supports horizontal scaling with federation, P2P clustering, and model sharding. For self-hosting teams that need a single platform covering every AI modality, LocalAI is the most comprehensive open-source option.