Local runtime - LocalAI team (Ettore Di Giacinto) - verified
LocalAI: install, API, platforms and features
LocalAI is a local LLM runtime from LocalAI team (Ettore Di Giacinto). As its docs describe it: LocalAI is an open-source, self-hosted engine that serves LLMs and other model types through OpenAI- and Anthropic-compatible APIs on CPU-only or GPU hardware. The latest release is v4.11.0, published October 2, 2026. License: MIT (open source).
- Latest release
- v4.11.0
- Released
- 2026-10-02
- License
- Open source
- OpenAI API
- Yes
Install
Copied verbatim from the page linked under each command. Labels are worded as the source words them.
docker (CPU only)
docker run -ti --name local-ai -p 8080:8080 localai/localai:latestSource: github.com/mudler/LocalAI
docker (NVIDIA CUDA 12)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12Source: github.com/mudler/LocalAI
docker (AMD ROCm)
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblasSource: github.com/mudler/LocalAI
docker (Vulkan)
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkanSource: github.com/mudler/LocalAI
Run a model
The first-run example from the official docs. OpenAI-compatible endpoint: http://localhost:8080 (chat completions at http://localhost:8080/v1/chat/completions)
local-ai run llama-3.2-1b-instruct:q4_k_mSource: github.com/mudler/LocalAI
Features
What the LocalAI docs say it supports. Not stated means the docs do not say either way.
| Feature | Value | Source |
|---|---|---|
| Interface | CLI, Server, Docker, Desktop app | See sources below |
| Platforms | macOS, Linux, Windows | localai.io/docs/installation/containers |
| Backends | CPU, CUDA, ROCm, SYCL, Metal, Vulkan | github.com/mudler/LocalAI |
| Model formats | GGUF, Safetensors, MLX | github.com/mudler/LocalAI |
| OpenAI-compatible API | Yeshttp://localhost:8080 (chat completions at http://localhost:8080/v1/chat/completions) | localai.io/docs/basics/getting_started |
| Anthropic-compatible API | Yes | localai.io/docs/basics/getting_started |
| Tool calling | Yes | github.com/mudler/LocalAI |
| Structured outputs | Yes | github.com/mudler/LocalAI |
| Vision (image input) | Yes | github.com/mudler/LocalAI |
| Multi-GPU | Yes | github.com/mudler/LocalAI |
| MCP client | Yes | localai.io/docs/features/mcp |
| License | MITOpen source | github.com/mudler/LocalAI/blob/master/LICENSE |
| Latest release | v4.11.0October 2, 2026 | github.com/mudler/LocalAI/releases/tag/v4.11.0 |
Models to run with LocalAI
Browse the open-weight models you can run locally for download sizes, licenses and memory fit; check which formats LocalAI loads in the table above.
Compare LocalAI
FAQ
Does LocalAI have an OpenAI-compatible API?
Yes. Endpoint, per its docs: http://localhost:8080 (chat completions at http://localhost:8080/v1/chat/completions). Source: https://localai.io/docs/basics/getting_started/
Does LocalAI have an Anthropic-compatible API?
Which platforms and GPUs does LocalAI support?
Its docs list macOS, Linux and Windows, with these backends: CPU, CUDA, ROCm, SYCL, Metal and Vulkan.
How do I install LocalAI?
On docker (CPU only): "docker run -ti --name local-ai -p 8080:8080 localai/localai:latest", as shown at https://github.com/mudler/LocalAI. This page lists 4 install commands: docker (CPU only), docker (NVIDIA CUDA 12), docker (AMD ROCm) and docker (Vulkan).
Is LocalAI open source?
Yes. It is released under MIT; the license is at https://github.com/mudler/LocalAI/blob/master/LICENSE.
What is the latest version of LocalAI?
v4.11.0, published October 2, 2026 (https://github.com/mudler/LocalAI/releases/tag/v4.11.0), as of October 18, 2026.
Which model formats does LocalAI run?
GGUF, Safetensors and MLX, per https://github.com/mudler/LocalAI
Sources
Last verified from the pages below.
- LocalAI GitHub README - github.com
- LocalAI release v4.11.0 - github.com
- LocalAI releases Atom feed - github.com
- LocalAI Quickstart - localai.io
- Install LocalAI - localai.io
- Linux installation - localai.io
- Containers - localai.io
- Model Context Protocol (MCP) - localai.io
- LICENSE - github.com
Newsletter
Local AI, every week
One email per week. New model releases, runtime updates and what they mean for what you build.
Keep exploring
Open-weight models you can run
Download sizes, licenses and the run command for each model.
All local LLM runtimes
Every runtime in one feature matrix, with sources.
Picking a runtime for coding agents
Ollama, LM Studio, vLLM and llama.cpp for agent workloads.
Setup guides
Point your coding tools at the model you picked.