Ollama vs LM Studio vs vLLM vs llama.cpp: Picking a Local Runtime for Coding Agents

Last updated: August 22, 2026 Running a coding agent against a model on your own machine instead of a hosted API changes the calculus: no per-token bill, no rate limit, and the model never leaves your network. It also means you are now responsible for the part a hosted API used to hide from you: picking a runtime, choosing a quantization, and living with whatever tokens-per-second your hardware actually delivers. Four names come up constantly when developers wire a local model into an agent harness (Claude Code, Cursor, Continue, Aider, or a custom loop): Ollama, LM Studio, vLLM, and llama.cpp directly. They are not four competitors doing the same job. One is a convenience layer, one is a GUI, one is a throughput engine for serving many requests, and one is the engine underneath most of the others.

This is a fair, sourced comparison: what each runtime actually is, how they relate to each other, and a decision guide for wiring one into a coding agent.

What each runtime is actually for#

Ollama is a daemon plus CLI that wraps llama.cpp (and its own backend work) behind a simple ollama run <model> and an OpenAI-compatible /v1/chat/completions endpoint. It handles model pulls, quantization defaults, and a Modelfile system for custom prompts and parameters. The tradeoff for that convenience is less direct control over batching and memory layout than you get going straight to llama.cpp or vLLM. It's a common first target for agent harnesses because the API surface matches OpenAI's, so baseURL swaps are close to a one-line change.

LM Studio is a desktop app with a chat UI, a model browser, and a built-in local server that also exposes an OpenAI-compatible endpoint. Under the hood it can run either llama.cpp (GGUF models) or Apple's MLX runtime on Apple Silicon, and its docs specify which engine a given model card uses. It is the easiest on-ramp for someone who wants to browse Hugging Face models, download a quant, and click "start server" without touching a terminal. It's less suited to headless CI or server deployments than the other three.

vLLM is built for throughput: it introduced PagedAttention for efficient KV-cache memory management and supports continuous batching, so it is the choice when many requests (or many agent instances) hit the same served model concurrently. It expects a GPU with real VRAM headroom and is typically run as a long-lived server process rather than a personal chat app. Model support and quantization options are documented in the vLLM supported models list.

llama.cpp is the C/C++ inference engine that Ollama and (partly) LM Studio build on top of. Running it directly means access to every flag and quantization option the moment it lands, at the cost of managing model files, context length, and GPU offload flags (-ngl, --ctx-size, etc.) yourself. The project's own server example exposes an OpenAI-compatible endpoint as well, so the "just point my agent at localhost" pattern works here too.

Where they actually overlap#

All four can, in the end, expose an OpenAI-style chat completions endpoint that a coding agent can talk to, which is why the comparison is more about workflow fit than raw capability:

• Ollama and llama.cpp share weights format (GGUF) and, in practice, much of the same underlying inference code.
• LM Studio's GGUF path is llama.cpp under a GUI; its MLX path is a separate runtime for Apple Silicon.
• vLLM does not use GGUF as its primary path; it favors safetensors-based models with its own quantization support (see the vLLM quantization docs), and its GGUF loader is documented as experimental, which matters if you're picking a model checkpoint before picking a runtime.

Decision guide for wiring one into a coding agent#

Solo developer on a laptop, want the fastest path from "nothing" to "agent talking to a local model": Ollama. The Modelfile and library of pre-quantized models mean less manual GGUF hunting, and most agent CLIs and IDE plugins (Continue, several Claude Code community configs) document an Ollama base URL out of the box.

You want to browse and compare models visually before committing, and you're on a Mac: LM Studio, especially if you want to try MLX-optimized builds alongside GGUF ones side by side. Its local server mode still gives you the same OpenAI-compatible endpoint for your agent once you've picked a model.

You're serving a model to a team, a CI fleet of agents, or multiple concurrent coding sessions and have real GPU capacity: vLLM. Continuous batching and PagedAttention exist specifically for the multi-request case; running it for a single interactive chat session on a laptop is using a truck to deliver one envelope.

You want every quantization and performance flag the moment it ships, and don't mind the terminal: llama.cpp directly. This is also the right layer to drop to when Ollama's abstraction hides a flag you need (custom RoPE scaling, specific -ngl tuning, draft-model speculative decoding) that hasn't been exposed through Ollama's own flags yet. The October 2026 MoE expert cache is a current example: see llama.cpp MoE offload: --n-cpu-moe vs the new expert cache for which flag to reach for when a model spills out of VRAM. New architectures are the other reason to drop down a layer: Aleph Alpha's Kolibri currently runs only on a patched llama.cpp build, not in Ollama or LM Studio, as covered in how to run Kolibri locally.

What doesn't change no matter which runtime you pick#

Quantization level and context window are usually a bigger factor in agent quality than the runtime choice. A coding agent doing multi-file edits needs enough context window to hold the files it's editing plus the conversation, and a heavily quantized model will degrade on the same tasks regardless of whether Ollama, LM Studio, vLLM, or llama.cpp served it. Runtime choice affects speed, concurrency, and workflow friction; model and quantization choice affects whether the agent's edits are any good. Pick the model first, then pick the runtime that fits how you'll actually use it.

FAQ#

Can I switch between Ollama, LM Studio, and llama.cpp without changing my agent's config?#

Often yes for the API shape, since all three can expose an OpenAI-compatible chat completions endpoint on localhost. What does change is the port, any auth headers, and sometimes the exact model name string the endpoint expects, so check each project's docs (linked above) rather than assuming a single config works everywhere unmodified.

Does vLLM support GGUF models like Ollama and llama.cpp do?#

vLLM does load GGUF models, but its own GGUF docs describe that path as highly experimental and under-optimized, and warn it may be incompatible with other vLLM features. Its primary, best-supported path is safetensors-based models with its own quantization stack, documented in the vLLM quantization docs. If you already have a GGUF file from the Ollama/llama.cpp ecosystem, check the current vLLM docs for compatibility status before assuming it will load cleanly.

Do I need a GPU to run any of these for a coding agent?#

No, all four support CPU-only inference, but speed drops sharply, especially for the larger models a coding agent benefits from. Apple Silicon Macs get a meaningful boost from LM Studio's MLX backend or llama.cpp's Metal support; on other hardware, GPU offload (partial or full, via -ngl in llama.cpp/Ollama) is what makes local agent latency tolerable for interactive use.

Is running a local model actually cheaper than an API for coding agent work?#

It shifts the cost from per-token billing to hardware and electricity you already own (or have to buy), plus your time managing the runtime. For light, occasional use, a hosted API is usually cheaper once you account for hardware amortization. For heavy, continuous agent workloads on hardware you already have, local inference can be materially cheaper per token, but you're trading a metered bill for a fixed capital and maintenance cost, and it's worth comparing tokens-per-second on your actual hardware against your actual usage before deciding.

Sources#

• Ollama docs
• Ollama OpenAI compatibility docs
• ollama/ollama on GitHub
• LM Studio docs
• ml-explore/mlx on GitHub
• vLLM docs
• vLLM PagedAttention design doc
• vLLM supported models
• vLLM quantization docs
• vLLM GGUF docs
• vllm-project/vllm on GitHub
• ggml-org/llama.cpp on GitHub
• llama.cpp server
• GGUF format spec

Continue Reading#

• Open-Weight Models You Can Run Locally - each model's Ollama tag and the vLLM, SGLang and llama.cpp commands copied from official pages, with download sizes
• How much VRAM do you need to run a local LLM in 2026? - why a model that fits on disk can still run out of memory at agent-length contexts
• Local LLM Runtimes Compared - Ollama, LM Studio, llama.cpp, vLLM and eight more in one feature matrix: platforms, OpenAI and Anthropic API compatibility, tool calling and MCP, each value sourced
• Kolibri Release Guide: Aleph Alpha's 78B Open Model - Aleph Alpha's English-German MoE, released October 3, with benchmarks and the vLLM command
• GitHub Copilot CLI With Ollama: Local Models and the New Sandbox - Copilot CLI now discovers Ollama models from /model, and the context setting that makes or breaks it
• vLLM vs TGI vs SGLang: Which Inference Server to Self-Host - production-scale self-hosted inference, rather than local agent runtimes
• Self-Hosted vs Managed AI Gateways: A Decision Guide - when to self-host at the gateway layer
• Best AI Agent Memory Providers in 2026 - memory layers that pair with local inference
• Jeff: Jev-Style Decision Models Trained at Home - a 0.8B model served with your own runtime, and what it costs in latency on an M4 Max
• llama.cpp MoE Offload: --n-cpu-moe vs the New Expert Cache - placing MoE experts across GPU and RAM, with the October 2026 GPU expert cache
• Magnitude: A Self-Tuning Inference Engine for Local Agents - a runtime that tunes kernels on your device, and where early testers measured it
• Hermes Agent Guide (2026): What It Is, Local Setup With Ollama, Skills, and Hermes vs OpenClaw - Nous Research's MIT-licensed self-improving agent, run fully locally with Ollama, and how it compares to OpenClaw
• Ollama 0.40 on Mac: MLX by Default, Disk Space Doubled - the October 2026 Ollama change that runs supported models on MLX and keeps backup copies on disk