LOCAL LLM

How much VRAM a local LLM needs: Q4 weights are about 30% of BF16, plus a KV cache that grows per token. Real numbers from 8 GB to 256 GB.

LFM2.5-24B-A2B vs Qwen3.6-35B-A3B for local agents: Liquid wins tool-use benchmarks, Qwen wins SWE-Bench Verified. Which one to run for which job.

Ollama 0.40 runs supported models on MLX on Apple Silicon and keeps a backup copy of each model you load. What it costs in disk, what is fixed, and the cleanup.

Copilot CLI 1.0.94 finds local Ollama models from /model and ships an OS-level sandbox. Setup, the 4k context trap, and why local is not offline.

llama.cpp now caches hot MoE experts on the GPU. In the PR author's RTX 5090 run, --cpu-moe plus --moe-cache-mib decoded 2.2x faster than --fit. When to use each flag.

LFM2.5-VL-3B is a 3B open-weights vision model from Liquid AI that reads screens, grounds objects and calls tools on a laptop. What shipped and how to run it.

LFM2.5-2.6B is an open-weight on-device agent model from Liquid AI that runs in under 2.5 GB. How it was trained, what the benchmarks say and how to run it.

TurboFieldfare runs Google's Gemma 4 26B MoE model in about 2 GB of RAM on any Apple Silicon Mac by streaming experts from SSD. How it works and what HN said.

Ollama vs LM Studio vs vLLM vs llama.cpp for coding agents: what each local runtime is for, where they overlap, and which to pick for your setup.

jamesob's local LLM guide: the $2k and $40k hardware paths, the BIOS and kernel settings that matter, and why multi-GPU setups fail at PCIe and IOMMU.

Local Qwen vs Claude Opus: Alex Ellis on running local LLMs in production, a 2-3 month hardware payback, and why local models are specialists, not substitutes.

Cohere North Mini Code: a 30B open-weight MoE coding model with 3B active parameters and Apache 2.0 weights. What it offers, where it runs and open questions.

Best local coding LLMs in 2026: which models to run on what hardware, how they benchmark, owned hardware versus cloud API costs, and setups by team size.

Gemma 4 ships byte-for-byte open weights from Google DeepMind. How developers deploy it locally, fine-tune it, and ship agents on top of it.