LOCAL AI
23 items
18 posts, 5 tools
Meta released Muse Spark Open weights on October 15, 2026: 816B total, 42B active, 1M context. Run recipes, hardware floor, license catch, and where it trails.
EmbeddingGemma 2 is a 740M open model that embeds code, images, video and audio in one space. Specs, Matryoshka storage math, and where it breaks.
Hermes Agent is Nous Research's open-source, self-improving AI agent. Run it locally with Ollama, see how its skills system works, and compare it with OpenClaw.
GitHub Copilot for JetBrains adds memory across chat sessions, Ollama as a BYOK provider, and enterprise settings for MCP access. What each feature does.
Cactus open-sourced Needle 2, a 45M-parameter agentic LLM in a single 14MB binary that runs a full tool-calling session in 28MB of RAM. 500 tok/s on a Raspberry Pi 5, ESP32-S3 class parts, Apache 2.0. Here is what the benchmarks actually show.
Muse Glimmer 30B is Meta's Apache 2.0 open-weight multimodal agent model that runs in 24GB. Benchmarks, hardware reality and what the numbers actually say.
Gemma 4 26B on a 2013 Xeon with no GPU: how a sub-$300 box reached about 5 tokens/sec, the silent MoE bug behind the gibberish, and the upstream fix.
A solo developer built a 1,300-line C inference engine that runs the 744B GLM 5.2 model on consumer hardware by streaming routed experts from disk.
An 82M parameter text-to-speech model that runs on CPU and produces high-quality speech across multiple languages - no cloud APIs or GPU required.
Program-as-Weights compiles prompts into compact local adapters that act as fuzzy functions. What the paper claims and where developers could use it first.
Run GLM-5.2 locally with Unsloth dynamic quantization: hardware math for a 256GB Mac or 24GB GPU, quantization tradeoffs and lessons from the HN thread.
DiffusionGemma is Google's 26B MoE open model that generates 256-token blocks in parallel. What diffusion generation means for latency and local inference.
Cline is a free, open-source VS Code extension for agentic AI coding. Bring your own model, a cloud API or a local one via Ollama, with no vendor lock-in.
Client-side tool calling keeps sensitive documents in the browser while the model requests narrow actions. How it works and what still leaves the machine.
The easiest way to run LLMs locally. One command to pull and run any model. OpenAI-compatible API. 52M+ monthly downloads. Supports GGUF, Safetensors, and custom Modelfiles.
Desktop app for discovering, downloading, and running local LLMs. Clean chat UI, OpenAI-compatible API server, and automatic GPU detection. MLX engine optimized for Apple Silicon.
Open-source OpenAI API replacement. Runs LLMs, vision, voice, image, and video models on any hardware, no GPU required. 60+ backends. Distributed mode for scaling.
DeepSeek R1 and V3 developer guide: use the MIT-licensed models through the API, run them locally with Ollama and decide when they beat closed-source models.

Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.