Topic
QUANTIZATION
All blog posts, tools, and guides about Quantization from Developers Digest.
4 resources - 3 posts, 1 tool
Blog Posts
View in blog →
How much VRAM do you need to run a local LLM in 2026?
How much VRAM a local LLM needs: Q4 weights are about 30% of BF16, plus a KV cache that grows per token. Real numbers from 8 GB to 256 GB.
13 min read
Bonsai 27B: How PrismML Fit a 27B Model on Your Phone
PrismML's Bonsai 27B uses 1-bit quantization to compress a 27B model to 3.9GB - small enough to run on an iPhone. Here's how it works and what HN thinks.
6 min read
GLM-5.2 Local Deployment: Running Z.ai's 744B Model on Consumer Hardware
Run GLM-5.2 locally with Unsloth dynamic quantization: hardware math for a 256GB Mac or 24GB GPU, quantization tradeoffs and lessons from the HN thread.
7 min read
Keep exploring
More on Quantization
- - llama.cpp - recommended Quantization tool from the Developers Digest directory
- - Compare Tools - dive deeper across the Developers Digest knowledge base
- - All Quantization articles in the blog archive
- - Developers Digest on YouTube - video tutorials covering Quantization and more
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever
Explore 838 topics
Browse All Topics