6 min read
Bonsai 27B: How PrismML Fit a 27B Model on Your Phone
PrismML's Bonsai 27B uses 1-bit quantization to compress a 27B model to 3.9GB - small enough to run on an iPhone. Here's how it works and what HN thinks.
Read more
QUANTIZATION
3 articles
LATEST
How much VRAM do you need to run a local LLM in 2026?
How much VRAM a local LLM needs: Q4 weights are about 30% of BF16, plus a KV cache that grows per token. Real numbers from 8 GB to 256 GB.
•13 min read
Read Article
7 min read
GLM-5.2 Local Deployment: Running Z.ai's 744B Model on Consumer Hardware
Run GLM-5.2 locally with Unsloth dynamic quantization: hardware math for a 256GB Mac or 24GB GPU, quantization tradeoffs and lessons from the HN thread.
Read more
Showing 2 of 2 articles
Keep exploring Quantization
- - Quantization Topic Hub - tools and guides for Quantization from the Developers Digest directory
- - Compare Tools - dive deeper across the Developers Digest knowledge base
- - Developers Digest on YouTube - video tutorials covering Quantization and more
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever
Explore 839 topics
Browse All Topics