QUANTIZATION

PrismML's Bonsai 27B uses 1-bit quantization to compress a 27B model to 3.9GB - small enough to run on an iPhone. Here's how it works and what HN thinks.

Unsloth's dynamic quantization makes GLM-5.2 runnable on a 256GB Mac or a 24GB GPU with CPU offloading. Here is the hardware math, the quantization tradeoffs, and what the HN community learned from actually running it.

C++ inference engine for LLMs. GGUF format, quantization, CPU and Metal/CUDA support. The foundation most local tools build on.