Bonsai 27B: How PrismML Fit a 27 Billion Parameter Model on Your Phone

A 27 billion parameter model running on a phone used to be a punchline. PrismML just made it reality.

Bonsai 27B, released on July 14, 2026, compresses a Qwen3.6 27B base into either a 5.9GB ternary variant or a 3.9GB 1-bit binary variant. The 1-bit version fits comfortably in an iPhone 17 Pro's memory constraints while retaining 90% of the full-precision model's performance across 15 benchmarks.

The Numbers That Matter#

The compression comes from extreme quantization. Instead of the typical 16 or 32 bits per weight, Bonsai goes to the absolute floor:

Both variants apply low-bit representation end to end - embeddings, attention, MLPs, and the language model head all use the compressed format. There are no higher-precision escape hatches.

Inference speeds tell the rest of the story:

• NVIDIA RTX 5090: 163 tok/s (1-bit), 134 tok/s (ternary)
• Apple M5 Max: 87 tok/s (1-bit), 58 tok/s (ternary)

That 87 tokens per second on an M5 Max puts Bonsai solidly in the usable range for real-time applications. For comparison, 87 tok/s is faster than many cloud API responses once you factor in network latency.

Why "Intelligence Density" Matters#

PrismML measures intelligence density as benchmark performance per gigabyte. The 1-bit Bonsai 27B hits 0.53 per GB - roughly 10x the full-precision baseline and 2.7x conventional low-bit alternatives at the same parameter count.

This metric matters because it directly translates to what you can run on consumer hardware. A model that needs 50GB of VRAM is a cloud-only proposition. A model that fits in 4GB can run on the device already in your pocket.

The Technical Approach#

The Bonsai models build on Qwen3.6 27B, which already supports multimodal inputs (images and text), agentic tool-calling, and a 262K token context window. PrismML's contribution is the quantization scheme that preserves these capabilities while collapsing model size.

The key insight is FP16 group-wise scaling. Each group of weights gets its own scaling factor stored at full precision, while the actual weight values collapse to ternary or binary. This hybrid approach sacrifices some compression for accuracy retention - pure 1-bit with no scaling would break down much faster.

What Hacker News Is Saying#

The HN thread surfaced several practical concerns and discoveries.

On the quantization math: One commenter clarified that "1-bit models are actually 1.58 bit with three values +1, 0 and -1" - technically correct for the ternary variant. The true 1-bit binary version does use just two values, hence the 1.125 effective bits.

Compared to alternatives: A user noted that "if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases." Post-training quantization versus native low-bit training appears to be a meaningful distinction.

Practical compatibility: Several users reported issues getting the models running in LM Studio - "I've tried a couple in LM Studio - the GGUF one and the MLX one - but neither worked there. Might be that LM Studio needs to upgrade their llama.cpp or MLX engines first." The models are available on Hugging Face but tooling support is still catching up.

Hardware fit: For those optimizing VRAM usage, one commenter asked what's possible with a 16GB GPU at 1.125 bits per weight. The math: 16GB / 1.125 bits = roughly 114B parameters, though real-world overhead reduces that.

Apple interest: According to CNBC reporting, Apple is "in talks" with PrismML about the compression technology - potentially for on-device AI features in future iPhones.

Edge cases: At least one Android user reported getting "!!!!!!!!!!!!!!" for answers - a reminder that bleeding-edge releases rarely work perfectly out of the box.

The Bigger Picture#

Bonsai 27B represents a milestone in the push toward on-device AI. A year ago, running models this size required cloud infrastructure or expensive workstations. Now it fits on a phone.

The implications extend beyond mobile. Edge deployment for privacy-sensitive applications, offline operation in areas with poor connectivity, reduced cloud costs for inference at scale - all become more feasible when model sizes drop by an order of magnitude.

Whether the 90% performance retention holds up for your specific use case is another question. Math and coding benchmarks reportedly stay near parity, but agentic tool-calling shows more degradation. Your mileage will vary.

The models are available under Apache 2.0 with native support for Apple devices (MLX) and NVIDIA GPUs (CUDA). Check the Hugging Face repository for the latest versions.

Sources#

• PrismML Bonsai 27B Announcement
• Hacker News Discussion
• Bonsai 27B on Hugging Face
• 9to5Mac Coverage