A 27-billion-parameter multimodal model that needs 53.8GB of memory at full precision now fits in 5.93GB, and PrismML says you give up less than two points of benchmark performance to get there. The Pasadena startup released Ternary Bonsai 2 27B on September 17 under an Apache 2.0 license, storing every weight in Alibaba's Qwen3.8 27B as one of three values (minus one, zero or plus one) and claiming 98.2% retention of the parent model's average across a 20-benchmark suite. The result is a 27B-class reasoning model with a 262K-token context window that runs on a 16GB laptop, a single 24GB consumer GPU, or, via a WebGPU demo, inside a browser tab.
The headline number is the compression ratio. Each ternary value carries roughly 1.585 bits of information, and PrismML adds one FP16 scale factor per group of 128 weights, which works out to about 1.71 bits per weight. Counting the 26.2 million parameters (0.0976% of the language model) that stay in higher precision brings the whole model to 1.72 bits per weight; the densest shipped GGUF packing, PTQ1_0, lands at 1.76 bits and 5.93GB, or about 9.1x smaller than FP16. The trick that keeps the model coherent at that bit depth is a blockwise Hadamard rotation with a block size of 1,024, applied to each weight matrix before ternary assignment so that no single weight carries too much information. The rotation is folded into the stored weights offline, but the runtime must apply the matching transform to activations, which is why stock llama.cpp either rejects the files or, for one packing, loads them and produces garbage. Users need PrismML's own llama.cpp fork, its MLX fork for Apple Silicon, or the Bonsai-demo repository that pins tested binaries.
“Bonsai 27B proved that powerful models do not have to be confined to cloud infrastructure,” said Babak Hassibi, PrismML's founder and CEO, in the launch announcement. “With Bonsai 2 27B, we are closing the quality gap while keeping the same deployment advantages: a dramatically smaller footprint, strong local performance, and the ability to support more demanding workflows such as agentic coding, multimodal understanding, and long-horizon task execution.”
The benchmark story is strongest exactly where conventional low-bit quantization tends to fall apart. PrismML evaluated everything in thinking mode with EvalScope and vLLM on H100s. On its 20-benchmark suite the ternary model scores 83.9 against 85.4 for full-precision Qwen3.8 27B. Math retains 99.5% (96.57 versus 97.06 across AIME 2025, AIME 2026, GSM8K and MATH-500), coding retains 99.3% (81.58 versus 82.17), and instruction following actually edges ahead at 82.66 versus 81.25. The comparison to a conventional IQ2_XXS build of the same base model is stark: that quant averages 75.2 at 7.3GB, scores 78.6 on AIME 2026 where Bonsai 2 scores 95.83, and 70.05 on LiveCodeBench v6 where Bonsai 2 scores 90.07.
Throughput figures, measured September 16 on PrismML's custom kernels at batch size one, put the model at 142.5 tokens per second on an RTX 5090 (0.582 mWh per token), 96.7 on an RTX 4090, 32.1 on a 72-watt L4, 46.8 on an Apple M5 Max and 27.7 on an M5 Pro. Hugging Face lists 1.5 million downloads of the GGUF repository in the past month, a figure that includes the prior Bonsai 27B generation from July, which retained about 95% of its own parent.
Why It Matters
The 98.2% figure is an average, and the losses are uneven in ways that matter for how people will actually use this model. Knowledge and reasoning retains 96.9% and vision 96.3%, but the sharper drops sit outside the headline suite entirely. On Terminal-Bench 2.1 the ternary model scores 52.8 against 69.7 for full precision; on SWE-bench Verified it scores 60.8 against 80.6. That is roughly 75% retention on exactly the long-horizon agentic work PrismML's demos showcase, where small errors compound over many steps. At medium reasoning effort the gap widens (79.3 versus 82.6). All numbers are PrismML's own and have not been independently reproduced.
Early hands-on testing backs up that caveat. MindStudio ran the model through a planted-bug hunt in a multi-service Docker application, a single-file HTML animation task, an image-description task and a multilingual translation prompt. Vision description was the clear pass. The debugging run reasoned its way to the correct time-zone mismatch and then talked itself out of the answer; the HTML task failed to produce a working reset button across three rounds of correction; and the translation task fell into a repetitive reasoning loop on Tamil and never finished, which the reviewers called a regression from the previous Bonsai release. MindStudio also measured over 12GB of VRAM with default llama.cpp settings, dropping to roughly 8GB with a single parallel slot.
Still, the direction is hard to dismiss. Ion Stoica, the UC Berkeley professor who advises PrismML, put it this way: “What is compelling about Bonsai 2 27B is how little capability is lost despite such a dramatic reduction in footprint. If that gap continues to close, it can fundamentally expand where highly capable models can be deployed.” The retention gap closed from 5 points to under 2 in two months. If sub-2-bit models keep converging on full precision for reasoning and code, the memory-bandwidth bottleneck that governs local inference, single-GPU serving and datacenter batching shifts all at once. PrismML, a Caltech spinout backed by Khosla Ventures, Cerberus, Google and Samsung, is betting the industry's scarce resource becomes memory and watts rather than parameters.
What to Watch
Independent reproductions of the 20-benchmark suite are the obvious next step, along with whether PrismML publishes how it assigns ternary values (the whitepaper describes the rotation but not the quantization algorithm itself). Watch for upstream llama.cpp support for the PTQ1_0 and PQ2_0 types, since the fork requirement is the biggest adoption friction today, and for whether the multilingual and self-correction gaps seen in early testing get patched in a point release or turn out to be structural costs of 1.7 bits per weight. EOF 1055 /Users/prajwalp/Projects/the-vault-ai/editions/2026/09/19/stories/08-prismml-ternary-bonsai-2-27b-compression.md
"What is compelling about Bonsai 2 27B is how little capability is lost despite such a dramatic reduction in footprint. If that gap continues to close, it can fundamentally expand where highly capable models can be deployed."— Ion Stoica, UC Berkeley professor and PrismML advisor