PrismML Releases Bonsai 2 27B with Ternary Quantization and 5.9GB Memory Footprint
PrismML has released Bonsai 2 27B, a ternary-quantized language model based on Qwen3.8 27B that slashes memory usage down to 5.9GB—less than 1/9th of the original—while retaining 98.2% of the base model's performance. The model is openly released under the Apache 2.0 license and supports a 262K token context window. This release demonstrates that advanced ternary quantization allows powerful 27B-parameter models to run locally on consumer hardware for tasks like agentic workflows and document parsing without relying on cloud APIs. Furthermore, it highlights impressive energy efficiency, consuming less power per token than standard 8B models. Bonsai 2 27B uses FP16 grouped scaling to achieve an effective size of 1.76 bits per weight, achieving throughputs of 143 tokens/sec on an NVIDIA RTX 5090 and 46.8 tokens/sec on Apple Silicon M5 Max via MLX. On an RTX 4090, its energy efficiency reaches 0.714 mWh per token, which is 40% lower than a full-precision 8B model.
## BACKGROUND
Model quantization compresses large language models by converting high-precision floating-point weights into lower-bit formats to drastically reduce memory and compute demands. Ternary quantization restricts model weights to just three discrete values (typically -1, 0, and +1), significantly shrinking memory requirements while striving to preserve accuracy.