Benchmarking Ultra-Low-Bit Ternary Model Ternary-Bonsai-2-27B Against 2-Bit Quantizations
A LocalLLaMA community member benchmarked the 27-billion parameter ternary model Ternary-Bonsai-2-27B-PQ2_0 against similarly sized ultra-low-bit models like Qwen3.8-27B IQ2_XXS. Using custom test suites including multi-hop decoy needles, phrase reconstruction, and long prose recall, the benchmark evaluated how much retrieval and reasoning capability survives extreme quantization. Extreme weight compression techniques allow large open-weights models to run locally on modest consumer VRAM budgets. Empirical benchmarks help developers determine whether novel ternary architectures perform better than standard post-training 2-bit quantizations under severe memory constraints. The evaluation revealed that expanding context windows severely degrades retrieval performance across models, indicating that large context capacities are ineffective if attention mechanisms fail over longer spans. Standard needle-in-a-haystack tests were replaced with a multi-hop passkey test with decoys to prevent modern LLMs from trivially achieving perfect scores.
## BACKGROUND
Ternary quantization restricts model weights to three discrete values (typically -1, 0, and +1), often described as 1.58-bit models to dramatically lower memory bandwidth and computation requirements. In contrast, GGUF importance-matrix quantizations like IQ2_XXS apply post-training compression down to roughly two bits per weight using matrix reconstruction techniques to preserve accuracy.