Benchmark Compares Qwen3.8 27B IQ3_XXS and Bonsai Ternary Quantization Performance
A user benchmarked Qwen3.8 27B quantized with IQ3_XXS (10.18 GiB) against Bonsai Ternary PQ2 (6.42 GiB) on consumer hardware with 16 GB VRAM across UI generation tasks. Results showed that while Bonsai saved memory, it was over 3 times slower in task completion time and generated 3.10 times more output tokens. This practical test demonstrates that extreme compression methods like ternary quantization can degrade output conciseness and execution speed, offsetting their VRAM savings. It highlights that smaller model file sizes do not automatically translate to better real-world throughput or efficiency during local inference. Qwen IQ3_XXS finished the test suite in 8:00 minutes using 27,197 output tokens, compared to Bonsai PQ2's 24:09 minutes and 84,176 tokens. Furthermore, IQ3 benefited from Multi-Token Prediction (MTP) in llama.cpp with a 39.76% acceptance rate, an optimization currently missing in the Bonsai model.
## BACKGROUND
LLM quantization reduces the precision of model weights to shrink memory footprint, with llama.cpp's IQ3_XXS using advanced 3-bit compression. Ternary quantization constrains weights to just three values (typically -1, 0, 1, or ~1.58 bits), achieving high compression at the cost of precision. Multi-Token Prediction (MTP) is an inference acceleration technique where the model predicts multiple future tokens simultaneously rather than one at a time.