Reddit Audit Exposes Discrepancies in Ternary Bonsai 2 Benchmark Claims

A community analysis revealed that while Bonsai advertised a headline claim of 98.2% intelligence retention for Ternary Bonsai 2 27B, their own whitepaper shows it retained only about 75% performance on key coding benchmarks. Specifically, the model scored 52.8 on Terminal-Bench 2.1 and 60.8 on SWE-bench Verified, down from the full-precision baseline scores of 69.7 and 80.6. This audit underscores growing skepticism around marketing metrics for quantized language models, emphasizing the need for independent evaluation. As low-bit quantization techniques gain popularity, misleading retention claims can misinform developers about actual coding and agentic capabilities. The whitepaper documents reveal that Ternary Bonsai 2 27B retained roughly three-quarters of full-precision performance on complex software engineering tasks rather than the advertised near-full retention. The disparity highlights how headline marketing prioritized high-scoring benchmarks while burying lower agentic coding results inside technical papers.