Retrained DFlash 2 Drafter Accelerates Ternary Bonsai 2 27B Up to 3.2x

Developer naklitechie fine-tuned the DFlash 2 speculative decoding draft model using 1.5 million tokens generated by PrismML's Ternary Bonsai 2 27B LLM. This re-alignment delivers inference speedups of up to 2.2x on standard benchmarks and up to 3.15x on code editing when combined with n-gram lookup on an NVIDIA L4 GPU. Standard draft models perform poorly with ternary-quantized LLMs due to mismatched output distributions, limiting the efficiency of speculative decoding. This work demonstrates that fine-tuning draft models on low-bit outputs restores high acceptance rates, enabling practical high-speed execution across NVIDIA GPUs, Apple Silicon, and WebGPU runtimes. The original DFlash 2 was trained on standard BF16 outputs, causing high rejection rates when paired with the ternary model. The optimization suite includes a custom 8-row 2-bit Metal GEMM kernel for macOS verification and a WGSL port for Chrome browsers, maintaining accuracy within 1–2 test problems.