Bonsai 1.7B Solves Physics Problems at 9.1 Tokens/Second on 12W Intel N97 CPU
A local AI enthusiast demonstrated running the Bonsai 1.7B reasoning model locally on an ultra-low-power 12-watt Intel N97 processor. The setup successfully solved simple physics problems while achieving an inference speed of approximately 9.1 tokens per second. This demonstration shows that highly quantized, small reasoning models can perform useful local AI tasks on basic budget hardware without requiring dedicated GPUs. It highlights the growing feasibility of running functional AI agents and private assistants on low-cost Mini PCs and edge devices. The Bonsai model series by Prism ML relies on extreme quantization formats (such as 1-bit or ternary weights) to dramatically lower memory bandwidth requirements. Running on an Intel N97—a budget 4-core chip with a 12W thermal design power—achieving over 9 tokens per second makes CPU-only LLM inference practical for interactive use.
## BACKGROUND
Small Language Models (SLMs) under 3 billion parameters are increasingly optimized for low-power edge computing. Memory bandwidth is typically the primary bottleneck when running LLMs on standard CPUs, which extreme quantization techniques help bypass. The Intel N97 is an entry-level processor commonly found in budget Mini PCs designed for basic everyday computing tasks.