1-Bit 27B LLM Runs In-Browser via WebGPU at 30 Tokens/Second
A developer built a custom WebGPU/WGSL inference engine called mentria.ai that runs Prism ML's 27B 1-bit quantized model (Bonsai-27B) entirely within a web browser without installation or server dependencies. The engine achieves decode speeds of 25–30 tokens/second on an RTX 3060 Laptop GPU by squeezing the 27-billion parameter model into just 3.8 GB of VRAM. This achievement proves that 27B-class large language models can run privately and locally on consumer-grade laptop hardware straight from a standard web browser. It removes server-side hosting costs and latency concerns while keeping user data entirely on-device, advancing the potential for browser-based edge AI. The engine stores model weights at ~1.14 bits per parameter and replaces standard matrix multiplications with precomputed lookup tables in GPU on-chip memory. Optimizations such as resolving shared memory bank conflicts on Ampere GPUs and retiling prompt layouts doubled the decode speed from 15 to over 30 tokens/second.
## BACKGROUND
WebGPU is a modern web API that enables high-performance parallel GPU computing within web browsers using the WGSL shader language. Quantization reduces the numerical precision of LLM weights, such as converting 16-bit floating-point numbers into 1-bit representations, to drastically cut VRAM requirements. Extremely low-bit architectures allow models with tens of billions of parameters to fit within the modest VRAM capacities of everyday laptops and smartphones.