LFM 2.5 230M Runs at 1440 Tokens/Second In-Browser via Custom WebGPU Backend
A developer demonstrated a custom WebGPU backend running the LFM 2.5 230M model in-browser at speeds up to 1440 tokens per second on an RTX 3090. The project achieves high performance by using hardware-specific kernel optimizations tailored for Nvidia and Apple Silicon architectures. This showcase highlights the feasibility of running highly efficient, low-latency AI models directly on client devices without relying on cloud servers. It paves the way for faster, privacy-focused edge AI applications running natively in web browsers or desktop frameworks like Electron and Tauri. The backend uses aggressively fused multi-pass kernels for Nvidia GPUs, while employing a fused mega-kernel for Apple Silicon to minimize Tile-Based Deferred Rendering (TBDR) overhead. The developer plans to integrate these optimizations into the open-source Sipp library, which is a WebGPU runtime built with Rust and C++.
## BACKGROUND
WebGPU is a modern web standard that allows web applications to access the device's GPU for accelerated graphics and computation. Liquid Foundation Models (LFMs) are a class of lightweight, compute-efficient AI architectures designed for fast inference and on-device deployment. Apple Silicon GPUs utilize Tile-Based Deferred Rendering (TBDR), a technique that processes rendering in small grid regions (tiles) to optimize power efficiency, which requires specific kernel design to avoid performance bottlenecks.