Running an 80B MoE Model in Chrome via llama.cpp's WebGPU
A developer successfully ran an 80B Mixture-of-Experts (Qwen3-Next) model inside a Chrome browser tab using llama.cpp's WebGPU backend by dynamically paging experts from local disk storage. This breakthrough demonstrates that massive language models far exceeding available VRAM can run locally inside a standard web browser without dedicated native apps, pushing the boundaries of edge AI. The system stores the GGUF model in the browser's OPFS storage, holding dense layers and a fixed number of expert slots in GPU memory while fetching missing experts on demand into an LRU cache slot.
## BACKGROUND
Mixture of Experts (MoE) is a neural network architecture where only a subset of network parameters, or experts, are activated for any given token, making it possible to run massive models efficiently. GGUF is a binary file format designed specifically for storing quantized neural network weights along with metadata to enable efficient local inference.