DeepGrove Releases Maple-Preview, a 20B MoE Model Reaching 127 Tokens/s on iPhone
US independent AI research lab DeepGrove has released Maple-Preview, a 20.2-billion-parameter Mixture of Experts (MoE) model that runs at 127 tokens/s locally on an iPhone. This represents a 13-fold speed increase compared to the Bonsai 27B model, which runs at 9.6 tokens/s on the same platform. This achievement marks a significant milestone for on-device AI, proving that large-scale models can run efficiently on mobile hardware. By using ternary weights instead of traditional quantization, it overcomes memory bandwidth bottlenecks without severely compromising model performance. The model features 24 layers, 256 experts (with 8 active), and utilizes a 3:1 SWA-512:GA hybrid attention mechanism. It maintains a low memory footprint of just 7.69 GB even when processing a long context of 131,000 tokens.
## BACKGROUND
Ternary weight models (often referred to as 1.58-bit LLMs) restrict weight values to {-1, 0, 1}, replacing energy-intensive multiplications with simple additions to drastically reduce memory usage. Additionally, hybrid attention mechanisms combine local Sliding Window Attention (SWA) with Global Attention (GA) to handle long-context sequences efficiently while minimizing KV-cache growth.