Taobao Open-Sources MNN-Pocket-0.3B Model for High-Performance Edge AI
Taobao's MNN team has open-sourced the MNN-Pocket-0.3B model, a 300-million-parameter language model optimized for mobile devices. It achieves a decoding speed of over 350 tokens/s on an iPhone 18 Pro and maintains peak memory usage below 300MB. This release sets a new benchmark for on-device AI efficiency, allowing complex mobile applications to run advanced language tasks locally without straining system memory or draining device batteries. It provides developers with a powerful, lightweight open-source foundation tailored specifically for resource-constrained environments. The model uses an INT4 MNN deployment file size of just 173MB and features a streamlined 64K vocabulary to reduce output projection overhead. In testing on an iPad Pro M5, it achieved a prefill throughput of 8,721 tokens/s and a decode throughput of 466 tokens/s.
## BACKGROUND
MNN (Mobile Neural Network) is an efficient, lightweight deep learning inference and training engine developed by Alibaba and battle-tested across dozens of their core apps like Taobao and Tmall. Edge AI refers to running machine learning models locally on end-user hardware—such as smartphones and tablets—rather than relying on remote cloud servers, offering lower latency and better privacy.