Qualcomm Runs 30B Parameter MoE AI Model On-Device via Snapdragon 8 Gen 6
Qualcomm partnered with StepFun and other tech companies to optimize and run the 30-billion-parameter StepEdge-Omni 30B-MoE model locally on the Snapdragon 8 Gen 6 mobile platform. The system achieved a prefill throughput exceeding 330 tokens per second and a decode throughput over 28 tokens per second. Running a large 30B model directly on mobile hardware represents a major step forward for edge AI, enabling intelligent agent features with low latency and enhanced privacy. By cutting memory requirements by over 50%, complex generative AI capabilities can run locally on smartphones without relying on cloud infrastructure. The performance gains were achieved through co-optimization across the inference engine, heterogeneous task scheduling, and storage management. The on-device model efficiently handles local tasks like email analysis, itinerary planning, calendar sync, and recommendation generation.
## BACKGROUND
Large language model inference comprises two phases: prefill, which ingests and processes the full prompt context, and decode, which generates the output text sequentially token by token. Mixture of Experts (MoE) is a specialized model architecture that routes tasks to smaller sub-networks (experts) rather than activating all parameters, reducing computation and memory bandwidth demands.