Running Qwen3.8-Flash-Next LLM locally on a smartphone CPU
A Reddit user demonstrated running the Qwen3.8-Flash-Next language model locally on a Xiaomi 14T Pro smartphone CPU. The setup used an aggressively quantized IQ3_XXS model variant running inside the BigMoeOnEdge mobile app. It demonstrates how combined advancements in low-bit quantization and specialized edge runtime tools allow large AI models to run on mobile hardware without cloud connectivity. This paves the way for private, offline AI assistants on everyday consumer devices. The setup leveraged the extreme `IQ3_XXS` GGUF quantization format to substantially reduce the model's RAM footprint, albeit with some loss in quality. The execution relies on BigMoeOnEdge, an open-source engine designed to run large Mixture-of-Experts (MoE) models on CPU-only hardware even when memory is limited.
## BACKGROUND
Quantization is a technique in machine learning that reduces the precision of model weights, drastically saving memory so models can run on smaller devices. BigMoeOnEdge is an open-source mobile and PC application specifically built to execute large Mixture-of-Experts (MoE) models that exceed standard device memory constraints.