Qwen3.8-Flash-Next: A Preview of the Qwen4 Multimodal MoE Architecture
Qwen has released Qwen3.8-Flash-Next, a new 125-billion parameter multimodal Mixture of Experts (MoE) open-weights model that serves as an early preview of the upcoming Qwen4 architecture. The model only utilizes 6 billion active parameters per token, providing a significant boost in inference performance. This release gives developers and researchers early access to the next-generation Qwen4 architecture in an open-weights format. By leveraging a sparse MoE design, it delivers high-capacity multimodal capabilities while remaining computationally efficient enough to run on accessible hardware. Early testers have successfully run quantized GGUF versions of the model, such as the 72.5GB UD-IQ1_S and 78.9GB UD-Q2_K_XL variants, on hardware like NVIDIA DGX Spark. These quantized versions, provided by Unsloth, allow the massive 125B model to be run locally with reduced memory requirements.
## BACKGROUND
Mixture of Experts (MoE) is an AI architecture that routes inputs to specialized sub-networks called "experts," activating only a fraction of the total parameters to save compute. GGUF is a binary file format optimized for quick loading and saving of models, making it highly efficient for local inference. Unsloth provides optimized tools and platforms to run and train these open models on local hardware.