Open Weights Released for Ling-3.0-flash-Fin MoE Model
The open weights for Ling-3.0-flash-Fin have been released, introducing a 124B parameter Mixture-of-Experts (MoE) architecture with only 5.1B active parameters per token. The model also features a massive native context window of 256K tokens. By serving a 124B model's knowledge capacity at the computational cost of a much smaller 5.1B parameter model, this release significantly lowers local deployment hardware requirements. It enables cost-effective long-document processing and domain-specific tasks without sacrificing performance. The architecture utilizes sparse routing to achieve only 5.1 billion activated parameters during inference out of its total 124 billion parameters. Additionally, the model's 256K native context window can be extended up to 1 million tokens for extreme long-context tasks.
## BACKGROUND
Mixture of Experts (MoE) is a model design where specialized sub-networks ('experts') are selectively triggered per token, offering high model capacity without requiring full computational power for every token. A context window determines how much input text a language model can process and remember at one time.