inclusionAI Releases Ling-3.0-flash Weights Under MIT License on Hugging Face
inclusionAI has released the weights for Ling-3.0-flash, a 127.5-billion parameter Mixture of Experts (MoE) model, on Hugging Face under the permissive MIT license. The model is available in both BF16 (~255GB) and official FP8 (~128GB) formats. This release provides the open-source community with a highly fine-grained MoE model under a permissive license, lowering the barrier for local deployment. The official FP8 version allows users with multi-GPU setups or unified-memory hardware to run a large model with reduced memory requirements. Ling-3.0-flash uses the BailingMoeV3 architecture with 512 total experts, of which only 8 are active per token, resulting in 5.1 billion active parameters. It requires custom code execution and currently raises questions about compatibility with inference engines like llama.cpp versus vLLM and SGLang.
## BACKGROUND
Mixture of Experts (MoE) is an architecture that activates only a subset of a model's parameters (experts) for each input, improving computational efficiency. FP8 quantization is a technique that reduces the precision of model weights to 8-bit floating-point numbers, cutting the memory footprint in half compared to 16-bit precision (BF16) and accelerating inference. SGLang and vLLM are high-performance serving frameworks designed to optimize LLM inference.