Ant Group Open-Sources Ling-3.0-tiny MoE Model for Local Deployment

Ant Group has open-sourced Ling-3.0-tiny, a lightweight 7.9B parameter Mixture of Experts (MoE) model designed for low-cost local inference. The model utilizes a hybrid architecture combining Kimi Delta Attention (KDA) and Multi-Head Latent Attention (MLA) to optimize performance on consumer hardware. This release advances the feasibility of running capable LLMs locally on consumer devices like Apple Silicon Macs, achieving high inference speeds of up to 90 tokens/s on an M4 Pro chip. It demonstrates the practical application of hybrid attention mechanisms to balance computational cost and context processing. Ling-3.0-tiny activates only 1.3B parameters per token and features a sparse MoE feed-forward network with 128 routed experts. It supports BF16, FP8, and INT4 quantization, requiring only about 8.34 GiB of peak memory for an 8K context length.