bartowski Releases GGUF Quantization for Xing4.0-29B-A4B MoE Model
A GGUF quantization of the Xing4.0-29B-A4B model has been released by community developer bartowski, making the 29B parameter Mixture of Experts model usable for local inference. The base model, developed by China Telecom, features 4 billion active parameters per token and was originally trained on Huawei's Ascend NPU platform. This release allows local LLM enthusiasts and developers to run a cutting-edge open-source MoE model on consumer or edge hardware using GGUF-compatible runtimes. It bridges enterprise-grade hardware training ecosystems with accessible local deployment workflows. The Xing4.0-29B-A4B model natively supports a 256K context length (extensible to 512K) and uses an agent-oriented architecture comprising mHC, MLA, and MTP. GGUF quantization formats enable users to run compressed versions of the weights efficiently with tools like llama.cpp.
## BACKGROUND
GGUF is a file format designed by the llama.cpp team to store language models for efficient local inference, particularly on consumer CPUs and GPUs through quantization. Mixture of Experts (MoE) is a neural network architecture that activates only a subset of its sub-networks (experts) for each token, achieving high capacity with lower active compute costs. Huawei's Ascend NPU and MindSpore framework form an alternative hardware and software ecosystem designed for large-scale AI training outside of traditional GPU architectures.