llama.cpp Adds Support for Tencent's Hy4 Preview Model Architecture
Developer Little0o0 submitted Pull Request #28127 to llama.cpp, adding support for Tencent's Hy4 (hy_v4) preview model architecture. This integration enables local inference and quantization of Tencent's latest open-weight model. Adding support for Hy4 brings Tencent's latest open-weights model to the local AI ecosystem, allowing developers and researchers to run advanced models without cloud API dependencies. This expands the range of high-performance Mixture-of-Experts architectures supported by llama.cpp. Tencent Hy4 Preview is a Mixture-of-Experts model featuring 770 billion total parameters, 49 billion active parameters per token, and a 1-million-token context window. It also incorporates native multi-token prediction to enhance token generation efficiency.
## BACKGROUND
llama.cpp is a widely used open-source C/C++ framework that enables efficient inference of large language models on local hardware with CPU and GPU acceleration. Mixture-of-Experts (MoE) is a neural network design that routes inputs to specific expert sub-networks, allowing high model capacity while maintaining manageable computation per token.