llama.cpp Release b10284 Fixes Memory Allocation for Multi-Token Prediction Layers
The llama.cpp project has released version b10284, which introduces a bug fix addressing memory allocation issues in Multi-Token Prediction (MTP) layers. Multi-Token Prediction is increasingly used in modern large language models like DeepSeek-V3 to speed up inference, making stable memory management for these layers crucial for local deployment. This release specifically addresses pull request #26605 to resolve memory allocation bugs. Additionally, the release notes indicate that KleidiAI support for macOS Apple Silicon (arm64) remains disabled.
## BACKGROUND
llama.cpp is a popular open-source software library that enables efficient LLM inference on various hardware, particularly consumer-grade CPUs and GPUs. Multi-Token Prediction (MTP) is a technique where a model predicts multiple tokens simultaneously instead of one by one, which helps improve training efficiency and generation speed.