ggerganov/llama.cpp released b12023 with server enhancements
llama.cpp release b12023 introduces server-side improvements that cap shared decision prefixes at the first output position. This update specifically resolves issue #30482 by ensuring decision output embeddings remain available in the final batch when questions share option tokens. This routine update optimizes how the server component handles complex prompt structures and batch processing for shared option tokens. Such improvements enhance the overall stability and performance of local large language model deployments running on llama.cpp. The fix limits the shared prefix to the required output position of both parent and child tasks via pull request #30710. Developers and users can download the latest binaries across various platforms including macOS, Linux, Android, and Windows.
## BACKGROUND
llama.cpp is an open-source software library developed by Georgi Gerganov that performs local inference on large language models like Meta's Llama using the GGML tensor library. It serves as the core backend for many popular local AI tools such as Ollama and LM Studio.