llama.cpp Release b12094 Introduces Hexagon Backend Top-K Sampling Optimizations

Llama.cpp release b12094 introduces performance optimizations for the Qualcomm Hexagon backend sampler, significantly speeding up TOP_K sampling. The update bypasses bitonic sorting for irrelevant tiles and replaces tile sorting with a top_k_scan function to process blocks more efficiently. This optimization improves text generation efficiency on edge devices powered by Qualcomm Snapdragon processors, enhancing local AI execution performance. It benefits developers and practitioners deploying large language models on mobile and embedded hardware. The optimization is applied for values of k between 2 and 64 on rows wider than 1024, for both single-row and multi-row paths, while eliminating scalar reads from vector tightly coupled memory (vtcm). Additionally, it treats NaN values as negative infinity (-INF) and maintains the first-seen index for ties.