K2 Horizon Benchmark Plots Mislead on Real RAM Needs Due to Inefficient KV Cache
A technical breakdown of the new K2 Horizon model lineup shows that Artificial Analysis benchmark plots mislead users by evaluating models solely by parameter count. In practice, K2 Horizon's inefficient KV cache design drastically inflates actual memory usage during long-context inference. Parameter count alone is no longer an accurate proxy for hardware requirements when context memory dominates overall RAM consumption. Users deploying local LLMs on consumer hardware (such as 16GB/24GB VRAM GPUs or Strix Halo APUs) must account for context memory footprint to select viable models. At a 128k context length with Q4_K_M quantization, K2 Horizon 7B consumes 5 GiB of RAM for context alongside 5.2 GiB for weights, whereas Qwen3.6-35B-A3B requires only 0.7 GiB for context. This context memory bloat shifts K2 Horizon models significantly to the right on RAM-demand charts, making them far less practical on VRAM-limited systems.
## BACKGROUND
During LLM inference, the key-value (KV) cache stores past token representations to speed up text generation, but its memory footprint scales linearly with context length. Quantization methods like Q4_K_M reduce weight memory down to 4-bit precision to fit within GPU VRAM, turning KV cache efficiency into the primary constraint for long-context tasks.