21llama.cpp Achieves 1.2k Tokens/s Prefill for Qwen Models on AMD Strix HaloREDDIT · /u/ilintar · reddit.com · Sep 12, 09:08 PM42d
22Dynamic VRAM Swapping Boosts MoE Prefill Speeds on Dual Consumer GPUsREDDIT · /u/Extension-Bid-639 · reddit.com · Sep 10, 02:39 AM45d
23Qwen3.8-Flash-Next Benchmark: SGLang Beats llama.cpp by 7.3x in Time-to-First-TokenREDDIT · /u/FantasticNature7590 · reddit.com · Sep 08, 07:26 PM47d
24LayerStoRm Enables Fast MoE Model Streaming Beyond GPU VRAM LimitsREDDIT · /u/CharacterBumblebee99 · reddit.com · Sep 07, 01:14 AM48d
25New Tool "cache-pressure" Validates Local LLM KV Cache Eviction under PressureREDDIT · /u/t4a8945 · reddit.com · Sep 06, 09:38 AM49d
26SGLang v0.5.19 Released with New Model Support and Performance OptimizationsGITHUB · Qiaolin-Yu · github.com · Sep 05, 02:27 AM50d
27MTP Merged into ik_llama.cpp, Doubling Qwen Inference Speed for CodingREDDIT · /u/Alternative_Will5974 · reddit.com · Sep 03, 04:30 PM52d
28Perplexity Open-Sources 'Lily' Inference Engine for Apple SiliconREDDIT · /u/Specter_Origin · reddit.com · Sep 02, 10:13 PM52d
29Why vLLM Achieves Drastically Faster Prefill Speeds Than llama.cpp on Multi-GPU SystemsREDDIT · /u/dangerous_inference · reddit.com · Sep 01, 05:05 PM54d
30Developer Pushes 27B LLM Inference to 2,000 Prefill Tokens/Sec on RTX 3090REDDIT · /u/iamMess · reddit.com · Sep 01, 11:43 AM54d
31ExLlamaV3 Update Adds MoE CPU Offloading and Self-Calibrated QuantizationREDDIT · /u/Unstable_Llama · reddit.com · Sep 01, 07:14 AM54d
32llama.cpp v0.3.0 Released with GLM-4.5-Air and DeepSeek 4 SupportGITHUB · github-actions[bot] · github.com · Aug 25, 10:22 AM61d
33Moore Threads Releases Prefill-as-a-Service Whitepaper for MTT S5000 GPUsRSS · IT HOME · ithome.com · Aug 24, 02:09 AM62d
34The Maturation and Adoption of Speculative Decoding in 2026 LLM InferenceREDDIT · /u/Ok-River5924 · reddit.com · Aug 10, 08:02 AM76d
35Developer Runs LFM-2.5 2.6B Model on Mobile CPU at 17 Tokens/SecondREDDIT · /u/trikboomie · reddit.com · Aug 05, 02:20 PM81d
36llama.cpp b10228 Released with Support for DeepSeek-V4 MTP and DSparkGITHUB · github-actions[bot] · github.com · Aug 02, 01:28 PM84d
37llama.cpp Adds MTP and DSpark Support for DeepSeek V4 FlashREDDIT · /u/rmhubbert · reddit.com · Aug 02, 12:58 PM84d
38OpenAI Uses GPT-5.6 Sol to Optimize Inference and Cut Serving Costs by 20%RSS · IT HOME · ithome.com · Jul 30, 06:06 AM87d
39llama.cpp Releases Build b10150 with Backend Offloading AdjustmentsGITHUB · github-actions[bot] · github.com · Jul 27, 12:45 PM90d