01Custom CUDA Megakernel Accelerates 27B Model to 140 tok/s on Single RTX 3090REDDIT · /u/Adorable_Weakness_39 · reddit.com · Oct 10, 01:01 PM15d
02High-Speed Local LLM Inference on iPadOS via External AMD GPU PortREDDIT · /u/TheOriginalG2 · reddit.com · Oct 09, 01:20 AM16d
03Developer Builds $2,800 256GB Rig Using Radeon Pro V620 GPUs and Custom vLLM ForkREDDIT · /u/_TheWolfOfWalmart_ · reddit.com · Oct 08, 05:12 PM17d
04Retrained DFlash 2 Drafter Accelerates Ternary Bonsai 2 27B Up to 3.2xREDDIT · /u/naklitechie · reddit.com · Oct 07, 08:13 AM18d
05Strata Engine Runs 125B Qwen3.8 MoE Model on 12GB Consumer GPUsRSS · IT HOME · ithome.com · Oct 06, 07:45 AM19d
06Fast Local LLM Decode Speeds Shift Performance Bottleneck to CPU Tool CallsREDDIT · /u/BringTea_666 · reddit.com · Oct 05, 06:21 PM20d
07Qwen3.8-Flash-Next (125B) Runs at 59 tok/s on AMD Strix Halo Mini PCREDDIT · /u/Yaniss916 · reddit.com · Oct 05, 03:25 PM20d
08Running 176B Qwen MoE Model on a 16GB VRAM Laptop with TensorSharpREDDIT · /u/fuzhongkai · reddit.com · Oct 03, 08:06 PM21d
09The Emerging Trend of Hyper-Specialized Overfit LLM Inference RuntimesREDDIT · /u/carteakey · reddit.com · Oct 03, 06:24 PM22d
10Strata Engine Boosts Local LLM Inference Speeds to 1,500 t/s Prompt ProcessingREDDIT · /u/MLDataScientist · reddit.com · Sep 30, 04:01 AM25d
11LlamAmpere v0.4 Achieves 95+ TPS for 27B LLM on Single RTX 3090REDDIT · /u/Brief-Tap-6616 · reddit.com · Sep 28, 06:35 PM26d
12OpenVINO Integration Boosts Laya CPU Inference Speed by 3.4xREDDIT · /u/simpleuserhere · reddit.com · Sep 25, 12:37 AM30d
13Strata Engine Delivers 65 Tokens/sec on 12GB VRAM via System RAM OffloadingREDDIT · /u/KnownAd4832 · reddit.com · Sep 24, 05:30 PM31d
14ExllamaV3 and EXL3 Deliver High Benchmark Speeds for Local LLM Flash Next InferenceREDDIT · /u/youcloudsofdoom · reddit.com · Sep 20, 06:10 PM34d
15Optimizing Local LLM Inference on Asymmetric NVIDIA Tesla V100 GPUsREDDIT · /u/OkBase5453 · reddit.com · Sep 19, 08:16 PM35d
16Optimizing Gemma Inference TPS via Custom vLLM Codebase PatchesREDDIT · /u/metalvendetta · reddit.com · Sep 19, 08:38 PM35d
17Seeking High-Speed LLM Inference Engines Like NInfer for 16GB GPUsREDDIT · /u/FerLuisxd · reddit.com · Sep 19, 12:31 PM36d
18SGLang v0.5.20 Released with New Models and Inference EnhancementsGITHUB · Qiaolin-Yu · github.com · Sep 18, 10:41 PM36d
19CrofAI Exposed as Fraudulent OpenRouter Wrapper Before Deleting Online PresenceREDDIT · /u/SorosAhaverom · reddit.com · Sep 15, 10:19 AM40d
20Developer Releases llamAmpere, an Ampere-Optimized llama.cpp Fork for High-Speed InferenceREDDIT · /u/Brief-Tap-6616 · reddit.com · Sep 15, 01:32 AM40d