Consumer GPU Comparison Guide for Local LLM Inference
A community member on Reddit created visual charts analyzing popular consumer GPUs for local LLM inference based on VRAM capacity per dollar, theoretical memory bandwidth, and bandwidth efficiency relative to price. The dataset incorporates both new and second-hand GPU pricing collected through automated scraping. Local LLM inference performance relies heavily on VRAM capacity to fit larger model weights and high memory bandwidth to maximize generation speeds. This guide provides a practical reference for developers and enthusiasts trying to maximize performance per dollar when choosing consumer hardware. The chart utilizes paper-spec theoretical bandwidth rather than real-world benchmark speeds (tokens per second) and relies on price data scraped via ChatGPT, which may contain inaccuracies. It focuses specifically on cards frequently discussed across subreddits like r/LocalLLaMA, r/LowEndLocalAI, and r/LocalLLM, using popular GPUs like the RTX 3090 as baselines.
## BACKGROUND
Unlike traditional graphics applications or AI training workloads that depend heavily on raw processing power (TFLOPS), LLM inference is primarily memory-bound. The model's parameter count and quantization level dictate the minimum VRAM required, while memory bandwidth acts as the main bottleneck for token generation speed. Consequently, users running local models prioritize VRAM capacity and memory bandwidth over theoretical compute throughput.