Budget-Friendly Local LLM Setup Using AMD Radeon 780m iGPU and 64GB RAM
A budget-friendly hardware configuration using a mini PC with an AMD Radeon 780m iGPU and 64GB DDR5 RAM has been demonstrated to run large language models like Qwen 35B. By configuring specific Linux kernel parameters, users can allocate up to 48GB of system memory as VRAM for GPU-accelerated inference. This approach offers a cost-effective alternative (under €1000) to expensive dedicated GPUs for running large local AI models. It democratizes access to local LLM inference by utilizing consumer-grade mini PCs and shared system memory. The setup achieves around 21 tokens per second on Qwen 35B (Q8 quantization) using llama.cpp with the Vulkan backend. Key Linux kernel parameters used to expand the VRAM allocation include amdgpu.gttsize=49152, amd_iommu=off, and ttm.pages_limit=16777216.
## BACKGROUND
Integrated GPUs (iGPUs) like the Radeon 780m share system RAM with the CPU instead of having dedicated VRAM. By default, operating systems limit how much system RAM can be allocated to the iGPU, but Linux kernel parameters like amdgpu.gttsize allow users to override these limits. Llama.cpp is a popular open-source inference engine optimized for running LLMs locally on consumer hardware.