Garlic LLM Enables Jailbroken PS4 to Run 9B Local AI Models
The open-source Garlic LLM project allows jailbroken PlayStation 4 consoles to run a quantized 9-billion parameter language model entirely offline at speeds up to 43 tokens per second. Developer Leonardo Alegria achieved this by bypassing official graphics drivers and directly interfacing with the console's AMD GPU for low-level AI inference. This breakthrough demonstrates impressive hardware hacking and edge AI optimization, turning legacy gaming hardware into capable offline AI workstations. It opens up new possibilities for repurposing consumer hardware to run large language models locally without relying on cloud infrastructure. To fit within the PS4's limited memory, Garlic LLM utilizes a 3-bit or 4-bit quantized version of the Qwen3.5-9B model and employs speculative decoding (MTP) to boost efficiency. While basic text generation reaches up to 43 tokens per second, enabling additional features like voice input and output reduces the generation speed to approximately 22 tokens per second.
## BACKGROUND
Model quantization reduces the memory footprint of large language models by storing weights in lower-precision formats, while speculative decoding accelerates text generation by using a lightweight process to draft multiple future tokens in parallel for the main model to verify. The AMD GCN architecture is a graphics and computing instruction set architecture utilized in the GPU of the PlayStation 4 console.