Developer Turns Jailbroken Sony PS5 into Local AI Server
A developer created PS5LM Plus, a native GPU application that allows a jailbroken Sony PlayStation 5 to run large language models locally, achieving an inference speed of 20.9 tokens per second for a quantized 27B parameter model. This project demonstrates the untapped potential of gaming console hardware for local AI tasks by leveraging high memory bandwidth for GPU acceleration. It offers a creative alternative for running powerful open-source models outside of traditional PC or server environments. By porting the AMD RADV driver and utilizing Vulkan compute via llama.cpp, the developer increased usable memory to 11.4GB and achieved 100% GPU utilization. The tool also includes a web interface, a local API server compatible with OpenAI and Anthropic formats, and supports even higher speeds for smaller models.
## BACKGROUND
Llama.cpp is a popular open-source software library that performs efficient local inference for large language models, serving as a core component for many AI tools. Vulkan is a cross-platform, open standard API that gives developers low-level control over modern GPUs for parallelized computing and graphics.