Google's Android Studio Uses llama.cpp to Power Native Local Gemma Models
Developers discovered that Android Studio's native integration of local Gemma models runs on the open-source llama.cpp inference engine under the hood. The implementation supports multi-GPU execution and context windows up to 128k tokens for Gemma 31B models. This discovery highlights how open-source LLM inference tooling like llama.cpp has become an industry standard, even adopted by tech giants like Google. It enables developer environments to offer fast, privacy-centric, and offline AI features without building proprietary inference stacks from scratch. Running the Gemma 31B model with a 128k context window requires approximately 34 GB of VRAM when fully loaded, likely leveraging Vulkan acceleration and Quantization-Aware Training (QAT). However, Android Studio currently lacks user interface options to adjust the context length or display prompt processing and token generation speeds.
## BACKGROUND
llama.cpp is a popular open-source C/C++ library designed for high-performance inference of large language models across diverse hardware setups. Gemma is a family of lightweight, open LLMs built by Google using the same research and technology as Gemini models. Quantization-Aware Training (QAT) is a method that optimizes models during training for lower-bit precision, reducing VRAM usage while preserving model quality.