Developer Runs LFM-2.5 2.6B Model on Mobile CPU at 17 Tokens/Second
A developer has successfully run the LFM-2.5 2.6B parameter model at 17 tokens per second on a OnePlus 13 smartphone using only the CPU. This was achieved using a custom-built, lightweight inference engine that is only 450kb in size. This demonstration showcases the feasibility of running relatively large language models locally on consumer mobile hardware without relying on cloud servers or mobile GPUs. It highlights the potential of highly optimized, tiny inference engines to enable efficient edge AI and agentic workflows on everyday devices. The model used is a Q4_K_M GGUF quantized version of the 2.69B parameter LFM-2.5, which features a 128K context window. The custom 450kb engine also supports other model architectures like Qwen, Gemma, and Bonsai, with the developer aiming to push performance to 30 tokens per second.
## BACKGROUND
Liquid Foundation Models (LFMs) are a class of generative AI models developed by Liquid AI, designed for high computational efficiency, low latency, and on-device deployment. GGUF is a file format used for storing models for inference, often utilizing quantization methods like Q4_K_M to compress model weights (reducing memory usage by over 70%) while preserving most of the model's accuracy.