LFM3-Omni-2B Runs Offline on iPhone at 80 Tok/s via Custom WebGPU
A developer successfully ran the LFM3-Omni-2B multimodal model fully offline on an iPhone at approximately 80 tokens per second. This performance was achieved using custom WebGPU kernels integrated into a React Native mobile application. This achievement highlights the growing viability of running advanced multimodal AI models locally on edge devices rather than relying on cloud infrastructure. It paves the way for faster, more private, and offline-capable AI applications on mobile hardware. The setup utilizes the 2-billion parameter LFM3-Omni-2B model optimized for local deployment and leverages custom WebGPU operations to accelerate execution on mobile GPUs. Running entirely offline ensures user data remains secure on the device without requiring server communication.
## BACKGROUND
WebGPU is a modern web standard designed to provide high-performance low-level access to a device's GPU, increasingly adopted for client-side machine learning acceleration. React Native is a popular framework for building native mobile apps using JavaScript and React, while LFM3-Omni models are end-to-end multimodal systems capable of handling text, images, video, and audio.