Evaluating Open-Weights VLMs on Egocentric Data Using HFlow
Researchers evaluated recent open-weights Vision-Language Models (VLMs) like Gemma 4 and Qwen 3.8 on egocentric video data using the open-source HFlow pipeline. The evaluation showed that Gemma 4 26B-A4B achieved a 90.87% agreement rate with the Gemini 2.5 Flash baseline (91.65%) while being 19 times more cost-effective. This demonstration proves that modern open-weights VLMs are now capable enough to handle large-scale egocentric data processing, which is crucial for robotics and physical AI. It allows organizations to self-host these models, ensuring data privacy and significantly reducing API costs compared to proprietary models. The evaluation utilized Build AI's Egocentric-10k dataset to measure hand visibility and active manipulation. While Gemini 2.5 Flash set the baseline at 91.65%, GLM 5.3 Flash scored 91.00%, Qwen 3.8 27B scored 90.79%, and Inkling Small trailed at 85.21%.
## BACKGROUND
Egocentric vision, or first-person vision, involves analyzing images and videos captured by wearable cameras (such as head-mounted devices) to simulate a person's visual field. The Egocentric-10k dataset is a large-scale collection of factory worker videos designed to benchmark hand perception and manipulation. HFlow is an open-source SDK developed by Hebbian Robotics for building scalable multimodal data pipelines in robotics.