Fast Voice Agents With ConvFill
ConvFill introduces a model collaboration technique that pairs a lightweight 0.6B talker model for rapid conversational fillers with a heavy reasoner model for complex responses. Developed by students from the University of Washington UbiComp Lab, this approach successfully bridges the gap between speed and intelligence in voice agents. This architecture solves a critical latency issue in real-time voice agents by allowing them to respond instantly without sacrificing domain knowledge or reasoning depth. It opens up new possibilities for creating more natural, fluid human-computer voice interactions. The system works by having the small talker model generate immediate filler speech using cheat notes from the background reasoner, buying time until the final response is ready. A live demo of ConvFill has been released on Hugging Face alongside its presentation at the EMNLP conference.
## BACKGROUND
Voice agents powered by large language models often struggle to balance responsiveness and depth, as cloud models introduce noticeable latency while on-device models lack sophistication. ConvFill addresses this task, known as conversational infill, by coordinating local low-latency models with powerful backend reasoning systems.