Tencent Releases UI-Mate-27B, an Open-Weight GUI Agent for Desktop Automation
Tencent has released UI-Mate-27B, an open-weight GUI agent model built on Qwen3.6-27B that can execute complex tasks on desktop interfaces. It operates by analyzing live screenshots and generating structured keyboard and mouse actions, supporting both general computer use and demonstration-guided workflows. The release provides a powerful, open-weight alternative for desktop automation and "computer use" tasks, which have traditionally been dominated by proprietary models. By supporting demonstration-guided planning, it allows users to teach the model new workflows from a single demonstration rather than relying on rigid scripts. UI-Mate-27B was trained using supervised fine-tuning followed by online reinforcement learning in executable GUI environments, and its structured actions are compatible with the pyautogui library. It features live-screen grounding instead of coordinate replay, allowing it to dynamically re-plan when the interface layout or application state changes.
## BACKGROUND
GUI (Graphical User Interface) agents are AI models designed to interact with computers in the same way humans do, by observing the screen and using virtual inputs like mouse clicks and keystrokes. Long-horizon tasks refer to complex goals that require an agent to plan and execute dozens or hundreds of sequential steps across multiple applications over an extended period.