WinMind: A Windows MCP Server for Accessibility-Based AI Computer Control
Developer Cool-Statistician880 released WinMind, an open-source Windows Model Context Protocol (MCP) server that enables AI agents to control desktop applications using native UI Automation accessibility trees instead of processing vision-based screenshots. By replacing screenshot-based coordinate guessing with direct accessibility interactions, WinMind significantly cuts token consumption, minimizes latency, and allows AI agents to interact with background windows without moving the mouse pointer. WinMind represents application interfaces as compact, numbered trees (e.g., executing actions via reference IDs like `ui_act(ref=30)`) and bundles approximately 222 native Windows tools for process, network, and system control. It also resolves complex edge cases, such as Chromium apps rendering blank when covered, WinUI 3/WebView2 recursion loops, and identifying apps by process name.
## BACKGROUND
The Model Context Protocol (MCP) is an open standard designed to seamlessly connect AI models with local tools, data sources, and workflows. Microsoft's Windows UI Automation framework exposes application user interface elements as a structured accessibility tree, primarily used by assistive technologies such as screen readers. Most existing computer-use AI agents rely on visual processing by repeatedly taking screenshots and estimating pixel coordinates, which is resource-intensive and prone to failure when windows overlap.