AI SummaryAn MCP server that lets AI agents interact with Windows desktop applications using the same patterns as Playwright's browser automation - structured accessibility snapshots with element refs, not screenshot-based guessing. The Playwright MCP pattern works extremely well for agents: This project brin
Install
Copy this and paste it into Claude Code, Cursor, or any AI assistant:
I want to add the "FlaUI-MCP — Copilot Instructions" prompt rules to my project. Repository: https://github.com/shanselman/FlaUI-MCP Please read the repo to find the rules/prompt file, then: 1. Download it to the correct location (.cursorrules, .windsurfrules, .github/prompts/, or project root — based on the file type) 2. If there's an existing rules file, merge the new rules in rather than overwriting 3. Confirm what was added
Description
MCP server for Windows desktop automation using FlaUI and UI Automation APIs
Windows App Automation MCP Server
An MCP server that lets AI agents interact with Windows desktop applications using the same patterns as Playwright's browser automation - structured accessibility snapshots with element refs, not screenshot-based guessing.
Why MCP Server (not CLI or library)
| Approach | Agent Experience | |----------|-----------------| | MCP Server | windows_snapshot → structured tree with refs → windows_click ref="btn7" ✅ | | CLI exe | Run command → parse text output → figure out element IDs → run another command | | Screenshot-only | Take screenshot → vision model guesses coordinates → hope click lands | The Playwright MCP pattern works extremely well for agents: • browser_snapshot returns an accessibility tree with refs like [ref=s1e5] • Agent picks element by semantic meaning, not coordinates • browser_click ref="s1e5" is precise and reliable This project brings that same pattern to Windows desktop apps.
Architecture
` ┌─────────────────────────────────────────────────────────────────┐ │ AI Agent (GitHub Copilot, Claude, etc.) │ │ - Calls MCP tools: windows_snapshot, windows_click, etc. │ │ - Sees structured element tree, picks refs by meaning │ └─────────────────────────────────────────────────────────────────┘ │ MCP Protocol (JSON-RPC over stdio) ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Windows MCP Server (.NET / FlaUI) │ │ - Implements MCP tool handlers │ │ - Translates UI Automation tree → agent-friendly snapshot │ │ - Manages element refs ↔ AutomationElement mapping │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ FlaUI (.NET) - github.com/FlaUI/FlaUI │ │ - UIA3Automation (WPF, UWP, Store Apps, modern Win32) │ │ - UIA2Automation (fallback for older WinForms) │ │ - Handles platform quirks, control patterns, tree walking │ └─────────────────────────────────────────────────────────────────┘ `
`windows_launch`
Launch a Windows application and return a session ID. `json { "app": "calc.exe" } { "app": "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App" } // UWP { "app": "C:\\Program Files\\MyApp\\app.exe", "args": ["--debug"] } `
Discussion
Health Signals
My Fox Den
Community Rating
Sign in to rate this booster
Works With
Any AI assistant that accepts custom rules or system prompts