YT · YOUTUBE// 21d agoOPENSOURCE RELEASE

Ghost OS gives agents macOS control

Ghost OS is a Swift-based open-source MCP server that lets AI agents operate native macOS apps through the accessibility tree, with a local ShowUI-2B vision fallback when the AX tree is not enough. It leans hard on self-learning recipes, so a workflow can be observed once and replayed as structured JSON forever.

// ANALYSIS

This is a smarter computer-use bet than screenshot-first agents: structured UI data, local fallback vision, and reusable recipes should make it faster, cheaper, and easier to audit.

–The repo says Ghost OS reads the macOS accessibility tree first, falls back to local vision for web or dynamic elements, and plugs into Claude Code, Cursor, VS Code, or any MCP client. Source: https://github.com/ghostwright/ghost-os
–Its self-learning flow is the interesting part: the agent watches a human do a task, synthesizes a parameterized recipe, and reuses it instead of reasoning from scratch every time.
–The tradeoff is obvious: this is powerful but Mac-specific, permission-heavy, and likely brittle on apps with messy accessibility trees or shifting UIs.
–Announcement video: https://www.youtube.com/watch?v=E7C2msOl2gQ

// TAGS

ghost-osagentcomputer-useautomationmcpopen-sourceself-hosted

DISCOVERED

21d ago

2026-03-21

PUBLISHED

21d ago

2026-03-21

RELEVANCE

9/ 10

AUTHOR

Github Awesome