Nous Research showcased their Hermes Agent autonomously learning and controlling TouchDesigner from scratch to build creative visual systems.
A demonstration shared by creator Amber Shen highlights the capabilities of Nous Research's Hermes Agent, which was able to autonomously learn and build in the node-based visual programming environment TouchDesigner. Operating via the Model Context Protocol (MCP), the local AI agent navigated the desktop environment to construct interactive real-time 3D visuals without manual user intervention.
Local desktop-controlling agents are transitioning complex creative software from a manual, high-barrier craft to a conversational, prompt-driven interface.
- –Hermes' ability to learn and drive a niche, node-based tool like TouchDesigner demonstrates that agents do not need custom GUI training for every unique application if they have structured API tools.
- –The use of MCP (Model Context Protocol) is key here, replacing fragile visual macro-clicking with robust, state-aware tool schemas.
- –This shift will likely democratize real-time media design, enabling creators to build complex interactive installations through conversational iterations.
DISCOVERED
52m ago
2026-06-13
PUBLISHED
1h ago
2026-06-13
RELEVANCE
AUTHOR
NousResearch