← Field Notes
EN/ES

Teaching AI Agents to Use Desktop Software

March 13, 2026via github · @HKUDS
AIautomationworkflowtoolsopen-source

The problem it solves

Most AI agents are great at browsing the web, writing text, or moving files around. But ask one to resize a batch of images in GIMP, or export a video from Blender, and it hits a wall. Those apps were built for humans with mice — not for software talking to software.

CLI-Anything changes that. You point it at a piece of desktop software, and it figures out how to operate that software through text commands instead of clicks. The result: an AI agent can now use Blender or LibreOffice the same way it uses a search engine — just by sending instructions and getting results back.

No screenshots. No simulated clicking. Just clean, reliable communication.

Why this matters for your business

Think about the software your team uses every day — design tools, video editors, office suites. Right now, automating those usually means hiring someone to do the repetitive parts. With something like CLI-Anything, an AI agent could handle those tasks directly: batch-export files, apply templates, generate reports — running overnight while nobody's in the office.

It already works with GIMP, Blender, LibreOffice, OBS Studio, and a few others. More are being added fast.

Words worth knowing

AI agent — A piece of software that can take actions on its own, not just answer questions. Like a very literal assistant that does what you tell it, step by step.

CLI (Command Line Interface) — A way to control software by typing instructions instead of clicking buttons. Computers have used this for decades. It's how software talks to other software.

GUI (Graphical User Interface) — The visual part of an app — the buttons, menus, and windows you interact with. Most desktop software is GUI-only, which makes it hard for agents to use.

JSON output — When a program returns its results in a tidy, structured format that other software can easily read and act on. Think of it like getting a receipt with every item clearly listed, instead of a handwritten note.


If your team does anything repetitive in desktop software — exporting, formatting, converting — it's worth asking: could an AI agent do this while we sleep?

Want us to audit your site? Takes 60 seconds →