Google's Agent That Tests Android Apps in English

Google open-sourced ARTEMIS, an AI agent that takes plain-English instructions and runs real Android app tests — no technical setup, 99%+ success rate.

A tester that understands plain English

Imagine you could describe a problem to a colleague and they'd go test your app — tapping through every screen, checking nothing broke — while you got on with your day. That's roughly what ARTEMIS does.

Google just published it for anyone to use. You tell it what to check (in plain English), and it goes through an Android phone or simulator, step by step, doing the thing. No developer needed to write the instructions in a special language. It just… understands.

In their own testing, it completed over a hundred complex, multi-step tasks with a 99%+ success rate. That's not a demo — that's a proper benchmark the industry uses to compare tools.

Why this matters if you have an app

Right now, making sure your app doesn't break after an update usually means either paying a QA team to click through everything manually, or paying developers to write automated tests — which takes time and expertise you may not have in-house.

ARTEMIS points toward a world where you describe your workflows in plain language and the agent handles the repetitive checking. You'd still need someone technical to set it up, but the ongoing work shrinks considerably.

It's early. It's a research tool. But the quality bar here is genuinely new.

Words worth knowing

Open-source — The code is free and public. Anyone can use it, inspect it, or build on it.

QA (Quality Assurance) — The process of checking that your software works as expected before real users see it.

Benchmark — A standardised test the tech industry uses to compare tools fairly, like a race with defined rules.

MCP — A way for AI tools to connect and talk to each other, like a shared plug socket.

If you have a mobile app and you're spending real money on manual testing, this is worth watching. Even bookmarking the project now — github.com/google/artemis — means you'll be ready when the right person asks.