Apple's Free On-Device AI Just Got a Lot More Real

What happened

At its developer conference last week, Apple slipped in something most tech journalists glossed over. They released a tool called fm serve — and with it, a way for developers to run an AI model directly on a Mac, completely offline, for free.

That might not sound dramatic. But think about it this way: until now, every time an app asked an AI a question, that question went to a server somewhere — probably in the US — and came back with an answer. That costs money per question. It also means your data travels.

With this, the AI lives on the device itself. The conversation never leaves the computer.

Why it matters for founders

Apple also announced that any developer with fewer than two million app downloads gets free access to their private cloud AI tier. That's basically every indie app maker on earth.

If you're working with someone building you a custom iOS or macOS app — a booking tool, a client portal, a kitchen management system — they can now add AI features without passing those costs on to you. And the data stays on your phone or Mac, not on someone else's servers.

The framework is going open source this summer, and it already plays nicely with Claude and Google's AI, so developers aren't locked in.

Something to sit with

We're quietly moving toward a world where AI doesn't need the internet to work. Ask your developer what it would take to run something like this in your next project.

Words worth knowing

On-device AI — The model runs on your computer or phone, not on a distant server. Faster, private, no ongoing cost per use.

API — A way for two software tools to talk to each other. When an app sends your data to an AI and gets an answer back, it's using an API.

Open source — The code is published publicly so anyone can read, verify, or build on it. Usually a good sign for longevity and trust.

Indie developer — A small, independent software maker. Not a big company. The person who might be building your app.