I'm exploring an AOSP-based Android launcher where an AI agent is the primary interface to the device.
I'm not trying to replace Android UI completely, and I'm not interested in giving an LLM unrestricted root/system access.
The architecture I'm considering is roughly:
User ↓ AI Launcher ├── Chat ├── Voice └── Vision ↓ AI Orchestrator ├── Local model ├── Cloud model ├── Context └── Memory ↓ Capability API ↓ Policy Engine ↓ Action Broker ↓ Android APIs / privileged APIs ↓ Android OS
Instead of letting the model directly interact with Android, the model would produce structured tool calls such as:
camera.capture() contacts.search() calendar.create() media.play() bluetooth.connect()
The Capability API would expose only explicitly defined operations.
Then a deterministic Policy Engine would decide whether an operation is:
allowed automatically
allowed only after user confirmation
not allowed at all
For example:
media.play() → automatic calendar.create() → automatic / policy dependent contacts.search() → automatic messages.send() → confirmation purchase() → confirmation delete.data() → restricted
The idea is to separate two things:
Android permission:
"Can this application access the resource?"
AI capability/policy:
"Can the AI autonomously perform this operation, under what conditions, and does it require user confirmation?"
I'm considering starting with a normal Android launcher using public APIs, then moving parts into a privileged/system app only when the required capability cannot be implemented safely otherwise.
Eventually I may test this on an AOSP-based system image, but I don't want to modify system_server unless there is a concrete reason to do so.
I'm mainly interested in the Android architecture/security side:
Does this separation between AI tool/capability layer and Android permissions make sense?
Is there already an Android mechanism that provides something conceptually similar?
Where would you put such a policy/capability layer: inside the application, a privileged system app, or the Android framework?
What security problems would you expect from an AI agent with this kind of architecture?
Is there a reason this would be fundamentally worse than implementing the same functionality directly inside a privileged Android application?
I'm deliberately looking for criticism here. If this is basically just a complicated version of an existing Android assistant, I'd rather find that out now.
submitted by /u/Ok_Middle_7371[link] [comments]