Aiden: a physical AI agent that controls phones over USB HID
• 1
AI & ML interests
AI hardware
Recent Activity
View all activity
Articles
Post
1991
Spent a while chasing a genuinely strange iOS bug while building Aiden (a physical agent that drives phones over USB HID): modifier-key shortcuts like Cmd+V would silently fail while plain keystrokes worked fine every time.
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
Article
Aiden: a physical AI agent that controls phones over USB HID
AidenAgent
• • 1Post
2444
Aiden: a physical AI agent that controls phones over USB HID
Most GUI agent work assumes the agent lives inside the device or
drives it through a debugging interface. We went the other way.
Aiden is a small board that sits outside the host. It captures the
screen over HDMI-to-CSI, runs the agent loop on-device, and sends
actions back as a standard USB HID device — the host sees a keyboard
and a mouse, nothing else. No app install, no root, no ADB, no cloud.
Runtime is Go. Frame capture, full-duplex audio with VAD, the agent
loop, and HID output all run as independent goroutines. There's no
backend — nothing leaves the device, which is the only defensible
design when the input is a live feed of someone's phone screen.
Open questions we haven't solved:
· Action verification — inferring success from a re-read of the
screen breaks when loading states lie
· Prompt injection — an agent that reads screens reads whatever an
attacker puts on them
· iOS pointer control requires AssistiveTouch
Repo, including the HID gadget config and capture pipeline:
github.com/AidenAI-IO/aiden-hardware-demo
Wrote up how this differs from cloud-based computer use agents here:
https://aidenai.io/blog/mobile-ai-agent-vs-computer-use-agent-whats-the-difference/
Note: current hardware is a dev board, not a finished product.
Most GUI agent work assumes the agent lives inside the device or
drives it through a debugging interface. We went the other way.
Aiden is a small board that sits outside the host. It captures the
screen over HDMI-to-CSI, runs the agent loop on-device, and sends
actions back as a standard USB HID device — the host sees a keyboard
and a mouse, nothing else. No app install, no root, no ADB, no cloud.
Runtime is Go. Frame capture, full-duplex audio with VAD, the agent
loop, and HID output all run as independent goroutines. There's no
backend — nothing leaves the device, which is the only defensible
design when the input is a live feed of someone's phone screen.
Open questions we haven't solved:
· Action verification — inferring success from a re-read of the
screen breaks when loading states lie
· Prompt injection — an agent that reads screens reads whatever an
attacker puts on them
· iOS pointer control requires AssistiveTouch
Repo, including the HID gadget config and capture pipeline:
github.com/AidenAI-IO/aiden-hardware-demo
Wrote up how this differs from cloud-based computer use agents here:
https://aidenai.io/blog/mobile-ai-agent-vs-computer-use-agent-whats-the-difference/
Note: current hardware is a dev board, not a finished product.
- 3 replies
Post
174
This is a short office demo showing how Aiden works in practice.
Aiden is a physical mobile AI agent device that plugs into any phone or computer via USB. It sees the screen, hears your voice, and operates the device for you — no app install required.
In this video Aiden is receiving a voice command and completing a multi-step task
Built for the AI-Native Era. Works on the phone you already have.
AidenAgent
aidenai.io
Aiden is a physical mobile AI agent device that plugs into any phone or computer via USB. It sees the screen, hears your voice, and operates the device for you — no app install required.
In this video Aiden is receiving a voice command and completing a multi-step task
Built for the AI-Native Era. Works on the phone you already have.
aidenai.io
Post
87
Launching Aiden Mobile Agent Prototype. Looks incredible. Thoughts?
https://github.com/AidenAI-IO/aiden-hardware-demo
https://github.com/AidenAI-IO/aiden-hardware-demo
NatalieY
updated a
Space about 2 months ago
Post
123
Introducing AidenAgent
We built a physical AI agent device. It plugs into any phone or computer via USB. It watches the screen through HDMI. It listens, speaks, and controls the connected device autonomously. No app install, no API, no permissions required.
The host just sees a keyboard and a mouse, the intelligence is in the device.
Full builder story + technical architecture at huggingface.co/blog/NatalieY/aiden-hardware-ai-agent-device
We built a physical AI agent device. It plugs into any phone or computer via USB. It watches the screen through HDMI. It listens, speaks, and controls the connected device autonomously. No app install, no API, no permissions required.
The host just sees a keyboard and a mouse, the intelligence is in the device.
Full builder story + technical architecture at huggingface.co/blog/NatalieY/aiden-hardware-ai-agent-device
- 2 replies
NatalieY
published a
Space about 2 months ago
NatalieY
published a
Space 9 months ago