⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
agent-android
Control Android over LAN without USB, ADB, or root.
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
// README
See the Chinese overview at README_CN.md.
AIVane AI RPA
Control an Android phone from your desktop or AI agent in minutes: inspect UI, tap, type, launch apps, and capture screenshots locally over LAN.
This repository is the public home for aivanelabs/ai-rpa. The current public surface is AIVane Android REPL Beta: a directly installable agent-android CLI plus a GitHub-installable agent-android skill for tools such as Codex, Claude Code, and OpenClaw.
Demo
Add a calendar event
https://github.com/user-attachments/assets/f4e48dd4-386c-4df5-936e-37c3fa650fd4
Search on Xiaohongshu
https://github.com/user-attachments/assets/7024b34b-73cd-4cc5-857e-90a84c341110
Start Here
- The beta already has a ready-to-install CLI and skill path; use the commands below instead of repo-local script paths.
- Download APK: GitHub Releases
- Install CLI:
uv tool install aivane-agent-android - Install skill:
npx skills add aivanelabs/ai-rpa --skill agent-android
3-Step Quickstart
- Install the APK on your phone and enable the AIVane accessibility service.
- Install the CLI:
uv tool install aivane-agent-android
If agent-android is not found afterwards, run:
uv tool update-shell
Then reopen the terminal. If you want the current shell to work immediately on Linux, run:
export PATH="$HOME/.local/bin:$PATH"
- Verify the device is reachable:
agent-android --health --url http://<device-ip>:8080
If that succeeds, start the REPL:
agent-android --repl --url http://<device-ip>:8080
Choose A Path
For Humans
Use the CLI directly when you want to manually explore the phone from your desktop:
- Download the APK from GitHub Releases
- Install the CLI with
uv tool install aivane-agent-android - Run
agent-android --repl --url http://<device-ip>:8080 - In the REPL, use the short loop:
health->apps->la <package>->list-> one action ->list
For AI Agents
Use the skill when you want Codex, Claude Code, or another coding agent to drive the phone:
- Install the CLI with
uv tool install aivane-agent-android - Install the skill:
npx skills add aivanelabs/ai-rpa --skill agent-android
- Give the agent a concrete task, for example:
Use the installed agent-android skill to:
1. check phone health
2. list launcher apps
3. launch Settings
4. inspect visible UI nodes
5. tap the Wi-Fi entry
First Success Path
This is the shortest practical smoke flow after the phone is reachable:
agent-android --repl --url http://<device-ip>:8080set url http://<device-ip>:8080healthappsla <package>listtap <refId>list
If you want the full setup path, see docs/install-agent-android.md and docs/quickstart.md.
Template And File Sync
Use an application bundle when a workflow has one main template plus child templates. The phone executes the zip as one scoped package, so child template names do not collide with phone-local templates:
agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080
Use the CLI upload command when you want to push a standalone image, ordinary binary file, or one-off local file from your computer to phone-local REPL storage:
agent-android --upload foo.json --remote-path Templates/foo.json --url http://<device-ip>:8080
Uploads overwrite by default. Add --no-overwrite to reject an existing target. In templates, use http.download when the phone should fetch and save an image, JSON template, zip, text file, or other binary file from a URL. Keep http.get for text or JSON responses that should be read into variables.
Template Validation
Install the template linter when you want to check a template or application bundle before running it on a device:
uv tool install aivane-template-linter
aivane-template-lint my-app -r
The linter validates template JSON, OperationType parameters, variable references, and application-level relationships such as app.json main template resolution and template.execute child template calls.
See docs/templates/ for authoring guidance and docs/operation-types/ for the OperationType reference.
What This Beta Is
- Local-first Android automation over LAN
- Human-friendly REPL plus agent-friendly skill
- Public CLI command:
agent-android - Designed for inspect -> act -> inspect workflows
What This Beta Is Not
- Not a cloud phone farm
- Not remote control over arbitrary networks by default
- Not iOS support
- Not a visual recorder workflow yet
Why The Phone Is The Web Server
- The phone runs the lightweight HTTP service locally and the desktop connects directly to
http://<device-ip>:8080. - UI inspection, taps, text input, and screenshots stay between the phone and the controlling machine.
- The first smoke flow works without a cloud relay.
- The tradeoff is that this beta is LAN-only.
Install Sources
- PyPI package:
aivane-agent-android - Console command:
agent-android - Template linter package:
aivane-template-linter - Template linter command:
aivane-template-lint - Skill:
skills/agent-android/ - APK builds: GitHub Releases
Repo Layout
clients/python/: publishable Python CLI package using a standardsrclayoutclients/template-linter/: publishable Python template linting CLI packagedocs/: quickstart, install, protocol, permissions, release, and support docsdocs/templates/: template authoring and linting guidancedocs/operation-types/: public OperationType referencedocs/assets/: README-linked media such as demo videos and screenshotsexamples/: smoke-flow examples and launch helpersskills/agent-android/: installable public skill definition
Additional Resources
- docs/quickstart.md
- docs/install-agent-android.md
- docs/agent-examples.md
- docs/templates/README.md
- docs/operation-types/README.md
- docs/release-checklist.md
- docs/known-limitations.md
Contact
For questions and light coordination, please email [email protected].
// HOW IT'S BUILT
// FILES
4 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.
skills/agent-android/SKILL.md9.3 KBec3616a5bf33upstream ↗
--- name: agent-android description: Control Android over LAN without USB, ADB, or root. metadata: {"openclaw":{"homepage":"https://github.com/aivanelabs/ai-rpa","requires":{"bins":["agent-android"]},"install":[{"id":"agent-android-uv","kind":"uv","package":"aivane-agent-android","bins":["agent-android"],"label":"Install agent-android CLI (uv)"}]}} --- # Android REPL Use this skill to drive an Android device through the public `agent-android` beta surface published in the `aivanelabs/ai-rpa` repo. Runtime prerequisites: - `agent-android` is available on `PATH` - if it is missing, install the CLI with `uv tool install aivane-agent-android`; then run `uv tool update-shell` if the command is still not found - the user has provided a trusted device URL such as `http://<device-ip>:8080`; use the phone-side configured port if it is not `8080` The public path is local-first: - the phone hosts the lightweight HTTP service locally - the desktop connects directly to the phone URL, usually `http://<device-ip>:8080` - UI reads, taps, inputs, and screenshots stay on the phone and controlling machine - the current tradeoff is LAN-only control; an optional server-side or relay path may arrive later If an `agent-android` command suddenly stops working, first check whether the AIVane app or its local API service has exited on the phone. ## Safety Boundaries - Connect only to a device URL explicitly provided by the user. - Do not scan local networks or guess device IP addresses. - Treat UI trees and screenshots as sensitive because they may contain private app content. - Save screenshots or raw UI dumps only when the user explicitly asks for a file output. - Ask for confirmation before operating sensitive apps, private content, account settings, or irreversible actions. - Do not expose the phone-side service on a public network. ## Core Workflow Every Android control task should follow the same short loop: 1. Confirm connectivity with `/health` 2. Discover the target app with `apps` if needed 3. Launch one app 4. Inspect the current UI tree 5. Perform one action 6. Inspect again Keep the loop short. Prefer inspect -> act -> inspect over long speculative command chains. ## Quick Start Start the REPL with the user-provided device URL: - `agent-android --repl --url http://<device-ip>:8080` Built-in help: - `agent-android --help` - In the REPL: `h` ## Essential CLI Commands Use the one-off CLI when you already know the exact action you want. Connectivity: - `agent-android --health --url http://<device-ip>:8080` Discovery: - `agent-android --apps --url http://<device-ip>:8080` - `agent-android --list --url http://<device-ip>:8080` - `agent-android --id com.example:id/search --url http://<device-ip>:8080` - `agent-android --text Search --url http://<device-ip>:8080` - `agent-android --inputs --url http://<device-ip>:8080` - `agent-android --refId 7 --url http://<device-ip>:8080` - `agent-android --xpath 7 --url http://<device-ip>:8080` - `agent-android --get-attr 7 text --url http://<device-ip>:8080` Actions: - `agent-android --launch com.xingin.xhs --url http://<device-ip>:8080` - `agent-android --tap 7 --url http://<device-ip>:8080` - `agent-android --input 7 "hello world" --url http://<device-ip>:8080` - `agent-android --swipe up --url http://<device-ip>:8080` - `agent-android --swipe up --swipe-refid 7 --url http://<device-ip>:8080` - `agent-android --swipe up --swipe-xpath "//RecyclerView[1]" --url http://<device-ip>:8080` - `agent-android --back --url http://<device-ip>:8080` - `agent-android --press home --url http://<device-ip>:8080` - `agent-android --screenshot --url http://<device-ip>:8080` - `agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080` - `agent-android --template template.json --async --url http://<device-ip>:8080` - `agent-android --application-bundle app.zip --main-template-file __main__.json --async --url http://<device-ip>:8080` - `agent-android --upload foo.json --remote-path Templates/foo.json --url http://<device-ip>:8080` Waiting and output: - `agent-android --wait-for Search --timeout 30 --url http://<device-ip>:8080` - `agent-android --list --raw --url http://<device-ip>:8080` - `agent-android --tasks --url http://<device-ip>:8080` - `agent-android --task TASK_ID --url http://<device-ip>:8080` - `agent-android --task-logs TASK_ID --url http://<device-ip>:8080` - `agent-android --stop-task TASK_ID --url http://<device-ip>:8080` ## REPL Command Reference Use the REPL for exploratory tasks and smoke runs. Short aliases and long names both work. ### Browse - `health` or `hl` Check `/health`. - `l [n]` or `list [n]` List the first `n` elements, or all cached elements when `n` is omitted. - `ss` or `snapshot` Force-refresh the UI tree and print it again. - `f <text>` or `find <text>` Filter by text or content description. - `id <resourceId>` Filter by Android resource ID. - `ref <refId>` Show the full element detail for one refId. - `x <refId>` or `xpath <refId>` Generate XPath candidates and validate their runtime match counts. - `xx <refId>` Tap by the best unique generated XPath candidate. Refuses ambiguous matches. - `vx <xpath> [idx]` or `validatex <xpath> [idx]` Validate one XPath against the runtime. Optionally inspect one match by zero-based index. ### Interact - `t <refId>` or `tap <refId>` Tap the element center point from the current tree. - `tx <xpath>` or `tapx <xpath>` Tap one runtime-resolved XPath target. - `i <refId> <text>` or `input <refId> <text>` Input text into a refId target. Use `--clear` or `""` to clear instead of typing. - `ix <xpath> <text>` or `inputx <xpath> <text>` Input text into one XPath target. Use `ix <xpath> --` or `--clear` to clear the field. - `sw <d|u|l|r> [refId] [--dur N] [--dist N]` or `swipe ...` Swipe down/up/left/right across the screen, or inside a refId target when provided. - `swx <d|u|l|r> <xpath> [--dur N] [--dist N]` or `swipex ...` Swipe inside one XPath target. - `p <back|home|recents>` or `press ...` Press a system key. - `b` or `back` Press Back. - `la <package>` or `launch <package>` Launch an app by package name. - `s [path]` or `screenshot [path]` Capture a screenshot to an auto-generated or explicit path. - `up <local> <path>` or `upload <local> <path>` Upload a local file to the phone. Overwrites by default; add `--no-overwrite` to reject an existing target. ### Wait And Inspect - `wf <text> [--t N]` or `waitfor ...` Wait for an element to appear. - `g <refId> <attr>` or `get <refId> <attr>` Read an attribute such as `text`, `class`, `bounds`, `x`, `y`, or `xpath`. - `apps` List launcher apps from `/apps`. ### Session - `raw` Toggle raw JSON mode. - `vars` Show current URL, timeout, raw mode, and tree cache state. - `set timeout 30` Set the default wait timeout in seconds. - `h` or `help` Show the built-in help text. - `q` or `quit` Exit the REPL. ## Common Patterns ### First smoke flow 1. Start the REPL with `agent-android --repl --url http://<device-ip>:8080`. 2. `health` 3. `apps` 4. `la <package>` 5. `l` 6. `t <refId>` 7. `i <refId> hello` 8. `b` 9. `s` ### Find the right package before launch - `apps` - `la com.example.app` - `l` ### Inspect an element before using XPath - `l` - `ref 12` - `x 12` - `vx //EditText[@text='Search']` ### Clear and refill an input - `i 7 --clear` - `i 7 hello world` - `ix //EditText[@text='Search'] --` - `ix //EditText[@text='Search'] -- hello world` ### Run a local template bundle - Zip the local folder containing the main template and child templates. - Run `agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080`. - Use `--upload` only for standalone images, ordinary binary files, or one-off file sync. ## Troubleshooting - If an `agent-android` command fails, first check whether the AIVane app or phone-side API service has exited. - If `agent-android` is not found, run `uv tool update-shell`, reopen the terminal, and retry. - Re-open the app or restart the phone-side service, then retry `curl http://<device-ip>:8080/health`. - If `health` works but UI commands fail, run `ss` to force-refresh the tree before tapping or inputting. - If `tx` or `ix` fails, run `vx <xpath>` and make sure the XPath resolves to exactly one runtime match. - If screenshots fail the first time, confirm the on-device MediaProjection permission prompt was accepted. - If everything suddenly stops responding, confirm the phone IP did not change and that the desktop is still on the same LAN. ## When To Stop Stop and ask for user help when: - the device is unreachable on LAN - the app is not running and cannot be restarted from the current path - required Android permissions are missing - launcher discovery returns nothing useful - the runtime UI no longer matches the expected screen after repeated refreshes ## References - Smoke checklist: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/skills/agent-android/references/smoke-flow.md) - Quickstart: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/quickstart.md) - Install guide: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/install-agent-android.md) - Public protocol: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/protocol-v1.md) - Known beta limits: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/known-limitations.md)skills/agent-android/references/smoke-flow.md801 Bc5923c5c9670upstream ↗
# Android REPL Smoke Flow Use this checklist when validating a fresh setup. ## Preconditions - Android REPL app installed - API service running - controlling machine on the same LAN - `agent-android` available on `PATH` ## Checklist 1. Save the device URL: ```text set url http://<device-ip>:8080 ``` 2. Check service health: ```bash curl http://<device-ip>:8080/health ``` 3. List launcher apps: ```bash agent-android --apps --url http://<device-ip>:8080 ``` 4. Launch the target app. 5. List the current UI tree. 6. Perform one tap. 7. Perform one text input if relevant. 8. Navigate back. ## Success Criteria - the device responds - launcher apps are visible - launch succeeds - the UI tree refreshes after actions - the user can complete at least one inspect -> act -> inspect loopskills/agent-android/agents/openai.yaml281 Bfb8bc0ed0b3aupstream ↗
interface: display_name: "agent-android" short_description: "agent-android for AI agents" default_prompt: "Use $agent-android from the aivanelabs/ai-rpa repo to connect to an AIVane device over LAN, inspect launcher apps and UI state, then control the phone step by step."README.mdupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
Plain-text copy, no login: /skill-bundles/aivanelabs-ai-rpa-skills-agent-android.md