⏳ This skill is pending AI review.

Scores will appear once the review pipeline completes.

version unknown

agent-android

aivanelabs@aivanelabs⭐ 14 stars· last commit 2mo ago· 1 open issues

Control Android over LAN without USB, ADB, or root.

Choose how to use this skill

You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.

1. Native installer

This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.

Do not guess an installer command or replace an existing version without reviewing the diff.

2. Complete package recommended

Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.

No complete ProSkills package is published for this listing yet.

3. Prompt-only

Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.

Need only the instruction file?

Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.

No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.

—/10

// RATINGS

⭐GitHub Stars
⭐⭐ 14 on GitHubGitHub ↗

Growing

🟢ProSkills Score
—
📍

Not yet listed on ClawHub or SkillsMP

// README

See the Chinese overview at README_CN.md.

AIVane AI RPA

Control an Android phone from your desktop or AI agent in minutes: inspect UI, tap, type, launch apps, and capture screenshots locally over LAN.

This repository is the public home for aivanelabs/ai-rpa. The current public surface is AIVane Android REPL Beta: a directly installable agent-android CLI plus a GitHub-installable agent-android skill for tools such as Codex, Claude Code, and OpenClaw.

Demo

Add a calendar event

https://github.com/user-attachments/assets/f4e48dd4-386c-4df5-936e-37c3fa650fd4

Search on Xiaohongshu

https://github.com/user-attachments/assets/7024b34b-73cd-4cc5-857e-90a84c341110

Start Here

  • The beta already has a ready-to-install CLI and skill path; use the commands below instead of repo-local script paths.
  • Download APK: GitHub Releases
  • Install CLI: uv tool install aivane-agent-android
  • Install skill: npx skills add aivanelabs/ai-rpa --skill agent-android

3-Step Quickstart

  1. Install the APK on your phone and enable the AIVane accessibility service.
  2. Install the CLI:
uv tool install aivane-agent-android

If agent-android is not found afterwards, run:

uv tool update-shell

Then reopen the terminal. If you want the current shell to work immediately on Linux, run:

export PATH="$HOME/.local/bin:$PATH"
  1. Verify the device is reachable:
agent-android --health --url http://<device-ip>:8080

If that succeeds, start the REPL:

agent-android --repl --url http://<device-ip>:8080

Choose A Path

For Humans

Use the CLI directly when you want to manually explore the phone from your desktop:

  1. Download the APK from GitHub Releases
  2. Install the CLI with uv tool install aivane-agent-android
  3. Run agent-android --repl --url http://<device-ip>:8080
  4. In the REPL, use the short loop: health -> apps -> la <package> -> list -> one action -> list

For AI Agents

Use the skill when you want Codex, Claude Code, or another coding agent to drive the phone:

  1. Install the CLI with uv tool install aivane-agent-android
  2. Install the skill:
npx skills add aivanelabs/ai-rpa --skill agent-android
  1. Give the agent a concrete task, for example:
Use the installed agent-android skill to:
1. check phone health
2. list launcher apps
3. launch Settings
4. inspect visible UI nodes
5. tap the Wi-Fi entry

First Success Path

This is the shortest practical smoke flow after the phone is reachable:

  1. agent-android --repl --url http://<device-ip>:8080
  2. set url http://<device-ip>:8080
  3. health
  4. apps
  5. la <package>
  6. list
  7. tap <refId>
  8. list

If you want the full setup path, see docs/install-agent-android.md and docs/quickstart.md.

Template And File Sync

Use an application bundle when a workflow has one main template plus child templates. The phone executes the zip as one scoped package, so child template names do not collide with phone-local templates:

agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080

Use the CLI upload command when you want to push a standalone image, ordinary binary file, or one-off local file from your computer to phone-local REPL storage:

agent-android --upload foo.json --remote-path Templates/foo.json --url http://<device-ip>:8080

Uploads overwrite by default. Add --no-overwrite to reject an existing target. In templates, use http.download when the phone should fetch and save an image, JSON template, zip, text file, or other binary file from a URL. Keep http.get for text or JSON responses that should be read into variables.

Template Validation

Install the template linter when you want to check a template or application bundle before running it on a device:

uv tool install aivane-template-linter
aivane-template-lint my-app -r

The linter validates template JSON, OperationType parameters, variable references, and application-level relationships such as app.json main template resolution and template.execute child template calls.

See docs/templates/ for authoring guidance and docs/operation-types/ for the OperationType reference.

What This Beta Is

  • Local-first Android automation over LAN
  • Human-friendly REPL plus agent-friendly skill
  • Public CLI command: agent-android
  • Designed for inspect -> act -> inspect workflows

What This Beta Is Not

  • Not a cloud phone farm
  • Not remote control over arbitrary networks by default
  • Not iOS support
  • Not a visual recorder workflow yet

Why The Phone Is The Web Server

  • The phone runs the lightweight HTTP service locally and the desktop connects directly to http://<device-ip>:8080.
  • UI inspection, taps, text input, and screenshots stay between the phone and the controlling machine.
  • The first smoke flow works without a cloud relay.
  • The tradeoff is that this beta is LAN-only.

Install Sources

  • PyPI package: aivane-agent-android
  • Console command: agent-android
  • Template linter package: aivane-template-linter
  • Template linter command: aivane-template-lint
  • Skill: skills/agent-android/
  • APK builds: GitHub Releases

Repo Layout

  • clients/python/: publishable Python CLI package using a standard src layout
  • clients/template-linter/: publishable Python template linting CLI package
  • docs/: quickstart, install, protocol, permissions, release, and support docs
  • docs/templates/: template authoring and linting guidance
  • docs/operation-types/: public OperationType reference
  • docs/assets/: README-linked media such as demo videos and screenshots
  • examples/: smoke-flow examples and launch helpers
  • skills/agent-android/: installable public skill definition

Additional Resources

Contact

For questions and light coordination, please email [email protected].

// HOW IT'S BUILT

// FILES

4 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

  • skills/agent-android/SKILL.md9.3 KBec3616a5bf33upstream ↗
    ---
    name: agent-android
    description: Control Android over LAN without USB, ADB, or root.
    metadata: {"openclaw":{"homepage":"https://github.com/aivanelabs/ai-rpa","requires":{"bins":["agent-android"]},"install":[{"id":"agent-android-uv","kind":"uv","package":"aivane-agent-android","bins":["agent-android"],"label":"Install agent-android CLI (uv)"}]}}
    ---
    
    # Android REPL
    
    Use this skill to drive an Android device through the public `agent-android` beta surface published in the `aivanelabs/ai-rpa` repo.
    
    Runtime prerequisites:
    
    - `agent-android` is available on `PATH`
    - if it is missing, install the CLI with `uv tool install aivane-agent-android`; then run `uv tool update-shell` if the command is still not found
    - the user has provided a trusted device URL such as `http://<device-ip>:8080`; use the phone-side configured port if it is not `8080`
    
    The public path is local-first:
    
    - the phone hosts the lightweight HTTP service locally
    - the desktop connects directly to the phone URL, usually `http://<device-ip>:8080`
    - UI reads, taps, inputs, and screenshots stay on the phone and controlling machine
    - the current tradeoff is LAN-only control; an optional server-side or relay path may arrive later
    
    If an `agent-android` command suddenly stops working, first check whether the AIVane app or its local API service has exited on the phone.
    
    ## Safety Boundaries
    
    - Connect only to a device URL explicitly provided by the user.
    - Do not scan local networks or guess device IP addresses.
    - Treat UI trees and screenshots as sensitive because they may contain private app content.
    - Save screenshots or raw UI dumps only when the user explicitly asks for a file output.
    - Ask for confirmation before operating sensitive apps, private content, account settings, or irreversible actions.
    - Do not expose the phone-side service on a public network.
    
    ## Core Workflow
    
    Every Android control task should follow the same short loop:
    
    1. Confirm connectivity with `/health`
    2. Discover the target app with `apps` if needed
    3. Launch one app
    4. Inspect the current UI tree
    5. Perform one action
    6. Inspect again
    
    Keep the loop short. Prefer inspect -> act -> inspect over long speculative command chains.
    
    ## Quick Start
    
    Start the REPL with the user-provided device URL:
    
    - `agent-android --repl --url http://<device-ip>:8080`
    
    Built-in help:
    
    - `agent-android --help`
    - In the REPL: `h`
    
    ## Essential CLI Commands
    
    Use the one-off CLI when you already know the exact action you want.
    
    Connectivity:
    
    - `agent-android --health --url http://<device-ip>:8080`
    
    Discovery:
    
    - `agent-android --apps --url http://<device-ip>:8080`
    - `agent-android --list --url http://<device-ip>:8080`
    - `agent-android --id com.example:id/search --url http://<device-ip>:8080`
    - `agent-android --text Search --url http://<device-ip>:8080`
    - `agent-android --inputs --url http://<device-ip>:8080`
    - `agent-android --refId 7 --url http://<device-ip>:8080`
    - `agent-android --xpath 7 --url http://<device-ip>:8080`
    - `agent-android --get-attr 7 text --url http://<device-ip>:8080`
    
    Actions:
    
    - `agent-android --launch com.xingin.xhs --url http://<device-ip>:8080`
    - `agent-android --tap 7 --url http://<device-ip>:8080`
    - `agent-android --input 7 "hello world" --url http://<device-ip>:8080`
    - `agent-android --swipe up --url http://<device-ip>:8080`
    - `agent-android --swipe up --swipe-refid 7 --url http://<device-ip>:8080`
    - `agent-android --swipe up --swipe-xpath "//RecyclerView[1]" --url http://<device-ip>:8080`
    - `agent-android --back --url http://<device-ip>:8080`
    - `agent-android --press home --url http://<device-ip>:8080`
    - `agent-android --screenshot --url http://<device-ip>:8080`
    - `agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080`
    - `agent-android --template template.json --async --url http://<device-ip>:8080`
    - `agent-android --application-bundle app.zip --main-template-file __main__.json --async --url http://<device-ip>:8080`
    - `agent-android --upload foo.json --remote-path Templates/foo.json --url http://<device-ip>:8080`
    
    Waiting and output:
    
    - `agent-android --wait-for Search --timeout 30 --url http://<device-ip>:8080`
    - `agent-android --list --raw --url http://<device-ip>:8080`
    - `agent-android --tasks --url http://<device-ip>:8080`
    - `agent-android --task TASK_ID --url http://<device-ip>:8080`
    - `agent-android --task-logs TASK_ID --url http://<device-ip>:8080`
    - `agent-android --stop-task TASK_ID --url http://<device-ip>:8080`
    
    ## REPL Command Reference
    
    Use the REPL for exploratory tasks and smoke runs. Short aliases and long names both work.
    
    ### Browse
    
    - `health` or `hl`
      Check `/health`.
    - `l [n]` or `list [n]`
      List the first `n` elements, or all cached elements when `n` is omitted.
    - `ss` or `snapshot`
      Force-refresh the UI tree and print it again.
    - `f <text>` or `find <text>`
      Filter by text or content description.
    - `id <resourceId>`
      Filter by Android resource ID.
    - `ref <refId>`
      Show the full element detail for one refId.
    - `x <refId>` or `xpath <refId>`
      Generate XPath candidates and validate their runtime match counts.
    - `xx <refId>`
      Tap by the best unique generated XPath candidate. Refuses ambiguous matches.
    - `vx <xpath> [idx]` or `validatex <xpath> [idx]`
      Validate one XPath against the runtime. Optionally inspect one match by zero-based index.
    
    ### Interact
    
    - `t <refId>` or `tap <refId>`
      Tap the element center point from the current tree.
    - `tx <xpath>` or `tapx <xpath>`
      Tap one runtime-resolved XPath target.
    - `i <refId> <text>` or `input <refId> <text>`
      Input text into a refId target.
      Use `--clear` or `""` to clear instead of typing.
    - `ix <xpath> <text>` or `inputx <xpath> <text>`
      Input text into one XPath target.
      Use `ix <xpath> --` or `--clear` to clear the field.
    - `sw <d|u|l|r> [refId] [--dur N] [--dist N]` or `swipe ...`
      Swipe down/up/left/right across the screen, or inside a refId target when provided.
    - `swx <d|u|l|r> <xpath> [--dur N] [--dist N]` or `swipex ...`
      Swipe inside one XPath target.
    - `p <back|home|recents>` or `press ...`
      Press a system key.
    - `b` or `back`
      Press Back.
    - `la <package>` or `launch <package>`
      Launch an app by package name.
    - `s [path]` or `screenshot [path]`
      Capture a screenshot to an auto-generated or explicit path.
    - `up <local> <path>` or `upload <local> <path>`
      Upload a local file to the phone. Overwrites by default; add `--no-overwrite` to reject an existing target.
    
    ### Wait And Inspect
    
    - `wf <text> [--t N]` or `waitfor ...`
      Wait for an element to appear.
    - `g <refId> <attr>` or `get <refId> <attr>`
      Read an attribute such as `text`, `class`, `bounds`, `x`, `y`, or `xpath`.
    - `apps`
      List launcher apps from `/apps`.
    
    ### Session
    
    - `raw`
      Toggle raw JSON mode.
    - `vars`
      Show current URL, timeout, raw mode, and tree cache state.
    - `set timeout 30`
      Set the default wait timeout in seconds.
    - `h` or `help`
      Show the built-in help text.
    - `q` or `quit`
      Exit the REPL.
    
    ## Common Patterns
    
    ### First smoke flow
    
    1. Start the REPL with `agent-android --repl --url http://<device-ip>:8080`.
    2. `health`
    3. `apps`
    4. `la <package>`
    5. `l`
    6. `t <refId>`
    7. `i <refId> hello`
    8. `b`
    9. `s`
    
    ### Find the right package before launch
    
    - `apps`
    - `la com.example.app`
    - `l`
    
    ### Inspect an element before using XPath
    
    - `l`
    - `ref 12`
    - `x 12`
    - `vx //EditText[@text='Search']`
    
    ### Clear and refill an input
    
    - `i 7 --clear`
    - `i 7 hello world`
    - `ix //EditText[@text='Search'] --`
    - `ix //EditText[@text='Search'] -- hello world`
    
    ### Run a local template bundle
    
    - Zip the local folder containing the main template and child templates.
    - Run `agent-android --application-bundle app.zip --main-template-file __main__.json --url http://<device-ip>:8080`.
    - Use `--upload` only for standalone images, ordinary binary files, or one-off file sync.
    
    ## Troubleshooting
    
    - If an `agent-android` command fails, first check whether the AIVane app or phone-side API service has exited.
    - If `agent-android` is not found, run `uv tool update-shell`, reopen the terminal, and retry.
    - Re-open the app or restart the phone-side service, then retry `curl http://<device-ip>:8080/health`.
    - If `health` works but UI commands fail, run `ss` to force-refresh the tree before tapping or inputting.
    - If `tx` or `ix` fails, run `vx <xpath>` and make sure the XPath resolves to exactly one runtime match.
    - If screenshots fail the first time, confirm the on-device MediaProjection permission prompt was accepted.
    - If everything suddenly stops responding, confirm the phone IP did not change and that the desktop is still on the same LAN.
    
    ## When To Stop
    
    Stop and ask for user help when:
    
    - the device is unreachable on LAN
    - the app is not running and cannot be restarted from the current path
    - required Android permissions are missing
    - launcher discovery returns nothing useful
    - the runtime UI no longer matches the expected screen after repeated refreshes
    
    ## References
    
    - Smoke checklist: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/skills/agent-android/references/smoke-flow.md)
    - Quickstart: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/quickstart.md)
    - Install guide: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/install-agent-android.md)
    - Public protocol: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/protocol-v1.md)
    - Known beta limits: [GitHub](https://github.com/aivanelabs/ai-rpa/blob/main/docs/known-limitations.md)
    
  • skills/agent-android/references/smoke-flow.md801 Bc5923c5c9670upstream ↗
    # Android REPL Smoke Flow
    
    Use this checklist when validating a fresh setup.
    
    ## Preconditions
    
    - Android REPL app installed
    - API service running
    - controlling machine on the same LAN
    - `agent-android` available on `PATH`
    
    ## Checklist
    
    1. Save the device URL:
    
    ```text
    set url http://<device-ip>:8080
    ```
    
    2. Check service health:
    
    ```bash
    curl http://<device-ip>:8080/health
    ```
    
    3. List launcher apps:
    
    ```bash
    agent-android --apps --url http://<device-ip>:8080
    ```
    
    4. Launch the target app.
    5. List the current UI tree.
    6. Perform one tap.
    7. Perform one text input if relevant.
    8. Navigate back.
    
    ## Success Criteria
    
    - the device responds
    - launcher apps are visible
    - launch succeeds
    - the UI tree refreshes after actions
    - the user can complete at least one inspect -> act -> inspect loop
    
    
    
  • skills/agent-android/agents/openai.yaml281 Bfb8bc0ed0b3aupstream ↗
    interface:
      display_name: "agent-android"
      short_description: "agent-android for AI agents"
      default_prompt: "Use $agent-android from the aivanelabs/ai-rpa repo to connect to an AIVane device over LAN, inspect launcher apps and UI state, then control the phone step by step."
    
    
    
  • README.mdupstream ↗

    Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/aivanelabs-ai-rpa-skills-agent-android.md

// REPO STATS

14 stars
1 open issues
Last commit: 2mo ago