code-delegate

🤖 Jarvis AI Desktop Agent

A self-hosted, autonomous AI agent for Linux — it plans, executes, and gets real work done.

Control your Linux desktop with natural language. Receive tasks via WhatsApp. Search your knowledge base. Automate everything.

Live Demo · Report Bug · Request Feature · Contribute

---

---

📋 Table of Contents

• Overview
• Screenshots
• Key Features
• Architecture
• Tech Stack
• Installation
• Configuration
• Multi-User Chat
• Multi-Agent System
• Skill System
• Email Automation & Outlook Add-in
• SAP Analysis Area
• Short Tracks
• WhatsApp Integration
• Knowledge Base
• Vision & Face Recognition
• AD/LDAP & Security
• Multimedia Attachments
• Feedback & Self-Improvement
• Cognitive Evolution
• Client Apps
• API Reference
• Contributing
• Third-Party Licenses
• License

---

Overview

Jarvis is a self-hosted, autonomous AI agent that runs on a Linux server. Give it a goal in plain language — through the web chat, the built-in Support portal, a task pane inside Outlook or Excel, or even WhatsApp — and it plans and executes: browsing the web, reading and writing files, running code, editing existing spreadsheets, generating Office documents & diagrams, answering email by rule, evaluating SAP data read-only, managing your calendar. Whenever you want, you can watch it work live on the desktop via an optional VNC view.

"Find all emails from last week about Project Alpha, summarize them,
 and create a calendar event for the follow-up meeting."

Jarvis handles it — and because it's multi-LLM, multi-user, and wrapped in a real security layer with sandboxed execution, you can safely open it to a whole team.

---

Screenshots

Interactive API console — every REST endpoint listed, explained, with a live test caller (admin-only, /api):

Security settings — attack prevention, sandbox status & incident log under Settings → Security:

---

Key Features

🖥️ Real Desktop Control (live VNC view)

Jarvis drives a real Linux desktop — launching apps, clicking, typing. Toggle the live desktop view (noVNC) to watch or take over at any time; screenshots feed straight back into the LLM context, so the agent sees what it's doing. No blind automation.

🔀 Multi-LLM Support

Switch between AI providers without restarting anything:

• Google Gemini (gemini-2.5-flash, gemini-2.0-flash, gemini-1.5-pro, …)
• Anthropic Claude (claude-opus-4, claude-sonnet-4-5, claude-haiku-4, …)
• OpenRouter (hundreds of models via one API)
• Local Ollama (llama3, mistral, qwen2.5, … — fully offline)
• Any OpenAI-compatible endpoint

Both native tool/function calling and prompt-based tool calling are supported — so even models without native tool support can use all of Jarvis's capabilities.

🤖 Multi-Agent System & Role Delegation

The main agent can spawn autonomous sub-agents for parallel or background tasks. Each sub-agent runs independently, reports back in real-time, and appears in the sidebar. Complex multi-step workflows run in parallel without blocking the main conversation.

On top of that, an admin can define named role agents — each with its own system prompt, tool subset, LLM profile, reasoning depth and step limit. The main agent gets a single delegate(role, task) tool, hands off a sub-task, waits, and continues with the result. A role can only ever narrow the caller's permissions, never widen them.

💬 Multi-User Chat

A built-in user-to-user chat (/userchat) lets all logged-in users communicate in real-time — with image galleries, audio/video players, file attachments, lightbox preview, and a forward/save context menu.

📎 Multimedia Attachments

Send images, audio, video, and PDFs directly in the Jarvis chat:

• Images are sent to the LLM for visual analysis (all providers supported)
• Audio/Video is transcribed locally via Whisper before the LLM sees it
• PDFs are extracted and injected as text context
• In-chat gallery with lightbox, right-click context menu, and mobile long-press support

📱 WhatsApp Agent

Send Jarvis a voice note or text message on WhatsApp, get a response back. Voice messages are transcribed via faster-whisper (runs locally, no cloud). Perfect for mobile task delegation.

📚 Knowledge Base (hybrid RAG)

Drop PDFs, DOCX files, or plain text into watched folders. Jarvis indexes them into a FAISS index (multilingual-e5-small, 384-dim, cosine) and answers with a hybrid search: two semantic channels plus a lexical BM25 channel, fused by Reciprocal Rank Fusion. Pure embeddings are structurally weak on exact identifiers such as error codes or @STR_UCASE; BM25 covers exactly that. Multi-folder support, incremental re-indexing on file changes, crash-safe resume, and a TF-IDF fallback when no vector stack is available.

🧩 Modular Skill System

Skills are self-contained Python packages that extend Jarvis with new capabilities. Install, enable, disable, and configure them through the UI without touching config files. Compatible with OpenClaw skills.

👁️ Vision & Face Recognition

The optional Vision Skill adds real-time face recognition via dlib/face_recognition. Define per-person actions (webhook, LLM prompt, log-only) with configurable cooldown and tolerance. Works with USB cameras or IP cameras.

🛡️ Security Layer & Sandbox

Built to be opened to a whole team — every restriction is enforced in code, not just requested in the prompt (so it can't be talked around, base64-encoded around, or "learned" around):

• Sandboxed execution for network/domain users — shell commands run as an unprivileged OS user; file access is confined (no system/root/secret paths, symlink-escape safe)
• Prompt-injection, jailbreak & Base64-obfuscation detection across chat, support & WhatsApp (heuristics + LLM classifier)
• Automatic account lockout on repeated attack attempts, with a full, itemized violation log
• A private /tmp per user — all network users share one OS account, so file permissions cannot separate them (0600 would lock out their own run). A bubblewrap mount namespace binds a per-user directory onto /tmp instead: another user's files are not unreadable, they are not present. --unshare-pid hides foreign processes too (measured: