⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
ai-toolkit-trainer
Train custom LoRAs with ostris AI-Toolkit. Covers WAN 2.2/2.1 (people, styles, video motion) and Z-Image (Turbo & Base, low-VRAM image LoRAs). Use when the user wants to train a WAN or Z-Image LoRA; covers local + RunPod setup, dataset prep, key params, and using the result in a ComfyUI workflow.
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
Not yet listed on ClawHub or SkillsMP
// README
comfyui-mcp — drive ComfyUI with ANY LLM
The local-first, agent-native control plane for ComfyUI. An MCP server plus a live sidebar agent that generates images, video and audio, authors and runs workflows, manages models and custom nodes, and edits your live ComfyUI graph in natural language. Bring whatever model you have: Claude or ChatGPT on your subscription, Gemini on your Google login, a free local model via Ollama (fully offline), or any hosted model over one API key (DeepSeek, GLM, MiMo, Kimi, GPT, Claude via OpenRouter). The tools and the panel are the same on every tier, and the built-in LLM Arena scores each model on real ComfyUI tasks so you know what yours can do. One config targets local installs, LAN, VPS, or Comfy Cloud.
One-click GPU pod. A ready-to-run ComfyUI with this project + Agent Panel + ComfyUI-Manager v2 preinstalled, on your own GPU. No setup.
Works on macOS, Linux, and Windows. Auto-detects your ComfyUI installation and port.
Stuck or have a question? Join the Discord for help, model tips, and release announcements.
38 MCP tools | 42 AI skills (Flux · WAN · LTX 2.3 video · MiniMax H3 · Qwen · Z-Image · Ideogram 4 · ERNIE · ANIMA · model registry · Civitai · node authoring · launch/perf flags) | 56 installer packs | 11 slash commands | 4 autonomous agents | 3 hooks
The plugin ships skills that grow with every release: model-specific generation guides with curated download URLs, workflow recipes, troubleshooting, and custom-node authoring. With them, Claude knows the right sampler, CFG, resolution, and model files for each architecture without trial and error.
Now available: the ComfyUI Agent Panel on ComfyUI-Manager & the Comfy Registry
An autonomous AI agent in your ComfyUI sidebar, running on Claude, ChatGPT, Gemini, or ANY local/hosted LLM (Ollama and every OpenAI-compatible endpoint). Subscriptions work with no API key; local models work with no account at all. Pick a provider and it drives your live graph: edits, spatial layout, one-shot workflow/pack loads, rewind/rollback, a pending-message tray, activity cards, multi-tab. It asks before spending paid API credits. Search
comfyui-agent-panelin ComfyUI-Manager to install. Read more →
Full documentation: comfyui-mcp.artokun.io/docs
Quick Start
1. Install ComfyUI (if you haven't already): ComfyUI Desktop or from source
2. Add the MCP server to your Claude Code config (~/.claude/settings.json):
{
"mcpServers": {
"comfyui": {
"command": "npx",
"args": ["-y", "comfyui-mcp"],
"env": {
"CIVITAI_API_TOKEN": ""
}
}
}
}
3. Start using it. With ComfyUI running, ask Claude to generate an image:
> Generate an image of a sunset over mountains
Claude will find (or download) a checkpoint, build a workflow, execute it, and return the image.
Note: This runs as a standalone MCP server, so there is no need to clone this repo.
npxdownloads and runs it.
Scope: local, remote, or Comfy Cloud
comfyui-mcp is local-first. A self-hosted ComfyUI on Mac/Linux/Windows is the primary target, and the same agent reaches remote installs (RunPod, VPS, LAN, reverse-proxied) from one config. Local-first, not local-only.
More than a bridge. Most ComfyUI MCP servers are thin connectors. They forward a prompt and hand back an image. comfyui-mcp is a full control plane. It authors and edits the graph node by node, runs and iterates on workflows, manages models and custom nodes, and ships model-specific expertise (samplers, CFG, resolutions, curated model URLs) so the agent gets it right without trial and error. If you want a minimal local relay, a lightweight server is fine. If you want an agent that operates ComfyUI, that's this project.
For Comfy Cloud users, Comfy-Org ships its own agent tooling: Comfy Cloud MCP (public beta, hosted on Comfy Cloud GPUs), the Comfy In-App Agent (private alpha, inside Comfy Cloud), and a first-party Comfy Local MCP (private test, not publicly available yet), all maintained by the Comfy team. If you don't have a GPU or you want zero setup, that's the better path; go use it.
Where this project differs is that it runs on your install and your choice of model, including a free local one via Ollama with no account and no network at all. comfyui-mcp also includes a community cloud mode (set COMFYUI_API_KEY, see Deployment modes) so a single MCP can target all three deployment shapes from one config.
Local vs. Comfy Cloud agent is a side-by-side comparison, including when Comfy Cloud is the right answer. (Statuses above are as of July 2026. Comfy-Org ships fast, so check their docs for the current state.)
Remote / hosted connector (one command)
To use comfyui-mcp from Claude Desktop's Custom Connectors or any remote
client, like Comfy's own cloud.comfy.org/mcp connector, run it as an
authenticated, publicly reachable Streamable-HTTP server with one flag:
npx -y comfyui-mcp@latest --tunnel
This forces the HTTP transport, generates an auth token, opens a
cloudflared quick tunnel, and prints
a ready-to-paste https://…/mcp URL + token + Claude Desktop connector snippet.
Auth accepts Authorization: Bearer <token> or X-API-Key: <token> (matching
Comfy Cloud's convention). See the
Remote / hosted connector guide
for the full walkthrough and headless usage.
Auth is opt-in. With no
COMFYUI_MCP_HTTP_TOKENset and no--tunnel, the default stdio (and plain--httpon loopback) behavior is unchanged. It stays open and local. OAuth (Comfy's browser sign-in flow) is a planned follow-up.
Claude Code Plugin
This package also ships as a Claude Code plugin that adds slash commands, skills, agents, and hooks on top of the MCP tools.
Install as a pl
// HOW IT'S BUILT
// FILES
2 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.
plugin/skills/ai-toolkit-trainer/SKILL.md11 KBbbba435e034dupstream ↗
--- name: ai-toolkit-trainer description: Train custom LoRAs with ostris AI-Toolkit. Covers WAN 2.2/2.1 (people, styles, video motion) and Z-Image (Turbo & Base, low-VRAM image LoRAs). Use when the user wants to train a WAN or Z-Image LoRA; covers local + RunPod setup, dataset prep, key params, and using the result in a ComfyUI workflow. globs: - "**/*.json" --- # AI-Toolkit LoRA Trainer (WAN 2.2 & Z-Image) ## Overview AI-Toolkit by ostris is an MIT-licensed trainer for finetuning diffusion models. It is a standalone trainer with its own web UI, not a ComfyUI custom node. It runs a Node.js UI front end over a Python (`run.py`) training backend and trains LoRAs for many model families. This skill covers the WAN 2.2 / 2.1 video models and Z-Image (Turbo & Base). - Repo: `https://github.com/ostris/ai-toolkit` (cloned by the installers). - Backend: `python run.py config/<job>.yml`. UI: a Node.js app under `ui/` that schedules and monitors jobs. You do not have to keep the UI open while a job runs. - Output: a standard `.safetensors` LoRA you drop into ComfyUI `models/loras/` and load with `LoraLoaderModelOnly`. Best for: - WAN LoRAs. A person or character, an art style, or a specific camera or video motion, trained from image or video clip datasets. For *using* WAN see wan-t2v-video / wan-flf-video. - Z-Image LoRAs. Fast, very low-VRAM image LoRAs (faces, characters, outfits, styles) on the 6B Z-Image base/turbo. For *using* Z-Image see z-image-base / z-image-turbo, and the z-image-xy-plot pack to compare trained LoRAs. For low-VRAM anime image LoRAs on a different stack (kohya `sd-scripts`), see the sibling anima-lora-trainer. > Two LoRA kinds for WAN. A WAN image LoRA trains on still images; it is cheaper (~24GB-class) and suits identity or style. A WAN video LoRA trains on short clips; it is heavier, best run on cloud, and suits *motion*. Z-Image is image-only. ## Install The installer comes in two generations. Both clone `ostris/ai-toolkit`, set up Torch for your GPU, and launch the web UI. Put it in a folder whose full path has no spaces (e.g. `C:\AI-Toolkit`). - V1, `AI-TOOLKIT_AUTO_INSTALL.bat`, expects Git, Python 3.10.x, and Node 18+ already in PATH. - V2, `AI-TOOLKIT_AUTO_INSTALL-V2.bat` (recommended), uses an embedded Python 3.10.11, auto-installs Git and Node, builds a clean PATH without your system Python, and adds aggressive pip/curl retries. It has far fewer prerequisites and fails less often. The Z-Image Turbo LoRA training release used it. Both are CUDA-aware and select the Torch wheel by GPU generation: | Choice | GPU | CUDA | Torch index | Torch packages | |--------|-----|------|-------------|----------------| | 1 | RTX 50-series (Blackwell) | **12.8** | `https://download.pytorch.org/whl/cu128` | `torch==2.7.0 torchvision==0.22.0` | | 2 | RTX 40 / 30 / 20 and older | **12.6** | `https://download.pytorch.org/whl/cu126` | `torch==2.7.0 torchvision==0.22.0` | Each then clones `ostris/ai-toolkit`, downloads two launcher scripts (`LAUNCHER-TOOLKIT.bat`, `SECURE_LAUNCHER-TOOLKIT.bat`, from `https://huggingface.co/Aitrepreneur/FLX/resolve/main/`), makes the venv, installs Torch from the chosen index, runs `pip install -r requirements.txt`, then `cd ui && npm run build_and_start`. ### RunPod / Linux — `AI-TOOLKIT_AUTO_INSTALL-RUNPOD.sh` (and `-V2.sh`) Installs into the persistent volume `/workspace/ai-toolkit`. It is idempotent; a re-run just relaunches the UI. Use RunPod's PyTorch 2.8.0 template and a 100GB disk. It installs apt deps, clones the repo, makes a venv, installs Torch (`torchaudio` included), installs nvm + Node 22, then builds and starts the UI. | Choice | GPU | Stream | Torch spec | |--------|-----|--------|-----------| | 1 | RTX 5000-series (Blackwell) | `cu128` | `torch==2.7.0+cu128 torchvision==0.22.0+cu128 torchaudio==2.7.0+cu128` | | 2 | Ada / Hopper / Ampere, older | `cu126` | `torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0` | The UI listens on 8675 and Jupyter on 8888. Set `AI_TOOLKIT_AUTH` (UI password) before launch. Reach it at `https://${RUNPOD_POD_ID}-8675.proxy.runpod.net`. Use an RTX 4090/5090 for image (WAN t2i/t2v, Z-Image) LoRAs and an RTX 6000 Pro (Blackwell) for heavy WAN video, high-res, or high-rank jobs. ## Launching the web UI - On Windows, run `LAUNCHER-TOOLKIT.bat` (local) or `SECURE_LAUNCHER-TOOLKIT.bat` (password-protected) from the `ai-toolkit` folder. - On RunPod, rerun the `.sh`. It detects the install and starts the UI on :8675. In the UI, create a Job, point it at a dataset folder, pick the model (WAN variant or Z-Image), set params, and start. Jobs run in the Python backend, so you can close the browser. To bypass the UI, copy a `config/examples/*.yml`, edit it, and run `python run.py config/<job>.yml`. ## Dataset preparation AI-Toolkit pairs each sample with a same-basename `.txt` caption and auto-resizes/buckets aspect ratios (no pre-cropping). ### Image LoRA (WAN identity/style, or Z-Image) ``` my_dataset/ 001.png 001.txt 002.jpg 002.txt ``` - Captions are natural language. Include a unique trigger word for a person or character. - Use about 15 to 40 varied images for a person, more for a broad style. ### Video LoRA (WAN motion only) Short clips plus a `.txt` per clip; caption the motion or camera move. Set per-clip frames via the job's `num_frames` (e.g. 81). This is markedly heavier, so prefer cloud GPUs. ## Key training params ### WAN 2.2 WAN 2.2 14B is a Mixture-of-Experts with a high-noise expert (structure/motion) and a low-noise expert (detail). AI-Toolkit trains both via Multi-stage. | Param | Default | Notes | |-------|---------|-------| | Linear rank / dim | **16** | 16 simple; 16–32 complex/cinematic | | Learning rate | **5e-5** (identity) | 7e-5–1e-4 style; high LR → plasticky skin | | Steps | **1500–2500** | stop before overbaking | | Resolution | **512** (or 768) | bucketed; 768 costs more VRAM | | `num_frames` (video) | **81** | per-clip frame count | | Multi-stage | **High + Low = ON** | trains both experts | | Switch Every | **10** | raise to 20–50 if offload swapping is slow | | Optimizer / Quant | AdamW8bit / 4-bit ARA or float8 | fits 14B on consumer cards | ### Z-Image (Turbo & Base) Z-Image is a ~6B single-stream model with no hi/lo multi-stage. Leave Multi-stage OFF; you train one model. It is the lightest target here. The headline of the Z-Image releases is training on very low VRAM. | Param | Starting point | Notes | |-------|----------------|-------| | Linear rank / dim | **16–32** | 32 for detailed characters/styles | | Learning rate | **1e-4** | lower (5e-5) for tighter identity | | Steps | **1500–3000** | dataset-dependent | | Resolution | **768** (or 1024) | Z-Image's native range | | Multi-stage | **OFF** | single-stream model, not WAN's MoE | | Optimizer / Quant | AdamW8bit / float8 | enables sub-12GB training | > Train on Base, deploy anywhere. Z-Image Base is the finetuning-friendly model; a LoRA trained on Base generally applies to the Turbo workflow too. Use the z-image-xy-plot pack to grid-compare your trained LoRAs. > The param tables are aggregated starting points from community and training-guide sources, not read from the repo's `config/examples/*.yml`. Open the actual WAN / Z-Image example config in your clone and tune. See "Unverified". ## VRAM / GPU guidance - Z-Image image LoRA is the lightest. It trains on modest consumer GPUs with quantization (the releases describe very-low-VRAM training); a 4090 is comfortable, and smaller cards work with float8 at 512 to 768 res. - WAN image LoRA (t2i/t2v) needs 24GB+ locally with quantization. Below that, use RunPod. - WAN video LoRA, high res, or high rank is heavier. Use cloud (RTX 5090, or RTX 6000 Pro Blackwell / H100). - Memory savers: quantization, batch size 1, 512 res, and (WAN) raising Switch Every. ## Using the trained LoRA in ComfyUI 1. Copy `<your_lora>.safetensors` into ComfyUI `models/loras/`. 2. Load with `LoraLoaderModelOnly`: - WAN 2.2 is dual hi/lo. Apply the LoRA to both the HighNoise and LowNoise model branches (like lightning/concept LoRAs in wan-t2v-video). Typical strength 0.5 to 1.0. - Z-Image is a single model. Use one `LoraLoaderModelOnly` on the Z-Image model path (see the z-image-base / z-image-turbo packs). Strength 0.7 to 1.0. ```json { "class_type": "LoraLoaderModelOnly", "inputs": { "model": ["<base_model>", 0], "lora_name": "<your_lora>.safetensors", "strength_model": 1.0 } } ``` 3. Prompt using the trigger word or caption style you trained with. For WAN motion LoRAs, describe the same camera or motion. ## Troubleshooting - **`No module named 'torchaudio'` when starting a job (AI-Toolkit).** The venv's Torch stack is mismatched. Activate the AI-Toolkit venv (`venv\Scripts\activate`), then `pip uninstall torch torchaudio torchvision -y` and `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121` (or your CUDA's index). This only affects the AI-Toolkit install, not ComfyUI. - **`self and mat2 must have the same dtype` (ComfyUI-WanVideoWrapper, WAN usage).** Re-clone `ComfyUI-WanVideoWrapper` in `custom_nodes/` and reinstall its `requirements.txt`, then restart ComfyUI. - **5000-series (Blackwell) onnxruntime "QuickGelu" / CUDA error.** `pip install onnxruntime==1.20.1` in the affected venv. - **Pascal/Maxwell GPUs (GTX 9xx/10xx).** Recent Torch (cu128/cu130) dropped them. Reinstall the cu126 Torch build into the venv. - **Path with spaces (Windows).** Keep the install path space-free or the build/launch fails. - **OOM during training.** Quantization (4-bit ARA / float8), 512 res, batch 1, (WAN) raise Switch Every, or a bigger RunPod GPU. - **RunPod UI won't load / asks for a password.** Confirm `AI_TOOLKIT_AUTH` is set and you're on the 8675 proxy URL. ## Unverified / verify before relying - The param tables (both WAN and Z-Image) are synthesized starting points, not read from the repo's `config/examples/*.yml`. Open the actual example config in your clone and adjust. - The release notes describe the Z-Image training VRAM floor only qualitatively ("very low VRAM"). Confirm against your card; quantization plus 512 to 768 res is the lever. - The Windows UI port is whatever the launcher binds (the installer doesn't print it; check the launcher window). RunPod 8675/8888 are per the template. - The launcher `.bat` files are downloaded from a third-party HuggingFace repo (`Aitrepreneur/FLX`); review before running on a security-sensitive machine. - Model weights are fetched at job time by AI-Toolkit/HF, not by the installer. Confirm the model selector lists your target WAN variant or Z-Image model before a long run. ## Sources - **Official:** https://github.com/ostris/ai-toolkit - **Empirical:** Windows/RunPod installer steps and VRAM notes from the pack installers, not the vendor's training guide.README.mdupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
Plain-text copy, no login: /skill-bundles/artokun-comfyui-mcp-plugin-skills-ai-toolkit-trainer.md