tein-video-workflow

A local UI and agent-assisted workflow for replacing video subjects using colored depth, SAM 3 masks and Seedance through Enhancor. Draft first; approve before 1080p. The final export restores the entire original source audio, not generated audio.

Get access

• Sign up at TEIN.ai for early access: “The cheapest APIs, open source workflows, and more.”
• Get the Seedance API at Enhancor.ai — this workflow uses its Seedance integration with human-face support (pass_faces: true), draft previews, and approved 1080p generation. Get your own key from the API dashboard. Current the lowest price in the market; with unrestricted use.

Easiest setup: let your coding agent install it

Extract the ZIP, open this folder in Codex or another coding agent with terminal/file access, and paste:

Set up this TEIN Genjustsu — Open Source Workflow project for me. Read docs/SETUP-AGENT.md and follow it end to end. Detect my OS, install missing Python/Rubber Band/cloudflared prerequisites using the supported package manager, create the main virtual environment, install dependencies, install Python 3.12 and run bash setup-face-mesh.sh for both mesh modes, and create .env without overwriting it. Open .env so I can enter my own keys privately. Run the local tests, launch the app and its callback-only tunnel, run the strict readiness check on a free localhost port, verify the UI and companion images, and give me the working URL. Work independently through setup, respecting system permission prompts. Do not submit paid generations. If keys are missing, launch the UI and clearly tell me generation is waiting for them.

The launcher starts a callback-only Quick Tunnel when no custom webhook is configured. Keep it running with the app. The agent uses the included setup runbook. You only need to supply your own provider keys and approve any required system installation. No separate agent install is required. Native Windows users should use WSL2; macOS/Linux launch scripts are included.

Manual setup

1. Install Python 3.12+ (the development runtime was Python 3.14) and Rubber Band CLI:
   • macOS: brew install rubberband cloudflared
   • Ubuntu/Debian or WSL: sudo apt install rubberband-cli python3-venv
   • Install cloudflared too, unless you already have a public HTTPS webhook receiver. On launch it creates a temporary callback-only tunnel; the UI and media stay on localhost.
2. In this folder, run bash setup.sh. This creates an isolated .venv, installs requirements.txt, creates a blank .env, and checks dependencies. To enable all three modes, install Python 3.12 and run bash setup-face-mesh.sh, then .venv/bin/python doctor.py --mesh. On macOS, install the worker interpreter with brew install [email protected] if needed.
3. Fill .env with your own ENHANCOR_API_KEY and REPLICATE_API_TOKEN. Get them from Enhancor and Replicate. Fund/enable the relevant API accounts. HF_TOKEN is optional for the Hugging Face depth alternative.
4. Run ./start.command (or bash start.command). Open http://127.0.0.1:8770. Use PORT=8771 bash start.command if the port is occupied.
   • Once running, check .venv/bin/python doctor.py --require-keys. Enhancor requires a webhook URL: launch.py supplies one through cloudflared, or you can set ENHANCOR_WEBHOOK_URL yourself. Keep the app/tunnel running.
5. Upload your own MP4/MOV, prepare it, inspect the entire mask and prepared video, approve that exact mask, provide your own character references, review the prompt, and create a draft.
6. When you like the draft, click the 1080p approval button. The completed draft and HD result remain separate; both final previews use source audio.

No local model weights or GPU are needed for the default hosted setup. Python packages and the Rubber Band executable are required locally. No API keys, signed media URLs, or old jobs are included. Four user-selected demonstration videos are included in examples/. The bundled TEIN companion artwork is part of the UI.

What happens

1. Save the source unchanged; extract audio and separate vocals/music with Demucs on Replicate.
2. Pitch isolated vocals +3 semitones, maintaining duration with Rubber Band.
3. Normalize the working video to 24 fps, up to 960 pixels; obtain colored depth with lucataco/depth-anything-video. Preserve returned colors.
4. Use SAM 3 raw PNG masks on depth. Check alignment, coverage and background spill. An agent can retry segmentation on the original RGB if depth loses detail.
5. Composite the depth subject over the original background. Mandatory full-clip visual mask review gates every generation. Approval is bound to hashes of the current artifacts.
6. Embed the pitched vocals into the composite MP4. Send this video, no standalone audio, with character references.
7. Enhancor Seedance Edit for image references; Omni / multi_reference for a video identity reference. Generate a draft by default.
8. Poll for completion, download the result, discard its audio and copy the complete original soundtrack into the final MP4.
9. After explicit HD approval, complete the accepted draft at 1080p and restore the soundtrack again.

Autonomous agent operation

Open this folder in your coding agent and ask it to follow AGENTS.md and .agents/skills/tein-video-workflow/SKILL.md. No particular agent vendor or model is required, but it must be able to inspect video and files, run Python, and access the network. An agent service is not bundled or silently started. The UI/server automates processing and result collection; the agent or user performs visual QA and creative review.

Example: “Use the Genjustsu workflow for my clip and references. Prepare and inspect the entire mask, repair it if needed, show the prompt, submit the authorized draft, and retrieve the final preview. Wait for my approval before HD.”

See agent operations, API integration, models, media hosting, and troubleshooting.

Recovery and checks

.venv/bin/python doctor.py --require-keys
.venv/bin/python test_composite.py
.venv/bin/python -m unittest discover -s tests
.venv/bin/python workflow.py status LOCAL_JOB_ID
.venv/bin/python workflow.py resume LOCAL_JOB_ID
.venv/bin/python workflow.py resume LOCAL_JOB_ID --hd

The server resumes saved Se