# video-translator Adopt this skill: https://proskills.md/skills/other/shang-zhu-violin-claude-skills-video-translator Page: https://proskills.md/skills/other/shang-zhu-violin-claude-skills-video-translator This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required. ## File: .claude/skills/video-translator/SKILL.md size: 4174 sha: f497937f4932d09ff327a682f21afeb2dcaec5b3 upstream: https://github.com/shang-zhu/violin/blob/main/.claude/skills/video-translator/SKILL.md ````text --- name: video-translator description: Dub a video into another language and generate subtitles using the default Together + Cartesia stack. Trigger when the user wants to translate / dub / voice-over a video file, or generate subtitles for it. Handles `.mp4` / `.mkv` / `.webm`. Installs as the `violin` CLI (and `violin-api` for the FastAPI server) via `uv tool install`. For alternative models (OpenAI / ElevenLabs) or custom configs, point the user to the repo: https://github.com/shang-zhu/violin. allowed-tools: Bash, Read --- # Violin — operating skill Always uses the default config (Together for translation, `cartesia/sonic-3` for TTS). If the user asks for OpenAI, ElevenLabs, or custom configs, **stop and point them to the Violin repo** — those flows aren't supported through the global CLI. ## Pre-flight Run these silently first. Abort if any fails: ```bash command -v violin # 1. CLI on PATH test -f "" # 2. Input exists printenv TOGETHER_API_KEY # 3. Key available ``` If `violin` is missing: tell the user to `uv tool install violin`, then `violin --install-skill` to refresh this skill file. Do not auto-install. If `TOGETHER_API_KEY` is missing: - Inside the Violin repo → populate `.env` (auto-loaded) - Elsewhere → `export TOGETHER_API_KEY=...` in `~/.zshrc` / `~/.bashrc`, then `source` it ## Decisions - **CLI vs API**: single run-and-wait file → CLI (`violin`). Multi-job / HTTP / web UI → API server (`violin-api`); print the command, don't auto-start it. - **Style** (`--style`): default `standard`. Kids content → `kids`, formal/lecture → `academic`, casual → `casual`, dramatic → `storyteller`, news → `news`. Run `violin --style list` if unsure. - **Voiceover**: keep default (mix dubbed audio over a quiet original). Use `--no-voiceover` only when the user explicitly says "replace audio entirely". ## Run ```bash violin --language [flags] ``` ## Flags | Flag | Default | When to set | |------|---------|-------------| | `--language` / `-l` | *required* | Target language (e.g. `Chinese`, `Spanish`, `Japanese`). | | `--voice` / `-v` | auto (native voice picked by `preferences.voice_gender`) | Only when the user names a specific voice from the catalog (e.g. `"warm female narrator"`). Otherwise omit and let the default kick in. | | `--source-language` | `auto-detect` | Only if Whisper mis-detects the source language. | | `--style` / `-s` | `standard` | See Decisions above. | | `--no-subtitles` | off | User says "no SRT" / "video only". | | `--no-voiceover` | off | User says "replace original audio entirely". | | `--config` / `-c` | `config/default.yaml` | Don't use through this skill — repo-only flow. | | `--timings-out` | off | Only when the user wants a per-step timing JSON for debugging / benchmarking. | ## Language coverage 33 target languages total. **16** ship with handpicked native-speaker voices: Chinese, Spanish, English, Hindi, Arabic, Portuguese, Russian, Japanese, Turkish, German, Korean, French, Italian, Polish, Dutch, Swedish. The other **17** fall back to the English voice catalog (multilingual under Cartesia Sonic 3) — quality is decent but the voice isn't a native speaker. Mention this caveat only if the user is translating to a fallback language and asks about voice quality. ## Report back - Output video path + SRT path (printed by the run). - Total cost (printed at end — surface, don't hide). - If voiceover was on, mention the `_original.m4a` sidecar. ## Don'ts - Don't run on multi-GB videos without first quoting the rough cost (audio length × per-provider rates in `pipeline/pricing.py`). - Don't fabricate a "subtitles-only" mode — the CLI requires the full pipeline. If the user only wants SRT, run the full pipeline and hand them just the `.srt`, warning them of the cost first. - Don't try to switch to OpenAI or ElevenLabs from this skill. Point the user to the repo + `--config config/other_api.yaml` (or their own override). - Don't paraphrase the README. For supported languages (33), voice catalog, and full flag docs, point them at `README.md` or `violin --help`. ```` ## File: README.md upstream: https://github.com/shang-zhu/violin/blob/main/README.md ````text # 🎻 Violin **Open-source Video Translation Skill.** [🌐 Live demo](https://www.violin-ai.com) · [📝 Blog post](https://www.together.ai/blog/violin-open-source-translation-skill) · [📜 MIT License](https://github.com/shang-zhu/violin/blob/main/LICENSE)

Violin logo

Upload a video. Violin transcribes the speech, translates it, synthesizes a native-sounding voice-over in the target language, and remuxes it back into the video — fully aligned, with optional SRT subtitles. Available as a **CLI**, a **FastAPI web app**, and a **Claude Code skill**. --- ## ✨ Features - **33 target languages** with handpicked native-speaker voices for the 16 most-used ones (Cartesia Sonic 3 + ElevenLabs) - **In-video Q&A** — ask questions about any moment in the dubbed video; answers use nearby subtitles plus sampled frames - **Natural-language voice picker** — describe the voice you want, an LLM picks from the catalog - **6 style profiles** *(experimental)* — standard / kids / academic / casual / storyteller / news - **Pluggable stack** — Together / OpenAI / ElevenLabs interchangeable for every stage, one YAML --- ## 🚀 Quick start ### Try it without installing anything The live demo runs at **** — drop a short clip in, get a dubbed video out in a few minutes. ### Run locally Requires **Python 3.10+** and **ffmpeg** on PATH. ```bash curl -LsSf https://astral.sh/uv/install.sh | sh # install uv if you don't have it uv tool install violin # recommended — faster, isolated # or: pip install violin # if you'd rather install into your current Python env export TOGETHER_API_KEY=... # get one at https://api.together.ai (add to ~/.zshrc to persist) ``` Three ways to use it: **1. CLI** — translate one file: ```bash violin lecture.mp4 lecture_zh.mp4 --language Chinese ``` **2. Web app** — full REST API + browser UI: ```bash violin-api # → http://127.0.0.1:8000 (browser UI) # → http://127.0.0.1:8000/docs (interactive API docs) ``` **3. Claude Code skill** — invoke from any Claude Code session: ```bash violin --install-skill # one-time: copies the skill into ~/.claude/skills/ claude > please use the violin skill to translate path/to/video.mp4 into Chinese ```
Run from source (for hacking on the pipeline) ```bash git clone https://github.com/shang-zhu/violin.git cd violin uv sync cp .env.example .env # then fill in TOGETHER_API_KEY uv run main.py lecture.mp4 lecture_zh.mp4 --language Chinese ``` To use the `violin` / `violin-api` commands globally while edits to your local source reflect immediately, install editable: ```bash uv tool uninstall violin # if you've installed the PyPI version uv tool install --editable . ``` After this, `violin` / `violin-api` run from your local checkout — edit any file and the next invocation picks it up; no rebuild needed. To switch back to PyPI: `uv tool uninstall violin && uv tool install violin`.
--- ## 📝 To Do List - [-] support voice cloning. - [-] lip sync generation. --- ## 🎬 How Violin works ``` Video │ ├─ ffmpeg ─────────────────────► Extract audio (16 kHz WAV) │ ├─ Whisper Large v3 ────────────► Word-level timestamps → sentence segments │ ├─ LLM (DeepSeek V4 Pro by default) ──► Translate each segment, respecting style profile │ ├─ TTS (Cartesia Sonic 3 by default) ─► Synthesize dubbed audio per segment │ └─ ffmpeg ─────────────────────► Speed-align video to dubbed audio, concat with freeze-frame fallback, single-pass AAC encode the audio track, write output mp4 + optional SRT ``` --- ## ⚙️ Configuration Override any default by writing your own YAML and passing it with `--config my.yaml` — only the keys you want to change need to appear; values deep-merge with the [built-in defaults](https://github.com/shang-zhu/violin/blob/main/config/default.yaml). ### Switch providers ```yaml # config/default.yaml — pick the stack you want models: transcription: provider: together # together | openai model: openai/whisper-large-v3 # together → openai/whisper-large-v3 | openai → whisper-1 translation: provider: together # together | openai model: deepseek-ai/DeepSeek-V4-Pro-0813 # together → deepseek-ai/DeepSeek-V4-Pro-0813 | openai → gpt-5.5 tts: provider: together # together | elevenlabs | openai model: cartesia/sonic-3 # together → cartesia/sonic-3 | elevenlabs → eleven_v3 | openai → tts-1-hd chat: provider: together # together | openai — must be vision-capable model: zai-org/GLM-5.3-Flash # together → zai-org/GLM-5.3-Flash | openai → gpt-5.5 ``` > The `chat` stage is a reasoning model by default, and it spends part of `chat.max_tokens` on thinking before it answers. If you point it at a different model, keep that budget generous (the shipped default is `4096`) — too tight and the API returns an empty answer with `finish_reason: length`. ### Production overrides A starter `config/prod.yaml` is included for public deployments. It adds upload limits, serializes jobs, and caps ffmpeg concurrency. The included `Dockerfile` + `docker-compose.yml` + `Caddyfile` are how the live demo is hosted — `docker compose up -d --build` after filling `.env` is enough to put a copy of Violin behind auto-HTTPS on any Docker host. ### Environment variables | Variable | When required | Description | |----------|---------------|-------------| | `TOGETHER_API_KEY` | **Recommended** — covers every stage with the default config | Together AI API key | | `OPENAI_API_KEY` | Any stage uses `provider: openai` | Covers `whisper-1`, GPT models, and `tts-1` | | `ELEVENLABS_API_KEY` | TTS uses `provider: elevenlabs` | ElevenLabs API key | | `CORS_ORIGINS` | Optional | Comma-separated allowed origins (default: `*`) | > You only need keys for the providers you actually pick. Pure-OpenAI deployments (all stages on `openai`) work too — `OPENAI_API_KEY` alone is enough. Same idea for ElevenLabs. --- ## 🎭 Style profiles Six built-in profiles tune both the translation LLM prompt and the TTS delivery. Use `--style ` on the CLI or pass `style` in API requests. | Style | Tone | TTS speed | Emotion | |-------|------|-----------|---------| | `standard` | Faithful translation, natural voice | 1.0× | — | | `kids` | Rewritten for a 7-year-old, plain language | 1.0× | excited | | `academic` | Formal register, preserves jargon and honorifics | 0.95× | calm | | `casual` | Spoken slang, contractions, friendly | 1.1× | content | | `storyteller` | Vivid, dramatic narration | 0.9× | enthusiastic | | `news` | Concise, declarative, broadcast-style | 1.0× | neutral | Add your own by editing `prompts/styles.yaml`. See all available styles: `violin --style list`. --- ## 💻 CLI usage > Examples use the PyPI-installed `violin` command. If you're running from a git checkout, substitute `uv run main.py` for `violin` (and `uv run run_api.py` for `violin-api`). ```bash # Basic violin lecture.mp4 lecture_es.mp4 --language Spanish # Pick a style violin talk.mp4 talk_zh.mp4 --language Chinese --style kids # Pick a specific voice violin lecture.mp4 lecture_fr.mp4 --language French --voice "french narrator man" # Skip SRT violin lecture.mp4 lecture_ja.mp4 --language Japanese --no-subtitles # Full replacement (no original audio underneath) violin lecture.mp4 lecture_ko.mp4 --language Korean --no-voiceover # Custom config (e.g. switch to OpenAI/ElevenLabs) violin lecture.mp4 lecture_it.mp4 --language Italian --config config/other_api.yaml ``` ### CLI flags | Flag | Default | Description | |------|------- ````