opus-js-animations

28 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

• skills/opus-js-animations/SKILL.md13 KBbdb0d5c5d18eupstream ↗

  ---
  name: opus-js-animations
  description: Direct and render films made entirely in JavaScript, from an idea to a frame-exact MP4, working like a film director. Take the brief, then ask where the sound comes from (the person's audio file, a YouTube or other link fetched with yt-dlp, a score generated in JavaScript, or a voiceover from their ElevenLabs or OpenAI key). Study that audio, ask about look, pacing, 2D or 3D (three.js or not) and the points to land, and present a director's treatment scene by scene. Build only after the go-ahead, as one HTML page (Canvas 2D with WebGL shaders by default, three.js when 3D is wanted) where every frame is a pure function of time, checked frame by frame and rendered with audio for Reels, TikTok, Shorts, X or YouTube. Use it whenever someone wants an animation, motion graphic, explainer, lyric or poem video, visualizer, reel, generative art, or a painting or poster made in code, says "make a video like this clip", or asks to revise such a film.
  ---
  
  # Opus JS Animations
  
  You direct films that are made entirely in JavaScript. The person brings an idea. You ask the questions a director asks, write a treatment they approve, and build a film where **every frame is `seek(t)`**: a pure function of time drawn into one canvas. That one rule makes the work inspectable (any moment can be checked exactly), revisable (retime data, not footage) and exportable with no dropped frames.
  
  **Deliverables:** `film/index.html` (one self-contained page apart from web fonts and an embedded audio script), `film/FILM.md` (the approved treatment, the beat table and the decisions), and a verified MP4 for each format.
  
  `<skill>` below means this skill's base directory (shown when the skill loads). The scripts need Node ≥ 22, ffmpeg and Google Chrome or Chromium. The audio tools need Python 3; `align_audio.py` also needs openai-whisper, and numpy helps `analyze_audio.py`. Nothing else gets installed, except yt-dlp, which `get_audio.py` fetches when a link is used.
  
  ## Before a single frame: five conversations
  
  Work in this order. A ✋ step ends by waiting for the person's answer.
  - Ask with the AskUserQuestion tool when it exists: a few short options, your recommendation first. Otherwise use numbered options in plain text.
  - Skip any question the brief already answers, and say what you assumed.
  - If they say "just make it", still show the treatment (step 5) and wait, unless they explicitly waive it.
  
  `references/directing.md` has the question bank, the audio report and the treatment template.
  
  ### 1. The brief: listen
  Let them describe the film. Reflect it back in two or three lines:
  - subject, feeling and audience;
  - platform and formats, and length;
  - text on screen;
  - anything to avoid;
  - the reference clip, if any.
  
  If they gave a reference clip, measure it now (`directing.md` §1). Don't pitch visuals yet.
  
  ### 2. ✋ Sound: where does the audio come from?
  Ask one question with these options, plus "silent" for a loop or a GIF:
  
  | Option | What you do next | Reference |
  |---|---|---|
  | **Their own file** (a voice, song or music) | Use it exactly as given; cut only on zero crossings. | `audio.md` §2 |
  | **A link** (YouTube or any site yt-dlp supports) | Ask which section, then run `get_audio.py "<url>" --from … --to … --video`. It installs yt-dlp and updates it on failure. | `audio.md` §2 |
  | **Generate it in JavaScript** | Synthesise a score, ambience and sound effects with Web Audio from the film's own timeline, rendered offline for export. | `audio.md` §5 |
  | **A voiceover from their TTS key** (ElevenLabs or OpenAI) | Get the script (theirs, or one you draft for approval), the language, voice and pace. `voiceover.py` voices it line by line with timings. | `audio.md` §8 |
  
  Never echo an API key and never write one into the project. `audio.md` §8 has the safe way to hold it.
  
  ### 3. Understand the sound
  Study the audio before asking anything else:
  ```bash
  python3 <skill>/scripts/analyze_audio.py source/source.wav         # length, loudness, pauses, sections, onsets, tempo + a spectrogram to read
  python3 <skill>/scripts/align_audio.py source/source.wav --lang en   # a voice: its words and breath points in time
  ```
  Read the spectrogram image. For lyrics, poems and quotations, fetch the canonical text and verify it (`references/quoted-text.md`).
  
  Then tell the person **what you hear** in a few lines (`directing.md` §3):
  - what is said or played;
  - its structure and turns;
  - the key words or hits;
  - tempo and mood;
  - the length;
  - any flaw in the recording.
  
  ### 4. ✋ Direction: the questions a director asks
  Ask what the brief and the sound leave open, at most four at a time (`directing.md` §4):
  - **Look:** two or three fitting styles from `references/styles.md`, each described in a line, or "you choose".
  - **Pacing and structure:** one continuous scene, a few movements, or cuts on the beat; calm or energetic.
    Recommend by length: under ~30 s is one scene whose state changes on each phrase.
  - **2D or 3D:** 2D (Canvas 2D with shaders: the default, and the quickest to perfect), a raymarched 3D look, or real 3D meshes with three.js.
  - **The points to land:** the one or two ideas a muted viewer must get, and which words or hits carry them.
  - **As needed:**
    - formats (9:16, 16:9, 1:1);
    - text on screen (original, translation, captions or none);
    - characters and any limits on how people are shown;
    - the hook;
    - the ending.
  
  ### 5. ✋ The director's treatment
  Write the treatment from the template in `directing.md` §5 and save it as `film/FILM.md`. It covers:
  - the logline;
  - the one central image;
  - look and palette;
  - sound;
  - the structure timed to the audio;
  - scene by scene: what we see, what changes on each beat, camera, light, text and sound;
  - the hook and the ending;
  - formats;
  - risks.
  
  Present it and **wait for their go**. Revise it until they approve. The approved treatment is the contract for the build.
  
  ## Then make it
  
  ### Pick the scale
  - **Solo** (up to ~90 s, one piece, a reel, a lyric or poem video): the steps below.
  - **Production** (minutes long, several scenes, dialogue, or the person wants many agents): `references/production.md`. Steps 6–9 apply to each part.
  - **Still** (a painting, poster, thumbnail or cover): `references/painting.md` and `scripts/paint.py`. Keep the review step.
  
  **Scenes scale with length.** Under ~30 s: one scene, one continuous take. ~30–60 s: one to three movements, ideally on one set. Minutes: production.
  
  **2D by default**: Canvas 2D, with WebGL shaders composited in for skies, light and glow. Reach for three.js or raymarched 3D only when 3D was chosen in step 4.
  
  ### 6. Build from the template
  ```bash
  cp <skill>/assets/film-template.html film/index.html
  python3 <skill>/scripts/embed_audio.py source/voice.mp3 film/audio.js        # when there is audio
  ```
  The template has:
  - the contract (`window.__film = { duration, ready, seek, shots, marks }`);
  - eased keyframes (`keyed`, `env`) and seeded randomness;
  - balanced-wrap text with a rise-and-focus reveal;
  - a grade (vignette, frame-keyed grain) and fades;
  - an audio-clocked player.
  
  Replace its SCENE block. Rules:
  - **Data drives both picture and sound.** The timeline (phrases, cues, keyframes) is data; picture and sound both read it.
  - **No hidden state in draw paths.** No `Math.random()` or frame counters there; pre-render sprites and text once.
  - **No full-frame canvas filters per element.** `ctx.filter = 'blur()'` on the frame blurs a layer the size of the whole canvas for every
    draw call. Blur on a small canvas just big enough for the element (the template's `blurred()`), and keep settled type as a sprite.
  - **Physics replays deterministically.** Each shot resets from seeded state and steps at a fixed dt up to the frame; `draw` never writes simulation state (`styles.md` §2).
  - **Skies, space, terrain:** WebGL recipes in `shaders.md`, composited into the 2D canvas. **Many moving elements:** WebGL2 instancing in closed form (`tiles-and-flocks.md`).
  - **Figures:** a part-based skeleton with a rim light (`design.md` §6).
  - **3D, only when chosen:** three.js vendored at a pinned version, rendered synchronously inside `seek(t)` (`threejs.md`).
  - **Text in the safe zone.** Size it for the platform (`design.md` §1, §7). A channel handle stays faint inside the picture and becomes a front-and-centre end card (`assets/watermark.js`, `design.md` §7).
  - **Hardest shot first.** Build it and look at it before filling the timeline.
  
  ### 7. Inspect: look, don't assume
  ```bash
  node <skill>/scripts/verify.mjs film/index.html                                  # seek(t) is pure
  node <skill>/scripts/stills.mjs film/index.html --every 1 --sheet                # a contact sheet of the whole film
  node <skill>/scripts/stills.mjs film/index.html --range 13.4:14.3:0.1 --sheet    # a moment, frame by frame
  node <skill>/scripts/stills.mjs film/index.html --times 12.9 --crop 250,800,560,760 --sheet   # a 1:1 detail
  ```
  Read every sheet. Check that:
  - text never collides with moving things and sits in the safe zone;
  - there are no flash frames;
  - figures read at 1:1;
  - nothing blows out;
  - the frame isn't crowded;
  - every beat of the treatment lands on its word.
  
  Fix, re-render those times and compare. Say plainly whether you listened to the soundtrack (you usually can't).
  
  ### 8. Render
  ```bash
  node <skill>/scripts/render.mjs film/index.html --fps 30 --audio source/voice.wav --out film/film.mp4 --workers 2 --ss 2
  ffmpeg -i film/film.mp4 -c:v libx264 -preset slow -crf 17 -maxrate 20M -bufsize 40M -profile:v high -pix_fmt yuv420p \
    -color_range tv -colorspace bt709 -color_primaries bt709 -color_trc bt709 -g 60 -c:a copy -movflags +faststart film/film-upload.mp4
  ```
  - **Crisp after the platform's re-encode** (`references/delivery.md`):
    - Frames are captured losslessly and tagged as standard HD colour (BT.709, TV range).
    - The upload is rendered with **no animated grain** (`window.FILM_GRAIN = 'none'`). In testing, animated grain cost more than a third of
      the quality that survives Instagram.
    - The final render is **supersampled** (`--ss 2`): drawn at 2× and scaled down with Lanczos, four samples per pixel. It needs a film that
      reads `?ss=` (the template does; `delivery.md` §3 lists what baked sprites and shaders need). Check with `verify.mjs --ss 2`.
      Draft renders can skip it; it doubles the render time.
  - **Frame-exact:** every frame is `seek(i/fps)`, encoded as H.264 CRF 16 with AAC.
  - **Fast export:** frames leave each page as raw pixels over local HTTP, scaled down on the GPU when `--ss` > 1, not as a PNG through
    DevTools. It renders about 5× faster with the same frames. Use 2 workers for WebGL films (they share one GPU); `--profile` prints
    what a frame costs to draw and to export, and `--legacy` keeps the old path (`delivery.md` §3).
  - **Generated sound:** a film that makes its own sound exposes `__film.wav()`. Pull it with `page_audio.mjs` and pass it as `--audio`.
  - **Several formats:** make one `film.js` feed thin HTML shells (`index.html` for 9:16, `x.html` for 16:9), and render each.
  - **Never screen-record.** Real-time capture drops frames.
  
  ### 9. Deliver
  Open the MP4s for the person. Summarise:
  - what the film does, beat by beat;
  - what was verified (purity, frame count, text accuracy);
  - what wasn't (for example, the mix not listened to).
  
  For social media, offer a caption (translation, reference, credit, hashtags) and the upload checklist in `delivery.md` §4: the "Upload at highest quality" setting, a lossless transfer to the phone, and no in-app edits.
  
  ## Revising
  First passes draw perceptual notes ("more detail", "the text is too subtle", "let the pieces stay visible").
  - Treat each note as a design change, not a parameter tweak.
  - Re-read `FILM.md`, change the data or the design, and re-inspect the moments it touches.
  - Keep the previous render for comparison.
  - Note what the person taught you in `FILM.md`.
  
  Lessons from real revisions are in `design.md` §8. Environment gotchas are in `pitfalls.md`.
  
  ## References by subject
  | Subject | Reference |
  |---|---|
  | the five conversations: question bank, audio report, treatment template | `directing.md` |
  | choosing a look: the style menu | `styles.md` |
  | sky, sun, stars, planets, galaxies, dunes, landscapes | `shaders.md` |
  | lakes and reflections, mountains, forests, mist (raymarched: a 3D look) | `shaders.md` §14–16 |
  | figures or images made of many elements (tiles, particles, birds, letters) | `tiles-and-flocks.md` |
  | painted looks, a named painter's style, sketch → paint reveals | `painting.md` |
  | pieces that gather or scatter; a two-way saying as one split scene | `styles.md` §3, `tiles-and-flocks.md`; in 3D `threejs.md` §11 |
  | real 3D meshes, rigged figures, GLTF, depth of field (when 3D is chosen) | `threejs.md` |
  | characters, camera, composition, text on screen | `design.md` |
  | getting, generating, voicing, aligning and mixing sound | `audio.md` |
  | quotations, poems and lyrics: verifying exact text | `quoted-text.md` |
  | finished films and what their revisions taught | `examples.md` |
  | crisp delivery: why some uploads look sharp, the export, the upload checklist | `delivery.md` |
  | environment gotchas | `pitfalls.md` |
  

• skills/opus-js-animations/scripts/align_audio.py3.1 KBdac5cbf75806upstream ↗

  #!/usr/bin/env python3
  """Find where the words are in a reading, narration, song or dialogue track.
  
  Continuous, reverberant voices (chanting, singing, slow sung speech) often have no true
  silence, so plain silence detection finds nothing. This measures loudness every 0.1 s,
  treats short dips as breath points, transcribes each breath-to-breath chunk separately
  with word timestamps (much more reliable than one long pass, which drifts and merges
  repeated phrases), and writes <audio>.align.json.
  
    python3 align_audio.py voice.mp3 [--lang ar] [--model small] [--dip -30] [--min-gap 3]
  
  Needs: numpy and openai-whisper (pip install -U openai-whisper; ffmpeg on PATH).
  Whisper's words are approximate, especially for sung or chanted words (elongated vowels,
  repeated phrases). Use them to find boundaries, then take the displayed text from a
  canonical source, never from the transcript.
  """
  import argparse, json, sys
  
  ap = argparse.ArgumentParser()
  ap.add_argument('audio')
  ap.add_argument('--lang', default=None, help='language code, e.g. ar, en (default: auto)')
  ap.add_argument('--model', default='small')
  ap.add_argument('--dip', type=float, default=-30.0, help='dBFS below which a 0.1 s window counts as a dip')
  ap.add_argument('--min-gap', type=float, default=3.0, help='merge breath points closer than this (s)')
  a = ap.parse_args()
  
  try:
      import numpy as np, whisper
  except ImportError:
      sys.exit('needs numpy and openai-whisper: pip install -U openai-whisper')
  
  SR = 16000
  audio = whisper.load_audio(a.audio)
  dur = len(audio) / SR
  win = SR // 10
  rms = [20 * np.log10(np.sqrt(np.mean(audio[i:i + win] ** 2)) + 1e-9) for i in range(0, len(audio) - win, win)]
  
  # breath points: centre of each run of quiet windows (leading silence ignored)
  dips, run = [], []
  for i, v in enumerate(rms):
      if v < a.dip: run.append(i)
      elif run:
          if run[0] > 0: dips.append(round((run[0] + run[-1]) / 2 / 10 + .05, 2))
          run = []
  start = next((i / 10 for i, v in enumerate(rms) if v > a.dip), 0.0)
  cuts = [start]
  for d in dips:
      if d - cuts[-1] >= a.min_gap and dur - d >= 1.0: cuts.append(d)
  cuts.append(round(dur, 2))
  
  print(f'{a.audio}: {dur:.2f}s, voice starts {start:.2f}s')
  print('quiet dips (breaths):', ', '.join(f'{d:.1f}' for d in dips) or 'none')
  
  model = whisper.load_model(a.model)
  chunks = []
  for c0, c1 in zip(cuts[:-1], cuts[1:]):
      seg = audio[int(c0 * SR):int(c1 * SR)]
      r = model.transcribe(seg, language=a.lang, fp16=False, word_timestamps=True)
      words = [{'w': w['word'].strip(), 't0': round(c0 + w['start'], 2), 't1': round(c0 + w['end'], 2)}
               for s in r['segments'] for w in s.get('words', [])]
      chunks.append({'t0': c0, 't1': c1, 'text': r['text'].strip(), 'words': words})
      print(f'\n[{c0:6.2f} – {c1:6.2f}] {r["text"].strip()}')
      print('   ' + ' | '.join(f'{w["w"]} {w["t0"]:.1f}' for w in words))
  
  out = a.audio.rsplit('.', 1)[0] + '.align.json'
  with open(out, 'w') as f:
      json.dump({'duration': dur, 'start': start, 'dips': dips, 'chunks': chunks}, f, ensure_ascii=False, indent=1)
  print(f'\n→ {out}')
  print('Check for repeated phrases (speakers and singers often repeat a line) before assigning text to times.')
  

• skills/opus-js-animations/scripts/analyze_audio.py7.3 KB7f26d8bd2fa2upstream ↗

  #!/usr/bin/env python3
  """Study a soundtrack before directing to it: length, loudness, pauses, sections, hits and tempo.
  
    python3 analyze_audio.py source/source.wav [--out source] [--silence -35] [--min-pause .25]
  
  Prints a short report and writes, next to the audio (or into --out):
    <name>.analysis.json   duration, loudness (LUFS, true peak), pauses, a 10 Hz loudness contour, sections by
                           energy, onsets and an estimated tempo with beat times (needs numpy)
    <name>.spectrum.png    a spectrogram with a labelled time axis: read it to see phrases, pauses, hits and swells
    <name>.wave.png        the waveform, for a quick look at the shape of the piece
  
  Use it on anything (a voice, a reading, a song, a score you generated). For words in time, run
  align_audio.py as well. ffmpeg on PATH; numpy is optional (without it: no onsets, tempo or sections).
  """
  import argparse, json, re, subprocess, sys
  from pathlib import Path
  
  ap = argparse.ArgumentParser()
  ap.add_argument('audio')
  ap.add_argument('--out')
  ap.add_argument('--silence', type=float, default=-35.0, help='dBFS below which audio counts as a pause')
  ap.add_argument('--min-pause', type=float, default=.25, help='shortest pause to report (s)')
  a = ap.parse_args()
  
  src = Path(a.audio)
  out = Path(a.out) if a.out else src.parent
  out.mkdir(parents=True, exist_ok=True)
  stem = out / src.stem
  
  
  def ff(*args):
      return subprocess.run(['ffmpeg', '-hide_banner', '-nostats', *args], capture_output=True, text=True).stderr
  
  
  # duration and format
  probe = json.loads(subprocess.run(['ffprobe', '-v', 'error', '-show_entries', 'format=duration:stream=sample_rate,channels,codec_name',
                                     '-of', 'json', str(src)], capture_output=True, text=True).stdout or '{}')
  dur = float(probe.get('format', {}).get('duration', 0))
  st = next((s for s in probe.get('streams', []) if s.get('sample_rate')), {})
  
  # loudness (EBU R128) and true peak
  log = ff('-i', str(src), '-af', 'ebur128=peak=true', '-f', 'null', '-')
  lufs = re.findall(r'I:\s+(-?[\d.]+) LUFS', log)
  peak = re.findall(r'Peak:\s+(-?[\d.]+) dBFS', log)
  lra = re.findall(r'LRA:\s+(-?[\d.]+) LU', log)
  
  # pauses
  log = ff('-i', str(src), '-af', f'silencedetect=noise={a.silence}dB:d={a.min_pause}', '-f', 'null', '-')
  starts = [float(x) for x in re.findall(r'silence_start: (-?[\d.]+)', log)]
  ends = [float(x) for x in re.findall(r'silence_end: ([\d.]+)', log)]
  pauses = [[round(max(0, s), 2), round(e, 2)] for s, e in zip(starts, ends + [dur] * (len(starts) - len(ends)))]
  
  report = {'file': str(src), 'duration': round(dur, 3), 'sample_rate': int(st.get('sample_rate', 0) or 0), 'channels': st.get('channels'),
            'lufs': float(lufs[-1]) if lufs else None, 'true_peak_dbfs': float(peak[-1]) if peak else None,
            'loudness_range_lu': float(lra[-1]) if lra else None, 'pauses': pauses}
  
  # contour, sections, onsets, tempo
  try:
      import numpy as np
      SR = 22050
      x = np.frombuffer(subprocess.run(['ffmpeg', '-v', 'error', '-i', str(src), '-ac', '1', '-ar', str(SR), '-f', 'f32le', '-'],
                                       capture_output=True).stdout, dtype=np.float32)
      hop = SR // 10
      n = len(x) // hop
      rms = np.array([np.sqrt(np.mean(x[i * hop:(i + 1) * hop] ** 2) + 1e-12) for i in range(n)])
      db = 20 * np.log10(rms + 1e-9)
      report['contour_db_10hz'] = [round(float(v), 1) for v in db]
      # sections: smooth the loudness over 2 s and split where it crosses its own median by a margin
      k = 20
      sm = np.convolve(db, np.ones(k) / k, mode='same') if n > k else db
      med = float(np.median(sm[sm > -60])) if np.any(sm > -60) else -30.0
      lvl = np.where(sm > med + 3, 'loud', np.where(sm < med - 6, 'quiet', 'mid'))
      sections, s0 = [], 0
      for i in range(1, n + 1):
          if i == n or lvl[i] != lvl[s0]:
              if (i - s0) / 10 >= 1.5 or not sections:
                  sections.append({'t0': round(s0 / 10, 1), 't1': round(i / 10, 1), 'level': str(lvl[s0]), 'mean_db': round(float(np.mean(db[s0:i])), 1)})
              else:
                  sections[-1]['t1'] = round(i / 10, 1)
              s0 = i
      report['sections'] = sections
      # onsets: positive spectral flux, peak-picked
      N, H = 1024, 256
      win = np.hanning(N)
      frames = 1 + max(0, (len(x) - N) // H)
      mag = np.abs(np.fft.rfft(np.stack([x[i * H:i * H + N] * win for i in range(frames)]), axis=1)) if frames > 2 else np.zeros((1, N // 2 + 1))
      flux = np.maximum(0, np.diff(np.log1p(mag), axis=0)).sum(axis=1)
      flux = (flux - flux.mean()) / (flux.std() + 1e-9)
      fr = SR / H
      thr = np.convolve(flux, np.ones(int(fr * .5)) / int(fr * .5), mode='same') + .8
      on = [i for i in range(1, len(flux) - 1) if flux[i] > thr[i] and flux[i] >= flux[i - 1] and flux[i] >= flux[i + 1]]
      onsets, last = [], -1
      for i in on:
          if i - last >= fr * .1: onsets.append(round((i + 1) / fr, 3)); last = i
      report['onsets'] = onsets
      # tempo: autocorrelation of the flux between 60 and 180 BPM
      if len(flux) > fr * 8:
          f = flux - flux.mean()
          ac = np.correlate(f, f, mode='full')[len(f) - 1:]
          lo, hi = int(fr * 60 / 180), int(fr * 60 / 60)
          lag = lo + int(np.argmax(ac[lo:hi]))
          bpm = 60 * fr / lag
          strength = float(ac[lag] / (ac[0] + 1e-9))
          report['tempo'] = {'bpm': round(bpm, 1), 'confidence': round(strength, 2),
                             'note': 'steady beat' if strength > .3 else 'no steady beat (speech, chant or free time)'}
          if strength > .3:
              period = lag / fr
              first = onsets[0] if onsets else 0.0
              report['beats'] = [round(first + k * period, 3) for k in range(int((dur - first) / period) + 1)]
  except ImportError:
      report['note'] = 'numpy not installed: no contour, sections, onsets or tempo (pip install numpy)'
  
  (stem.parent / f'{stem.name}.analysis.json').write_text(json.dumps(report, indent=1))
  ff('-y', '-i', str(src), '-lavfi', 'showspectrumpic=s=1600x420:legend=1:scale=log:fscale=log:stop=8000', str(stem) + '.spectrum.png')
  ff('-y', '-i', str(src), '-filter_complex', 'aformat=channel_layouts=mono,showwavespic=s=1600x240:colors=#d9b25a', '-frames:v', '1', str(stem) + '.wave.png')
  
  # the report
  print(f"{src.name}: {dur:.2f} s · {report['sample_rate']} Hz · {report['channels']} ch")
  if report['lufs'] is not None:
      print(f"loudness {report['lufs']:.1f} LUFS · true peak {report['true_peak_dbfs']:.1f} dBFS · range {report['loudness_range_lu']:.1f} LU")
  if pauses:
      print('pauses: ' + ', '.join(f'{p[0]:.2f}–{p[1]:.2f}' for p in pauses[:40]) + (' …' if len(pauses) > 40 else ''))
  else:
      print(f'pauses: none below {a.silence} dB (reverberant voice or continuous music: use align_audio.py for breath points)')
  for s in report.get('sections', []):
      print(f"  {s['t0']:6.1f}–{s['t1']:6.1f} s  {s['level']:<5} {s['mean_db']:6.1f} dB")
  if 'tempo' in report:
      print(f"tempo ≈ {report['tempo']['bpm']} BPM ({report['tempo']['note']}, confidence {report['tempo']['confidence']})")
  if report.get('onsets'):
      print(f"{len(report['onsets'])} onsets (hits and syllable attacks); first few: " + ', '.join(f'{t:.2f}' for t in report['onsets'][:12]))
  if report['true_peak_dbfs'] is not None and report['true_peak_dbfs'] > -.1:
      print('warning: the peak touches 0 dBFS; listen for clipping before building to it')
  print(f"→ {stem}.analysis.json, {stem}.spectrum.png, {stem}.wave.png (read the spectrogram: phrases, pauses and hits are visible)")
  

• skills/opus-js-animations/scripts/embed_audio.py715 Bbfbc8f2faef3upstream ↗

  #!/usr/bin/env python3
  """Embed an audio file as window.FILM_AUDIO_B64 in an audio.js beside the film.
  
  A page opened from file:// cannot fetch() files, and routing an <audio> element through
  Web Audio yields silence (opaque origin). A base64 script works everywhere, including the
  render tools, at ~1.37x the file size. Keep the source as a compressed MP3/AAC.
  
    python3 embed_audio.py voice.mp3 film/audio.js
  """
  import base64, sys
  
  src, out = sys.argv[1], sys.argv[2]
  data = base64.b64encode(open(src, 'rb').read()).decode()
  with open(out, 'w') as f:
      f.write(f'// {src.split("/")[-1]} as base64 so the film works from file://\nwindow.FILM_AUDIO_B64 = "{data}";\n')
  print(f'{out}: {len(data) / 1e6:.2f} MB')
  

• skills/opus-js-animations/scripts/get_audio.py7.9 KB5211750ddfd3upstream ↗

  #!/usr/bin/env python3
  """Get source audio (a reading, song, speech) from YouTube or any site yt-dlp supports.
  Installs yt-dlp (and checks ffmpeg) if they're missing, updates yt-dlp and retries once when
  a download fails (sites change often and old versions break), saves the metadata, and warns
  when the voice is still sounding at the end of the section (a cut-off last word).
  
    python3 get_audio.py "<url>" [--from 0] [--to 75] [--out source] [--video]
    python3 get_audio.py --search "dylan thomas do not go gentle"   list candidates, download nothing
    python3 get_audio.py --install                                 just make sure the tools are there
  
  Writes into --out (default ./source):
    source.wav   the audio section, 48 kHz (lossless from the site's stream)
    meta.txt     title | uploader | duration | url, then the description
    clip.mp4     with --video: 720p video of the section, to read on-screen text/translation
    sheet.png    with --video: one frame every 3 s tiled, to read that text quickly
  
  Take a section a little longer than you need (--to +10 s) and trim later at a measured quiet
  point; the end check below tells you if the section ends mid-word.
  Stdlib only. Say where the audio came from, and let the user decide about reuse rights.
  """
  import argparse, json, os, platform, shutil, subprocess, sys, urllib.request
  from pathlib import Path
  
  ap = argparse.ArgumentParser()
  ap.add_argument('url', nargs='?')
  ap.add_argument('--from', dest='t0', type=float, default=None, help='section start (s)')
  ap.add_argument('--to', dest='t1', type=float, default=None, help='section end (s)')
  ap.add_argument('--out', default='source')
  ap.add_argument('--video', action='store_true', help='also keep a 720p clip + frame sheet')
  ap.add_argument('--search', help='list the top results for a query and exit')
  ap.add_argument('--install', action='store_true', help='only install/check yt-dlp and ffmpeg')
  a = ap.parse_args()
  
  LOCAL_BIN = Path.home() / '.local' / 'bin'
  
  
  def run(cmd, **kw):
      return subprocess.run(cmd, text=True, capture_output=True, **kw)
  
  
  def works(exe):
      try:
          return run([exe, '--version'], timeout=60).returncode == 0
      except (OSError, subprocess.TimeoutExpired):
          return False
  
  
  def find_ytdlp():
      for exe in [shutil.which('yt-dlp'), str(LOCAL_BIN / 'yt-dlp')]:
          if exe and works(exe):
              return exe
      return None
  
  
  def install_ytdlp():
      """Try, in order: Homebrew, pipx, pip --user, then the official standalone binary."""
      attempts = []
      if shutil.which('brew'):
          attempts.append(['brew', 'install', 'yt-dlp'])
      if shutil.which('pipx'):
          attempts.append(['pipx', 'install', 'yt-dlp'])
      attempts.append([sys.executable, '-m', 'pip', 'install', '--user', '-U', 'yt-dlp[default]'])
      for cmd in attempts:
          print('installing yt-dlp:', ' '.join(cmd))
          r = run(cmd)
          exe = find_ytdlp()
          if exe:
              return exe
          print('  not usable afterwards' + (' (installed, but not on PATH)' if r.returncode == 0 else ': ' + (r.stderr or r.stdout).strip()[-200:]))
      # standalone binary: needs no Python at all
      name = {'Darwin': 'yt-dlp_macos', 'Linux': 'yt-dlp_linux'}.get(platform.system(), 'yt-dlp.exe')
      if platform.system() == 'Linux' and platform.machine() in ('aarch64', 'arm64'):
          name = 'yt-dlp_linux_aarch64'
      url = f'https://github.com/yt-dlp/yt-dlp/releases/latest/download/{name}'
      LOCAL_BIN.mkdir(parents=True, exist_ok=True)
      dest = LOCAL_BIN / 'yt-dlp'
      print('downloading the standalone binary:', url)
      urllib.request.urlretrieve(url, dest)
      dest.chmod(0o755)
      if works(str(dest)):
          if str(LOCAL_BIN) not in os.environ.get('PATH', ''):
              print(f'  note: add {LOCAL_BIN} to PATH to call yt-dlp directly')
          return str(dest)
      sys.exit('could not install yt-dlp; see https://github.com/yt-dlp/yt-dlp#installation')
  
  
  def update_ytdlp(exe):
      """Newer yt-dlp fixes most 'Unable to extract' / 403 / signature errors."""
      real = os.path.realpath(exe)
      with open(real, 'rb') as f:
          is_script = f.read(2) == b'#!'                 # pip/pipx installs are Python entry scripts
      if 'Cellar' in real or '/homebrew/' in real:
          cmd = ['brew', 'upgrade', 'yt-dlp']
      elif 'pipx' in real:
          cmd = ['pipx', 'upgrade', 'yt-dlp']
      elif not is_script:
          cmd = [exe, '-U']                              # the standalone binary updates itself
      else:
          cmd = [sys.executable, '-m', 'pip', 'install', '--user', '-U', 'yt-dlp[default]']
      print('updating yt-dlp:', ' '.join(cmd))
      run(cmd)
  
  
  def need_ffmpeg():
      if shutil.which('ffmpeg') and shutil.which('ffprobe'):
          return
      if shutil.which('brew'):
          print('installing ffmpeg: brew install ffmpeg')
          run(['brew', 'install', 'ffmpeg'])
      if not shutil.which('ffmpeg'):
          sys.exit('ffmpeg is required: macOS `brew install ffmpeg`, Debian/Ubuntu `sudo apt install ffmpeg`, '
                   'Windows `winget install ffmpeg`')
  
  
  exe = find_ytdlp() or install_ytdlp()
  need_ffmpeg()
  print('yt-dlp', run([exe, '--version']).stdout.strip(), '·', exe)
  if a.install:
      sys.exit(0)
  
  if a.search:
      r = run([exe, '--flat-playlist', '--print', '%(id)s | %(duration_string)s | %(uploader)s | %(title)s',
               f'ytsearch8:{a.search}'])
      print(r.stdout or r.stderr)
      print('pick one and run: get_audio.py "https://youtu.be/<id>" --from S --to E')
      sys.exit(0)
  
  if not a.url:
      ap.error('give a URL, or --search "query"')
  
  out = Path(a.out)
  out.mkdir(parents=True, exist_ok=True)
  section = []
  if a.t0 is not None or a.t1 is not None:
      section = ['--download-sections', f'*{a.t0 or 0}-{a.t1 if a.t1 is not None else "inf"}', '--force-keyframes-at-cuts']
  
  
  def fetch(args):
      r = run([exe, '-q', '--no-playlist', '--no-warnings', *args, a.url])
      if r.returncode != 0:
          update_ytdlp(exe)
          r = run([exe, '-q', '--no-playlist', '--no-warnings', *args, a.url])
      if r.returncode != 0:
          sys.exit('download failed:\n' + r.stderr.strip()[-1500:])
  
  
  # metadata first: the title and description often identify the content exactly
  m = run([exe, '--no-playlist', '--skip-download', '--dump-single-json', a.url])
  if m.returncode != 0:
      update_ytdlp(exe)
      m = run([exe, '--no-playlist', '--skip-download', '--dump-single-json', a.url])
  if m.returncode == 0:
      info = json.loads(m.stdout)
      head = f"{info.get('title')} | {info.get('uploader')} | {info.get('duration')} s | {info.get('webpage_url')}"
      (out / 'meta.txt').write_text(head + '\n\n' + (info.get('description') or '') + '\n')
      print(head)
  
  for f in out.glob('source.*'):
      f.unlink()
  fetch(['-f', 'ba/b', '-x', '--audio-format', 'wav', '--postprocessor-args', 'ExtractAudio:-ar 48000',
         *section, '-o', str(out / 'source.%(ext)s')])
  wav = out / 'source.wav'
  if not wav.exists():
      sys.exit('no audio file was produced')
  
  if a.video:
      for f in out.glob('clip.*'):
          f.unlink()
      fetch(['-f', 'bv*[height<=720]+ba/b[height<=720]', '--merge-output-format', 'mp4', *section,
             '-o', str(out / 'clip.%(ext)s')])
      run(['ffmpeg', '-v', 'error', '-y', '-i', str(out / 'clip.mp4'), '-vf', 'fps=1/3,scale=480:-1,tile=6x3',
           '-frames:v', '1', str(out / 'sheet.png')])
  
  dur = float(run(['ffprobe', '-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', str(wav)]).stdout or 0)
  tail = run(['ffmpeg', '-hide_banner', '-sseof', '-0.4', '-i', str(wav), '-af', 'volumedetect', '-f', 'null', '-'])
  mean = next((float(l.split(':')[1].split()[0]) for l in tail.stderr.splitlines() if 'mean_volume' in l), -99.0)
  print(f'→ {wav} ({dur:.2f} s)')
  if mean > -35:
      print(f'WARNING: still {mean:.0f} dB in the last 0.4 s: the section likely ends mid-word or mid-phrase. '
            'If you need the audio up to here, download a longer section (--to later) and trim at a measured quiet point.')
  else:
      print(f'end check: quiet tail ({mean:.0f} dB)')
  print('next: python3 align_audio.py', wav, '--lang <code>')
  

• skills/opus-js-animations/scripts/lib.mjs6.7 KBae639845d115upstream ↗

  // Shared helpers: drive headless Chrome over the DevTools protocol with no npm installs
  // (Node ≥ 22 has fetch and WebSocket built in). Every tool loads a film page with
  // ?capture=1 and calls window.__film.seek(t).
  import { spawn } from 'node:child_process';
  import http from 'node:http';
  import { mkdtempSync, existsSync, readFileSync } from 'node:fs';
  import { tmpdir } from 'node:os';
  import { join, resolve, sep, extname } from 'node:path';
  import { pathToFileURL } from 'node:url';
  
  export const sleep = ms => new Promise(r => setTimeout(r, ms));
  
  const CHROME_CANDIDATES = [
    process.env.CHROME,
    '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
    '/Applications/Chromium.app/Contents/MacOS/Chromium',
    '/usr/bin/google-chrome', '/usr/bin/chromium', '/usr/bin/chromium-browser',
  ].filter(Boolean);
  
  export function findChrome() {
    const c = CHROME_CANDIDATES.find(p => existsSync(p));
    if (!c) throw new Error('No Chrome found; set CHROME=/path/to/chrome');
    return c;
  }
  
  export function parseArgs(argv) {
    const out = { _: [] };
    for (let i = 0; i < argv.length; i++) {
      const a = argv[i];
      if (a.startsWith('--')) {
        const k = a.slice(2), next = argv[i + 1];
        if (next === undefined || next.startsWith('--')) out[k] = true; else { out[k] = next; i++; }
      } else out._.push(a);
    }
    return out;
  }
  
  // Launch a headless Chrome with GPU (Metal on macOS; the shader layers need it to be fast)
  // and open the film. ss > 1 asks a supersampling-aware film for an ss× canvas (?ss=2). Returns { ev, close, info }.
  // url: open the film from a local server (serveDir) instead of file://, so the page can send frames back over HTTP.
  export async function openFilm(htmlPath, { port = 9400 + Math.floor(Math.random() * 400), gpu = true, ss = 1, url: pageUrl = null } = {}) {
    const flags = ['--headless=new', `--remote-debugging-port=${port}`, `--user-data-dir=${mkdtempSync(join(tmpdir(), 'film-'))}`,
      '--hide-scrollbars', '--autoplay-policy=no-user-gesture-required', '--ignore-gpu-blocklist'];
    if (gpu && process.platform === 'darwin') flags.push('--use-angle=metal');
    if (!gpu) flags.push('--disable-gpu');
    const chrome = spawn(findChrome(), [...flags, 'about:blank'], { stdio: 'ignore' });
  
    let targets;
    for (let i = 0; i < 150; i++) {
      try { targets = await (await fetch(`http://127.0.0.1:${port}/json/list`)).json(); if (targets.some(t => t.type === 'page')) break; } catch {}
      await sleep(100);
    }
    const ws = new WebSocket(targets.find(t => t.type === 'page').webSocketDebuggerUrl);
    await new Promise(r => ws.onopen = r);
    let id = 0; const pending = new Map(), logs = [];
    ws.onmessage = m => {
      const d = JSON.parse(m.data);
      if (d.id && pending.has(d.id)) { pending.get(d.id)(d); pending.delete(d.id); }
      if (d.method === 'Runtime.exceptionThrown') logs.push(d.params.exceptionDetails.exception?.description || d.params.exceptionDetails.text);
    };
    const send = (method, params = {}) => new Promise(r => { const i = ++id; pending.set(i, r); ws.send(JSON.stringify({ id: i, method, params })); });
    const ev = async expr => {
      const r = await send('Runtime.evaluate', { expression: expr, returnByValue: true, awaitPromise: true });
      if (r.result?.exceptionDetails) throw new Error(r.result.exceptionDetails.exception?.description || r.result.exceptionDetails.text);
      return r.result.result.value;
    };
    await send('Runtime.enable');
    const url = (pageUrl || pathToFileURL(resolve(htmlPath)).href) + '?capture=1' + (ss > 1 ? `&ss=${ss}` : '');
    await send('Page.navigate', { url });
    // Navigation returns before the page loads; poll for the contract, not a timer.
    let ok = false;
    for (let i = 0; i < 300; i++) {
      try { ok = await ev('!!(window.__film && window.__film.ready)'); } catch {}
      if (ok) break;
      await sleep(100);
    }
    if (!ok) throw new Error(`window.__film never became ready.${logs.length ? ' Page errors:\n' + logs.join('\n') : ''}`);
    const info = await ev(`({ duration: __film.duration, w: document.getElementById('c')?.width ?? document.querySelector('canvas').width,
      h: document.getElementById('c')?.height ?? document.querySelector('canvas').height, gpu: (() => { try {
        const g = document.createElement('canvas').getContext('webgl'); const e = g.getExtension('WEBGL_debug_renderer_info');
        return e ? g.getParameter(e.UNMASKED_RENDERER_WEBGL) : g.getParameter(g.RENDERER); } catch { return 'none'; } })() })`);
    return { ev, logs, info, close: () => { try { ws.close(); } catch {} chrome.kill(); } };
  }
  
  // Seek and grab the canvas as JPEG (or PNG) bytes.
  export async function grab(ev, t, type = 'image/jpeg', q = .95) {
    const url = await ev(`(__film.seek(${t}), (document.getElementById('c') || document.querySelector('canvas')).toDataURL('${type}', ${q}))`);
    return Buffer.from(url.slice(url.indexOf(',') + 1), 'base64');
  }
  
  // A local server for a film folder: GET serves its files (the film, its scripts, fonts, audio), and POST /frame hands each
  // frame's raw bytes to onFrame(query, body, done). render.mjs uses it to take frames out of the page as raw pixels: a PNG
  // data URL through DevTools costs most of a second for a 2× frame, a POST of raw RGBA a few milliseconds.
  const MIME = { '.html': 'text/html', '.js': 'text/javascript', '.mjs': 'text/javascript', '.json': 'application/json', '.css': 'text/css',
    '.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.webp': 'image/webp', '.svg': 'image/svg+xml', '.gif': 'image/gif',
    '.wav': 'audio/wav', '.mp3': 'audio/mpeg', '.m4a': 'audio/mp4', '.ogg': 'audio/ogg', '.mp4': 'video/mp4',
    '.woff': 'font/woff', '.woff2': 'font/woff2', '.ttf': 'font/ttf', '.otf': 'font/otf', '.glb': 'model/gltf-binary', '.gltf': 'model/gltf+json', '.hdr': 'application/octet-stream' };
  export async function serveDir(root, onFrame = null) {
    root = resolve(root);
    const server = http.createServer((req, res) => {
      const u = new URL(req.url, 'http://local');
      if (req.method === 'POST' && u.pathname === '/frame' && onFrame) {
        const chunks = []; req.on('data', c => chunks.push(c));
        req.on('end', () => onFrame(u.searchParams, Buffer.concat(chunks), () => res.end('ok')));
        return;
      }
      const f = resolve(root, '.' + decodeURIComponent(u.pathname));
      if (f !== root && !f.startsWith(root + sep)) { res.writeHead(403); res.end(); return; }
      try { const body = readFileSync(f); res.writeHead(200, { 'content-type': MIME[extname(f).toLowerCase()] || 'application/octet-stream' }); res.end(body); }
      catch { res.writeHead(404); res.end(); }
    });
    await new Promise(r => server.listen(0, '127.0.0.1', r));
    const port = server.address().port;
    return { port, url: rel => `http://127.0.0.1:${port}/${rel.split(sep).map(encodeURIComponent).join('/')}`, close: () => server.close() };
  }
  
  

• skills/opus-js-animations/scripts/page_audio.mjs1.6 KBe6873882e7d8upstream ↗

  #!/usr/bin/env node
  // Pull a film's own procedural soundtrack out of the page as a WAV, for render.mjs --audio.
  // The page must expose __film.wav(): a Promise of a base64 WAV rendered with OfflineAudioContext
  // from the same score (and the same physics-born events) the picture uses.
  //
  //   node page_audio.mjs film/index.html --out film/mix.wav
  //   node render.mjs film/index.html --fps 30 --audio film/mix.wav --out film/film.mp4
  import { writeFileSync } from 'node:fs';
  import { resolve, dirname, join } from 'node:path';
  import { spawnSync } from 'node:child_process';
  import { openFilm, parseArgs } from './lib.mjs';
  
  const args = parseArgs(process.argv.slice(2));
  const html = args._[0];
  if (!html) { console.error('usage: node page_audio.mjs <film.html> [--out mix.wav]'); process.exit(1); }
  const out = resolve(args.out || join(dirname(resolve(html)), 'mix.wav'));
  const film = await openFilm(html, { gpu: !args['no-gpu'] });
  if (!(await film.ev('typeof __film.wav === "function"'))) { console.error('page has no __film.wav()'); film.close(); process.exit(1); }
  const b64 = await film.ev('__film.wav()');
  writeFileSync(out, Buffer.from(b64, 'base64'));
  film.close();
  console.log(out);
  // levels: you usually can't listen, so at least read them
  const r = spawnSync('ffmpeg', ['-hide_banner', '-nostats', '-i', out, '-af', 'ebur128=peak=true', '-f', 'null', '-'], { encoding: 'utf8' });
  const s = r.stderr.slice(r.stderr.lastIndexOf('Summary:'));
  const I = /I:\s+(-?[\d.]+) LUFS/.exec(s)?.[1], P = /Peak:\s+(-?[\d.]+) dBFS/.exec(s)?.[1];
  if (I) console.log(`integrated ${I} LUFS · true peak ${P} dBFS${+P > -1 ? '  (hot: normalise before muxing, audio.md §7)' : ''}`);
  

• skills/opus-js-animations/scripts/paint.py18 KBdd179abd0991upstream ↗

  #!/usr/bin/env python3
  """Paint a still pixel by pixel in a painter's manner: no image model, no reference pictures,
  no art software. numpy + scipy + Pillow only.
  
    python3 paint.py --demo storm --size 1080 --out storm.png     # vortex sea storm, impasto, rain
    python3 paint.py --demo pool --size 1080 --out pool.png       # flat acrylic, hard edges, caustic lines
  
  As a library (write your own scene in a new file next to this one):
    from paint import Canvas, fbm, vortex_flow, noise_flow, lay, stroke, finish, sketch
    c = Canvas(1080, 1080, ground=(.86, .82, .72), seed=1)
    c.wash(color_at)                                   # underpainting from a colour function
    lay(c, flow, color_at, n=1500, length=120, width=38)   # big brushes first, then smaller
    finish(c, light=(-.6, -.8), impasto=1.0, weave=.05).save('out.png')
  
  Model: strokes are polylines traced along a flow field. Each stroke has bristles (streaks
  across its width), a paint load that runs out along its length (dry-brush breaks), picks up
  wet paint from the canvas at its start (wet-in-wet), and deposits height for impasto, which
  `finish` lights like a raking lamp. Colour comes from a function of position (the scene),
  never from an image. Work coarse to fine, as a painter does: wash, masses, forms, accents.
  """
  import argparse, math
  import numpy as np
  from PIL import Image
  from scipy.ndimage import gaussian_filter
  
  
  def fbm(h, w, scale, octaves=5, seed=0):
      """Smooth noise in [-1, 1]-ish: blurred white noise summed over octaves (scale = largest feature, px)."""
      rng = np.random.default_rng(seed)
      out = np.zeros((h, w), np.float32)
      amp, sig, tot = 1.0, scale / 3.0, 0.0
      for _ in range(octaves):
          n = gaussian_filter(rng.standard_normal((h, w)).astype(np.float32), max(sig, .6), mode='wrap')
          out += amp * n / (n.std() + 1e-6)
          tot += amp; amp *= .5; sig *= .5
      return out / tot
  
  
  def vortex_flow(h, w, cx, cy, inward=.25, noise=.35, seed=0):
      """Angle field (radians) circling (cx, cy) counter-clockwise and spiralling in."""
      y, x = np.mgrid[0:h, 0:w].astype(np.float32)
      th = np.arctan2(y - cy, x - cx)
      return th + np.pi / 2 + inward + noise * fbm(h, w, w / 5, 4, seed)
  
  
  def noise_flow(h, w, base=0.0, amount=1.0, scale=None, seed=0):
      return base + amount * fbm(h, w, scale or w / 4, 4, seed)
  
  
  class Canvas:
      def __init__(self, w, h, ground=(.9, .88, .82), seed=0):
          self.w, self.h = w, h
          self.rgb = np.ones((h, w, 3), np.float32) * np.array(ground, np.float32)
          self.height = np.zeros((h, w), np.float32)
          self.rng = np.random.default_rng(seed)
  
      def grid(self):
          y, x = np.mgrid[0:self.h, 0:self.w].astype(np.float32)
          return x / self.w, y / self.h
  
      def wash(self, color_at, blur=12, opacity=1.0):
          """Thin underpainting: the scene colour, softened."""
          x, y = self.grid()
          col = np.clip(color_at(x, y), 0, 1).astype(np.float32)
          col = np.stack([gaussian_filter(col[..., i], blur) for i in range(3)], -1)
          self.rgb = self.rgb * (1 - opacity) + col * opacity
  
  
  def trace(flow, x, y, length, step=5.0, sign=1):
      """Polyline from (x, y) following the angle field."""
      h, w = flow.shape
      pts = [(x, y)]
      for _ in range(max(1, int(length / step))):
          a = flow[int(min(max(y, 0), h - 1)), int(min(max(x, 0), w - 1))]
          x += sign * math.cos(a) * step; y += sign * math.sin(a) * step
          if not (-20 < x < w + 20 and -20 < y < h + 20):
              break
          pts.append((x, y))
      return np.array(pts, np.float32)
  
  
  def stroke(c, pts, width, color, opacity=.92, bristle=.55, load=1.0, dry=.6, pickup=.25, impasto=.0, rng=None):
      """One brush stroke along pts (N×2, pixels). Rendered segment by segment into a patch."""
      rng = rng or c.rng
      if len(pts) < 2:
          return
      r = width / 2
      x0, y0 = np.floor(pts.min(0) - r - 2).astype(int); x1, y1 = np.ceil(pts.max(0) + r + 2).astype(int)
      x0, y0, x1, y1 = max(x0, 0), max(y0, 0), min(x1, c.w), min(y1, c.h)
      if x1 <= x0 or y1 <= y0:
          return
      ph, pw = y1 - y0, x1 - x0
      dist = np.full((ph, pw), np.inf, np.float32); along = np.zeros((ph, pw), np.float32); across = np.zeros((ph, pw), np.float32)
      seg = np.diff(pts, axis=0); seglen = np.hypot(seg[:, 0], seg[:, 1]) + 1e-6
      cum = np.concatenate([[0], np.cumsum(seglen)]); total = cum[-1]
      for i in range(len(seg)):
          a, d, L = pts[i], seg[i], seglen[i]
          bx0 = int(max(min(a[0], a[0] + d[0]) - r - 1, x0)); bx1 = int(min(max(a[0], a[0] + d[0]) + r + 2, x1))
          by0 = int(max(min(a[1], a[1] + d[1]) - r - 1, y0)); by1 = int(min(max(a[1], a[1] + d[1]) + r + 2, y1))
          if bx1 <= bx0 or by1 <= by0:
              continue
          yy, xx = np.mgrid[by0:by1, bx0:bx1].astype(np.float32)
          px, py = xx - a[0], yy - a[1]
          t = np.clip((px * d[0] + py * d[1]) / (L * L), 0, 1)
          qx, qy = px - t * d[0], py - t * d[1]
          dd = np.hypot(qx, qy)
          sub = (slice(by0 - y0, by1 - y0), slice(bx0 - x0, bx1 - x0))
          m = dd < dist[sub]
          dist[sub] = np.where(m, dd, dist[sub])
          along[sub] = np.where(m, cum[i] + t * L, along[sub])
          across[sub] = np.where(m, (px * d[1] - py * d[0]) / L, across[sub])     # signed distance across
      u = np.clip(across / r, -1, 1)
      s = along / total
      # bristles: fixed streak pattern across the brush; paint runs out along the stroke
      K = 48
      bprof = 1 - bristle * rng.random(K).astype(np.float32)
      bprof = np.convolve(np.r_[bprof, bprof[:2]], [.25, .5, .25], 'same')[:K]
      b = np.interp((u + 1) / 2 * (K - 1), np.arange(K), bprof)
      paint = load * (1 - dry * s ** 1.5)
      edge = np.clip((1 - np.abs(u)) * r / 1.2, 0, 1) * (dist < r + 1)
      ends = np.clip(along / (r * .6 + 1), 0, 1) * np.clip((total - along) / (r * .9 + 1), 0, 1)
      alpha = opacity * edge * ends * np.clip((b - (1 - paint)) * 3.5, 0, 1)
      if not alpha.any():
          return
      region = c.rgb[y0:y1, x0:x1]
      under = region[alpha > .05].mean(0) if (alpha > .05).any() else np.array(color)
      col = np.array(color, np.float32) * (1 - pickup) + under * pickup                 # wet-in-wet
      col = col[None, None, :] * (.94 + .1 * b[..., None])                              # bristle value shifts
      c.rgb[y0:y1, x0:x1] = region * (1 - alpha[..., None]) + col * alpha[..., None]
      if impasto:
          c.height[y0:y1, x0:x1] += impasto * alpha * (b - .5 + .6 * np.abs(u) ** 6)      # ridges and rims
  
  
  def lay(c, flow, color_at, n, length, width, jitter=.06, hue=.012, mask=None, opacity=.92, impasto=.0, both_ways=True, **kw):
      """n strokes: random starts (inside mask, if given), traced along flow, coloured by the scene.
      jitter varies value per stroke, hue adds a little temperature noise per channel."""
      rng = c.rng
      done = 0
      while done < n:
          x, y = rng.random() * c.w, rng.random() * c.h
          if mask is not None and not mask[int(y), int(x)]:
              if rng.random() < .98:
                  continue
          L = length * (.6 + .8 * rng.random())
          pts = trace(flow, x, y, L, step=max(3.0, width / 4), sign=1 if (not both_ways or rng.random() < .5) else -1)
          base = np.asarray(color_at(np.float32(x / c.w), np.float32(y / c.h)), np.float32)
          col = np.clip(base * (1 + rng.normal(0, jitter)) + rng.normal(0, hue, 3), 0, 1)
          stroke(c, pts, width * (.75 + .5 * rng.random()), col, opacity=opacity, impasto=impasto, **kw)
          done += 1
  
  
  def finish(c, light=(-.6, -.8), impasto=1.0, weave=.04, vignette=.25, grain=.02):
      """Rake light across the paint relief, add canvas weave, return a PIL image."""
      hmap = gaussian_filter(c.height, 1.0)
      gy, gx = np.gradient(hmap)
      lx, ly = light
      shade = 1 + impasto * np.clip(-(gx * lx + gy * ly), -1.5, 1.5) * .35
      spec = np.clip(-(gx * lx + gy * ly) - .25, 0, None) ** 2 * impasto * .5
      img = c.rgb * shade[..., None] + spec[..., None]
      y, x = np.mgrid[0:c.h, 0:c.w].astype(np.float32)
      if weave:
          img *= (1 - weave * (.5 + .25 * np.sin(x * 1.9) + .25 * np.sin(y * 2.1)))[..., None]
      if vignette:
          r = np.hypot((x - c.w / 2) / c.w, (y - c.h / 2) / c.h) * 1.5
          img *= (1 - vignette * np.clip(r, 0, 1) ** 2)[..., None]
      if grain:
          img += c.rng.normal(0, grain, img.shape[:2])[..., None].astype(np.float32)
      return Image.fromarray((np.clip(img, 0, 1) * 255).astype(np.uint8))
  
  
  def sketch(rgb, flow=None, seed=0, paper=(.93, .9, .84), density=1.0):
      """A graphite/ink drawing of a painted image: contour lines from value edges, hatching whose
      density follows darkness, cross-hatching in the darkest areas. Returns a Canvas-ready float
      array (h, w, 3). Used for 'paper → sketch → paint' reveals and for drawn stills."""
      h, w = rgb.shape[:2]
      c = Canvas(w, h, ground=paper, seed=seed)
      c.rgb *= (1 + .025 * fbm(h, w, 6, 2, seed + 1))[..., None]                 # paper tooth
      L = rgb @ np.array([.3, .59, .11], np.float32)
      Ls = gaussian_filter(L, max(2.0, w / 250))                                  # structure, not brush texture
      gy, gx = np.gradient(Ls)
      edge = np.hypot(gx, gy)
      ink = np.clip((edge - np.percentile(edge, 88)) / (np.percentile(edge, 99.5) - np.percentile(edge, 88) + 1e-6), 0, 1)
      if flow is None:                                                             # hatch along the contours
          a = np.arctan2(gy, gx) + np.pi / 2
          cs, sn = gaussian_filter(np.cos(2 * a), w / 60), gaussian_filter(np.sin(2 * a), w / 60)
          flow = .5 * np.arctan2(sn, cs)
      lo, hi = np.percentile(Ls, 15), np.percentile(Ls, 60)
      dark = np.clip((hi - Ls) / (hi - lo + 1e-6), 0, 1) ** 1.2                  # only the darker half is hatched
      rng = c.rng
      n = int(w * h / 160 * density)
      ys, xs = rng.random(n) * h, rng.random(n) * w
      keep = rng.random(n) < dark[ys.astype(int), xs.astype(int)] * 1.1
      graphite = np.array([.2, .19, .18], np.float32)
      for x, y in zip(xs[keep], ys[keep]):
          d = dark[int(y), int(x)]
          for k, off in enumerate([0.0, .9] if d > .62 else [0.0]):             # cross-hatch the darks
              px, py = (x, y) if k == 0 else (x + rng.normal(0, 7), y + rng.normal(0, 7))   # separate starts: no V shapes
              L = 10 + 22 * d
              fwd, back = trace(flow + off, px, py, L / 2, step=3.0), trace(flow + off, px, py, L / 2, step=3.0, sign=-1)
              pts = np.vstack([back[::-1], fwd[1:]])                                # centred on the point
              stroke(c, pts, 1.1 + .6 * d, graphite, opacity=.25 + .5 * d, bristle=.25, dry=.5, pickup=0, rng=rng)
      c.rgb *= (1 - .8 * gaussian_filter(ink, .7))[..., None] * .85 + .15             # contours
      return np.clip(c.rgb, 0, 1)
  
  
  def palette(stops):
      """stops: [(t, (r,g,b)), ...] → vectorised lookup."""
      ts = np.array([s[0] for s in stops], np.float32); cs = np.array([s[1] for s in stops], np.float32)
      return lambda v: np.stack([np.interp(v, ts, cs[:, i]) for i in range(3)], -1)
  
  
  # ── demos ────────────────────────────────────────────────────────────────────
  def demo_storm(size, seed):
      """Sea storm in a romantic manner: a vortex of light around a small steamboat, impasto, rain."""
      W = H = size
      c = Canvas(W, H, ground=(.55, .5, .4), seed=seed)
      cx, cy = .5, .55
      ramp = palette([(0, (.14, .13, .11)), (.25, (.29, .28, .23)), (.45, (.48, .45, .36)), (.65, (.7, .65, .51)),
                      (.82, (.88, .84, .7)), (1, (.98, .96, .88))])
      swirl = fbm(H, W, W / 6, 5, seed + 1)
      def color_at(x, y):
          dx, dy = x - cx, (y - cy) * 1.25
          r = np.sqrt(dx * dx + dy * dy); th = np.arctan2(dy, dx)
          xi = np.clip((np.asarray(x) * W).astype(int), 0, W - 1); yi = np.clip((np.asarray(y) * H).astype(int), 0, H - 1)
          spiral = np.sin(th * 2 + r * 22 + swirl[yi, xi] * 2.2)
          v = .95 * np.exp(-r * r / .05) + .16 * spiral * np.exp(-r / .35) + .12 * swirl[yi, xi]
          sea = np.clip((np.asarray(y) - .68) / .3, 0, 1)
          v = v * (1 - .7 * sea) + .08
          col = ramp(np.clip(v, 0, 1))
          green = np.array([.2, .27, .23], np.float32)
          return col * (1 - .55 * sea[..., None]) + green * .55 * sea[..., None]
      c.wash(color_at, blur=18)
      flow = vortex_flow(H, W, cx * W, cy * H, inward=.35, noise=.3, seed=seed)
      y, x = np.mgrid[0:H, 0:W]
      seaflow = noise_flow(H, W, base=-.15, amount=.35, seed=seed + 3)
      k = np.clip((y / H - .66) / .1, 0, 1)
      flow = flow * (1 - k) + seaflow * k
      lay(c, flow, color_at, n=650, length=size * .24, width=size * .065, opacity=.7, pickup=.45, bristle=.35, dry=.3, impasto=.4)
      lay(c, flow, color_at, n=2600, length=size * .1, width=size * .02, opacity=.8, pickup=.35, impasto=.7, dry=.55)
      near = np.hypot(x / W - cx, (y / H - cy) * 1.25) < .28
      lay(c, flow, color_at, n=2600, length=size * .05, width=size * .007, impasto=.9, mask=near, jitter=.05)
      # the boat: a dark hull in horizontal dabs, a mast, a lamp, and smoke pulled into the vortex
      bx, by = cx * W, cy * H + size * .02
      tilt = -.14
      for i in range(90):                                   # hull: rows of dabs, wide deck tapering to the keel
          v = c.rng.random()
          half = size * (.045 - .03 * v)
          u = (c.rng.random() * 2 - 1) * half
          yy = by + v * size * .016 + u * tilt
          pts = np.array([[bx + u - size * .008, yy - size * .008 * tilt], [bx + u + size * .008, yy + size * .008 * tilt]], np.float32)
          stroke(c, pts, size * .005, (.06, .05, .045), opacity=.95, bristle=.2, dry=.1, pickup=.05)
      for i in range(18):                                   # deckhouse and paddle box
          u = (c.rng.random() * 2 - 1) * size * .012
          pts = np.array([[bx + u, by - size * .012], [bx + u, by - size * .001]], np.float32)
          stroke(c, pts, size * .005, (.07, .06, .05), opacity=.95, bristle=.2, dry=.1, pickup=.05)
      stroke(c, np.array([[bx + size * .01, by], [bx + size * .012, by - size * .06]], np.float32), size * .003, (.06, .05, .045), bristle=.1, dry=0, pickup=0)
      sm = np.array([[bx - size * .005, by - size * .01]], np.float32)
      for i in range(40):
          sm = np.vstack([sm, sm[-1] + [-size * .004 - i * .1, -size * .004 + i * .12]])
      for i in range(14):
          stroke(c, sm + c.rng.normal(0, size * .003, sm.shape), size * (.012 + i * .0008), (.12, .11, .1), opacity=.5, bristle=.6, dry=.7, pickup=.4, impasto=.3)
      ly, lx = int(by - size * .045), int(bx + size * .012)
      glow = np.exp(-((x - lx) ** 2 + (y - ly) ** 2) / (2 * (size * .006) ** 2))
      c.rgb += (glow[..., None] * np.array([1, .85, .55]) * .9).astype(np.float32)
      # rain: long thin pale streaks on a slant
      rain = np.zeros((H, W), np.float32)
      for i in range(420):
          x0, y0 = c.rng.random() * W * 1.3 - W * .15, c.rng.random() * H * 1.2 - H * .2
          Lr = size * (.1 + .25 * c.rng.random())
          pts = np.array([[x0, y0], [x0 + Lr * .32, y0 + Lr]], np.float32)
          stroke(c, pts, size * .0018, (.86, .86, .8), opacity=.18 + .2 * c.rng.random(), bristle=.3, dry=.5, pickup=.5)
      return finish(c, light=(-.55, -.85), impasto=1.2, weave=.04, vignette=.35), flow
  
  
  def demo_pool(size, seed):
      """Flat acrylic in a sunlit Californian manner: hard edges, unmodulated fields, a caustic line net."""
      W = H = size
      c = Canvas(W, H, ground=(.93, .91, .86), seed=seed)
      x, y = c.grid()
      m = .06                                                         # unpainted border
      inside = (x > m) & (x < 1 - m) & (y > m) & (y < 1 - m)
      def fill(mask, col, roll=.025):                                 # flat field with a faint roller texture
          tex = 1 + roll * fbm(H, W, 30, 3, int(c.rng.integers(1e6)))
          c.rgb[mask] = (np.array(col, np.float32)[None, :] * tex[mask][:, None])
      fill(inside & (y < .5), (.47, .7, .88), roll=.01)               # sky, flat
      fill(inside & (y >= .5) & (y < .56) & (x >= .62), (.86, .84, .8))   # low wall
      fill(inside & (y >= .46) & (y < .56) & (x < .62), (.94, .94, .92))   # house
      for i in range(7):                                              # window slats
          fill(inside & (y > .475) & (y < .54) & (np.abs(x - (.12 + i * .02)) < .004), (.72, .62, .42))
      fill(inside & (y >= .56) & (y < .6), (.3, .5, .28))             # hedge line
      fill(inside & (y >= .6) & (y < .66), (.93, .8, .76))            # pink paving
      pool = inside & (y >= .66)
      fill(pool, (.26, .66, .86))
      n = fbm(H, W, W / 4, 2, seed + 5) + 1.4 * (y - .66)
      for lvl, col, wdt in [(0, (.72, .9, .97), .07), (.5, (.14, .48, .74), .05)]:
          f = np.abs(((n * 4.5 + lvl) % 1) - .5)
          c.rgb[pool & (f < wdt)] = col                               # the wavy interlocking lines
      board = pool & (np.abs((x - .1) - (y - .72) * .55) < .045) & (y < .98)
      fill(board, (.86, .72, .38))
      for px_, top in [(.22, .12), (.78, .2), (.9, .08)]:             # palms: trunk, fronds
          trunk = inside & (np.abs(x - px_ - (y - .5) * .03) < .006) & (y > top) & (y < .5)
          fill(trunk, (.45, .36, .25))
          for k in range(9):
              a = k / 9 * 2 * np.pi
              fx, fy = px_ + np.cos(a) * .07, top + np.sin(a) * .035 + .02
              t = np.clip(((x - px_) * (fx - px_) + (y - top) * (fy - top)) / ((fx - px_) ** 2 + (fy - top) ** 2), 0, 1)
              d = np.hypot(x - px_ - t * (fx - px_), y - top - t * (fy - top) - .02 * np.sin(t * np.pi))
              fill(inside & (d < .01 * (1 - t) + .002), (.16, .42, .2))
      return finish(c, impasto=0, weave=.015, vignette=0, grain=.008), None
  
  
  if __name__ == '__main__':
      ap = argparse.ArgumentParser()
      ap.add_argument('--demo', choices=['storm', 'pool'], default='storm')
      ap.add_argument('--size', type=int, default=1080)
      ap.add_argument('--seed', type=int, default=3)
      ap.add_argument('--out', default='painting.png')
      ap.add_argument('--sketch', action='store_true', help='also write <out>-sketch.png: the drawing under the painting')
      a = ap.parse_args()
      img, flow = {'storm': demo_storm, 'pool': demo_pool}[a.demo](a.size, a.seed)
      img.save(a.out)
      print(a.out)
      if a.sketch:
          out = a.out.rsplit('.', 1)[0] + '-sketch.png'
          Image.fromarray((sketch(np.asarray(img, np.float32) / 255, flow=flow, seed=a.seed) * 255).astype(np.uint8)).save(out)
          print(out)
  

• skills/opus-js-animations/scripts/render.mjs14 KB163f64068d19upstream ↗

  #!/usr/bin/env node
  // Frame-exact MP4 export. Every frame is __film.seek(i / fps), so nothing drops or drifts,
  // whatever the machine speed. Long films split across parallel Chrome workers.
  //
  //   node render.mjs film/index.html [--fps 30] [--audio mix.wav] [--out film.mp4]
  //                   [--from 0 --to 12] [--workers 2] [--crf 16] [--ss 2] [--root dir] [--no-gpu] [--fast] [--legacy] [--profile]
  //
  // --audio is muxed and the video is cut to the shorter of the two (-shortest).
  // Without --audio the output is silent (use for section previews).
  //
  // How frames leave the page. The film is served from its folder over http://127.0.0.1 (--root serves a parent folder when the
  // film loads files from outside its own). In each worker the page runs its own loop: seek(t), then a small WebGL2 exporter
  // copies the canvas to a texture, scales it down with a 3-lobe Lanczos when --ss > 1 (the kernel ffmpeg's "lanczos" uses), reads
  // the raw RGBA and POSTs it back. Node pipes it into ffmpeg as rawvideo. A few frames are kept in flight, so drawing,
  // copying and encoding overlap. Measured on an Apple M1 at 2160×3840: the PNG data URL the old path sent through DevTools cost
  // ~0.9 s a frame; this costs ~30 ms. Frames match the old path to PSNR 60–68 dB (differences of a level or so on hard edges).
  // --legacy keeps the old path (a lossless PNG of the canvas through DevTools, scaled in ffmpeg) for comparison.
  //
  // Frames are converted to standard HD video: BT.709, limited (TV) range, tagged, so phones and Instagram/TikTok/X
  // transcoders read the colours and blacks correctly. --fast encodes with x264 "veryfast" at CRF 20, for previews only.
  // --ss 2 supersamples: the film draws every frame at 2× (?ss=2; the film must support it, see references/delivery.md) and each
  // frame is scaled back down. Four samples per pixel: smoother edges, text and fine detail.
  // Workers: the GPU is shared, so a WebGL film saturates it at about 2 workers on an M1; more only queue. --profile prints what
  // one frame costs (drawing, export) before you choose.
  import { spawn, spawnSync } from 'node:child_process';
  import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
  import { tmpdir } from 'node:os';
  import { join, dirname, resolve, relative } from 'node:path';
  import { openFilm, grab, parseArgs, serveDir } from './lib.mjs';
  
  const args = parseArgs(process.argv.slice(2));
  const html = args._[0];
  if (!html) { console.error('usage: node render.mjs <film.html> [--fps 30] [--audio file] [--out file.mp4] [--from s --to s] [--workers n] [--ss 2] [--root dir]'); process.exit(1); }
  const fps = Number(args.fps || 30), workers = Math.max(1, Number(args.workers || 2)), fast = !!args.fast, legacy = !!args.legacy;
  const crf = String(args.crf || (fast ? 20 : 16)), preset = fast ? 'veryfast' : 'slow';
  const ss = Math.max(1, Math.round(Number(args.ss || 1)));
  const out = resolve(args.out || join(dirname(resolve(html)), 'film.mp4'));
  const root = resolve(args.root || dirname(resolve(html)));
  const rel = relative(root, resolve(html));
  if (rel.startsWith('..')) { console.error(`--root ${root} must contain the film`); process.exit(1); }
  
  // the local server: the film's files, and the door frames come back through (one ordered queue per worker)
  const sinks = new Map();
  function onFrame(q, body, done) {
    const s = sinks.get(Number(q.get('w'))); s.waiting.set(Number(q.get('i')), { body, done });
    const pump = () => {
      while (s.waiting.has(s.next)) {
        const f = s.waiting.get(s.next); s.waiting.delete(s.next); s.next++; s.count++;
        // answer each POST once its frame is in ffmpeg's pipe, so a page never runs far ahead of the encoder
        if (!s.ff.stdin.write(f.body)) { s.ff.stdin.once('drain', () => { f.done(); pump(); }); return; }
        f.done();
      }
    };
    pump();
  }
  const server = await serveDir(root, onFrame);
  const pageUrl = server.url(rel);
  const open = (opts = {}) => openFilm(html, { gpu: !args['no-gpu'], url: legacy ? null : pageUrl, ...opts });
  
  const probe = await open();
  const { duration, w: ow, h: oh, gpu } = probe.info;                  // the film's own size
  probe.close();
  let w = ow, h = oh;
  if (ss > 1) {                                                          // the canvas must hold exactly ss× the pixels
    const p2 = await open({ ss }); ({ w, h } = p2.info); p2.close();
    if (w !== ow * ss || h !== oh * ss) { console.error(`--ss ${ss}: the film drew ${w}×${h}, not ${ow * ss}×${oh * ss}. Does it read ?ss= (references/delivery.md §3)?`); process.exit(1); }
  }
  
  // The exporter, put into each page: the canvas → a texture → Lanczos ↓ss in two separable passes (float) → RGBA8, top row
  // first. At ss = 1 it reads the texture as it is.
  const EXPORTER = String.raw`(() => {
    const src = document.getElementById('c') || document.querySelector('canvas'), SSX = ${ss}, OW = src.width / SSX, OH = src.height / SSX;
    const cv = document.createElement('canvas'); cv.width = OW; cv.height = OH;
    const g = cv.getContext('webgl2', { antialias: false, premultipliedAlpha: false, preserveDrawingBuffer: false });
    if (!g) throw new Error('render.mjs: no WebGL2 for the exporter (use --legacy)');
    if (SSX > 1 && !g.getExtension('EXT_color_buffer_float')) throw new Error('render.mjs: no float render targets (use --legacy)');
    const params = t => { for (const [k, v] of [[g.TEXTURE_MIN_FILTER, g.NEAREST], [g.TEXTURE_MAG_FILTER, g.NEAREST], [g.TEXTURE_WRAP_S, g.CLAMP_TO_EDGE], [g.TEXTURE_WRAP_T, g.CLAMP_TO_EDGE]]) g.texParameteri(g.TEXTURE_2D, k, v); };
    const fbOf = t => { const f = g.createFramebuffer(); g.bindFramebuffer(g.FRAMEBUFFER, f); g.framebufferTexture2D(g.FRAMEBUFFER, g.COLOR_ATTACHMENT0, g.TEXTURE_2D, t, 0); return f; };
    const tIn = g.createTexture(); g.bindTexture(g.TEXTURE_2D, tIn); params(tIn);
    g.pixelStorei(g.UNPACK_FLIP_Y_WEBGL, false); g.pixelStorei(g.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
    const px = new Uint8Array(OW * OH * 4);
    let draw = null;
    if (SSX > 1) {
      const VS = '#version 300 es\nin vec2 p; void main(){ gl_Position = vec4(p, 0., 1.); }';
      const FS = dir => '#version 300 es\n' +
        'precision highp float; uniform sampler2D t; uniform vec2 inSize; out vec4 o;\n' +
        'float L(float x){ x = abs(x); if (x < 1e-5) return 1.; if (x >= 3.) return 0.; float a = 3.14159265*x; return 3.*sin(a)*sin(a/3.)/(a*a); }\n' +
        'void main(){\n' +
        '  float c = ' + (dir === 'x' ? 'gl_FragCoord.x' : 'gl_FragCoord.y') + '*' + SSX + '.;\n' +
        '  vec4 s = vec4(0.); float ws = 0.;\n' +
        '  for (int k = -' + 3 * SSX + '; k <= ' + 3 * SSX + '; k++){\n' +
        '    float x = floor(c) + float(k) + .5, w = L((x - c)/' + SSX + '.);\n' +
        '    vec2 uv = ' + (dir === 'x' ? 'vec2(clamp(x, .5, inSize.x - .5), gl_FragCoord.y)' : 'vec2(gl_FragCoord.x, clamp(x, .5, inSize.y - .5))') + '/inSize;\n' +
        '    s += texture(t, uv)*w; ws += w;\n' +
        '  }\n' +
        '  o = s/ws;\n' +
        '}';
      const prog = fs => { const sh = (ty, s) => { const x = g.createShader(ty); g.shaderSource(x, s); g.compileShader(x); if (!g.getShaderParameter(x, g.COMPILE_STATUS)) throw new Error(g.getShaderInfoLog(x)); return x; };
        const p = g.createProgram(); g.attachShader(p, sh(g.VERTEX_SHADER, VS)); g.attachShader(p, sh(g.FRAGMENT_SHADER, fs)); g.bindAttribLocation(p, 0, 'p'); g.linkProgram(p); return p; };
      const PX = prog(FS('x')), PY = prog(FS('y'));
      const vb = g.createBuffer(); g.bindBuffer(g.ARRAY_BUFFER, vb); g.bufferData(g.ARRAY_BUFFER, new Float32Array([-1, -1, 3, -1, -1, 3]), g.STATIC_DRAW);
      g.enableVertexAttribArray(0); g.vertexAttribPointer(0, 2, g.FLOAT, false, 0, 0);
      const store = (w, h, fmt) => { const t = g.createTexture(); g.bindTexture(g.TEXTURE_2D, t); g.texStorage2D(g.TEXTURE_2D, 1, fmt, w, h); params(t); return t; };
      const tMid = store(OW, src.height, g.RGBA32F), tOut = store(OW, OH, g.RGBA8);
      const fMid = fbOf(tMid), fOut = fbOf(tOut);
      const pass = (p, tex, fb, w, h, inW, inH) => { g.bindTexture(g.TEXTURE_2D, tex); g.bindFramebuffer(g.FRAMEBUFFER, fb); g.viewport(0, 0, w, h);
        g.useProgram(p); g.uniform1i(g.getUniformLocation(p, 't'), 0); g.uniform2f(g.getUniformLocation(p, 'inSize'), inW, inH); g.drawArrays(g.TRIANGLES, 0, 3); };
      draw = () => { pass(PX, tIn, fMid, OW, src.height, src.width, src.height); pass(PY, tMid, fOut, OW, OH, OW, src.height); };
    }
    const fIn = fbOf(tIn);
    window.__export = () => {
      g.bindTexture(g.TEXTURE_2D, tIn); g.texImage2D(g.TEXTURE_2D, 0, g.RGBA, g.RGBA, g.UNSIGNED_BYTE, src);   // row 0 = the top of the frame
      if (draw) draw(); else g.bindFramebuffer(g.FRAMEBUFFER, fIn);
      g.readPixels(0, 0, OW, OH, g.RGBA, g.UNSIGNED_BYTE, px);                                               // so the rows come out top first
      return px;
    };
    window.__runRange = async (a, b, fps, w, depth) => {
      const inflight = [];
      for (let i = a; i <= b; i++) {
        __film.seek(i / fps);
        inflight.push(fetch('/frame?w=' + w + '&i=' + i, { method: 'POST', body: __export() }).then(r => r.text()));   // the body is copied at the call
        if (inflight.length >= depth) await inflight.shift();
      }
      await Promise.all(inflight);
      return true;
    };
    return [OW, OH];
  })()`;
  
  if (args.profile) {                                                    // what one frame costs, in one page
    const film = await open({ ss });
    if (!legacy) await film.ev(EXPORTER);
    const r = await film.ev(`(() => {
      const c = document.getElementById('c') || document.querySelector('canvas'), T = []; for (let i = 0; i < 40; i++) T.push(${Number(args.profile) > 1 ? Number(args.profile) : duration / 3} + i / ${fps});
      const sync = () => { try { c.getContext('2d') ? c.getContext('2d').getImageData(0, 0, 1, 1) : c.toDataURL('image/jpeg', .1); } catch { c.toDataURL('image/jpeg', .1); } };
      let draw = 0, exp = 0;
      for (const t of T) { let a = performance.now(); __film.seek(t); sync(); let b = performance.now(); draw += b - a;
        ${legacy ? "c.toDataURL('image/png');" : '__export();'} exp += performance.now() - b; }
      return { draw_ms: +(draw / T.length).toFixed(1), export_ms: +(exp / T.length).toFixed(1) };
    })()`);
    console.log(`one frame at ${w}×${h}:`, r, legacy ? '(legacy export; the DevTools transfer comes on top)' : '');
    film.close(); server.close(); process.exit(0);
  }
  
  const from = Number(args.from || 0), to = Math.min(Number(args.to ?? duration), duration);
  const first = Math.round(from * fps), last = Math.ceil(to * fps) - 1, total = last - first + 1;
  console.log(`${ow}×${oh}${ss > 1 ? ` (drawn at ${w}×${h}, ${ss}× supersampled)` : ''} · ${fps} fps · frames ${first}–${last} (${total}) · ${workers} worker(s) · ${legacy ? 'legacy PNG export' : 'raw export'} · GPU: ${gpu}`);
  
  const tmp = mkdtempSync(join(tmpdir(), 'render-'));
  const t0 = Date.now();
  let done = 0;
  const tick = () => { const n = legacy ? done : [...sinks.values()].reduce((a, s) => a + s.count, 0), sec = (Date.now() - t0) / 1000;
    process.stdout.write(`\r${n}/${total} frames · ${sec.toFixed(0)}s · ${(n / Math.max(sec, .001)).toFixed(1)} fps`); };
  const timer = setInterval(tick, 2000);
  const toHD = 'scale=out_range=tv:out_color_matrix=bt709:flags=accurate_rnd+full_chroma_int,format=yuv420p,setparams=range=tv:colorspace=bt709:color_primaries=bt709:color_trc=bt709';
  const enc = seg => ['-color_range', 'tv', '-colorspace', 'bt709', '-color_primaries', 'bt709', '-color_trc', 'bt709',
    '-c:v', 'libx264', '-preset', preset, '-crf', crf, '-profile:v', 'high', seg];
  
  async function renderRange(k, a, b, seg) {
    const film = await open({ ss });
    if (legacy) {
      const down = ss > 1 ? `scale=${ow}:${oh}:flags=lanczos+accurate_rnd,format=rgb48le,` : '';
      const ff = spawn('ffmpeg', ['-v', 'error', '-y', '-f', 'image2pipe', '-framerate', String(fps), '-c:v', 'png', '-i', '-', '-vf', down + toHD, ...enc(seg)], { stdio: ['pipe', 'inherit', 'inherit'] });
      for (let i = a; i <= b; i++) {
        const buf = await grab(film.ev, i / fps, 'image/png');
        if (!ff.stdin.write(buf)) await new Promise(r => ff.stdin.once('drain', r));
        done++;
      }
      ff.stdin.end(); await new Promise(r => ff.on('close', r));
    } else {
      const [OW, OH] = await film.ev(EXPORTER);
      const ff = spawn('ffmpeg', ['-v', 'error', '-y', '-f', 'rawvideo', '-pix_fmt', 'rgba', '-s', `${OW}x${OH}`, '-framerate', String(fps), '-i', '-', '-vf', toHD, ...enc(seg)],
        { stdio: ['pipe', 'inherit', 'inherit'] });
      sinks.set(k, { next: a, waiting: new Map(), ff, count: 0 });
      await film.ev(`__runRange(${a}, ${b}, ${fps}, ${k}, 3)`);
      ff.stdin.end(); await new Promise(r => ff.on('close', r));
    }
    if (film.logs.length) console.error('\npage errors:\n' + film.logs.join('\n'));
    film.close();
  }
  
  const per = Math.ceil(total / workers), segs = [];
  await Promise.all(Array.from({ length: workers }, (_, k) => {
    const a = first + k * per, b = Math.min(last, a + per - 1);
    if (a > b) return null;
    const seg = join(tmp, `seg${String(k).padStart(3, '0')}.mp4`); segs.push(seg);
    return renderRange(k, a, b, seg);
  }));
  clearInterval(timer); tick(); console.log();
  server.close();
  
  segs.sort();
  writeFileSync(join(tmp, 'list.txt'), segs.map(s => `file '${s}'`).join('\n'));
  const video = join(tmp, 'video.mp4');
  spawnSync('ffmpeg', ['-v', 'error', '-y', '-f', 'concat', '-safe', '0', '-i', join(tmp, 'list.txt'), '-c', 'copy', video], { stdio: 'inherit' });
  const mux = args.audio
    ? ['-v', 'error', '-y', '-i', video, '-ss', String(from), '-i', resolve(args.audio), '-map', '0:v', '-map', '1:a', '-c:v', 'copy',
       '-c:a', 'aac', '-b:a', '192k', '-shortest', '-movflags', '+faststart', out]
    : ['-v', 'error', '-y', '-i', video, '-c', 'copy', '-movflags', '+faststart', out];
  spawnSync('ffmpeg', mux, { stdio: 'inherit' });
  rmSync(tmp, { recursive: true, force: true });
  
  const p = spawnSync('ffprobe', ['-v', 'error', '-count_packets', '-show_entries', 'stream=codec_type,width,height,nb_read_packets:format=duration',
    '-of', 'compact', out]).stdout.toString().trim();
  console.log(`done → ${out} in ${((Date.now() - t0) / 1000).toFixed(0)} s\n${p}`);
  process.exit(0);
  

• skills/opus-js-animations/scripts/stills.mjs3.7 KB69fbb327aa9cupstream ↗

  #!/usr/bin/env node
  // Exact stills and labelled contact sheets, drawn by seek(t) (never a realtime screenshot).
  //
  //   node stills.mjs film.html --times 0.5,3,7.2              individual JPEGs
  //   node stills.mjs film.html --every 2 --sheet              whole film, one frame per 2 s
  //   node stills.mjs film.html --range 6.6:7.2:0.0333 --sheet frame-by-frame strip at a handoff
  //   node stills.mjs film.html --times 12,18 --crop 380,1040,440,380 --sheet   1:1 detail crops
  //
  // Options: --out dir (default <film dir>/stills)  --cell 300 (sheet cell width)  --cols 6
  //          --png (lossless frames)  --ss 2 (supersampled, as render.mjs --ss 2)  --no-gpu
  // With --ss, crops and cells stay in film pixels (the sheet boxes the supersampled frame down); single stills keep the full canvas.
  // Read every sheet you make; a sheet can't judge pacing, so also watch a render.
  import { mkdirSync, writeFileSync } from 'node:fs';
  import { join, dirname, resolve } from 'node:path';
  import { openFilm, grab, parseArgs } from './lib.mjs';
  
  const args = parseArgs(process.argv.slice(2));
  const html = args._[0];
  if (!html) { console.error('usage: node stills.mjs <film.html> (--times a,b | --range a:b:step | --every s) [--sheet] [--crop x,y,w,h] [--ss 2]'); process.exit(1); }
  const ss = Math.max(1, Math.round(Number(args.ss || 1)));
  const film = await openFilm(html, { gpu: !args['no-gpu'], ss });
  const { duration } = film.info, w = film.info.w / ss, h = film.info.h / ss;
  
  let times;
  if (args.times) times = String(args.times).split(',').map(Number);
  else if (args.range) { const [a, b, s] = String(args.range).split(':').map(Number); times = []; for (let t = a; t <= b + 1e-9; t += s) times.push(+t.toFixed(4)); }
  else { const s = Number(args.every || Math.max(1, duration / 12)); times = []; for (let t = s / 2; t < duration; t += s) times.push(+t.toFixed(3)); }
  
  const outDir = resolve(args.out || join(dirname(resolve(html)), 'stills'));
  mkdirSync(outDir, { recursive: true });
  const crop = args.crop ? String(args.crop).split(',').map(Number) : null;
  
  if (args.sheet) {
    const cols = Number(args.cols || Math.min(times.length, crop ? 4 : 6));
    const cellW = Number(args.cell || (crop ? crop[2] : 300));
    const src = crop || [0, 0, w, h];
    const cellH = Math.round(cellW * src[3] / src[2]);
    const url = await film.ev(`(() => {
      const times = ${JSON.stringify(times)}, src = ${JSON.stringify(src.map(v => v * ss))}, cols = ${cols}, cw = ${cellW}, ch = ${cellH}, lab = 26;
      const rows = Math.ceil(times.length / cols), main = document.getElementById('c') || document.querySelector('canvas');
      const s = document.createElement('canvas'); s.width = cols * cw; s.height = rows * (ch + lab);
      const x = s.getContext('2d'); x.fillStyle = '#111'; x.fillRect(0, 0, s.width, s.height);
      x.font = '15px ui-monospace, Menlo, monospace'; x.textBaseline = 'middle';
      times.forEach((t, i) => {
        __film.seek(t);
        const cx = (i % cols) * cw, cy = Math.floor(i / cols) * (ch + lab);
        x.drawImage(main, src[0], src[1], src[2], src[3], cx, cy, cw, ch);
        x.fillStyle = '#ddd'; x.fillText(t.toFixed(t % 1 ? 3 : 1) + 's', cx + 8, cy + ch + lab / 2);
      });
      return s.toDataURL('image/jpeg', .9);
    })()`);
    const name = `sheet_${times[0]}-${times[times.length - 1]}${crop ? '_crop' : ''}.jpg`;
    writeFileSync(join(outDir, name), Buffer.from(url.split(',')[1], 'base64'));
    console.log(join(outDir, name));
  } else {
    for (const t of times) {
      const ext = args.png ? 'png' : 'jpg';
      const buf = await grab(film.ev, t, args.png ? 'image/png' : 'image/jpeg', .92);
      writeFileSync(join(outDir, `t_${t}.${ext}`), buf);
    }
    console.log(`${times.length} stills → ${outDir}`);
  }
  if (film.logs.length) console.error('page errors:\n' + film.logs.join('\n'));
  film.close();
  

• skills/opus-js-animations/scripts/verify.mjs1.8 KB23e9e5ae44e3upstream ↗

  #!/usr/bin/env node
  // Proves the contract: seek(t) is pure. The same t twice, and a cold jump to t after
  // seeking elsewhere, must give identical pixels. A failure means something in the draw
  // path depends on history (Math.random, frame counters, mutable state), which shows up as
  // flicker and makes the exported MP4 differ from what you inspected.
  //
  //   node verify.mjs film.html [--times 0.5,10,20] [--ss 2]   (default: 9 times across the film; --ss checks the supersampled path)
  import { createHash } from 'node:crypto';
  import { openFilm, parseArgs } from './lib.mjs';
  
  const args = parseArgs(process.argv.slice(2));
  const film = await openFilm(args._[0], { gpu: !args['no-gpu'], ss: Math.max(1, Math.round(Number(args.ss || 1))) });
  const { duration, w, h, gpu } = film.info;
  const times = args.times ? String(args.times).split(',').map(Number)
    : Array.from({ length: 9 }, (_, i) => +(duration * (i + .5) / 9).toFixed(3));
  
  const hash = async t => {
    const px = await film.ev(`(() => { __film.seek(${t}); const c = document.getElementById('c') || document.querySelector('canvas');
      return c.toDataURL('image/png'); })()`);
    return createHash('sha1').update(px).digest('hex').slice(0, 12);
  };
  let fail = 0;
  console.log(`${w}×${h}, ${duration}s, GPU: ${gpu}`);
  for (const t of times) {
    const a = await hash(t), b = await hash(t);
    await hash((t + duration / 2) % duration);        // wander off
    const c = await hash(t);                           // cold jump back
    const ok = a === b && a === c;
    if (!ok) fail++;
    console.log(`${ok ? 'ok  ' : 'FAIL'} t=${t}  ${a} ${b} ${c}`);
  }
  if (film.logs.length) { console.log('page errors:\n' + film.logs.join('\n')); fail++; }
  film.close();
  console.log(fail ? `${fail} problem(s)` : 'seek(t) is pure at every tested time');
  process.exit(fail ? 1 : 0);
  

• skills/opus-js-animations/scripts/voiceover.py11 KBd5343cce3bd2upstream ↗

  #!/usr/bin/env python3
  """Voice a script with a text-to-speech API (ElevenLabs or OpenAI), line by line, with timings.
  
    python3 voiceover.py script.txt --provider elevenlabs [--voice "Rachel" | <voice_id>] [--model eleven_multilingual_v2]
    python3 voiceover.py script.txt --provider openai [--voice coral] [--model gpt-4o-mini-tts] [--instructions "warm, unhurried"]
    python3 voiceover.py --provider elevenlabs --list-voices
    python3 voiceover.py script.txt --provider openai --only 4          re-voice line 4 only, keep the other takes
  
  The script is plain text: one phrase per line, in the order spoken. A blank line is a longer pause (a paragraph).
  Each line is voiced on its own, so phrase boundaries are exact, one line can be re-voiced without redoing the rest,
  and the picture can be timed to the measured durations instead of guesses.
  
  The key comes from the environment (ELEVENLABS_API_KEY or OPENAI_API_KEY), from --key-file (a file holding only
  the key), or from ~/.config/opus-js-animations/keys.env (KEY=value lines; keep it chmod 600). It is never printed or written.
  
  Writes into --out (default ./source):
    voice.wav          the whole voiceover: 48 kHz mono, lines joined with --gap / --para seconds of silence
    voice.mp3          the same, for embed_audio.py
    voice.align.json   the same format as align_audio.py: one chunk per line with t0/t1 and, from ElevenLabs,
                       word times from the API's character alignment (OpenAI returns none: run align_audio.py)
    lines/NN.mp3       each line's take, reused unless --only or --redo names it
  
  Stdlib only (plus ffmpeg on PATH). Voice names, models and prices change: --list-voices shows what the key can use.
  Say which provider and voice made the audio, and check its terms for the platform the film is for.
  """
  import argparse, base64, json, os, shutil, subprocess, sys, urllib.error, urllib.request, wave
  from pathlib import Path
  
  ap = argparse.ArgumentParser()
  ap.add_argument('script', nargs='?', help='text file: one phrase per line, blank line = paragraph pause')
  ap.add_argument('--provider', choices=['elevenlabs', 'openai'], required=True)
  ap.add_argument('--voice', help='ElevenLabs: voice name or id (default Rachel). OpenAI: alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer, verse (default coral)')
  ap.add_argument('--model', help='ElevenLabs default eleven_multilingual_v2; OpenAI default gpt-4o-mini-tts')
  ap.add_argument('--instructions', help='OpenAI gpt-4o-mini-tts only: how to speak (tone, pace, accent)')
  ap.add_argument('--stability', type=float, default=.5, help='ElevenLabs voice stability 0–1 (lower = more expressive)')
  ap.add_argument('--similarity', type=float, default=.75, help='ElevenLabs similarity boost 0–1')
  ap.add_argument('--style', type=float, default=0.0, help='ElevenLabs style exaggeration 0–1')
  ap.add_argument('--speed', type=float, default=1.0, help='speaking rate (OpenAI 0.25–4; ElevenLabs 0.7–1.2)')
  ap.add_argument('--gap', type=float, default=.35, help='silence between lines (s)')
  ap.add_argument('--para', type=float, default=.9, help='silence at a blank line (s)')
  ap.add_argument('--lead', type=float, default=.25, help='silence before the first line (s)')
  ap.add_argument('--tail', type=float, default=.8, help='silence after the last line (s)')
  ap.add_argument('--out', default='source')
  ap.add_argument('--only', type=int, nargs='*', help='re-voice only these line numbers (1-based), keep the rest')
  ap.add_argument('--redo', action='store_true', help='re-voice every line')
  ap.add_argument('--key-file')
  ap.add_argument('--list-voices', action='store_true')
  a = ap.parse_args()
  
  ENV = {'elevenlabs': 'ELEVENLABS_API_KEY', 'openai': 'OPENAI_API_KEY'}[a.provider]
  SR = 48000
  
  
  def api_key():
      if a.key_file:
          return Path(a.key_file).expanduser().read_text().strip()
      if os.environ.get(ENV):
          return os.environ[ENV].strip()
      cfg = Path.home() / '.config' / 'opus-js-animations' / 'keys.env'
      if cfg.exists():
          for line in cfg.read_text().splitlines():
              k, _, v = line.partition('=')
              if k.strip() == ENV and v.strip():
                  return v.strip().strip('"\'')
      sys.exit(f'No {ENV}. Set it in the environment, pass --key-file, or add {ENV}=... to {cfg} (chmod 600).')
  
  
  def request(url, key, body=None, accept='application/json'):
      headers = {'Accept': accept}
      if a.provider == 'elevenlabs': headers['xi-api-key'] = key
      else: headers['Authorization'] = f'Bearer {key}'
      data = None
      if body is not None:
          headers['Content-Type'] = 'application/json'
          data = json.dumps(body).encode()
      req = urllib.request.Request(url, data=data, headers=headers, method='POST' if data else 'GET')
      try:
          with urllib.request.urlopen(req, timeout=180) as r:
              return r.read()
      except urllib.error.HTTPError as e:
          msg = e.read().decode(errors='replace')[:600]
          sys.exit(f'{a.provider} API error {e.code}: {msg}')
      except urllib.error.URLError as e:
          sys.exit(f'{a.provider} API unreachable: {e.reason}')
  
  
  def list_voices(key):
      if a.provider == 'openai':
          print('OpenAI voices: alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer, verse (see the API docs for new ones)')
          return
      vs = json.loads(request('https://api.elevenlabs.io/v1/voices', key)).get('voices', [])
      for v in vs:
          labels = ', '.join(f'{k}: {x}' for k, x in (v.get('labels') or {}).items())
          print(f"{v['voice_id']}  {v['name']:<22} {v.get('category', ''):<12} {labels}")
  
  
  def eleven_voice_id(key, want):
      want = want or 'Rachel'
      if len(want) >= 20 and want.isalnum():
          return want
      vs = json.loads(request('https://api.elevenlabs.io/v1/voices', key)).get('voices', [])
      for v in vs:
          if v['name'].lower() == want.lower() or v['name'].lower().startswith(want.lower() + ' '):
              return v['voice_id']
      sys.exit(f'No ElevenLabs voice named "{want}". Run with --list-voices.')
  
  
  def voice_line(key, text, voice_id):
      """Returns (mp3 bytes, [(word, t0, t1)] or None)."""
      if a.provider == 'elevenlabs':
          body = {'text': text, 'model_id': a.model or 'eleven_multilingual_v2',
                  'voice_settings': {'stability': a.stability, 'similarity_boost': a.similarity, 'style': a.style,
                                     'use_speaker_boost': True, **({'speed': a.speed} if a.speed != 1.0 else {})}}
          r = json.loads(request(f'https://api.elevenlabs.io/v1/text-to-speech/{voice_id}/with-timestamps?output_format=mp3_44100_128', key, body))
          al = r.get('alignment') or r.get('normalized_alignment') or {}
          return base64.b64decode(r['audio_base64']), chars_to_words(al)
      body = {'model': a.model or 'gpt-4o-mini-tts', 'voice': a.voice or 'coral', 'input': text, 'response_format': 'mp3'}
      if a.speed != 1.0: body['speed'] = a.speed
      if a.instructions: body['instructions'] = a.instructions
      return request('https://api.openai.com/v1/audio/speech', key, body, accept='audio/mpeg'), None
  
  
  def chars_to_words(al):
      cs, s0, s1 = al.get('characters') or [], al.get('character_start_times_seconds') or [], al.get('character_end_times_seconds') or []
      words, cur, t0 = [], '', None
      for c, a0, a1 in zip(cs, s0, s1):
          if c.isspace():
              if cur: words.append((cur, t0, prev)); cur = ''
              continue
          if not cur: t0 = a0
          cur += c; prev = a1
      if cur: words.append((cur, t0, prev))
      return words
  
  
  def pcm(path):
      """Decode any audio file to 48 kHz mono s16 PCM bytes."""
      return subprocess.run(['ffmpeg', '-v', 'error', '-i', str(path), '-ac', '1', '-ar', str(SR), '-f', 's16le', '-'],
                            capture_output=True, check=True).stdout
  
  
  if not shutil.which('ffmpeg'):
      sys.exit('needs ffmpeg on PATH (macOS: brew install ffmpeg)')
  key = api_key()
  if a.list_voices:
      list_voices(key); sys.exit()
  if not a.script:
      sys.exit('give a script file (one phrase per line)')
  
  raw = Path(a.script).read_text(encoding='utf-8').splitlines()
  lines, breaks = [], set()                      # breaks: indices of lines that follow a blank line
  for s in raw:
      if not s.strip():
          if lines: breaks.add(len(lines))
          continue
      lines.append(s.strip())
  if not lines:
      sys.exit('the script is empty')
  
  out = Path(a.out); (out / 'lines').mkdir(parents=True, exist_ok=True)
  voice_id = eleven_voice_id(key, a.voice) if a.provider == 'elevenlabs' else None
  todo = set(range(1, len(lines) + 1)) if a.redo else set(a.only or [])
  meta_path = out / 'lines' / 'takes.json'
  takes = json.loads(meta_path.read_text()) if meta_path.exists() else {}
  
  for i, text in enumerate(lines, 1):
      f = out / 'lines' / f'{i:02d}.mp3'
      same = takes.get(str(i), {}).get('text') == text
      if f.exists() and same and i not in todo:
          continue
      print(f'voicing {i}/{len(lines)}: {text[:70]}')
      audio, words = voice_line(key, text, voice_id)
      f.write_bytes(audio)
      takes[str(i)] = {'text': text, 'provider': a.provider, 'voice': a.voice or ('Rachel' if a.provider == 'elevenlabs' else 'coral'),
                       'model': a.model, 'words': words}
  meta_path.write_text(json.dumps(takes, ensure_ascii=False, indent=1))
  
  # join the takes with measured silences
  silence = lambda s: b'\x00\x00' * int(round(s * SR))
  buf, chunks, t = bytearray(silence(a.lead)), [], a.lead
  for i, text in enumerate(lines, 1):
      if i > 1:
          g = a.para if (i - 1) in breaks else a.gap
          buf += silence(g); t += g
      p = pcm(out / 'lines' / f'{i:02d}.mp3')
      d = len(p) / 2 / SR
      ws = takes[str(i)].get('words')
      chunks.append({'t0': round(t, 3), 't1': round(t + d, 3), 'text': text,
                     'words': [{'w': w, 't0': round(t + w0, 3), 't1': round(t + w1, 3)} for w, w0, w1 in ws] if ws else []})
      buf += p; t += d
  buf += silence(a.tail); t += a.tail
  
  with wave.open(str(out / 'voice.wav'), 'wb') as w:
      w.setnchannels(1); w.setsampwidth(2); w.setframerate(SR); w.writeframes(bytes(buf))
  subprocess.run(['ffmpeg', '-v', 'error', '-y', '-i', str(out / 'voice.wav'), '-c:a', 'libmp3lame', '-q:a', '2', str(out / 'voice.mp3')], check=True)
  (out / 'voice.align.json').write_text(json.dumps({'duration': round(t, 3), 'start': a.lead, 'dips': [], 'chunks': chunks,
                                                    'source': {'provider': a.provider, 'voice': a.voice, 'model': a.model}},
                                                   ensure_ascii=False, indent=1))
  for c in chunks:
      print(f"[{c['t0']:6.2f} – {c['t1']:6.2f}] {c['text']}")
  print(f'\n{len(lines)} lines, {t:.2f} s → {out / "voice.wav"}, {out / "voice.mp3"}, {out / "voice.align.json"}')
  if a.provider == 'openai':
      print('OpenAI returns no word times: run align_audio.py on voice.wav if picture events must land on words.')
  

• skills/opus-js-animations/references/audio.md14 KB91ab0b19945fupstream ↗

  # Audio: sourcing, understanding, generating, voicing, mixing
  
  ## Contents
  1. The four sources
  2. Getting and trimming source audio (a file or a link)
  3. Understanding it: analysis and aligning text to a voice
  4. Using audio in the page
  5. Generating the sound in JavaScript (Web Audio)
  6. Dialogue, sound effects and the mix (longer films)
  7. Mux and checks
  8. Voiceover from a script (ElevenLabs or OpenAI)
  
  ## 1. The four sources
  
  Ask which one it is before anything else (`directing.md` §2):
  
  | Source | Picture follows | Tool | Notes |
  |---|---|---|---|
  | Their own file (voice, reading, song, music) | the audio: every beat is a word or hit | none; `ffmpeg` to take audio from a video | use it untouched (§7) |
  | A link (YouTube, or any site yt-dlp supports) | the audio | `get_audio.py` (§2) | say where it came from; reuse rights are the person's call |
  | Generated in JavaScript | the story; the score reads the picture's event times | Web Audio in the page, `page_audio.mjs` (§5) | deterministic, rendered offline for export |
  | A voiceover from their TTS key | the script: each line is a sync point | `voiceover.py` (§8) | approve the script before voicing it |
  
  Longer films mix several of these (dialogue plus score plus effects, §6). No music under a voice wherever the person asks for none.
  
  The picture and the sound share one timeline defined as data (phrase list, cue list, event
  times). Never copy numbers between them by hand; retiming one must retime the other.
  
  ## 2. Getting and trimming source audio
  
  Use `scripts/get_audio.py`; it does the steps below and handles the tools:
  - **Install**: finds a working yt-dlp, else installs it with Homebrew → pipx → `pip --user` →
    the official standalone binary into `~/.local/bin` (no Python needed). ffmpeg is installed
    with Homebrew when available; otherwise it stops with the install command for the platform.
  - **Stale yt-dlp**: most failures ("Unable to extract", HTTP 403, signature errors) are fixed by
    a newer version, so on failure it updates (brew/pipx/pip, or `yt-dlp -U` for the binary) and
    retries once.
  - **Output**: `source.wav` (48 kHz), `meta.txt`, and with `--video` a 720p `clip.mp4` plus `sheet.png`.
  - **End check**: loudness of the last 0.4 s. A loud tail means the section stops mid-word. This
    happened in practice: a 55 s section cut the last word of a line, found only after the film
    was finished. Take ~10 s more than you need and trim at a measured quiet point.
  - `--search "query"` lists the top 8 results (id, length, uploader, title) when there is no URL.
  
  The manual equivalent:
  
  ```bash
  # a section of an online video (video kept for reading on-screen text; audio extracted)
  yt-dlp -q --no-playlist --download-sections "*0-55" --force-keyframes-at-cuts \
    -f "bv*[height<=720]+ba/b[height<=720]" --merge-output-format mp4 -o "clip.%(ext)s" "<url>"
  yt-dlp --skip-download --print "%(title)s|%(uploader)s|%(duration)s" --print description "<url>"  # metadata first
  ffmpeg -v error -y -i clip.mp4 -vn -c:a libmp3lame -q:a 2 voice.mp3
  ffmpeg -v error -y -i clip.mp4 -vf "fps=1/3,scale=640:-1,tile=3x3" sheet_%d.png   # read on-screen text/translation
  ```
  
  The title, description and on-screen text often identify the content exactly (title,
  author, lyrics, speaker). Read them before transcribing.
  
  Trim at a measured quiet point, never mid-word, on a zero crossing, so no fade is needed (§7):
  ```bash
  ffmpeg -v error -y -ss 7.35 -i voice.mp3 -c:a libmp3lame -q:a 2 voice-trimmed.mp3   # add -af "afade=t=in:d=0.005" only if a click remains
  ```
  When trimming the start, keep scene times in the original clip's seconds and apply one
  constant offset (`SHIFT`) in the player and renderer; this keeps every measured time valid.
  
  Respect rights: say where the audio came from and let the user decide about reuse.
  
  ## 3. Understanding it: analysis and aligning text to a voice
  
  Start with the whole shape of the sound:
  ```bash
  python3 scripts/analyze_audio.py source/source.wav
  ```
  It prints:
  - the length, loudness (LUFS) and true peak;
  - the pauses;
  - sections by energy (quiet, mid, loud), with their times;
  - onsets (hits and syllable attacks);
  - an estimated tempo with beat times, when there is a steady beat.
  
  It writes `source.analysis.json`, a spectrogram with a labelled time axis (`source.spectrum.png`) and a waveform (`source.wave.png`).
  Read the spectrogram: phrases, pauses, beats, swells and faint endings are visible at a glance. Report what you hear before you design
  (`directing.md` §3). For music, the beat times are candidate cut points and event times. For a voice, align the words:
  
  
  **Quiet endings are real.** A speaker or singer often says the last words softly or under a pad. Whisper-small can drop them, so a transcript
  that stops early proves nothing. Re-transcribe the last few seconds with a larger model on a loudness-normalised clip before deciding
  what the audio contains:
  ```bash
  ffmpeg -ss 15.9 -i clip.mp4 -vn -ac 1 -ar 16000 -af loudnorm=I=-16 tail.wav
  python3 -c "import whisper; r = whisper.load_model('medium').transcribe('tail.wav', language='en', word_timestamps=True, condition_on_previous_text=False); print(r['text'])"
  ```
  Keep the user's audio whole: no tail fades or trims beyond a few-millisecond de-click, unless they ask.
  
  ```bash
  python3 scripts/align_audio.py voice.mp3 --lang en      # or another language code, or omit for auto
  ```
  It prints breath points (loudness dips), a transcript per breath-to-breath chunk with word
  times, and writes `voice.align.json`. Then:
  
  1. Take the **displayed text from a canonical source**, not the transcript (see
     `quoted-text.md`). The transcript only locates the words.
  2. Look for repeats: speakers and singers often repeat a phrase (a chorus, a
     line said twice for weight). A phrase list must show
     the repeat.
  3. Split into phrases at breath points; each phrase gets `t0` (word onset − ~0.2 s) and
     `t1` (next phrase's onset).
  4. Silence detection (`silencedetect`) fails on reverberant voices (chant, sung or elongated speech); loudness dips at
     0.1 s resolution work (`astats` RMS per 1600 samples at 16 kHz, or the script above).
  5. **Refine cues that picture events land on** (a posture change on a word) from the signal
     itself. Whisper word times can be off by up to ~1 s on elongated or sung words: the first word of
     a window is pinned to the window start, and neighbouring words share boundaries. Two passes
     with different models disagreed by 0.7–0.9 s. Use:
     - **stop consonants** (q, k, t, b, j): a sharp 30–60 ms loudness dip (10 ms RMS windows, local
       minimum > 7 dB below its ±80 ms neighbourhood);
     - **fricatives** (s, sh, f, h): a peak in the ratio of energy above 4 kHz to energy below it (2048-point FFT every 10 ms).
     ```python
     S = np.abs(np.fft.rfft(seg * np.hanning(2048))) ** 2; f = np.fft.rfftfreq(2048, 1 / sr)
     hf = 10 * np.log10(S[f > 4000].sum() / S[(f > 80) & (f < 4000)].sum())   # > ~5 dB: /s/-like
     ```
     Cross-check each cue with two independent measures before timing picture to it.
  
  ## 4. Using audio in the page
  
  Pages opened from `file://` can't `fetch()` local files, and an `<audio>` element routed
  through Web Audio outputs silence. Embed the file as base64:
  ```bash
  python3 scripts/embed_audio.py voice-trimmed.mp3 film/audio.js
  ```
  The template decodes `window.FILM_AUDIO_B64` with `decodeAudioData` and plays it with an
  `AudioBufferSourceNode`; `ac.currentTime - t0` is the film clock, so picture and sound
  can't drift. Autoplay needs a click; keep a "click to play" card.
  
  ## 5. Procedural score (Web Audio)
  
  Build instruments from oscillators, filters and envelopes; schedule notes from an event list:
  - Pad: 2 detuned saws per chord note → low-pass (600→1500→700 Hz sweep) → slow envelope.
  - Bell / music box: sine at f + sine at 4.01f (fast decay) + triangle at 2f; reverb and a dotted-eighth delay.
  - Pluck arpeggio: triangle → low-pass envelope 3200→500 Hz, 0.7 s decay, alternating pan.
  - Bass: sine plus low-passed triangle; kick: sine 130→42 Hz in 0.14 s.
  - Reverb: `ConvolverNode` with a generated impulse (noise × (1 − i/n)^2.6, 3.4 s).
  - SFX from noise: whoosh (band-pass sweep), firework (whistle + low-pass boom + crackles),
    rain (high-pass + band-pass loop), thunder (low-pass 170 Hz, 3 s decay).
  
  - Foley kit for small physical worlds (music box, Karplus–Strong wire, wing flaps, chirps, paper
    rustle, ceramic clink, creak, per-shot room tones): `styles.md` §2.
  - Sounds born from physics (a landing, a petal touching down): `emit()` them from the simulation;
    for export, simulate the whole film once with a baking flag to record them with their times, then
    render them with the score.
  
  Live playback: a look-ahead scheduler (every 40 ms, schedule events up to 1 s ahead) on a
  per-session bus; restarting fades the old bus out. Export: render the same event list in an
  `OfflineAudioContext`, return it from `__film.wav()` as a base64 WAV, and pull it out with
  `scripts/page_audio.mjs` (it also prints integrated LUFS and true peak).
  Check levels offline: peak < 1.0, per-section RMS around −17 dBFS. If you can't listen,
  say so; meters do not prove the music sounds good.
  
  ## 6. Dialogue, sound effects and the mix (longer films)
  
  The pipeline that delivered a 4-minute film with dialogue:
  1. **Script with timings**: each line has a speaker, text, target start and emotion.
     Picture timing reads these.
  2. **Voices**: `voiceover.py` with the person's ElevenLabs or OpenAI key (§8), one voice per character; or a local TTS
     (macOS `say -v <voice> -o line.aiff`). Render each line to its own file and measure its real duration; the picture's
     timing uses measured durations, not guesses.
  3. **Cue sheet**: every visible event (door, footstep, impact, UI beep) exported from the
     picture as `{t, type, x}` data; the sound designer renders a cue per event and pans by
     screen x.
  4. **Score**: the composer gets the dialogue map and the dramatic arc (silence at bad news,
     swell at the crest) and leaves space under every line.
  5. **Mix** in ffmpeg with separate buses (dialogue, music, SFX): duck music under dialogue
     (`sidechaincompress`, or keyframed volume), keep music ~10 dB under speech in talky
     scenes and let it swell above in action. Master a mix you built to −16 LUFS integrated, true peak ≤ −1 dBTP
     (a user's own recording stays as it is, §7):
     ```bash
     ffmpeg -i mix.wav -af loudnorm=I=-16:TP=-1:LRA=11:print_format=summary -ar 48000 master.wav
     ```
  6. **Plot loudness over time** and check the arc lands (dips where planned, peaks where planned).
  
  ## 7. Mux and checks
  
  `scripts/render.mjs film.html --audio master.wav` muxes AAC 192k with `-shortest`.
  Afterwards:
  ```bash
  ffprobe -v error -count_packets -show_entries stream=codec_type,nb_read_packets:format=duration -of compact film.mp4
  ```
  Frame count must equal duration × fps; the audio stream must exist; spot-check a decoded
  frame from the MP4 itself (encoding can band gradients: use CRF ≤ 16 and grain).
  
  **Default to the source audio untouched.** Cut the section on zero crossings (no fade needed), pad silence for holds, and
  don't add EQ, denoise, loudness normalisation or reverb. A user asked "why even have to edit the audio instead of using it
  as is from the reference video" after a talk clip had been cleaned. If the recording has a flaw, name it and offer a repair
  as an option. Diagnose first with a spectrogram and the peak distribution per speaker. In that clip the second speaker's voice
  was squashed by the phone's limiter whenever van rumble pushed the signal to the ceiling (3 % of his samples above 0.7 against
  0.002 % for the first speaker), so removing the rumble afterwards couldn't undo the distortion. Only a restoration model
  (e.g. VoiceFixer, installed in a throwaway venv) could, and the user preferred the original.
  
  ## 8. Voiceover from a script (ElevenLabs or OpenAI)
  
  When the person has no audio but has a text-to-speech key.
  
  **1. The script.** Theirs, or one you draft from the brief, as plain text with one phrase per line (a phrase is what one picture
  beat will carry) and a blank line where a longer pause belongs. Aim for ~2.3–2.8 words a second for calm narration,
  ~3.2 for brisk explainers, and read the timing back to them (20 s is ~50 words). Get the script approved before voicing it,
  because every call costs them money.
  
  **2. The key.** The script reads it, in order, from:
  - `--key-file <path>`;
  - the environment variable (`ELEVENLABS_API_KEY`, `OPENAI_API_KEY`);
  - `~/.config/opus-js-animations/keys.env`, as `KEY=value` lines.
  
  If they paste a key in the chat, write that file yourself and `chmod 600` it. Never echo it, and never put it in the project, a command line or a log.
  
  **3. The voice.** List what their key can use and offer two or three voices that suit the film:
  ```bash
  python3 scripts/voiceover.py --provider elevenlabs --list-voices
  ```
  - **OpenAI:** `alloy`, `ash`, `ballad`, `coral`, `echo`, `fable`, `nova`, `onyx`, `sage`, `shimmer`, `verse`.
    `gpt-4o-mini-tts` takes `--instructions` for tone and pace ("warm, unhurried, a slight smile").
  - **ElevenLabs:** `eleven_multilingual_v2` covers most languages. `--stability` (lower is more expressive), `--similarity` and
    `--style` shape the read.
  
  **4. Voice it.**
  ```bash
  python3 scripts/voiceover.py source/script.txt --provider elevenlabs --voice "Rachel" --out source
  python3 scripts/voiceover.py source/script.txt --provider openai --voice onyx --instructions "slow, reverent, low" --out source
  python3 scripts/voiceover.py source/script.txt --provider openai --voice onyx --only 3 --out source    # re-voice line 3 only
  ```
  Each line is voiced on its own and joined with measured silences (`--gap` between lines, `--para` at blank lines).
  - **Outputs:** `voice.wav`, `voice.mp3`, and `voice.align.json`, which has the same shape as `align_audio.py` output, with one chunk per
    line and exact line boundaries.
  - **Word times:** ElevenLabs returns them from its character alignment. For OpenAI, run `align_audio.py` when picture events must
    land on words.
  - **Re-takes:** takes are cached in `lines/`, so changing one line re-voices only that line.
  
  **5. Then treat it like any recorded voice:** analyse it (§3), report what you hear, and time the treatment to the measured lines.
  Name the provider and voice in the post description when their terms ask for it.
  

• skills/opus-js-animations/references/delivery.md11 KB36cd7b3342e3upstream ↗

  # Delivery: crisp on Instagram, TikTok and X
  
  Every platform re-encodes the upload, to a few Mbps for a 1080×1920 Reel. Two videos at the same resolution can look very different
  afterwards. What decides it: how much of that small budget the picture needs, how cleanly the file arrives, and choices the platform makes.
  
  ## Contents
  1. Why some videos look crisp and others don't
  2. What the film should be (design for the re-encode)
  3. The export (what `render.mjs` does, supersampling with `--ss 2`, and the upload copy)
  4. The upload checklist for the person
  5. Covers and thumbnails
  6. How the grain and supersampling tests were run
  
  ## 1. Why some videos look crisp and others don't
  
  | Factor | Effect | Who controls it |
  |---|---|---|
  | **Compressibility of the picture** | Noise, animated grain, fine twinkling particles and full-frame chaos eat the bitrate. Clean flat colour, sharp edges and steady motion survive. | the film |
  | **The upload file** | A lossy or mis-tagged file loses twice. JPEG frame capture, full-range or BT.601 colour flags, and a 60 fps file splitting the same bits over twice the frames all cost quality. | the export |
  | **The app's upload setting** | With "Upload at highest quality" off, the phone compresses the video *before* uploading, and Instagram then compresses it again. | the person |
  | **Popularity** | Instagram re-encodes videos that get more views with costlier encoders (VP9, and AV1 for the most popular), and falls back to a cheaper encode when views stop. Mosseri confirmed it; he says the difference "isn't huge". | the audience |
  | **The viewer's connection** | Adaptive streaming drops to a lower rung on slow networks or in data-saver mode. | nobody |
  
  ## 2. What the film should be (design for the re-encode)
  
  - **No animated grain in the upload.** Grain that changes every frame is noise the encoder must spend bits on. It can't be kept at a few
    Mbps anyway; it becomes smeared blocks, and it takes the bits that text and edges needed.
    - **Measured on the opening burst of a 20 s reel:** at near-lossless quality the section needs 116 Mbps with animated grain, 51 Mbps with
      static grain and 31 Mbps with none.
    - **After a simulated Instagram re-encode**, the grain-free version kept SSIM .94 (H.264, 3 Mbps) and .95 (AV1, 1.5 Mbps) against its own
      upload; the animated-grain version kept .85–.86.
    - **What to do:** films expose `window.FILM_GRAIN = 'animated' | 'static' | 'none'` and the delivery shells set `'none'`. The paper
      textures inside the artwork stay; they move with the picture and compress well.
  - **Bold, generous text.** Thin light type at 32 px smears. Medium weight at ~44 px (1080 wide) stays sharp (`design.md` §7).
  - **Big shapes carry the frame; tiny sparkles are garnish.** Dozens of 2 px twinkling points turn to mush; fewer and larger ones read.
  - **30 fps.** At the same bitrate 30 fps gives each frame twice the bits of 60.
  - **Dark gradients:** the smooth night skies held without banding in the test once grain was removed. If banding appears, use a *static*
    dither at ≤ 4/255, not animated grain.
  
  ## 3. The export
  
  `render.mjs` (default):
  - serves the film from its folder over `http://127.0.0.1` (`--root` serves a parent folder when the film loads files from outside its own);
  - runs the frame loop inside each worker's page: `seek(t)`, then a small WebGL2 exporter copies the canvas to a texture, scales it down
    on the GPU when `--ss` > 1, reads the raw RGBA and POSTs it back; Node pipes it into ffmpeg as rawvideo. Three frames stay in flight,
    so drawing, copying and encoding overlap;
  - `--legacy` keeps the old path (a lossless PNG of the canvas through DevTools, scaled in ffmpeg) for comparison; `--fast` encodes with
    x264 "veryfast" at CRF 20 for previews; `--profile` prints what one frame costs to draw and to export;
  - converts to standard HD video: `yuv420p`, BT.709 matrix, limited (TV) range;
  - tags primaries, transfer, matrix and range as BT.709/TV, so phones and platform transcoders don't shift colours or lift or crush the
    blacks. The old path produced full-range, BT.601-flagged files.
  
  **Supersampling (`--ss 2`)**: the film draws every frame at 2× (2160×3840, `?ss=2`) and `render.mjs` scales it back down with a 3-lobe
  Lanczos on the GPU, in float, before the BT.709 conversion (`--legacy` does it in ffmpeg in 16-bit RGB; the two agree to PSNR 60–68 dB
  on most frames and about 50 dB on frames covered in per-frame grain, where they round differently). Four samples per pixel give smoother edges, rounder type and less shimmer on moving detail.
  - **Measured** on the opening burst of the same 20 s reel, against a 4× render boxed down to 1080×1920 (the nearest thing to the ideal picture):
    - the master moved from 39.1 to 45.0 dB PSNR (SSIM .985 → .996): a quarter of the error;
    - it stays ahead after a simulated Instagram re-encode: 32.7 → 33.5 dB (H.264, 3 Mbps) and 34.3 → 35.0 dB (AV1, 1.5 Mbps);
    - the upload needs slightly fewer bits (17.2 → 16.7 Mbps at CRF 17) and survives the re-encode as well as before (SSIM .94/.95
      against itself). Lanczos and a box filter scored the same; Lanczos looks crisper.
  - **Cost:** about twice the render time for Canvas 2D. Heavy shaders cost up to four times (every fragment runs four times); give WebGL
    films 2 workers at 2×.
  - **What a film needs** (`assets/film-template.html` has it):
    - read `ss` from the URL and make the canvas `W·SS × H·SS`, but keep all drawing in film pixels;
    - wrap the main context with `supersample(ctx, SS)`. It scales `setTransform`, `resetTransform`, shadow blur and offsets, and `filter`
      lengths. The last three are canvas pixels that the transform does not touch;
    - bake every sprite, text line and buffer at SS× its density, and multiply shadow and filter lengths inside a bake by SS yourself;
    - draw every baked canvas at an explicit film-pixel size. `drawImage(c, x, y)` uses its pixel size, which is now SS× too big;
    - multiply by SS the pixel numbers of pixel-space steps: offset copies for rims, source rectangles, reads of the main canvas. A bloom
      that read the main canvas at film size boxes it down to film size first;
    - shaders keep `uRes` and focal lengths in film pixels and map each fragment back,
      `vec2 fc = vec2(gl_FragCoord.x, uRes.y * uSS - gl_FragCoord.y) / uSS;`, so pixel-sized features (star radii, detail fades) keep
      their size and are simply sampled four times.
  - **Check:**
    - at `ss=1` the frames stay byte-identical to the film before the change (hash a dozen times);
    - `verify.mjs --ss 2` passes;
    - a 2× frame scaled down matches the 1× frame at SSIM ≥ .95, with a difference map that lights only edges and fine texture.
  - **Covers:** grab at `ss=2` and scale down the same way.
  
  **Render time.** Time a short range (`--from --to`) before quoting a number, and report progress when asked.
  - **Where the time went** (Apple M1, a 61 s 1080×1920 film at 60 fps and `--ss 2`: WebGL2 with ~190k instanced particles and bloom,
    plus Canvas 2D type). Drawing a frame took about 80 ms. The old export then spent ~220 ms encoding a PNG of the 2160×3840 canvas and
    ~600 ms sending it as base64 through DevTools, and titles revealed with a per-letter `ctx.filter` blur took ~0.9 s a frame on their
    own (`pitfalls.md`). The whole film took 21 min. With the raw export and the type blurred on small canvases it took **3 min 45 s**
    on 2 workers, with the same frames.
  - **Workers:** every worker shares one GPU. A WebGL film saturates an M1 at about 2 workers; 3 or 4 only queue. Light Canvas 2D films
    scale further. Run `render.mjs film.html --ss 2 --profile` first: it prints the draw and export cost of one frame.
  - **Progress:** `render.mjs` prints frames done and the frame rate every 2 s; the slowest worker sets the finish time.
  - **Speed-ups that keep the picture:**
    - blur on a small canvas, never with `ctx.filter` on the full-size frame (`pitfalls.md`), and keep settled type as a sprite drawn
      1:1 instead of setting it again every frame;
    - bake anything that doesn't change with time once (a still grain texture, a static mask) instead of computing it per pixel per
      frame;
    - work out soft passes such as bloom at a quarter of the size and sample them back up (it measured 69 dB against full size).
  
  The upload copy for each format, 1080×1920 (or 1920×1080) at 30 fps:
  ```bash
  ffmpeg -i film/film.mp4 -c:v libx264 -preset slow -crf 17 -maxrate 20M -bufsize 40M -profile:v high -pix_fmt yuv420p \
    -color_range tv -colorspace bt709 -color_primaries bt709 -color_trc bt709 -g 60 -c:a copy -movflags +faststart film/film-upload.mp4
  ```
  Upload at exactly the platform's size (1080×1920 for Reels, TikTok and Shorts). Bigger gets downscaled by the platform's scaler; smaller
  gets upscaled and looks soft. Check the result with
  `ffprobe -show_entries stream=pix_fmt,color_range,color_space,color_primaries,color_transfer film-upload.mp4`.
  
  ## 4. The upload checklist for the person
  
  1. In Instagram: **Profile → ☰ → Media quality** (on some builds, Data usage / Cellular data use): turn on **Upload at highest quality**.
  2. **Get the file onto the phone without recompression:** AirDrop, a cable, or iCloud Photos with "Download and Keep Originals".
     Not WhatsApp, Telegram or other messengers.
  3. **Upload on Wi-Fi.**
  4. **Post the finished file as it is.** Don't trim, filter, add text or stickers, or re-crop in the app; each edit is another processing
     pass. Choose the cover frame instead.
  5. **Pick a strong cover and give the first hour its best push.** Views earn the better encode (§1).
  6. **Same file for TikTok.** TikTok also has an "upload HD" option in its post settings; turn it on.
  
  ## 5. Covers and thumbnails
  
  - **Where they're seen:**
    - A Reel's cover shows mostly in the profile grid, which crops the **centre 3:4** of the 9:16 image (1080×1440 of 1080×1920).
      Keep every word inside y ≈ 240–1680.
    - The 16:9 thumbnail is for X and YouTube, wherever a custom thumbnail is allowed.
  - **How to make them:** render a *clean* frame of the film's strongest moment in cover mode (`window.FILM_COVER = true`: no subtitles, no
    watermark), then set a title in the film's own type.
    - One idea, big: a two-word contrast ("NOW | LATER") or an imperative ("Look up tonight.").
    - A small reference line above it.
    - One line of hook below it, plus the handle.
  - **Text and output:**
    - Take any quotation on a cover from the verified text (`quoted-text.md`). Match words by their base letters, because the order of
      combining marks can vary.
    - Capture the cover losslessly (PNG) and save a JPG at quality 95 for phones.
    - In Instagram: Edit cover → Add from camera roll.
  
  ## 6. How the grain and supersampling tests were run
  
  1. Render the hardest 3.5 s (the opening burst) four ways: the old JPEG path with animated grain, then PNG capture with animated, static and
     no grain.
  2. Encode each near-lossless, then simulate the platform: H.264 at 3 Mbps (`-b:v 3000k -maxrate 3500k`) and SVT-AV1 at 1.5 Mbps.
  3. Measure SSIM and PSNR of each simulated copy against its own upload and against the clean picture.
  4. Look at 1:1 crops of the text and pieces.
  
  For supersampling:
  1. Capture the same burst as PNG frames at 1× and 2×, plus six frames at 4× boxed down to film size as the reference.
  2. Make the master and upload copy of each (2× scaled down with Lanczos and with a box filter), then the two simulated re-encodes.
  3. Score every stage against the 4× reference, not against its own upload. Only the reference shows which version is closer to the ideal
     picture.
  
  Repeat these tests for a new style whenever its look depends on fine texture.
  

• skills/opus-js-animations/references/design.md16 KB8250362a5e12upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/directing.md10 KBe72633341615upstream ↗

  # Directing: the five conversations before a single frame
  
  A film made in code costs little to change while it is still words and a lot once it is 1,500 lines of JavaScript.
  So direct first: listen, get the sound, study it, ask, write the treatment, and build only on a "go".
  
  ## Contents
  1. The brief and a reference clip
  2. Asking about the sound
  3. Understanding the sound: the audio report
  4. The question bank
  5. The director's treatment: template and example
  6. Skipping, shortcuts and revisions
  
  ## 1. The brief and a reference clip
  
  Listen to the whole brief before proposing anything. Reflect it back in two or three lines:
  - subject, feeling and audience;
  - platform and formats, and length;
  - text on screen;
  - anything to avoid.
  
  Note what is still unknown; those become questions in §4.
  
  When they bring a reference clip ("make something like this"), measure it; don't guess from memory (`design.md` §2):
  ```bash
  f="clip.mp4"                                                   # quote paths: macOS names can contain spaces and U+202F
  ffprobe -v error -show_entries format=duration:stream=width,height,r_frame_rate -of compact "$f"
  ffmpeg -v error -i "$f" -vf "fps=1,scale=320:-1,tile=6x4" sheet.png                        # one frame a second, tiled
  ffmpeg -v error -i "$f" -vf "select='gt(scene,0.08)',showinfo" -f null - 2>&1 | grep -o "pts_time:[0-9.]*"   # its cuts
  ```
  Read the sheet, then name the reference precisely:
  - the style (e.g. "silhouettes on gradient skies with a tiny-planet horizon");
  - the likely stack;
  - its one big idea;
  - its rhythm (cuts per second, or one take).
  
  Keep its identity and drop its defects.
  
  ## 2. Asking about the sound
  
  Everything downstream is timed to the sound, so settle it first. One question, four options (plus "silent" for loops and GIFs):
  
  | They choose | Follow-up | Then |
  |---|---|---|
  | **My own file** | The path. If it's a video, you'll take its audio. | `ffmpeg -i in.mp4 -vn -c:a pcm_s16le -ar 48000 source/source.wav`. Use it untouched (`audio.md` §7). |
  | **A link** | Which section (from and to, in seconds), or all of it if it's short. | `get_audio.py "<url>" --from 12 --to 44 --out source --video`, then read `meta.txt` and the frame sheet. Say where the audio came from; reuse rights are theirs to judge. |
  | **Generate it in JavaScript** | The mood and genre, a tempo if they care, instruments or "you choose", and whether sound effects should follow the action. | The score is written into the page from the film's own timeline (`audio.md` §5), so music and picture can't drift. Draft its tempo and structure in the treatment; render the mix with `page_audio.mjs`. |
  | **A voiceover from my TTS key** | The provider (ElevenLabs or OpenAI) and the key; the script (theirs, or one you draft); the language, a voice and the pace. | Approve the script in writing before voicing it, because each call costs them money. `voiceover.py` (`audio.md` §8). |
  
  **Keys.** Ask them to put the key where the script reads it:
  - `export ELEVENLABS_API_KEY=…` in their shell profile, or
  - a line in `~/.config/opus-js-animations/keys.env` (chmod 600).
  
  If they paste a key into the chat, write it to that file yourself with `chmod 600`. Never put it in the project, a command line, a log or a reply.
  
  ## 3. Understanding the sound: the audio report
  
  ```bash
  python3 <skill>/scripts/analyze_audio.py source/source.wav         # report + source.analysis.json + source.spectrum.png
  python3 <skill>/scripts/align_audio.py source/source.wav --lang en   # words and breath points (voice only)
  ```
  Read the spectrogram image, not only the numbers:
  - phrases are islands of energy and pauses are gaps;
  - a beat shows as vertical stripes;
  - a swell is a widening band;
  - a quiet last word is a faint smudge at the end.
  
  Whisper can miss a quiet ending; check it (`audio.md` §3) before saying the audio stops early. For lyrics, poems and quotations, fetch the canonical text now (`quoted-text.md`).
  
  Then give the person a short report in this shape:
  
  > **What I hear** (20.4 s; one voice reading in English; no music)
  > - **Structure:** two halves of four clauses each; the turn comes at 8.6 s ("And whoever lives for more").
  > - **Key words:** "together" at 2.4 s, "heart" at 5.4 s, "anyway" at 7.9 s, "want" at 11.5 s, "apart" at 13.6 s, "share" at 17.6 s (spoken softly).
  > - **Tempo and mood:** unhurried and warm, with breaths of 0.3–0.6 s between clauses.
  > - **Flaws:** a soft tonal bed under the voice; the last words are quiet but present.
  
  For music, report the tempo and bar length, the sections (intro, verse, drop, outro) with times, the hits worth landing a cut or event on, and the loudest moment.
  For a voiceover you generated, report the measured length of each line against the plan.
  
  ## 4. The question bank
  
  Ask what the brief and the sound leave open: at most four questions per round, and ideally one round. Put your recommendation first, and say why in its description.
  
  **Look.** Offer two or three styles from `styles.md` that fit the subject, each with a one-line description. Include "you choose".
  
  **Pacing and structure.** Recommend by length:
  
  | Length | Recommend | Why |
  |---|---|---|
  | under ~30 s | one continuous scene whose state changes on each phrase | a cut every phrase feels frantic and says less |
  | ~30–60 s | one to three movements, ideally on one set | one per idea; cut at the turns of the argument |
  | music with a beat | movement locked to the beat, cuts on downbeats | the picture should play the music |
  | minutes | production with scenes (`production.md`) | |
  
  Also ask whether it should feel calm or energetic.
  
  **2D or 3D.**
  - **2D** (Canvas 2D with WebGL shaders for light and sky) is the default: the quickest to perfect and the most controllable.
  - A **raymarched 3D look** suits landscapes, water and sky.
  - **three.js meshes** suit a built world, rigged figures or an exploded model.
  
  Offer 3D when the idea needs depth, not by default.
  
  **The points to land.** Ask directly: what must a viewer with the sound off understand, and on which words or hits? Offer your reading of the audio as the default.
  
  **As needed:**
  - **Formats:** 9:16 1080×1920 (Reels, TikTok, Shorts), 16:9 1920×1080 (X, YouTube), 1:1, or several from one source.
  - **Text:** the original only, original plus translation, captions of the voice, or none; which translation.
  - **Characters:** a faceless figure, silhouettes, no figures (symbols, nature, objects), or a mascot.
    For sensitive content, ask how people may be shown (`quoted-text.md`).
  - **The hook** (for social feeds): a bright, moving, strange first frame is the default; a calm open is the alternative.
  - **The ending:** resolve on the central image, a text card or line, or a loop back to frame 0.
  - **A watermark:** their handle or logo. By default it stays faint inside the picture and becomes an end card (`design.md` §7).
  - **Avoid:** anything they never want to see (music under a voice, faces, a brand's colours).
  
  ## 5. The director's treatment: template and example
  
  Write it in `film/FILM.md` and present it in the chat. Keep it to one screen or two. It is a promise of what the film will do, beat by beat, so a person can say "yes" or "change scene 3" to it.
  
  ```markdown
  # <Title>: treatment
  
  **Logline.** One sentence: who or what, what changes, and what it means.
  **The central image.** The one picture that carries the idea; the thumbnail.
  **Look.** A style from the menu, 5–7 colours with their roles, texture, light. 2D or 3D, and why.
  **Sound.** Its source and length, what drives the picture (words, beats), and any score or effects.
  **Structure.** Scenes or movements with their times, and the turn.
  
  | Time | Words / beat | What we see | What changes | Camera | Light and colour | Text |
  |---|---|---|---|---|---|---|
  
  **The hook.** Frame 0 and the first second.
  **The ending.** The last image and how it goes out.
  **Formats.** Canvas sizes and where the text sits in each.
  **Risks.** What might not read, and how it will be checked.
  ```
  
  An example, abridged:
  
  > **Logline.** Two men stand back to back under one gold medallion split between them. The half of the one who lives for what lasts gathers
  > and lights up; the half of the one who lives for more falls apart into a mess over his head.
  > **The central image.** The split medallion over the two men: a gold half over a lit hall, a broken half over a neon market.
  > **Look.** A paper lightbox at night in 2D: indigo sky; gold, amber and rose on his side; magenta and cyan neon on the other; a wet floor mirroring both.
  > **Sound.** The client's 20.4 s reading, untouched. Every change lands on a clause.
  >
  > | Time | Words | What changes | Camera |
  > |---|---|---|---|
  > | 0–0.5 | (hook) | the medallion bursts into spiral arms around a white-gold sun | close on the core, pulling back |
  > | 2.4–4.1 | "his days come together" | his half locks home piece by piece; lanterns light outward from him | wide, drifting to him |
  > | 4.1–6.6 | "his heart is full" | a light kindles in his chest; his half lights from within | push in on him |
  > | 6.6–8.6 | "the world comes to him" | coins roll from the market past the other man and lie down at his feet | ease back |
  > | 10.9–13.5 | "want before his eyes" | an empty coin hangs before the other man's eyes, always a hand ahead | push in |
  > | 13.5–15.4 | "his days come apart" | his half bursts into a mess that stays in his sky; bags and phone among it | pull back |
  > | 17.2–20.4 | "only his share" | one piece comes down into his palm; pull back to both | wide, fade |
  >
  > **Risks.** A muted viewer must read "gathered versus scattered" from the medallion alone, so the loose half has to look loose at thumbnail size.
  
  ## 6. Skipping, shortcuts and revisions
  
  - **Detailed briefs:** if the brief already answers a question, don't ask it; state the assumption in the treatment.
  - **"Just make it":** skip the questions, not the treatment. Show it and wait, unless they say plainly that they don't want to see a plan.
  - **Revisions of a delivered film:** don't rerun the five conversations. Take the note, restate the change in one line if it is ambiguous, update `FILM.md`, and rebuild.
  - **Changes that break the contract** after approval (a new scene, a new style, a different length) go back to them as a revised treatment.
  - **Record what they teach you** in `FILM.md` ("Revisions" with their words), so a later session revises in the same direction.
  

• skills/opus-js-animations/references/examples.md3.9 KB0b967c1ca3f0upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/painting.md11 KB4ccd4f68e498upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/pitfalls.md10 KB806385b396f6upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/production.md6.9 KBea5a245eb17cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/quoted-text.md2.5 KBbc564e1f5348upstream ↗

  # Quotations, poems and lyrics
  
  For films built on a poem, lyrics, a speech, a literary passage or any quotation where exact wording matters.
  
  ## Identify, then verify
  
  1. Identify the passage from the source's metadata and on-screen text (title, author, book and line, song and verse), then confirm it by
     transcription (`align_audio.py`).
  2. Fetch the canonical text from an authoritative source (the publisher, an official lyrics source, a critical edition, a public-domain
     archive); never type it from memory.
  3. Build phrase strings as **exact word slices of the fetched text** (split on spaces, join word ranges), not retyped. Combining marks can
     be ordered differently and still look the same, so compare byte for byte:
     ```js
     words.slice(0, 3).join(' ') === canonical   // the displayed phrases re-join to the source text
     ```
     Verify again after every edit to the phrase list. Parse the list with a robust boundary, because translations contain brackets like
     "[above such a thing];", which can break naive `];` matching.
  4. If the recording differs from the canonical wording, follow the recording on screen and list the differences in `FILM.md`.
  5. Keep a translation's own words and brackets, and name the translation in the post description.
  
  ## Display
  
  - Use a font with full support for the script and its marks; check that small marks render, not as boxes.
  - Don't break joined scripts (Arabic, Devanagari and others) into letters for animation: reveal whole words or lines, or use a gradient
    mask moving in the writing direction.
  - If the speaker or singer repeats a phrase, show it again at the repeat.
  - Credits, references and hashtags: ask; many people prefer them in the caption rather than in the video.
  
  ## Depiction and sound
  
  - When the content is sensitive (a real person, grief, a community's own texts), ask how people may be shown: faceless figures,
    silhouettes, or no figures at all (symbols, nature, sky) are all options, and the person decides.
  - Illustrate the meaning, not your gloss: "the alternation of night and day" is shown as day and night passing, not an unrelated symbol.
    For dark themes, prefer restraint (a distant glow, then relief) over graphic imagery.
  - Ask whether music may play under a voice; some people want none.
  
  ## Post description
  
  Offer a ready caption: the text or its translation, the reference, the speaker or performer, and a few relevant hashtags. Mention where
  the recording came from so the person can judge reuse rights.
  

• skills/opus-js-animations/references/shaders.md27 KB432e31a61ee4upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/styles.md15 KBbbb1c39f8d5fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/threejs.md18 KB7c8dc3e112afupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/references/tiles-and-flocks.md11 KBd3bd1898b959upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/assets/film-template.html14 KB69cce8dc958cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/opus-js-animations/assets/watermark.js6.2 KBce1781efef38upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• README.mdupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/klsoen-opus-js-animations-skills-opus-js-animations.md