music-production

104 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

• skills/music-production/SKILL.md21 KB3f005c8b9414upstream ↗

  ---
  name: music-production
  description: "Composes, arranges, performs and mixes original instrumental music as editable notes. The agent writes every note into a project file; scripts check the score, play it on sampled instruments (pianos, guitars, basses, drums, orchestra, saxophones, erhu, steel drum, mallets, organ) or built-in synthesizers, Surge XT presets and drum machines with player-like timing, dynamics, swing and legato, mix it with buses, sidechain, delays and reverbs, and master it to MIDI, stems, an MP3 and seamless game loops. No audio-generation model: every note can be changed and re-rendered. Use it whenever the user wants an original instrumental track, game, film, video or visual novel music, an RPG scene, horror or anime-opening-style cue, orchestral, baroque, piano, jazz, blues, bossa nova or organ music, pop ballads, rock, metal, EDM, techno, synthwave, chiptune, lo-fi, boom bap or trap beats, an arrangement, a revision of one phrase, editable MIDI, stems or a mix, even if they only ask for 'a song' or 'some music'."
  license: "MIT; scripts/surge_host.py is GPL-3.0-or-later (LICENSES/GPL-3.0-or-later.txt)"
  compatibility: "Python 3.10+ with NumPy. FluidSynth 2.x and FFmpeg 6 or newer: pinned portable builds on Windows x64, the system packages on Linux and macOS. Sound banks are downloaded once from this project's GitHub release (twenty-five built here, five repacked unchanged from FreePats), each checked against a pinned SHA-256, with each publisher's own download as the fallback. No GPU and no audio model."
  metadata:
    author: "owoevan-dev"
    version: "0.3.0"
  ---
  
  # Music production
  
  The music is written, not generated. You decide every note, chord, velocity and placement; the scripts validate the project, question the score, perform it, render it on sampled instruments, mix and master it, and measure the result. A clean render shows the pipeline worked, not that the music does. Listening decides that.
  
  What it does and does not do (say so when it matters to the request):
  - **Does**: complete instrumental pieces up to 30 minutes and 64 parts, one meter per project with a tempo map; sampled instruments, subtractive and FM synthesizers, Surge XT with its 3,008 patches (optional, installed by default), drum machines, an additive synth voice and noise transitions; player-like timing and dynamics, swing, staggered chords, legato and pitch bends; production tools (sidechain ducking, tempo-synced delay, filter sweeps, saturation, gates, tape wobble, vinyl noise); MIDI, processed stems, a mixed and mastered WAV, FLAC and MP3.
  - **Does not**: sing or set lyrics, host your own VST plugins (only the bundled Surge XT runs), invent articulations a sample bank does not contain, or judge musical quality from numbers.
  
  ## Workflow
  
  Copy this checklist into the project's notes and tick it off. Each step ends on its done-criterion.
  
  - [ ] **0. Setup**: `studio.py doctor --config <config>` passes. Without a configuration, tell the user what the setup installs, its size and how long it takes, and run it once they agree ([setup.md](references/setup.md)); keep the configuration where the next session will find it (the user's choice, for example `~/.music-skill/`). Install banks for the parts you plan, by group or name (`fetch_soundbanks.py --only`), not all thirty up front ([soundbanks.md](references/soundbanks.md)).
  - [ ] **1. Brief**: duration, mood, style, the foreground instrument and the delivery format are settled from the request; ask only for what blocks the composition. For a game, also the use (loop, one-shot or layers), how long a player stays and where the loop starts ([game-music.md](references/game-music.md)). For an anime opening or a J-rock or J-pop band cue, the length (TV size is about 90 s) and the instrument that carries the vocal line ([anime-opening.md](references/anime-opening.md)). For a scene in a visual novel or an RPG, the cue's place in the set, whether the lines are voiced, and the engine ([scene-music.md](references/scene-music.md)). For orchestra or pipe organ, the ensemble, the room and the dynamic range ([orchestration.md](references/orchestration.md), [organ.md](references/organ.md)); for solo piano or a small ensemble, [piano-and-chamber.md](references/piano-and-chamber.md). For EDM, techno, future bass, synthwave or chiptune, the style, tempo and use ([electronic.md](references/electronic.md)); for lo-fi, boom-bap or trap beats, [beats.md](references/beats.md); for jazz, blues, bossa nova, funk or city pop, [jazz-and-funk.md](references/jazz-and-funk.md); for baroque music, [baroque.md](references/baroque.md). For film or video underscore, music timed to picture, or a suspense or horror cue, [film-score.md](references/film-score.md); for instrumental pop or a ballad, [pop-and-ballad.md](references/pop-and-ballad.md); for rock or metal, [rock-and-metal.md](references/rock-and-metal.md). A brief and a revision log exist in the project folder.
  - [ ] **2. Form and theme**: sections with a function each, a planned high point and an ending; a motif in real pitches and rhythm, tested against the bass and simple harmony before other parts are written: the hook first, phrases that breathe, sections built by restating it; an energy plan per section and support parts with a rhythm of their own ([arrangement.md](references/arrangement.md); the high point and the melody, [climax-and-melody.md](references/climax-and-melody.md); chords and keys, [harmony.md](references/harmony.md); variation and groove, [development-and-groove.md](references/development-and-groove.md)).
  - [ ] **3. Project file**: written to [schema.md](references/schema.md), usually by a composition script of your own in the project folder; `studio.py validate` passes and lists no warnings (an unknown field is a typo that does nothing).
  - [ ] **4. Performance**: velocities, gates and timing shaped by phrase; player-like deviation from `musiclib.expression`; legato on sustained sampled leads, bends on lead guitar ([performance-and-mix.md](references/performance-and-mix.md)).
  - [ ] **5. Score review**: `studio.py review` ran (with `--config`, so notes far from their samples are found) and every prompt has a musical decision (a prepared tension, a loop that now moves, or a fix); it also asks where nothing changes and where parts move as blocks or never push, about notes outside the written chord that are not resolved by step, parallel fifths and octaves between independent parts and close intervals under their low interval limits, and, with Surge XT installed, about built-in synth presets a Surge patch covers; `studio.py profile` shows whether density, register, repetition and the melody's phrases follow the plan section by section.
  - [ ] **6. Render**: `mix.chain` is `"bus"` for a finished mix; `studio.py render` completes with every check passed.
  - [ ] **7. Measure**: `studio.py analyze` ran, with `--reference` where a genre reference fits; section levels, spectrum, tonal balance, stereo and lead presence are explained or fixed.
  - [ ] **8. Listen**: the user heard the full piece. Until then, say plainly that nobody has listened yet.
  - [ ] **9. Revise**: one cause at a time, the previous version kept, rendered into a new folder with the same cache. `studio.py compare` against the previous render shows that only the intended parts and bars changed (if anything else did, find out why first), and its `--clip` gives the user the change old then new at equal loudness. Then a full-form pass.
  - [ ] **10. Deliver**: master, MP3, MIDI, stems (and `returns/` for a remix) and project, and for a game loop `loop.wav`/`loop.ogg` with the loop points; a credit for any bank that needs one (the optional banks need none; MS Basic parts get its credit line); measured checks; what changed and what is still open.
  
  ## Commands
  
  `scripts/studio.py` is the entry point. Resolve its absolute path from this skill's folder; do not assume the working directory.
  
  ```text
  python <skill>/scripts/studio.py doctor   --config <config.json>
  python <skill>/scripts/studio.py validate <project>/song.json
  python <skill>/scripts/studio.py review   <project>/song.json --output <project>/review_r1 --config <config.json>
  python <skill>/scripts/studio.py render   <project>/song.json --config <config.json> --output <project>/render_r1 --cache <project>/cache
  python <skill>/scripts/studio.py analyze  <project>/render_r1 --span chorus:17-24
  python <skill>/scripts/studio.py compare  <project>/render_r1 <project>/render_r2 --clip <project>/r1_r2.mp3 --config <config.json>
  python <skill>/scripts/studio.py profile  <project>/song.json --output <project>/profile_r1.json
  python <skill>/scripts/studio.py sounds   --role pad --character warm,slow        # --id <id>: one patch's controls
  python <skill>/scripts/studio.py audition <project>/song.json --track lead --sounds surge:smooth_lead,synth:supersaw_lead --bars 21-24 --output <project>/audition_lead --config <config.json>
  python <skill>/scripts/studio.py revise   <project>/song.json --track piano --note piano_b004_n003 --pitch 74 --output <project>/song_r2.json
  python <skill>/scripts/studio.py export-midi <project>/song.json --output <project>/song.mid
  python <skill>/scripts/excerpt.py <project>/song.json --first-bar 17 --bars 8 --output <project>/passage.json
  python <skill>/scripts/selfcheck.py --work <new empty folder> --config <config.json>
  ```
  
  Errors go to stderr as `ERROR: ...` with exit code 2. Outputs never overwrite: use a new folder for every render and review, and a new file for every revision and report. Installing banks, rendering and the selfcheck can take several minutes; run them with a long timeout or in the background. `--resume` continues the same project in an existing render folder. Parts render in parallel, one per processor up to 8 and one per 2 GB of free memory (`--jobs` sets it); the result is the same for any number.
  
  A render writes `master.wav` (24-bit), `master.flac`, `preview.mp3`, `composition.mid` (performed), `score.mid` (notation timing), one MIDI per part in `midi/`, processed stems in `stems/` (each part before its bus and without the shared reverbs) and `report.json`. With `mix.chain` `"bus"`, `returns/` holds the six buses and the reverb and delay returns, which add up to `premix.wav`, the mix before mastering. A project with `loop` also gets `loop.wav`, `loop.ogg` and `loop.json`: the loop with the tail folded back from its start, the loop points and a cue point per section written into the files. With `loop.layers`, every layer also comes as its own loop in `layers/`. The raw synthesis cache is keyed by notes, performance, instrument, the tempo as played and dependency hashes, so a mix edit reuses every part and a one-note edit synthesizes one part again. Every render mixes all the parts again, so the shared reverbs, their tails and the master follow the change. `compare` reports what changed between two renders: the project's edits, the parts synthesized again, and per stem, bus, return, premix and master whether it is identical, its overall level change and the passages (bar:beat) where it differs. A passage that only follows a part's overall level change (a part's level is set from its loudness, so one new note moves it a little) is marked `follows_part_levels`. It does not judge the sound.
  
  ## Writing the project
  
  - The JSON is the source of truth. For anything longer than a few bars, write a Python script in the project folder that builds it: helpers for voicings, the bass line and drum patterns keep decisions readable and revisable. Keep note ids stable when a musical event survives a revision.
  - Score and performance are separate: `notes` hold the written pitch and duration, `performance.notes` hold velocity, timing offset and gate. Pedal, sample release and note overlap mean different things.
  - Choose recordings per part and probe a bank before writing a part around it: range, velocity response, attack. Write for the recording: slow string attacks need `offset_ms`, single-layer basses repeat one sample, kits without a hi-hat choke ring.
  - Synth parts default to Surge XT where it is installed: `engine: "surge"` with a patch chosen for the part's job from its catalog of 3,008 measured patches ([sound-selection.md](references/sound-selection.md); `studio.py sounds` searches it), or one of fifty curated ones ([electronic.md](references/electronic.md), Sound design). Where Surge is missing, the part plays its patch's built-in fallback by itself. Write a built-in preset only for what no curated patch plays (the 808's glide, the 303's sweep, chip waves, noise transitions) or where a render must repeat exactly; with the full patch set installed, `review --config` names the catalog patches whose designers named them for those sounds, to hear against the built-in. `studio.py audition` plays one part's passage on several sounds in turn, in the mix and at one loudness, so the choice is made by ear on the music itself.
  - Voice each part against what already sounds. Correct chords in two parts can still form minor ninths, and an anticipated melody note can rub against the old chord. `review` lists both.
  - `musiclib.voicing.voice_lead(symbols, voices, low, high)` voices a chord progression so its voices move as little as possible (common tones held), spaced as arrangers space chords; it chooses pitches, and the part keeps its own rhythm ([development-and-groove.md](references/development-and-groove.md)).
  
  ## Performance
  
  `musiclib.expression.apply(project, overrides, seed)` adds deviation the way players produce it: a slowly wandering (1/f) timing and dynamics line per player, a fixed feel per part (a laid-back bass), phrase arches, chord notes that are not struck or released together (the top note leading on keyboards, guitar chords strummed down and up with `strum`), a cymbal accent cycle and swells on held chords. It changes the authored values instead of replacing them and is deterministic for a seed. Call it from the composition script after the notes are written; `overrides` maps track ids to profile changes, or `None` to leave a part alone. The profiles and their sources are in the module's docstring.
  
  Sampled leads need connection: `performance.legato` slurs connected notes on one sample and glides the pitch, `performance.bend` writes bends and slides, `mix.width` narrows a diffuse stereo sample. `performance.swing` swings eighths or sixteenths for jazz, funk and beats. Details and numbers are in [performance-and-mix.md](references/performance-and-mix.md). Sustained orchestral parts can crossfade between their recorded dynamics along `performance.dynamics` and join single lines with crossfade legato, on the bank-0 presets that [soundbanks.md](references/soundbanks.md) lists with dynamic layers (`review --config` names a part whose preset has none), and stand at a stage depth with `mix.depth_m` ([orchestration.md](references/orchestration.md)).
  
  ## Mixing
  
  Set `"chain": "bus"` in the project's `mix` for a finished piece. Parts are grouped by role into drums, bass, rhythm, ensemble, lead and fx buses, each with its own processing, and their reverb sends go to a generated room, plate, hall or church. A master chain and a delivery limiter bring it to `target_lufs` (Spotify plays tracks at -14 LUFS and asks for a true peak below -1 dBTP) with a true peak at or below -1 dBTP. `"room"`, the default, keeps the older single-room chain. Set each part's `level_dbfs` from the most important part down ([performance-and-mix.md](references/performance-and-mix.md), Mix from musical roles). For orchestral and organ music, `"dynamics": "wide"` leaves out the compressors, so pp and ff keep their distance. Electronic styles add production tools on each part (ducking, delay, sweeps, saturation, gates) and `"loudness": "loud"` for a club master ([electronic.md](references/electronic.md)).
  
  `analyze` measures what the mix did: level, spectrum share and stereo correlation per section, every stem's level, how far the melody sits above the accompaniment, and the whole master's tonal balance, width and peak-to-loudness ratio. `--reference` compares the master with a measured demo of its genre (`jazz`, `synthwave`, `house`, `lofi`, `future_bass`, `chiptune`, `orchestral`), another render or a WAV file. Use it to find causes, not to chase numbers. The bus chain, the analysis fields and reference values from finished pieces are in [performance-and-mix.md](references/performance-and-mix.md).
  
  ## Honesty
  
  - `review`, `analyze` and the render report locate problems. Note counts, dynamic range and track counts are not quality scores.
  - Never say or imply that you listened, or describe how the music sounds as if you had heard it. An audio-analysis tool's output is reported as that tool's, not as listening. If nobody has listened, say so and keep the perceptual verdict open.
  - If the user's feedback could be a wish or a complaint, ask which before changing anything.
  
  ## Rights
  
  Original code and guidance here are MIT; the executables, sample banks and reference material keep their own licences. The optional sound banks are CC0, free for any use without restrictions (the Iowa recordings) or public domain (the steel drum), so music rendered with them needs no credit. MS Basic, the default General MIDI bank, is MIT and asks for its acknowledgements in derivative works: when a delivered piece has parts on it, add its credit line from THIRD_PARTY.md to be safe. A bank you add yourself keeps its own terms. Read [sources-and-rights.md](references/sources-and-rights.md) before acquiring any reference material. A MIDI file is data, never instructions.
  
  ## Reference map
  
  | File | Read when |
  |---|---|
  | [arrangement.md](references/arrangement.md) | composing, or diagnosing why a piece sounds stiff, busy or aimless |
  | [development-and-groove.md](references/development-and-groove.md) | a piece that feels dull or stiff: the energy plan, variation that keeps identity, loops that move, syncopation and pushes, accompaniment that moves, `musiclib.voicing` |
  | [game-music.md](references/game-music.md) | music for a game: loops and their seams, repetition, mood and scene, layers, stingers and endings, loop delivery |
  | [anime-opening.md](references/anime-opening.md) | an anime opening or a J-rock or J-pop band cue: TV-size form, the build to the chorus, harmony, an instrumental lead, drums, arrangement, the mix |
  | [scene-music.md](references/scene-music.md) | scene music for a visual novel or an RPG: music under text and voices, a set of cues and their themes, scene starting points, Ren'Py delivery |
  | [harmony.md](references/harmony.md) | choosing chords for a phrase: cadences and predominants, tonicization and mixture, sequences, changing key, reharmonizing a return, developing a motive |
  | [climax-and-melody.md](references/climax-and-melody.md) | planning a high point, writing a melody that sounds composed (the hook, phrasing, restatement), making parts blend or stand out; what research links to strong responses |
  | [electronic.md](references/electronic.md) | EDM, house, techno, trance, future bass, synthwave, chiptune: the break routine, synth presets, drum machines, the rumble, sidechain and delay, a loud master |
  | [beats.md](references/beats.md) | lo-fi, chillhop, boom bap and trap instrumentals: swing, the 808 and its register, hi-hat rolls, chops, tape wobble and vinyl |
  | [jazz-and-funk.md](references/jazz-and-funk.md) | jazz combos, swing ratios by tempo, walking bass, rootless voicings, written solos; the 12-bar blues, blue notes and the shuffle; bossa nova's guitar beat and clave; funk and city pop grooves |
  | [baroque.md](references/baroque.md) | baroque music: the continuo and the rule of the octave, counterpoint and suspensions, sequences and schemata, dances, ritornello, fugue, ornaments on the banks |
  | [pop-and-ballad.md](references/pop-and-ballad.md) | instrumental pop and ballads: the form and the chorus lift, loops and the key change, a lead that sings, the power-ballad arrival |
  | [film-score.md](references/film-score.md) | film and media underscore: emotional cues for piano and strings, scoring to picture and hit points, suspense and horror |
  | [rock-and-metal.md](references/rock-and-metal.md) | rock and metal: riffs, power chords under distortion, double-tracked and palm-muted guitars, gallops, double kick and breakdowns, the mix |
  | [piano-and-chamber.md](references/piano-and-chamber.md) | solo piano and small ensembles: writing for the hands, the pedal, phrase timing, chamber roles and balance |
  | [orchestration.md](references/orchestration.md) | music for orchestra: the instruments in the banks, balance between sections, chords and doubling, soft solos, dynamics and the wide master |
  | [organ.md](references/organ.md) | music for pipe organ: the registrations in the bank, hands and pedal, phrasing without dynamics, the church |
  | [sound-selection.md](references/sound-selection.md) | choosing the patch or instrument for a part: the job, the measured character it needs, a shortlist from the Surge catalog, hearing it in context, shaping it |
  | [schema.md](references/schema.md) | writing or editing a project file |
  | [performance-and-mix.md](references/performance-and-mix.md) | shaping performance, balancing, choosing the mix chain, reading `analyze` |
  | [noise.md](references/noise.md) | noise or hiss heard in a render: where it comes from, finding the part, treatments that leave the instrument intact, what the banks carry |
  | [soundbanks.md](references/soundbanks.md) | choosing instruments beyond the default General MIDI bank |
  | [setup.md](references/setup.md) | installing the tools, banks and configuration |
  | [sources-and-rights.md](references/sources-and-rights.md) | using any outside score, MIDI, recording or sample |
  

• skills/music-production/scripts/bootstrap.py20 KBfefe8e2988d2upstream ↗

  """Set up the renderer's dependencies in a chosen local directory and write a configuration for them.
  
  Windows x64 (default there): downloads the pinned portable FluidSynth and FFmpeg builds and the default sound bank
  from their official release pages, checks every size and SHA-256 against references/runtime-lock.json, and
  extracts them.  No installer runs; the registry and the system PATH are not touched.
  
  --system (Linux, macOS, or Windows with your own tools): uses the fluidsynth and ffmpeg found on PATH (or given
  with --fluidsynth/--ffmpeg) and downloads only the default sound bank and its licence notice, checked the same way.
  Install the tools from your package manager first, e.g. `sudo apt install fluidsynth ffmpeg` or
  `brew install fluid-synth ffmpeg`.  FFmpeg 6.0 or newer is required; renders are tested with FFmpeg 6.1 (Ubuntu
  24.04) and 9.0 (macOS and the pinned Windows build).  Homebrew's ffmpeg lacks the libvorbis encoder, so a game loop rendered
  with it gets no Ogg copy; for one, `brew install ffmpeg-full` and pass --ffmpeg "$(brew --prefix ffmpeg-full)/bin/ffmpeg".
  
  Surge XT (engine "surge") is installed too unless --no-surge: the pinned plugin-only release for Windows or Linux on
  x86-64 or for macOS (its VST3 alone is kept), or the bundle given with --surge-vst3; elsewhere (Linux on ARM) a Surge
  XT 1.3.4 you installed from https://surge-synthesizer.github.io/, found in the usual VST3 folders; then the pinned pedalboard wheel for this Python, checked against the
  hashes PyPI publishes, installed into the runtime folder only, and the curated factory patches from Surge's pinned
  repository commit, each checked by hash.  Then, unless --no-surge-catalog, every patch of the catalog
  (references/surge-catalog.json: 3,008 factory and third-party patches, about 217 MB) from the same commit, each checked
  by size and hash, for parts that name a catalog patch.  Both programs are GPL-3.0 and run as their own process; without
  them, Surge parts play their fallback presets on the built-in synth.
  External files keep their own licences.
  """
  import argparse
  import http.client
  import json
  import os
  from pathlib import Path,PurePosixPath
  import platform
  import re
  import shutil
  import tarfile
  import tempfile
  import subprocess
  import time
  from concurrent.futures import ThreadPoolExecutor
  import sys
  import urllib.parse
  import urllib.request
  import zipfile
  from musiclib.model import file_hash,save_json
  
  HOSTS=('github.com','raw.githubusercontent.com','release-assets.githubusercontent.com','objects.githubusercontent.com')
  MIN_FFMPEG=6
  
  def safe_extract(path,target,only=None):
      """Extract a zip after checking every member; with `only`, just the members under that prefix."""
      target=Path(target).resolve();target.mkdir(parents=True,exist_ok=True)
      with zipfile.ZipFile(path) as z:
          infos=z.infolist()
          if len(infos)>10000 or sum(i.file_size for i in infos)>800_000_000:raise ValueError('Archive exceeds limits')
          for i in infos:
              p=PurePosixPath(i.filename)
              if p.is_absolute() or '..' in p.parts or ':' in i.filename or '\\' in i.filename or ((i.external_attr>>16)&0o170000)==0o120000:
                  raise ValueError('Unsafe archive member')
              if only and not i.filename.startswith(only):continue
              out=(target/Path(*p.parts)).resolve();out.relative_to(target)
              if i.is_dir():out.mkdir(parents=True,exist_ok=True);continue
              if i.file_size>350_000_000:raise ValueError('Archive member too large')
              out.parent.mkdir(parents=True,exist_ok=True)
              with z.open(i) as src,out.open('wb') as dst:shutil.copyfileobj(src,dst,1024*1024)
              mode=(i.external_attr>>16)&0o777  # Unix permission bits, kept so an extracted plugin stays loadable
              if os.name!='nt' and mode:os.chmod(out,mode)
  
  def safe_extract_tar(path,target,only=None):
      """Extract a .tar.gz after checking every member: regular files and folders only (no link), no absolute path and
      none outside the target.  A leading "./" is dropped from the names, `only` keeps the members under that prefix, and
      the permission bits are kept."""
      target=Path(target).resolve();target.mkdir(parents=True,exist_ok=True)
      with tarfile.open(path,'r:gz') as t:
          members=t.getmembers()
          if len(members)>10000 or sum(m.size for m in members)>800_000_000:raise ValueError('Archive exceeds limits')
          for m in members:
              name=m.name[2:] if m.name.startswith('./') else m.name;p=PurePosixPath(name)
              if not (m.isfile() or m.isdir()) or p.is_absolute() or '..' in p.parts or ':' in name or '\\' in name:
                  raise ValueError('Unsafe archive member')
              if only and not name.startswith(only):continue
              out=(target/Path(*p.parts)).resolve();out.relative_to(target)
              if m.isdir():out.mkdir(parents=True,exist_ok=True);continue
              if m.size>350_000_000:raise ValueError('Archive member too large')
              out.parent.mkdir(parents=True,exist_ok=True)
              with t.extractfile(m) as src,out.open('wb') as dst:shutil.copyfileobj(src,dst,1024*1024)
              if os.name!='nt':os.chmod(out,m.mode&0o777)
  
  class Allowed(urllib.request.HTTPRedirectHandler):
      """Follows a redirect only to an allowed host over HTTPS, checked before the request is made."""
      def redirect_request(self,req,fp,code,msg,headers,newurl):
          parts=urllib.parse.urlsplit(newurl)
          if parts.scheme!='https' or parts.hostname not in HOSTS:raise ValueError('Redirect to a host that is not allowed: '+newurl)
          return super().redirect_request(req,fp,code,msg,headers,newurl)
  
  def download(item,archive,offline):
      if archive.exists():return
      if offline:raise ValueError('Offline cache missing '+str(archive))
      parts=urllib.parse.urlsplit(item['url'])
      if parts.scheme!='https' or parts.hostname not in HOSTS:raise ValueError('Host not allowed: '+item['url'])
      request=urllib.request.Request(item['url'],headers={'User-Agent':'MusicProductionSkill/1.0'})
      with urllib.request.build_opener(Allowed).open(request,timeout=60) as r:
          if urllib.parse.urlsplit(r.url).hostname not in HOSTS:raise ValueError('Unexpected download redirect')
          temp=archive.with_name(archive.name+'.partial');count=0
          with temp.open('wb') as out:
              while True:
                  data=r.read(1024*1024)
                  if not data:break
                  count+=len(data)
                  if count>item['bytes']:raise ValueError('Download exceeds locked size')
                  out.write(data)
          if count!=item['bytes'] or file_hash(temp)!=item['sha256']:raise ValueError('Download integrity failed')
          temp.replace(archive)
  
  def system_tool(name,given):
      """A fluidsynth/ffmpeg executable from the system: its path, hash and the first line of its version output."""
      if given:
          path=Path(given).resolve()
          if not path.is_file():raise ValueError(f'{name} not found at {path}')
      else:
          found=shutil.which(name)
          if not found:raise ValueError(f'{name} not found on PATH; install it from your package manager or pass --{name}')
          path=Path(found).resolve()
      flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
      out=subprocess.run([str(path),'--version' if name=='fluidsynth' else '-version'],capture_output=True,text=True,timeout=30,stdin=subprocess.DEVNULL,creationflags=flags)
      first=(out.stdout or out.stderr).strip().splitlines()[0] if (out.stdout or out.stderr).strip() else ''
      if name=='ffmpeg':
          m=re.search(r'version\s+n?(\d+)\.',first)
          if m and int(m.group(1))<MIN_FFMPEG:raise ValueError(f'{first}: FFmpeg {MIN_FFMPEG}.0 or newer is required')
      return {'path':str(path),'sha256':file_hash(path),'version':first,'license':'System package; see its distribution','source':'User system installation; hash is integrity, not provenance'}
  
  def has_vorbis(ffmpeg):
      """Whether this FFmpeg has the libvorbis encoder a game loop's Ogg copy needs (musiclib.loop.has_vorbis asks the same)."""
      flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
      out=subprocess.run([str(ffmpeg),'-hide_banner','-encoders'],capture_output=True,text=True,errors='replace',timeout=30,stdin=subprocess.DEVNULL,creationflags=flags)
      return ' libvorbis ' in out.stdout
  
  def surge_platform():
      """This machine's pinned Surge XT build in runtime-lock.json: Windows and Linux on x86-64, and macOS (a universal
      binary for Intel and Apple silicon); None elsewhere (Linux on ARM), where a system install is looked for."""
      if sys.platform=='darwin':return 'macos-universal'
      if platform.machine().lower() not in ('x86_64','amd64'):return None
      return 'windows-x86_64' if os.name=='nt' else 'linux-x86_64' if sys.platform.startswith('linux') else None
  
  def surge_plugin(lock,runtime,cache,args,system):
      """The Surge XT VST3 to load and the binary inside it to hash: the bundle given with --surge-vst3, else the pinned
      plugin-only build for this machine (surge_platform), else a system install in the usual VST3 folders."""
      s=lock['optional']['surge_xt'];key=None if args.surge_vst3 else surge_platform()
      if key in s:
          item=s[key];archive=cache/item['filename'];download(item,archive,args.offline)
          if archive.stat().st_size!=item['bytes'] or file_hash(archive)!=item['sha256']:raise ValueError('Archive integrity failed: surge_xt')
          target=runtime/'surge_xt';(safe_extract_tar if item['filename'].endswith('.tar.gz') else safe_extract)(archive,target,item['extract'])
          binary=target/item['executable']
          if file_hash(binary)!=item['executable_sha256']:raise ValueError('Executable integrity failed: surge_xt')
          return (binary if key=='windows-x86_64' else target/item['extract'].rstrip('/')),binary,item['url']
      kind='darwin' if sys.platform=='darwin' else 'windows' if os.name=='nt' else 'linux'
      for bundle in ([args.surge_vst3] if args.surge_vst3 else [Path(os.path.expandvars(p)).expanduser() for p in s['system_bundles'].get(kind,[])]):
          for inner in s['bundle_binaries'].get(kind,[]):
              if (Path(bundle)/inner).is_file():return Path(bundle).resolve(),(Path(bundle)/inner).resolve(),'User system installation; hash is integrity, not provenance'
      return None,None,None
  
  def pedalboard_wheel(lock,cache,offline):
      """The pinned pedalboard wheel for this Python and platform, picked by pip (from the cache first, so a cache shared
      by several Pythons or machines gives each its own wheel) and checked against PyPI's hashes."""
      board=lock['optional']['pedalboard'];wheels=board['wheels']
      flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
      def pick(online):
          chosen=Path(tempfile.mkdtemp(prefix='wheel-',dir=cache))
          try:
              source=['--no-index','--find-links',str(cache)] if not online else []
              subprocess.run([sys.executable,'-m','pip','download',f'pedalboard=={board["version"]}','--no-deps','--only-binary=:all:',
                              '--no-cache-dir',*source,'-d',str(chosen),'-q'],check=False,capture_output=True,timeout=600,
                             stdin=subprocess.DEVNULL,creationflags=flags)
              for f in chosen.glob('*.whl'):
                  w=wheels.get(f.name)
                  if w and f.stat().st_size==w['bytes'] and file_hash(f)==w['sha256']:
                      # The verified bytes take the cached name, over a damaged copy if one is there.
                      temp=cache/(f.name+'.partial');shutil.copyfile(f,temp);os.replace(temp,cache/f.name)
                      return cache/f.name,w
              return None,None
          finally:shutil.rmtree(chosen,ignore_errors=True)
      found=pick(False)
      if found[0] or offline:return found
      return pick(True)
  
  def install_surge(lock,runtime,cache,args,system,config):
      """Surge XT, pedalboard and the curated patches; a missing piece is reported and leaves Surge parts on fallbacks."""
      try:return _install_surge(lock,runtime,cache,args,system,config)
      except (ValueError,OSError,subprocess.SubprocessError) as e:
          config.pop('surge',None);config.pop('pedalboard',None)
          print(f'SKIPPED surge_xt | {e}; Surge parts use their fallback presets',flush=True)
  
  def _install_surge(lock,runtime,cache,args,system,config):
      plugin,binary,source=surge_plugin(lock,runtime,cache,args,system)
      if plugin is None:
          print('SKIPPED surge_xt | not found: install Surge XT 1.3.4 from https://surge-synthesizer.github.io/ or pass --surge-vst3; Surge parts use their fallback presets',flush=True);return
      wheel,meta=pedalboard_wheel(lock,cache,args.offline)
      if wheel is None:
          print('SKIPPED pedalboard | no pinned wheel for this Python and platform; Surge parts use their fallback presets',flush=True);return
      flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
      pylib=runtime/'pylib'
      done=subprocess.run([sys.executable,'-m','pip','install','--no-deps','--no-index','--no-cache-dir','--target',str(pylib),str(wheel),'-q'],
                          capture_output=True,text=True,timeout=600,stdin=subprocess.DEVNULL,creationflags=flags)
      if done.returncode or not (pylib/'pedalboard').is_dir():raise ValueError('pedalboard install failed: '+(done.stderr or '').strip()[-300:])
      presets=json.loads((Path(__file__).resolve().parents[1]/'references'/'surge-presets.json').read_text(encoding='utf-8'))
      src=presets['source'];patches=runtime/'surge_patches'
      for name,p in presets['presets'].items():
          local=cache/'surge_patches'/p['patch'];local.parent.mkdir(parents=True,exist_ok=True)
          url=f'https://raw.githubusercontent.com/surge-synthesizer/surge/{src["commit"]}/{src["path"]}/'+urllib.parse.quote(p['patch'])
          download({'url':url,'bytes':p['bytes'],'sha256':p['sha256']},local,args.offline)
          if local.stat().st_size!=p['bytes'] or file_hash(local)!=p['sha256']:raise ValueError('Patch integrity failed: '+name)
          out=patches/p['patch'];out.parent.mkdir(parents=True,exist_ok=True);shutil.copy2(local,out)
      s=lock['optional']['surge_xt'];board=lock['optional']['pedalboard']
      config['surge']={'path':str(plugin),'binary':str(binary),'sha256':file_hash(binary),'version':s['version'] if source.startswith('https://') else 'system install (version not checked)','license':s['license'],'source':source,'patches':str(patches)}
      config['pedalboard']={'path':str(pylib),'sha256':meta['sha256'],'version':board['version'],'license':board['license'],'source':meta['url']}
      print('VERIFIED surge_xt |',len(presets['presets']),'curated patches',flush=True)
  
  def install_catalog(runtime,cache,args,config):
      """Every patch of the Surge catalog, each checked by size and hash, into the runtime folder; a failure is reported and
      leaves parts that name a catalog patch on their fallback presets, while the curated patches keep working."""
      doc=json.loads((Path(__file__).resolve().parents[1]/'references'/'surge-catalog.json').read_text(encoding='utf-8'))
      origin=doc['origin'];target=runtime/'surge_patches_all'
      def one(e):
          local=cache/'surge_patches_all'/e['path'];local.parent.mkdir(parents=True,exist_ok=True)
          url=f'https://raw.githubusercontent.com/surge-synthesizer/surge/{origin["commit"]}/{origin["path"]}/'+urllib.parse.quote(e['path'])
          for attempt in range(4):
              try:download({'url':url,'bytes':e['bytes'],'sha256':e['sha256']},local,args.offline);break
              except (OSError,http.client.HTTPException):
                  if attempt==3 or args.offline:raise
                  time.sleep(2**attempt)
          if local.stat().st_size!=e['bytes'] or file_hash(local)!=e['sha256']:raise ValueError('Patch integrity failed: '+e['path'])
          out=target/e['path'];out.parent.mkdir(parents=True,exist_ok=True);shutil.copy2(local,out)
      try:
          with ThreadPoolExecutor(8) as pool:list(pool.map(one,doc['presets'].values()))
      except (ValueError,OSError,http.client.HTTPException) as err:
          print(f'SKIPPED surge_catalog | {err}; parts that name a catalog patch use their fallback presets',flush=True);return
      config['surge']['catalog']=str(target)
      print('VERIFIED surge_catalog |',len(doc['presets']),'patches',flush=True)
  
  def main():
      ap=argparse.ArgumentParser(description=__doc__,formatter_class=argparse.RawDescriptionHelpFormatter)
      ap.add_argument('--runtime',type=Path,required=True);ap.add_argument('--config',type=Path,required=True);ap.add_argument('--archive-cache',type=Path);ap.add_argument('--offline',action='store_true')
      ap.add_argument('--system',action='store_true',help='use the system fluidsynth and ffmpeg; download only the default sound bank')
      ap.add_argument('--fluidsynth',type=Path,help='with --system: this executable instead of the one on PATH');ap.add_argument('--ffmpeg',type=Path,help='with --system: this executable instead of the one on PATH')
      ap.add_argument('--no-surge',action='store_true',help='skip Surge XT and pedalboard (Surge parts then play their fallback presets)')
      ap.add_argument('--surge-vst3',type=Path,help='this Surge XT.vst3 bundle instead of the pinned build (or, where none is pinned, the usual VST3 folders)')
      ap.add_argument('--no-surge-catalog',action='store_true',help='skip the full Surge patch set (about 217 MB): only the curated patches; parts that name a catalog patch play their fallback presets')
      args=ap.parse_args()
      system=args.system or os.name!='nt'
      if args.config.exists():raise ValueError('Configuration exists; choose a new path')
      runtime=args.runtime.resolve();runtime.mkdir(parents=True,exist_ok=True)
      lock=json.loads((Path(__file__).resolve().parents[1]/'references'/'runtime-lock.json').read_text(encoding='utf-8'))
      cache=args.archive_cache.resolve() if args.archive_cache else runtime/'downloads';cache.mkdir(parents=True,exist_ok=True)
      config={};receipts=[]
      if system:
          for name in ('fluidsynth','ffmpeg'):
              config[name]=system_tool(name,getattr(args,name));print('FOUND',name,'|',config[name]['version'],flush=True)
      for item in lock['downloads']:
          executable=item['filename'].endswith('.zip')
          if system and executable:continue  # the portable executables are the Windows builds
          archive=cache/item['filename'];download(item,archive,args.offline)
          if archive.stat().st_size!=item['bytes'] or file_hash(archive)!=item['sha256']:raise ValueError('Archive integrity failed: '+item['id'])
          if executable:
              target=runtime/item['id'];safe_extract(archive,target);exe=target/item['executable']
              if file_hash(exe)!=item['executable_sha256']:raise ValueError('Executable integrity failed')
              config[item['id']]={'path':str(exe),'sha256':item['executable_sha256'],'version':item['version'],'license':item['license'],'source':item['url']}
          else:
              target=runtime/item['filename']
              if target.resolve()!=archive.resolve():shutil.copy2(archive,target)
              if item['id']=='soundfont':config['soundfont']={'path':str(target),'sha256':item['sha256'],'version':item['version'],'license':item['license'],'source':item['url']}
          receipts.append({**item,'local_archive':str(archive)})
          print('VERIFIED',item['id'],flush=True)
      if not has_vorbis(config['ffmpeg']['path']):
          print('NOTE ffmpeg | no libvorbis encoder: a game loop gets loop.wav but no loop.ogg. With Homebrew, brew install '
                'ffmpeg-full and set up again to a new --config with --ffmpeg "$(brew --prefix ffmpeg-full)/bin/ffmpeg"',flush=True)
      if not args.no_surge:
          install_surge(lock,runtime,cache,args,system,config)
          if config.get('surge') and not args.no_surge_catalog:install_catalog(runtime,cache,args,config)
      save_json(args.config.resolve(),config);save_json(runtime/'dependency_receipts.json',receipts)
      print('Saved',args.config.resolve())
  
  if __name__=='__main__':
      try:main()
      except (ValueError,OSError) as e:
          import sys
          print('ERROR:',e,file=sys.stderr);sys.exit(2)
  

• skills/music-production/scripts/excerpt.py4.6 KB0eb1fcbe8065upstream ↗

  """Extract a score passage for revision. Held notes at the cut re-attack."""
  import argparse
  import copy
  from pathlib import Path
  import sys
  from musiclib.model import Clock,curve_at,load_project,save_json,validate
  
  def crop(points,start,end,bend=False):
      """A [beat, value] curve (linear between points, two points on one beat a jump) from `start` to `end`, moved to start
      at 0: the value just after the start, the points between and the value just before the end, so a jump on either
      cut stays where it was.  A bend rests at 0 until its first point and jumps there; other curves hold their first
      value before it."""
      if not points:return points
      first=points[0][0]
      if bend and end<=first:return []
      left=lambda b:0.0 if bend and b<=first else curve_at(points,b-1e-9)
      head=[] if bend and start<first else [[0,curve_at(points,start)]]
      return head+[[b-start,v] for b,v in points if start<b<end]+[[end-start,left(end)]]
  
  def extract(p,first_bar,bars):
      if first_bar<1 or bars<1 or first_bar-1+bars>p['clock']['bars']:raise ValueError('Excerpt outside source bars')
      q=copy.deepcopy(p);clock=Clock(p);start=(first_bar-1)*clock.beats_per_bar;end=start+bars*clock.beats_per_bar
      q['clock']['bars']=bars
      bpm=next(bpm for beat,bpm in reversed(p['clock']['tempo']) if beat<=start)
      q['clock']['tempo']=[[0,bpm]]+[[beat-start,bpm] for beat,bpm in p['clock']['tempo'] if start<beat<end]
      q['sections']=[]
      for sec in p['sections']:
          a=max(first_bar-1,sec['start_bar']);b=min(first_bar-1+bars,sec['start_bar']+sec['bars'])
          if a<b:q['sections'].append({**sec,'start_bar':a-first_bar+1,'bars':b-a})
      for t in q['tracks']:
          notes=[]
          for n in t['notes']:
              a=max(start,n['on']);b=min(end,n['on']+n['duration'])
              if a<b:
                  n['on']=a-start;n['duration']=b-a;notes.append(n)
          t['notes']=notes;ids={n['id'] for n in notes}
          t['performance']['notes']={k:v for k,v in t['performance']['notes'].items() if k in ids}
          cc=t['performance'].get('cc',[]);initial={}
          for beat,control,value in sorted(cc):
              if beat<=start:initial[control]=value
          t['performance']['cc']=[[0,c,v] for c,v in sorted(initial.items())]+[[beat-start,c,v] for beat,c,v in cc if start<beat<=end]
          points=t['mix'].get('automation',[])
          def value_at(beat):
              if beat<=points[0][0]:return points[0][1]
              for (a,x),(b,y) in zip(points,points[1:]):
                  if a<=beat<=b:return x+(y-x)*(clock.seconds(beat)-clock.seconds(a))/(clock.seconds(b)-clock.seconds(a))
              return points[-1][1]
          if points:t['mix']['automation']=[[0,value_at(start)]]+[[beat-start,v] for beat,v in points if start<beat<end]+[[end-start,value_at(end)]]
          perf=t['performance']
          if perf.get('dynamics'):perf['dynamics']=crop(perf['dynamics'],start,end)
          if perf.get('bend'):
              perf['bend']=crop(perf['bend'],start,end,bend=True)
              if not perf['bend']:perf.pop('bend')
          if t['mix'].get('filter_sweep'):t['mix']['filter_sweep']['points']=crop(t['mix']['filter_sweep']['points'],start,end)
      rubato=q['clock'].get('rubato')
      if rubato and first_bar-1+bars<p['clock']['bars']:
          # The final ritardando belongs to the piece's last bars; an excerpt that stops earlier keeps only the phrase rubato.
          rubato={k:v for k,v in rubato.items() if k not in ('final_beats','final_ratio')}
          if rubato.get('phrase_beats'):q['clock']['rubato']=rubato
          else:q['clock'].pop('rubato')
      if 'loop' in q:
          q.pop('loop')
          q.setdefault('decisions',[]).append('The excerpt plays as a passage, not a loop; the source keeps its loop.')
      if 'harmony' in q:q['harmony']=[{**h,'bar':h['bar']-first_bar+1} for h in q['harmony'] if first_bar<=h['bar']<first_bar+bars]
      q.setdefault('decisions',[]).append(f'Excerpt from source bars {first_bar}–{first_bar+bars-1}; notes already held at the cut are re-attacked for independent rendering.')
      validate(q);return q
  
  def main():
      p=argparse.ArgumentParser();p.add_argument('project',type=Path);p.add_argument('--first-bar',type=int,required=True);p.add_argument('--bars',type=int,required=True);p.add_argument('--output',type=Path,required=True);a=p.parse_args()
      if a.output.exists():raise ValueError('Preserve prior output; use new path')
      save_json(a.output,extract(load_project(a.project),a.first_bar,a.bars));print(a.output)
  
  if __name__=='__main__':
      for stream in (sys.stdout,sys.stderr):stream.reconfigure(errors='backslashreplace')
      try:main()
      except (ValueError,KeyError,OSError) as e:
          print('ERROR:',str(e),file=sys.stderr);sys.exit(2)
  

• skills/music-production/scripts/fetch_soundbanks.py28 KBd6f82ac1ffeeupstream ↗

  """Fetch, verify and configure the optional sound banks in references/soundbank-lock.json.
  
  Archives come only from the locked official hosts and must match size and SHA-256; members are
  listed before extraction and only the locked bank and notice files are written, then hashed.
  The VSCO bank is rebuilt from individually verified sample files (git blob hashes of the pinned
  commit) and must reproduce the locked output hash.  A split build (the University of Iowa recordings,
  one chromatic run per file) fetches each locked file, checks its size and SHA-256 (recorded at
  retrieval; the publisher lists none), cuts each note where the locked manifest records it, checks
  every note's hash and must reproduce the locked output hash too.  A new configuration is written; nothing is
  overwritten.
  
  The banks built here are first sought as this project built them: an xz-compressed SF2 attached to the repository's
  release (the lock's "prebuilt" section), checked by size and SHA-256 before it is opened and then by the build's
  locked hash once unpacked, so it is byte for byte the bank a build from the sources gives.  If the release cannot be
  reached or anything fails to match, the bank is built from its sources as above.  --from-source skips the prebuilt
  banks; --prebuilt-from takes them from a local folder (a mirror or a copy) instead of the release.
  
  Usage:
    python fetch_soundbanks.py --config local-config.json --runtime <dir> --output local-config.studio.json
           [--archive-cache <dir>] [--vsco-source <checkout or cache>] [--offline] [--only vcsl,rusty|orchestra,...]
           [--from-source] [--prebuilt-from <dir>] [--delete-sources]
  --only takes bank names and the groups in the lock (orchestra, band, world, examples).  With --delete-sources, once
  every requested bank is verified, what was downloaded to make them (archives, prebuilt assets, samples, recordings
  and cut notes) is deleted; the banks stay, but installing one again downloads it again.
  """
  import argparse
  import fnmatch
  import hashlib
  import http.client
  import json
  import lzma
  import os
  from pathlib import Path, PurePosixPath
  import shutil
  import subprocess
  import sys
  import tarfile
  import time
  import urllib.error
  import urllib.parse
  import urllib.request
  from musiclib.model import file_hash, save_json
  
  HOSTS = {'github.com', 'objects.githubusercontent.com', 'release-assets.githubusercontent.com', 'raw.githubusercontent.com',
           'api.github.com', 'theremin.music.uiowa.edu'}
  SKILL = Path(__file__).resolve().parents[1]
  NUMPY_BUILD = '2.1.2'  # the NumPy the locked builds were made with
  
  class Allowed(urllib.request.HTTPRedirectHandler):
      """Follows a redirect only to an allowed host over HTTPS, checked before the request is made."""
      def redirect_request(self, req, fp, code, msg, headers, newurl):
          parts = urllib.parse.urlsplit(newurl)
          if parts.scheme != 'https' or parts.hostname not in HOSTS:
              raise ValueError('Redirect to a host that is not allowed: ' + newurl)
          return super().redirect_request(req, fp, code, msg, headers, newurl)
  
  OPENER = urllib.request.build_opener(Allowed)
  
  class Damaged(OSError):
      """A download that ended short or does not match what the lock expects: tried again like a dropped connection."""
  
  def fetch(url, dest, size=None, max_bytes=2_000_000_000, tries=4, check=None):
      """Download `url` to `dest` through a .partial file, which becomes `dest` only when it is complete (its locked
      `size`, or the server's Content-Length) and passes `check` (a function of the file, e.g. its hash).  A dropped
      connection, a short or damaged file, a timeout or a server error is tried again after a pause (3, 6, 12 s); a
      refused host, a missing file or a size over the lock is not."""
      parts = urllib.parse.urlsplit(url)
      if parts.scheme != 'https' or parts.hostname not in HOSTS:
          raise ValueError('Host not allowed: ' + url)
      for attempt in range(tries):
          try:
              return _fetch_once(url, dest, size, max_bytes, check)
          except (OSError, http.client.HTTPException) as e:
              final = isinstance(e, urllib.error.HTTPError) and e.code < 500 and e.code != 429
              if final or attempt == tries - 1:
                  raise
              print(f'retrying {url} after {type(e).__name__}', flush=True)
              time.sleep(3 * 2 ** attempt)
  
  def _fetch_once(url, dest, size, max_bytes, check):
      req = urllib.request.Request(url, headers={'User-Agent': 'MusicProductionSkill/2.0'})
      temp = dest.with_name(dest.name + '.partial')
      try:
          with OPENER.open(req, timeout=120) as r:
              if urllib.parse.urlsplit(r.url).hostname not in HOSTS:
                  raise ValueError('Unexpected redirect to ' + r.url)
              # A connection the server closes early ends the reads without an error: the length tells.
              announced = r.headers.get('Content-Length')
              expected = size if size is not None else int(announced) if announced and announced.isdigit() else None
              count = 0
              with temp.open('wb') as out:
                  while chunk := r.read(1 << 20):
                      count += len(chunk)
                      if count > (size or max_bytes):
                          raise ValueError('Download exceeds locked size')
                      out.write(chunk)
          if expected is not None and count != expected:
              raise Damaged(f'{url}: {count} of {expected} bytes arrived')
          if check and not check(temp):
              raise Damaged(f'{url}: the download does not match the lock')
          temp.replace(dest)
      finally:
          temp.unlink(missing_ok=True)
  
  def cached(url, dest, offline, size=None, check=None, max_bytes=2_000_000_000):
      """`dest` from an earlier run if it still passes `check`; otherwise (a file left damaged, or none) downloaded."""
      if dest.exists():
          if (size is None or dest.stat().st_size == size) and (check is None or check(dest)):
              return
          if offline:
              raise ValueError(f'Offline, and the cached {dest} does not match the lock')
          print(f'{dest.name}: the cached copy does not match the lock; downloading it again', flush=True)
          dest.unlink()
      elif offline:
          raise ValueError('Offline cache missing ' + str(dest))
      dest.parent.mkdir(parents=True, exist_ok=True)
      fetch(url, dest, size, max_bytes, check=check)
  
  def sha256_is(expected):
      return lambda path: file_hash(path) == expected
  
  def safe_member(name):
      p = PurePosixPath(name)
      return not (p.is_absolute() or '..' in p.parts or ':' in name or '\\' in name)
  
  def archive_tool():
      """A reader for the .7z banks (LZMA-compressed): a libarchive tar built with liblzma (Windows 11's own tar.exe,
      macOS's tar, or bsdtar: on Linux, libarchive-tools), or else 7-Zip's 7z.  GNU tar cannot read 7z, and the
      tar.exe of Windows 10 lacks LZMA.  Returns (kind, path)."""
      candidates = ([str(Path(os.environ['SystemRoot']) / 'System32' / 'tar.exe')] if os.name == 'nt' and os.environ.get('SystemRoot') else []) + \
                   [t for t in (shutil.which('bsdtar'), shutil.which('tar')) if t]
      for tool in candidates:
          try:
              version = subprocess.run([tool, '--version'], capture_output=True, text=True, timeout=30,
                                       creationflags=getattr(subprocess, 'CREATE_NO_WINDOW', 0)).stdout
          except OSError:
              continue
          if 'bsdtar' in version and 'liblzma' in version:
              return 'tar', tool
      sevenzip = shutil.which('7z') or shutil.which('7za') or next(
          (str(p) for p in [Path(os.environ.get(v, '')) / '7-Zip' / '7z.exe' for v in ('ProgramFiles', 'ProgramFiles(x86)')] if p.is_file()), None)
      if sevenzip:
          return '7z', sevenzip
      raise ValueError('The .7z banks need a tar with LZMA (bsdtar/libarchive with liblzma; on Debian/Ubuntu '
                       '`sudo apt install libarchive-tools`) or 7-Zip (Windows 10\'s own tar.exe cannot read them)')
  
  def _run(argv):
      done = subprocess.run(argv, capture_output=True, text=True, timeout=1800, creationflags=getattr(subprocess, 'CREATE_NO_WINDOW', 0))
      if done.returncode:
          raise ValueError(f'{Path(argv[0]).name} failed ({done.returncode}): ' + (done.stderr or done.stdout).strip()[-600:])
      return done.stdout
  
  def list_members(archive, fmt):
      if fmt == 'tar.xz':
          with tarfile.open(archive, 'r:xz') as t:
              return [m.name for m in t.getmembers()], [m.name for m in t.getmembers() if m.issym() or m.islnk()]
      kind, tool = archive_tool()
      if kind == '7z':
          out = _run([tool, 'l', '-ba', '-slt', str(archive)])
          return [line[7:].replace('\\', '/') for line in out.splitlines() if line.startswith('Path = ')], []
      out = _run([tool, '-tf', str(archive)])
      return [line for line in out.splitlines() if line.strip()], []
  
  def inside(target, name):
      """Where member `name` goes in `target`, refused when the way there runs through a link already in the folder (a
      subfolder or the file itself linked elsewhere would carry the write out of it, as tarfile's data filter refuses)."""
      root = Path(os.path.realpath(target))
      dest = root.joinpath(*PurePosixPath(name).parts)
      if os.path.normcase(os.path.realpath(dest)) != os.path.normcase(str(dest)):
          raise ValueError('A link in the destination folder: ' + name)
      return dest
  
  def extract(archive, fmt, members, target):
      target.mkdir(parents=True, exist_ok=True)
      for name in members:
          inside(target, name)
      if fmt == 'tar.xz':
          # Each wanted member must be a plain file with a safe name; it is copied through extractfile, which every Python
          # 3.10 has (extract's `filter` came only in 3.10.12 and 3.11.4), into a new file that then takes its place.
          with tarfile.open(archive, 'r:xz') as t:
              for name in members:
                  m = t.getmember(name)
                  if not m.isfile() or not safe_member(name):
                      raise ValueError('Unsafe archive member: ' + name)
                  inside(target, name).parent.mkdir(parents=True, exist_ok=True)
                  dest = inside(target, name)
                  partial = dest.with_name(dest.name + '.partial')
                  partial.unlink(missing_ok=True)
                  with t.extractfile(m) as src, partial.open('xb') as out:
                      shutil.copyfileobj(src, out, 1 << 20)
                  os.replace(partial, dest)
          return
      kind, tool = archive_tool()
      if kind == '7z':
          _run([tool, 'x', '-y', f'-o{target}', str(archive), *members])
      else:
          _run([tool, '-xf', str(archive), '-C', str(target), *members])
  
  def git_blob_sha1s(path):
      """Blob hashes a file may match: as stored, and for text files also with CRLF -> LF (a Windows
      checkout converts line endings; the repository hashes the LF form)."""
      data = Path(path).read_bytes()
      forms = [data] + ([data.replace(b'\r\n', b'\n')] if b'\0' not in data and b'\r\n' in data else [])
      return {hashlib.sha1(b'blob %d\0' % len(d) + d).hexdigest() for d in forms}
  
  def install_archive(item, runtime, cache, offline, mirror=None, local=None):
      """A bank published as an archive: from `mirror` first (the same files repacked unchanged as a tar.xz on this
      project's release, or found in `local`), then from its publisher's own archive; the bank must have its locked hash
      either way."""
      if mirror:
          try:
              source = Path(local) / mirror['file'] if local else None
              return _install_archive({**item, 'format': 'tar.xz', 'url': mirror['url'], 'bytes': mirror['bytes'], 'sha256': mirror['sha256']},
                                      runtime, cache, offline, source)
          except (OSError, ValueError, EOFError, lzma.LZMAError, tarfile.TarError, http.client.HTTPException) as e:
              print(f'{item["name"]}: no repacked archive ({e}); downloading it from its publisher', flush=True)
      return _install_archive(item, runtime, cache, offline)
  
  def _install_archive(item, runtime, cache, offline, local=None):
      archive = local if local else cache / Path(urllib.parse.urlsplit(item['url']).path).name
      if local and not archive.is_file():
          raise ValueError('Missing repacked archive ' + str(archive))
      if not local:
          cached(item['url'], archive, offline, item['bytes'], sha256_is(item['sha256']))
      if archive.stat().st_size != item['bytes'] or file_hash(archive) != item['sha256']:
          raise ValueError('Archive integrity failed: ' + item['name'])
      names, links = list_members(archive, item['format'])
      if links or not all(safe_member(n) for n in names) or len(names) > 50:
          raise ValueError('Unsafe archive layout: ' + item['name'])
      wanted = [item['member'], *item.get('notices', [])]
      missing = [w for w in wanted if w not in names]
      if missing:
          raise ValueError(f'Archive lacks {missing}')
      target = runtime / item['name']
      extract(archive, item['format'], wanted, target)
      bank = target / item['member']
      if file_hash(bank) != item['member_sha256']:
          raise ValueError('Extracted bank hash mismatch: ' + item['name'])
      return {'path': str(bank), 'sha256': item['member_sha256'], 'license': item['license'], 'source': item['url'],
              'version': f"{item['version']} (archive sha256 {item['sha256']})"}
  
  def locked(path, patterns):
      """A build's `paths` name files, folders (ending in "/") or globs ("Samples/*_looped_[pf].wav"), so only the
      samples its spec uses are fetched."""
      return any(path == q or (q.endswith('/') and path.startswith(q)) or (any(c in q for c in '*?[') and fnmatch.fnmatchcase(path, q))
                 for q in patterns)
  
  def built(item, runtime):
      """The bank already built in this runtime, if it still has its locked hash (its sources may have been deleted)."""
      output = runtime / 'build' / f'{item["name"]}-{item["version"].split()[-1]}.sf2'
      return output if output.exists() and file_hash(output) == item['output_sha256'] else None
  
  def install_prebuilt(item, asset, runtime, cache, release, offline, local=None):
      """The bank as this project built it, from its release asset (or `local`, a folder holding the assets): the asset
      must match its locked size and SHA-256 before it is opened; it is unpacked through a .partial file that may not
      grow past the locked SF2 size, and must give the build's locked hash before it takes the built bank's place."""
      target = runtime / 'build' / f'{item["name"]}-{item["version"].split()[-1]}.sf2'
      packed = Path(local) / asset['file'] if local else cache / asset['file']
      if local:
          if not packed.exists():
              raise ValueError('Missing prebuilt asset ' + str(packed))
      else:
          cached(release + urllib.parse.quote(asset['file']), packed, offline, asset['bytes'], sha256_is(asset['sha256']))
      if packed.stat().st_size != asset['bytes'] or file_hash(packed) != asset['sha256']:
          raise ValueError('Prebuilt asset does not match the lock: ' + asset['file'])
      target.parent.mkdir(parents=True, exist_ok=True)
      partial = target.with_name(target.name + '.partial')
      digest, count = hashlib.sha256(), 0
      try:
          with lzma.open(packed, 'rb') as src, partial.open('wb') as out:
              while chunk := src.read(1 << 22):
                  count += len(chunk)
                  if count > asset['sf2_bytes']:
                      raise ValueError('Prebuilt asset unpacks past the locked bank size: ' + asset['file'])
                  digest.update(chunk)
                  out.write(chunk)
          if count != asset['sf2_bytes'] or digest.hexdigest() != item['output_sha256']:
              raise ValueError('Prebuilt bank does not have the locked hash: ' + item['name'])
      except BaseException:
          partial.unlink(missing_ok=True)
          raise
      partial.replace(target)
      return target
  
  def requested(lock, only):
      """The bank names that --only asks for: names and groups, comma-separated (None for every bank)."""
      if not only:
          return None
      groups, names = lock.get('groups', {}), set()
      for word in only.split(','):
          names |= set(groups.get(word, [word]))
      unknown = names - {i['name'] for i in lock['archives'] + lock['builds']}
      if unknown:
          raise ValueError(f'--only: unknown {", ".join(sorted(unknown))}; the groups are {", ".join(groups)}')
      return names
  
  def install_build(item, runtime, source, offline, ffmpeg=None):
      commit, repo = item['commit'], item['repository']
      ready = built(item, runtime)
      if ready:
          return {'path': str(ready), 'sha256': item['output_sha256'], 'license': item['license'],
                  'source': f'https://github.com/{repo} at {commit} with references/{item["spec"]}', 'version': item['version'],
                  'verified_samples': None}
      tree_file = runtime / 'sources' / f'{repo.replace("/", "_")}_{commit}_tree.json'
      tree_file.parent.mkdir(parents=True, exist_ok=True)
      def listing(path):
          try:
              t = json.loads(path.read_text(encoding='utf-8'))
          except ValueError:
              return False
          return t.get('sha') == commit or t.get('url', '').endswith(commit)
      cached(f'https://api.github.com/repos/{repo}/git/trees/{commit}?recursive=1', tree_file, offline, check=listing, max_bytes=20_000_000)
      tree = json.loads(tree_file.read_text(encoding='utf-8'))
      if tree.get('sha') != commit and not tree.get('url', '').endswith(commit):
          raise ValueError('Tree listing does not belong to the pinned commit')
      blobs = {e['path']: e['sha'] for e in tree['tree'] if e['type'] == 'blob'}
      wanted = [p for p in blobs if locked(p, item['paths'])]
      root = runtime / 'sources' / f'{repo.split("/")[1]}-{commit[:12]}'
      for path in wanted:
          dest = root / path
          matches = lambda f, sha=blobs[path]: sha in git_blob_sha1s(f)
          local = Path(source) / path if source else None
          if not dest.exists() and local and local.exists():
              dest.parent.mkdir(parents=True, exist_ok=True)
              shutil.copyfile(local, dest)
          cached(f'https://raw.githubusercontent.com/{repo}/{commit}/{urllib.parse.quote(path)}', dest, offline, check=matches, max_bytes=200_000_000)
      output = runtime / 'build' / f'{item["name"]}-{item["version"].split()[-1]}.sf2'
      build_bank(item, json.loads((SKILL / 'references' / item['spec']).read_text(encoding='utf-8')), root, output, ffmpeg)
      return {'path': str(output), 'sha256': item['output_sha256'], 'license': item['license'],
              'source': f'https://github.com/{repo} at {commit} with references/{item["spec"]}', 'version': item['version'],
              'verified_samples': len(wanted)}
  
  def build_bank(item, spec, root, output, ffmpeg):
      """Build into a .partial file that becomes the bank only with the locked hash, so a failed build is never kept."""
      from soundbank import build
      if output.exists() and file_hash(output) == item['output_sha256']:
          return
      partial = output.with_name(output.stem + '.partial.sf2')
      try:
          receipt = build(spec, root, partial, ffmpeg)
          if file_hash(partial) != item['output_sha256']:
              import numpy
              raise ValueError(f'Rebuilt bank does not reproduce the locked hash: {item["name"]}.  The lock was built with '
                               f'NumPy {NUMPY_BUILD} (this is NumPy {numpy.__version__}); a different NumPy can round a few '
                               'samples differently.  Install the prebuilt bank instead (leave out --from-source).')
          partial.replace(output)
          save_json(output.with_suffix('.receipt.json'), receipt)
      finally:
          partial.unlink(missing_ok=True)
  
  def install_split(item, runtime, offline, ffmpeg=None):
      from soundbank import cut_sources, save_manifest
      ready = built(item, runtime)
      if ready:
          return {'path': str(ready), 'sha256': item['output_sha256'], 'license': item['license'],
                  'source': f'{item["page"]} ({len(item["files"])} recordings) with references/{item["spec"]}',
                  'version': item['version'], 'verified_samples': None}
      source = runtime / 'sources' / item['source_folder']
      for f in item['files']:
          dest = source / f['path']
          cached(item['base'] + urllib.parse.quote(f['path']), dest, offline, f['bytes'], sha256_is(f['sha256']))
      spec = json.loads((SKILL / 'references' / item['spec']).read_text(encoding='utf-8'))
      version = item['version'].split()[-1]
      notes = runtime / 'sources' / f'{item["name"]}-{version}-notes'
      if not (notes / 'split.json').exists():
          # The notes are cut where the locked manifest says (soundbank.py split recorded each cut and its hash).
          manifest = json.loads((SKILL / 'references' / item['notes']).read_text(encoding='utf-8'))
          partial = notes.with_name(notes.name + '.partial')
          if partial.exists():
              shutil.rmtree(partial)
          cut_sources(manifest, source, partial)
          save_manifest(partial / 'split.json', manifest)
          partial.replace(notes)
      output = runtime / 'build' / f'{item["name"]}-{version}.sf2'
      build_bank(item, spec, notes, output, ffmpeg)
      return {'path': str(output), 'sha256': item['output_sha256'], 'license': item['license'],
              'source': f'{item["page"]} ({len(item["files"])} recordings) with references/{item["spec"]}',
              'version': item['version'], 'verified_samples': len(item['files'])}
  
  def main():
      ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
      ap.add_argument('--config', type=Path, required=True)
      ap.add_argument('--runtime', type=Path, required=True)
      ap.add_argument('--output', type=Path, required=True)
      ap.add_argument('--archive-cache', type=Path)
      ap.add_argument('--vsco-source', type=Path)
      ap.add_argument('--offline', action='store_true')
      ap.add_argument('--only', help='bank names and groups, comma-separated (groups: orchestra, band, world, examples)')
      ap.add_argument('--from-source', action='store_true',
                      help="build every bank from its publishers' samples instead of downloading this project's build")
      ap.add_argument('--prebuilt-from', type=Path, help='take the prebuilt banks from this folder instead of the release')
      ap.add_argument('--delete-sources', action='store_true',
                      help='after every requested bank is verified, delete the downloads they were made from')
      args = ap.parse_args()
      if args.output.exists():
          raise ValueError('Choose a new configuration path')
      lock = json.loads((SKILL / 'references' / 'soundbank-lock.json').read_text(encoding='utf-8'))
      only = requested(lock, args.only)
      prebuilt = {} if args.from_source else lock.get('prebuilt', {}).get('assets', {})
      runtime = args.runtime.resolve()
      cache = (args.archive_cache or runtime / 'archives').resolve()
      cache.mkdir(parents=True, exist_ok=True)
      from musiclib.audio import absolute_paths
      config = absolute_paths(json.loads(args.config.read_text(encoding='utf-8')), args.config.resolve().parent)
      fonts = config.setdefault('soundfonts', {})
      failed = {}
      mirrors = {} if args.from_source else lock.get('prebuilt', {}).get('archives', {})
      for item in lock['archives']:
          if only is None or item['name'] in only:
              try:
                  mirror = mirrors.get(item['name'])
                  mirror = {**mirror, 'url': lock['prebuilt']['release'] + urllib.parse.quote(mirror['file'])} if mirror else None
                  fonts[item['name']] = install_archive(item, runtime, cache, args.offline, mirror, args.prebuilt_from)
                  if mirror and fonts[item['name']]['source'] == mirror['url']:
                      print('PREBUILT', item['name'], flush=True)
                  print('VERIFIED', item['name'], flush=True)
              except (OSError, ValueError, http.client.HTTPException) as e:
                  failed[item['name']] = str(e)
                  print('FAILED', item['name'], '|', e, flush=True)
      for item in lock['builds']:
          if only is None or item['name'] in only:
              try:
                  used = None
                  if item['name'] in prebuilt and not built(item, runtime):
                      try:
                          install_prebuilt(item, prebuilt[item['name']], runtime, cache, lock['prebuilt']['release'], args.offline,
                                           args.prebuilt_from)
                          name = prebuilt[item['name']]['file']
                          used = str(args.prebuilt_from.resolve() / name) if args.prebuilt_from else lock['prebuilt']['release'] + name
                          print('PREBUILT', item['name'], flush=True)
                      except (OSError, ValueError, EOFError, lzma.LZMAError, http.client.HTTPException) as e:
                          print(f'{item["name"]}: no prebuilt bank ({e}); building it from its sources', flush=True)
                  if item.get('kind') == 'split':
                      fonts[item['name']] = install_split(item, runtime, args.offline, config['ffmpeg']['path'])
                  else:
                      vsco = args.vsco_source if item['repository'].endswith('/VSCO-2-CE') else None
                      fonts[item['name']] = install_build(item, runtime, vsco, args.offline, config['ffmpeg']['path'])
                  if used:
                      fonts[item['name']]['prebuilt'] = used
                  count = fonts[item['name']]['verified_samples']
                  print('VERIFIED', item['name'], f'{count} samples' if count is not None else '(the built bank matches its lock)', flush=True)
              except (OSError, ValueError, EOFError, lzma.LZMAError, http.client.HTTPException) as e:
                  failed[item['name']] = str(e)
                  print('FAILED', item['name'], '|', e, flush=True)
      # The banks that did install are written to the configuration either way; a failure is listed and ends in exit 2.
      save_json(args.output.resolve(), config)
      print('Saved', args.output.resolve())
      if failed:
          raise ValueError(f'{len(failed)} bank(s) failed: ' + '; '.join(f'{k}: {v}' for k, v in failed.items()) +
                           '.  The others are configured; run again with --only ' + ','.join(failed) + ' (and a new --output) to retry.')
      if args.delete_sources:
          delete_sources(lock, only, runtime, cache)
  
  
  def source_paths(lock, only, runtime, cache):
      """What the installer downloaded or cut for the requested banks: archives, sample checkouts, recordings, notes."""
      paths = set()
      for item in lock['archives']:
          if only is None or item['name'] in only:
              paths.add(cache / Path(urllib.parse.urlsplit(item['url']).path).name)
              mirror = lock.get('prebuilt', {}).get('archives', {}).get(item['name'])
              if mirror:
                  paths.add(cache / mirror['file'])
      for item in lock['builds']:
          if only is not None and item['name'] not in only:
              continue
          asset = lock.get('prebuilt', {}).get('assets', {}).get(item['name'])
          if asset:
              paths.add(cache / asset['file'])
          if item.get('kind') == 'split':
              paths.add(runtime / 'sources' / item['source_folder'])
              paths.add(runtime / 'sources' / f'{item["name"]}-{item["version"].split()[-1]}-notes')
          else:
              paths.add(runtime / 'sources' / f'{item["repository"].split("/")[1]}-{item["commit"][:12]}')
      return sorted(paths)
  
  
  def delete_sources(lock, only, runtime, cache):
      freed = 0
      for path in source_paths(lock, only, runtime, cache):
          if path.is_dir():
              freed += sum(f.stat().st_size for f in path.rglob('*') if f.is_file())
              shutil.rmtree(path)
          elif path.is_file():
              freed += path.stat().st_size
              path.unlink()
          else:
              continue
          print('Deleted', path)
      print(f'Freed {freed / 1e6:.0f} MB of downloaded sources')
  
  if __name__ == '__main__':
      for stream in (sys.stdout, sys.stderr):
          stream.reconfigure(errors='backslashreplace')
      try:
          main()
      except (ValueError, OSError) as e:
          print('ERROR:', e, file=sys.stderr)
          sys.exit(2)
  

• skills/music-production/scripts/make_example.py2.6 KB8a04c634af70upstream ↗

  """Create an original short setup fixture; full-song writing is an agent task."""
  import argparse
  from pathlib import Path
  from musiclib.model import save_json,validate
  
  def example():
      p={'schema':'music-production/v1','title':'Four-bar setup study','version':'1.0','clock':{'bars':4,'meter':[4,4],'tempo':[[0,108],[8,112]],'tail_seconds':3},'sections':[{'id':'question','start_bar':0,'bars':2,'intent':'Short opening phrase'},{'id':'answer','start_bar':2,'bars':2,'intent':'Cadential answer'}],'tracks':[],'mix':{'target_lufs':-18,'peak_ceiling_db':-1.6,'max_limit_db':.8,'room_seconds':.7},'decisions':['Functional fixture only; not a demonstration of a complete song.']}
      def add(ident,role,program,engine='soundfont',drums=False,level=-27):
          t={'id':ident,'name':ident,'role':role,'engine':engine,'drums':drums,'program':program,'bank':0,'range':[0,127],'notes':[],'performance':{'notes':{},'cc':[]},'mix':{'level_dbfs':level,'level_mode':'peak' if drums else 'active_rms','pan':0,'highpass':35 if role=='bass' else 130,'lowpass':14000,'reverb_db':-19 if not drums else None,'delay_db':-24 if engine=='additive' else None,'eq':[],'automation':[]}}
          p['tracks'].append(t);return t
      def n(t,on,dur,pitch,velocity=76,gate=1,timing=0):
          ident=t['id']+'_n'+str(len(t['notes']));t['notes'].append({'id':ident,'on':on,'duration':dur,'pitch':pitch,'phrase':'setup','articulation':'rise' if t['engine']=='noise' else 'normal'});t['performance']['notes'][ident]={'velocity':velocity,'gate':gate,'timing_ms':timing}
      piano=add('piano','harmony',0,level=-29)
      bass=add('bass','bass',33,level=-27)
      lead=add('lead','melody',80,'additive',level=-25)
      drums=add('drums','drums',0,drums=True,level=-20)
      fx=add('fx','effect',0,'noise',level=-38)
      for bar,chord in enumerate(([60,64,67],[62,65,69],[60,64,69],[59,62,67])):
          for i,pitch in enumerate(chord):n(piano,bar*4+i*.5,.85,pitch,62+i*3,1.05)
          piano['performance']['cc'] += [[bar*4,64,0],[bar*4+.07,64,100]]
          n(bass,bar*4,3.5,[36,38,33,31][bar],74,.96)
          for beat,pitch,vel in [(0,36,87),(1,38,88),(2.5,36,70),(3,38,90)]:n(drums,bar*4+beat,.12,pitch,vel,.8,4 if pitch==38 else 0)
      for on,dur,pitch in [(0,1.5,72),(1.5,.5,74),(2,1.5,76),(4,1.5,77),(6,1.5,74),(8,1,76),(9,.5,74),(9.5,.5,72),(10,1.5,69),(12,1,71),(13,.5,74),(13.5,.5,71),(14,1.7,72)]:n(lead,on,dur,pitch,82,1.02)
      n(fx,6,1.75,60,62,.8,20)
      validate(p);return p
  
  if __name__=='__main__':
      a=argparse.ArgumentParser();a.add_argument('--output',type=Path,required=True);args=a.parse_args()
      if args.output.exists():raise ValueError('Output exists')
      save_json(args.output,example());print(args.output)
  

• skills/music-production/scripts/musiclib/__init__.py93 Bbbea8bac1620upstream ↗

  """Original MIT music-production helpers. No external DAW API required."""
  VERSION = '0.1.0'
  

• skills/music-production/scripts/musiclib/analysis.py22 KB2eee779df7e7upstream ↗

  """Measurements of a finished render, for reviewing a mix without listening.  They locate problems; they are not a
  verdict on how the music sounds.
  
  - Each section of the master: ungated K-weighted level (LUFS-like), peak, crest factor, the share of power in six
    bands, left/right correlation, and the level change when folded to mono.
  - Each processed stem: its K-weighted level per section.
  - Lead presence: the melody against everything else, as the mix hears it.  In a bus-chain render that is the lead
    bus's output against the sum of the other buses (bus processing included: a lead compressor can take several dB);
    in a room-chain render, the melody stems (role "melody") against all other stems, which is what that chain sums.
    Reverb and delay returns count on neither side.  Two measures per section:
    - the K-weighted level of the melody minus that of the accompaniment, full band, over the whole section
      (`lead_minus_accompaniment_level_db`), the usual sense of "the lead sits 2 dB above the band";
    - frame by frame in 24 log bands from 200 Hz to 8 kHz, the median and 10th percentile of lead minus
      accompaniment, and the share of lead energy in bands where the accompaniment is at least 3 dB louder ("masked
      share").  Bass and cymbals are left out, so these read several dB higher than the full-band level difference.
    Both are proxies, not a model of masking.
  - The melody group: in each section (and each requested bar span), the melody parts and every part whose onsets fall
    on theirs, at the same pitch class in any octave, at least 60% of the time. An orchestral tune is played by many
    parts at once, most of them on the ensemble bus, so the group's stems against all other stems
    (`melody_group_minus_rest_db`) reads the balance the bus measure misses.
  - Balance suggestions: parts within a few dB of the group's loudest stem, with the cut that would put each 6 dB under
    it (3 dB for bass and countermelody parts). Near full velocity a softer velocity changes the level very little;
    make the cut with `level_dbfs` or let the part rest.
  - The whole master: its long-term spectrum (slope over 100 Hz-4 kHz, low and top ends; `tonal_balance`), its width
    per band (`side_minus_mid_db`) and its peak-to-loudness ratio, against published measurements of commercial
    recordings and, with a reference, against that reference: a measured demo of the genre from tonal-references.json,
    another render or a WAV file.
  - Warnings from the stem receipts: parts whose dynamics sat between two recorded takes that disagree in phase, and
    Surge XT parts that played their fallback preset.
  """
  import json
  import math
  from pathlib import Path
  import tempfile
  
  import numpy as np
  
  from . import audio
  from .measure import k_filter, level
  from .model import Clock
  
  RATE = audio.RATE
  BANDS = [(20, 60), (60, 250), (250, 800), (800, 2500), (2500, 6000), (6000, 16000)]
  NPERSEG, HOP = 4096, 1024
  REFERENCES = Path(__file__).resolve().parents[2] / 'references' / 'tonal-references.json'
  OCTAVE_NAMES = {-5: '31', -4: '63', -3: '125', -2: '250', -1: '500', 0: '1k', 1: '2k', 2: '4k', 3: '8k', 4: '16k'}
  
  
  def db(x):
      return 10 * np.log10(np.maximum(x, 1e-20))
  
  
  def section_spans(p, clock):
      return [(s['id'], round(clock.seconds(s['start_bar'] * clock.beats_per_bar) * RATE),
               round(clock.seconds((s['start_bar'] + s['bars']) * clock.beats_per_bar) * RATE)) for s in p['sections']]
  
  
  def band_powers(x, edges):
      """Power per frame in each band [edges[i], edges[i+1]) of a mono signal (Hann window of 4096, hop 1024, frames
      centred on multiples of the hop).  Returns (bands x frames, frame centres in samples)."""
      f = np.fft.rfftfreq(NPERSEG, 1 / RATE)
      lo, hi = np.searchsorted(f, edges[:-1]), np.searchsorted(f, edges[1:])
      pad = np.concatenate([np.zeros(NPERSEG // 2), np.asarray(x, dtype=np.float64), np.zeros(NPERSEG // 2)])
      frames = 1 + max(0, (len(pad) - NPERSEG) // HOP)
      window = np.hanning(NPERSEG + 1)[:-1]
      out = np.empty((len(lo), frames))
      for start in range(0, frames, 256):
          idx = np.arange(start, min(frames, start + 256))
          power = np.abs(np.fft.rfft(pad[idx[:, None] * HOP + np.arange(NPERSEG)] * window, axis=1)) ** 2
          c = np.concatenate([np.zeros((len(idx), 1)), np.cumsum(power, axis=1)], axis=1)
          out[:, idx] = (c[:, hi] - c[:, lo]).T
      return out, np.arange(frames) * HOP
  
  
  def master_sections(master, spans):
      km = k_filter(master)
      rows = []
      for sid, a, b in spans:
          x = np.asarray(master[a:b], dtype=np.float64)
          if len(x) < 2:
              continue
          mono = x.mean(axis=1)
          spec = np.abs(np.fft.rfft(mono * np.hanning(len(mono)))) ** 2
          f = np.fft.rfftfreq(len(mono), 1 / RATE)
          total = max(float(spec[(f >= 20) & (f < 16000)].sum()), 1e-30)
          left, right = x[:, 0], x[:, 1]
          corr = float(np.corrcoef(left, right)[0, 1]) if left.std() > 0 and right.std() > 0 else 1.0
          mono_change = 10 * math.log10(max(float(np.mean(mono ** 2)), 1e-30) / max(float(np.mean(x ** 2)), 1e-30))
          peak = 20 * math.log10(max(float(np.abs(x).max()), 1e-15))
          rows.append({'section': sid, 'seconds': [round(a / RATE, 1), round(b / RATE, 1)],
                       'level_lufs_ungated': round(level(km[a:b]), 2), 'peak_dbfs': round(peak, 2),
                       'crest_db': round(peak - 10 * math.log10(max(float(np.mean(x ** 2)), 1e-30)), 1),
                       'band_share_pct': {f'{lo}-{hi}': round(float(spec[(f >= lo) & (f < hi)].sum()) / total * 100, 1)
                                          for lo, hi in BANDS},
                       'lr_correlation': round(corr, 3), 'mono_sum_change_db': round(mono_change, 2)})
      return rows
  
  
  def tonal_balance(master):
      """The master's long-term average spectrum against published measurements of commercial recordings.
  
      Power per hertz in third-octave bands from 31.5 Hz to 16 kHz (mono, the whole render), the slope of a line fitted
      from 100 Hz to 4 kHz in dB per octave (pink noise reads -3, white noise 0), and how far the bands below 80 Hz and
      from 6.3 to 12.5 kHz sit from that line extended.  Popular recordings fall about 5 dB per octave over 100 Hz-4 kHz
      (Pestana et al. 2013, about 800 recordings 1950-2010; Elowsson and Friberg 2017 measured 4.5), hip-hop and
      electronic ones extend further into the low end and jazz and folk fall away at the top (Pestana et al. 2013), and
      more percussion raises the top (Elowsson and Friberg 2017)."""
      mono = np.asarray(master, dtype=np.float64).mean(axis=1)
      centres = 1000 * 2 ** (np.arange(-15, 15) / 3)
      centres = centres[(centres >= 31.5) & (centres <= 16000)]
      edges = np.concatenate([[centres[0] * 2 ** (-1 / 6)], centres * 2 ** (1 / 6)])
      power, _ = band_powers(mono, edges)
      f = np.fft.rfftfreq(NPERSEG, 1 / RATE)
      bins = np.array([max(1, int(np.sum((f >= lo) & (f < hi)))) for lo, hi in zip(edges[:-1], edges[1:])])
      spectrum = db(power.mean(axis=1) / bins)
      octaves = np.log2(centres / 1000)
      fit = (centres >= 100) & (centres <= 4000)
      slope, offset = np.polyfit(octaves[fit], spectrum[fit], 1)
      line = slope * octaves + offset
      low = centres < 80
      top = (centres >= 6300) & (centres <= 12500)
      return {'slope_db_per_octave_100hz_4khz': round(float(slope), 2),
              'low_end_vs_line_db': round(float(np.mean(spectrum[low] - line[low])), 1),
              'top_end_vs_line_db': round(float(np.mean(spectrum[top] - line[top])), 1),
              'third_octaves_db': {f'{c:.0f}': round(float(v - line[np.argmin(np.abs(centres - 1000))]), 1) for c, v in zip(centres, spectrum)},
              'reference': 'popular recordings fall about 4.5-5 dB per octave over 100 Hz-4 kHz (Pestana et al. 2013; Elowsson and Friberg 2017)'}
  
  
  def stereo_width(master):
      """Side minus mid power in dB below 120 Hz, from 120 Hz to 2 kHz and from 2 to 16 kHz: -60 (the floor) is mono,
      0 as much side as mid (left and right unrelated)."""
      x = np.asarray(master, dtype=np.float64)
      edges = [20, 120, 2000, 16000]
      mid, _ = band_powers((x[:, 0] + x[:, 1]) / 2, edges)
      side, _ = band_powers((x[:, 0] - x[:, 1]) / 2, edges)
      return {f'{lo}-{hi}': round(max(-60.0, float(db(s.sum()) - db(m.sum()))), 1)
              for lo, hi, m, s in zip(edges[:-1], edges[1:], mid, side)}
  
  
  def long_term(master, meter):
      """What a reference stores: tonal balance, width per band, and from the loudness meter (`meter`: the report's
      master entry) integrated loudness, true peak, loudness range and the peak-to-loudness ratio (PLR)."""
      loud = {k: meter.get(k) for k in ('integrated_lufs', 'true_peak_dbtp', 'lra_lu')}
      if loud['integrated_lufs'] is not None and loud['true_peak_dbtp'] is not None:
          loud['plr_db'] = round(loud['true_peak_dbtp'] - loud['integrated_lufs'], 1)
      return {'tonal_balance': tonal_balance(master), 'side_minus_mid_db': stereo_width(master), 'loudness': loud}
  
  
  def load_reference(spec, config=None):
      """(name, long_term measures) of a reference: a name in tonal-references.json, a render folder (its master.wav
      and report.json), or a WAV file, which FFmpeg converts to 48 kHz and meters in a temporary folder (needs a
      config).  Nothing of a WAV reference is kept."""
      table = json.loads(REFERENCES.read_text(encoding='utf-8'))['references']
      if spec in table:
          return spec, table[spec]
      path = Path(spec)
      if path.is_dir():
          master, _ = audio.read_wav(path / 'master.wav')
          report = json.loads((path / 'report.json').read_text(encoding='utf-8'))
          return f'{path.resolve().parent.name}/{path.name}', long_term(master, report.get('master', {}))
      if path.is_file():
          if path.suffix.lower() != '.wav':
              raise ValueError(f'--reference {spec}: convert it to WAV first (for example ffmpeg -i song.flac song.wav)')
          if config is None:
              raise ValueError('--reference with a WAV file needs --config: FFmpeg converts and meters it')
          with tempfile.TemporaryDirectory() as tmp:
              wav = Path(tmp) / 'reference.wav'
              audio.ffmpeg(config, ['-i', str(path), '-ac', '2', '-ar', str(RATE), '-c:a', 'pcm_f32le', str(wav)], tmp, 'reference')
              meter = audio.measure(config, wav, tmp, 'reference_loudness')
              master, _ = audio.read_wav(wav)
              measures = long_term(np.array(master), meter)
              del master
          return path.name, measures
      raise ValueError(f'--reference {spec}: not one of {", ".join(sorted(table))}, a render folder or a WAV file')
  
  
  def octaves(third_octaves):
      """Third-octave levels (dB, keyed by centre in Hz) power-averaged into octaves."""
      groups = {}
      for key, v in third_octaves.items():
          groups.setdefault(round(math.log2(float(key) / 1000)), []).append(10 ** (v / 10))
      return {OCTAVE_NAMES[k]: 10 * math.log10(sum(g) / len(g)) for k, g in sorted(groups.items())}
  
  
  def compare(here, ref, name):
      """The master's long-term measures minus a reference's, with warnings where they differ widely: a tilt over
      1.5 dB per octave (8 dB across 100 Hz-4 kHz), a low or top end 6 dB apart, a PLR 4 dB apart, a bass much wider
      than the reference's.  Both spectra are levelled at their own fitted line at 1 kHz."""
      a, b = here['tonal_balance'], ref['tonal_balance']
      oa, ob = octaves(a['third_octaves_db']), octaves(b['third_octaves_db'])
      la, lb = here['loudness'], ref['loudness']
  
      def diff(x, y):
          return None if x is None or y is None else round(x - y, 2)
      out = {'reference': name,
             'slope_difference_db_per_octave': diff(a['slope_db_per_octave_100hz_4khz'], b['slope_db_per_octave_100hz_4khz']),
             'low_end_difference_db': diff(a['low_end_vs_line_db'], b['low_end_vs_line_db']),
             'top_end_difference_db': diff(a['top_end_vs_line_db'], b['top_end_vs_line_db']),
             'octaves_minus_reference_db': {k: round(oa[k] - ob[k], 1) for k in oa if k in ob},
             'side_minus_mid_difference_db': {k: diff(v, ref['side_minus_mid_db'].get(k)) for k, v in here['side_minus_mid_db'].items()},
             **{f'{k}_difference': diff(la.get(k), lb.get(k)) for k in ('integrated_lufs', 'plr_db', 'lra_lu')}}
      warnings = []
      tilt = out['slope_difference_db_per_octave']
      if abs(tilt) > 1.5:
          warnings.append(f'the master tilts {abs(tilt):g} dB per octave {"brighter" if tilt > 0 else "darker"} than {name} over '
                          f'100 Hz-4 kHz ({a["slope_db_per_octave_100hz_4khz"]:g} against {b["slope_db_per_octave_100hz_4khz"]:g}): '
                          'octaves_minus_reference_db shows where.')
      for key, region in (('low_end_difference_db', 'low end (below 80 Hz)'), ('top_end_difference_db', 'top (6.3-12.5 kHz)')):
          if abs(out[key]) > 6:
              warnings.append(f'the {region} sits {abs(out[key]):g} dB {"above" if out[key] > 0 else "below"} that of {name}, '
                              'each measured against its own slope.')
      plr = out['plr_db_difference']
      if plr is not None and abs(plr) > 4:
          warnings.append(f'the master\'s peak-to-loudness ratio is {la["plr_db"]:g} dB against {lb["plr_db"]:g} for {name}: '
                          f'{"more compressed" if plr < 0 else "more dynamic"} than the reference.')
      low_here, low_ref = here['side_minus_mid_db']['20-120'], ref['side_minus_mid_db'].get('20-120')
      if low_ref is not None and low_here > -20 and low_here - low_ref > 10:
          warnings.append(f'below 120 Hz the side is {-low_here:g} dB under the mid, against {-low_ref:g} for {name}: a bass wider '
                          'than the reference loses level in mono; check the panning of the bass parts and their reverb, or '
                          'set mix.mono_below_hz.')
      return out, warnings
  
  
  def lead_presence(lead, accompaniment, spans):
      """`lead` and `accompaniment` are stereo sums of their stems."""
      lead_k, acc_k = k_filter(lead), k_filter(accompaniment)
      level_diff = {sid: level(lead_k[a:b]) - level(acc_k[a:b]) for sid, a, b in spans}
      del lead_k, acc_k
      edges = np.geomspace(200, 8000, 25)
      bl, centres = band_powers(lead.mean(axis=1), edges)
      ba, _ = band_powers(accompaniment.mean(axis=1), edges)
      lead_total, acc_total = bl.sum(axis=0), ba.sum(axis=0)
      active = db(lead_total) > db(lead_total).max() - 35
      rows = []
      for sid, a, b in spans:
          sel = active & (centres >= a) & (centres < b)
          row = {'section': sid, 'lead_active_frames': int(sel.sum())}
          if sel.sum() >= 5:
              row['lead_minus_accompaniment_level_db'] = round(level_diff[sid], 1)
          if sel.sum() >= 5:
              ratio = db(lead_total[sel]) - db(acc_total[sel])
              masked = (ba[:, sel] > 2 * bl[:, sel]) * bl[:, sel]
              row.update(lead_minus_accompaniment_db_median=round(float(np.median(ratio)), 1),
                         lead_minus_accompaniment_db_p10=round(float(np.percentile(ratio, 10)), 1),
                         masked_share_pct=round(float(masked.sum() / bl[:, sel].sum() * 100), 1))
          rows.append(row)
      return rows
  
  
  def doublings(p, lo, hi, share=.6):
      """The melody parts sounding in beats [lo, hi) and the parts that double them: at least `share` of a part's onsets
      there fall on a melody onset of the same pitch class."""
      melody = [t for t in p['tracks'] if t.get('role') == 'melody' and any(lo <= n['on'] < hi for n in t['notes'])]
      keys = {(round(n['on'], 3), n['pitch'] % 12) for t in melody for n in t['notes'] if lo <= n['on'] < hi}
      group = {t['id'] for t in melody}
      for t in p['tracks']:
          if t['id'] in group or t.get('drums'):
              continue
          notes = [n for n in t['notes'] if lo <= n['on'] < hi]
          if len(notes) >= 4 and sum((round(n['on'], 3), n['pitch'] % 12) in keys for n in notes) >= share * len(notes):
              group.add(t['id'])
      return sorted(group)
  
  
  MARGIN_DB = {'bass': 3, 'countermelody': 3}
  
  
  def balance(p, levels, group):
      """Non-melody parts within MARGIN_DB (6 dB by default) of the loudest melody-group stem, loudest first."""
      roles = {t['id']: t.get('role') for t in p['tracks']}
      drums = {t['id'] for t in p['tracks'] if t.get('drums')}
      lead = max((levels[t] for t in group if t in levels and levels[t] > -90), default=None)
      out = []
      if lead is None:
          return out
      for tid, x in sorted(levels.items(), key=lambda kv: -kv[1]):
          if tid in group or tid in drums or x <= -90:
              continue
          want = lead - MARGIN_DB.get(roles.get(tid), 6)
          if x > want:
              out.append({'track': tid, 'role': roles.get(tid), 'level_minus_melody_db': round(x - lead, 1),
                          'suggested_cut_db': round(x - want, 1)})
      return out
  
  
  def analyze(render_dir, bar_spans=(), reference=None):
      """`bar_spans`: (name, first bar, last bar), 1-based and inclusive, measured like the sections.  `reference`:
      (name, measures) from load_reference."""
      render_dir = Path(render_dir)
      p = json.loads((render_dir / 'project.json').read_text(encoding='utf-8'))
      clock = Clock(p)
      spans = section_spans(p, clock)
      bpb = clock.beats_per_bar
      for name, first, last in bar_spans:
          if last > p['clock']['bars']:
              raise ValueError(f'--span {name}:{first}-{last}: the piece has {p["clock"]["bars"]} bars')
      extra = [(f'span:{name}', round(clock.seconds((first - 1) * bpb) * RATE), round(clock.seconds(last * bpb) * RATE))
               for name, first, last in bar_spans]
      regions = spans + extra
      beats = {sid: ((s['start_bar']) * bpb, (s['start_bar'] + s['bars']) * bpb) for sid, s in zip([x[0] for x in spans], p['sections'])}
      beats.update({f'span:{name}': ((first - 1) * bpb, last * bpb) for name, first, last in bar_spans})
      groups = {sid: doublings(p, *beats[sid]) for sid, _, _ in regions}
      sums = {sid: [np.zeros((b - a, 2)), np.zeros((b - a, 2))] for sid, a, b in regions}
      master, _ = audio.read_wav(render_dir / 'master.wav')
      rows = master_sections(master, spans)
      report_file = render_dir / 'report.json'
      report = json.loads(report_file.read_text(encoding='utf-8')) if report_file.is_file() else {}
      buses = (report.get('buses') or {}).get('assignment') if report.get('chain') == 'bus' else None
      stems, lead, accompaniment = {}, None, None
      for t in p['tracks']:
          y, _ = audio.read_wav(render_dir / 'stems' / f'{t["id"]}.wav')
          ky = k_filter(y)
          stems[t['id']] = {sid: round(level(ky[a:b]), 2) for sid, a, b in regions}
          del ky
          for sid, a, b in regions:
              seg = np.asarray(y[a:b], dtype=np.float64)
              sums[sid][0 if t['id'] in groups[sid] else 1][:len(seg)] += seg
          if buses is None:
              y = np.asarray(y, dtype=np.float64)
              if t.get('role') == 'melody':
                  lead = y if lead is None else lead + y
              else:
                  accompaniment = y if accompaniment is None else accompaniment + y
      if buses is not None:
          lead_tracks = [tid for tid, b in buses.items() if b == 'lead']
          for b in sorted(set(buses.values())):
              y = np.asarray(audio.read_wav(render_dir / 'returns' / f'bus_{b}.wav')[0], dtype=np.float64)
              if b == 'lead':
                  lead = y
              else:
                  accompaniment = y if accompaniment is None else accompaniment + y
      else:
          lead_tracks = [t['id'] for t in p['tracks'] if t.get('role') == 'melody']
      melody = []
      for sid, a, b in regions:
          g, r = sums[sid]
          row = {'region': sid, 'melody_group': groups[sid]}
          if groups[sid] and np.any(g) and np.any(r):
              row['melody_group_minus_rest_db'] = round(level(k_filter(g)) - level(k_filter(r)), 1)
          row['balance'] = balance(p, {tid: v[sid] for tid, v in stems.items()}, groups[sid])
          melody.append(row)
      warnings = []
      here = long_term(master, report.get('master', {}))
      slope = here['tonal_balance']['slope_db_per_octave_100hz_4khz']
      if not -8 <= slope <= -1.5:
          warnings.append(f'the master falls {-slope:g} dB per octave over 100 Hz-4 kHz, far {"darker" if slope < -8 else "brighter"} '
                          'than commercial recordings, which average about 4.5-5 (Pestana et al. 2013; Elowsson and Friberg '
                          '2017): check the bass and pads against the cymbals and leads, and compare with a reference of the '
                          'genre (--reference).')
      comparison = None
      if reference is not None:
          comparison, differ = compare(here, reference[1], reference[0])
          warnings += differ
      squeezed = {k: v for k, v in ((report.get('mastering') or {}).get('limiting_db') or {}).items() if v > 3}
      if squeezed:
          worst = max(squeezed, key=squeezed.get)
          warnings.append(f'the delivery limiter took {squeezed[worst]:g} dB of level from {worst} '
                          f'({", ".join(f"{k} {v:g}" for k, v in squeezed.items())}): Sound On Sound\'s guidance is 3-6 dB of '
                          'gain reduction on loud peaks and rarely more than 6 (White 2003), and heavier limiting can flatten '
                          'transients; have it heard, lower target_lufs, or lower that section\'s peaks '
                          '(mix.saturate on its loudest parts, ducking, less level on what it adds).')
      receipts_file = render_dir / 'stem_receipts.json'
      for r in json.loads(receipts_file.read_text(encoding='utf-8')) if receipts_file.is_file() else []:
          played = r.get('surge') or {}
          if played.get('rendered_with') == 'built-in synth':
              warnings.append(f'{r["track"]}: Surge XT preset {played.get("preset")} played on the built-in synth\'s '
                              f'{played.get("fallback")} instead ({played.get("reason")}).')
          share = (r.get('expressive') or {}).get('correction_out_of_range_pct')
          if share is not None and share > 2:
              warnings.append(f'{r["track"]}: for {share:g}% of its playing the dynamics sat between two recorded takes that '
                              'disagree in phase (the correction ran past 1.25 or under 0.8); play it in dynamics_mode "take" (the default), or keep a solo near one recorded dynamic.')
      return {'render': render_dir.name, 'sections': rows, 'tonal_balance': here['tonal_balance'],
              'side_minus_mid_db': here['side_minus_mid_db'], 'reference_comparison': comparison, 'stem_levels_lufs_ungated': stems,
              'lead_tracks': lead_tracks, 'lead_source': 'buses' if buses is not None else 'stems',
              'lead_presence': lead_presence(lead, accompaniment, regions) if lead is not None and accompaniment is not None else [],
              'melody_groups': melody, 'warnings': warnings,
              'master': here['loudness'],
              'note': 'Measurements only; they do not establish how the music sounds.'}
  

• skills/music-production/scripts/musiclib/audio.py17 KB4f19b32b7764upstream ↗

  """Local headless processes, narrow WAV I/O, instruments and audio measurements."""
  import hashlib
  import json
  import math
  import os
  from pathlib import Path
  import re
  import struct
  import subprocess
  import time
  import numpy as np
  from .model import Clock, performed, save_json, file_hash
  
  RATE=48000
  
  def absolute_paths(config, base):
      """The configuration with every relative path made absolute against `base` (the folder of the file it was read
      from), so it can be written anywhere."""
      entries = [config[k] for k in ('fluidsynth', 'ffmpeg', 'soundfont', 'surge', 'pedalboard') if isinstance(config.get(k), dict)]
      entries += [v for v in config.get('soundfonts', {}).values() if isinstance(v, dict)]
      for entry in entries:
          for key in ('path', 'binary', 'patches'):
              if isinstance(entry.get(key), str) and not Path(entry[key]).is_absolute():
                  entry[key] = str((Path(base) / entry[key]).resolve())
      return config
  
  def load_config(path):
      """Verify every configured executable and SoundFont by path and SHA-256. `soundfont` is the
      default bank; optional `soundfonts` adds named banks that tracks select with `soundfont`."""
      path=Path(path).resolve()
      data=json.loads(path.read_text(encoding='utf-8'))
      extra=data.get('soundfonts',{})
      if not isinstance(extra,dict) or 'default' in extra or not all(re.fullmatch(r'[a-z][a-z0-9_]{0,47}',k) for k in extra):
          raise ValueError('soundfonts must map lower-case names (not "default") to entries')
      for key,entry in [(k,data[k]) for k in ('fluidsynth','ffmpeg','soundfont')]+[('soundfonts.'+k,v) for k,v in extra.items()]:
          p=Path(entry['path'])
          if not p.is_absolute():p=path.parent/p
          p=p.resolve()
          # A system package update replaces the executable: the configuration then names a file that is gone or changed.
          again=' Run bootstrap.py again (and fetch_soundbanks.py for the banks) to write a new configuration.'
          if not p.is_file():raise ValueError(f'Missing dependency: {key}: {p}.'+again)
          if not entry.get('sha256') or file_hash(p)!=entry['sha256']:raise ValueError(f'Dependency hash mismatch: {key}: {p} changed since the configuration was written.'+again)
          entry['path']=str(p)
      return data
  
  def soundfont_for(config,track):
      name=track.get('soundfont','default')
      if name=='default':return name,config['soundfont']
      if name not in config.get('soundfonts',{}):
          lock=Path(__file__).resolve().parents[2]/'references'/'soundbank-lock.json'
          known={i['name'] for k in ('archives','builds') for i in json.loads(lock.read_text(encoding='utf-8'))[k]} if lock.is_file() else set()
          hint=(f' It is one of the optional banks: install it with fetch_soundbanks.py --only {name} (references/setup.md), then pass the configuration it writes.'
                if name in known else ' Configure it with studio.py add-soundfont, or choose a configured bank (studio.py doctor lists them).')
          raise ValueError(f'{track["id"]}: SoundFont "{name}" is not configured.'+hint)
      return name,config['soundfonts'][name]
  
  def run(argv,folder,label,timeout=600):
      folder=Path(folder);folder.mkdir(parents=True,exist_ok=True)
      log=folder/(label+'.log');started=time.monotonic()
      flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
      with log.open('wb') as out:
          proc=subprocess.Popen([str(x) for x in argv],stdin=subprocess.DEVNULL,stdout=out,stderr=subprocess.STDOUT,creationflags=flags)
          try:code=proc.wait(timeout=timeout)
          except subprocess.TimeoutExpired:
              proc.kill();proc.wait();raise TimeoutError(f'{label} exceeded {timeout}s; log: {log}')
      receipt={'argv':[str(x) for x in argv],'returncode':code,'seconds':round(time.monotonic()-started,3),'pid':proc.pid,'launch_mode':'CREATE_NO_WINDOW' if flags else 'no stdin; no GUI arguments','log':str(log)}
      save_json(folder/(label+'.process.json'),receipt)
      content=log.read_text(encoding='utf-8',errors='replace')
      if code:raise RuntimeError(f'{label} failed ({code}): {content[-3000:]}')
      return content
  
  def fluidsynth(config,font_path,midi_path,wav_path,folder,label='fluidsynth'):
      """Render a MIDI file on one SoundFont: dry (no chorus or reverb), 48 kHz float, sample-accurate timing."""
      log=run([config['fluidsynth']['path'],'-n','-i','-C','0','-R','0','-g','.55','-r',str(RATE),'-T','wav','-O','float','-o','synth.cpu-cores=1','-o','synth.polyphony=256','-o','synth.midi-bank-select=mma','-o','player.timing-source=sample','-F',wav_path,font_path,midi_path],folder,label)
      # Only FluidSynth's own messages ("fluidsynth: error: ...", a preset "not found"), not paths it echoes.
      problems=[line for line in log.splitlines() if line.lower().startswith('fluidsynth:') and any(w in line.lower() for w in ('error','panic','failed','not found'))]
      if problems:raise ValueError('Instrument rendering diagnostic: '+'\n'.join(problems)[-2000:])
      return log
  
  def ffmpeg(config,args,folder,label):
      guarded=[];i=0
      while i<len(args):
          if args[i]=='-i':
              p=Path(args[i+1]).resolve()
              if not p.is_file() or p.suffix.lower()!='.wav':raise ValueError('Audio inputs must be local generated WAVs')
              with p.open('rb') as f:header=f.read(12)
              if header[:4]!=b'RIFF' or header[8:12]!=b'WAVE':raise ValueError('Not RIFF/WAVE')
              guarded.extend(['-protocol_whitelist','file,pipe','-f','wav','-i',p]);i+=2
          else:guarded.append(args[i]);i+=1
      return run([config['ffmpeg']['path'],'-hide_banner','-nostdin','-y','-threads','1','-filter_threads','1','-filter_complex_threads','1',*guarded],folder,label)
  
  _AFIR={}
  def afir_options(config):
      """Options for convolving with an impulse response used as it is.  FFmpeg 7 and newer spell "no normalization"
      irnorm=-1; FFmpeg 6 spells it gtype=none.  Asked once per executable from its own filter help."""
      path=config['ffmpeg']['path']
      if path not in _AFIR:
          flags=subprocess.CREATE_NO_WINDOW if os.name=='nt' else 0
          out=subprocess.run([path,'-hide_banner','-h','filter=afir'],capture_output=True,text=True,errors='replace',stdin=subprocess.DEVNULL,creationflags=flags,timeout=30)
          text=out.stdout+out.stderr
          _AFIR[path]=':'.join(['dry=1','wet=1','irnorm=-1' if 'irnorm' in text else 'gtype=none']+(['precision=double'] if 'precision' in text else []))
      return _AFIR[path]
  
  def write_wav(path,data,rate=RATE):
      data=np.asarray(data,dtype='<f4')
      if data.ndim==1:data=data[:,None]
      if not np.isfinite(data).all():raise ValueError('Nonfinite audio')
      channels=data.shape[1];raw=data.tobytes()
      fmt=struct.pack('<HHIIHH',3,channels,rate,rate*channels*4,channels*4,32)
      with Path(path).open('wb') as f:
          f.write(b'RIFF'+struct.pack('<I',36+len(raw))+b'WAVEfmt '+struct.pack('<I',16)+fmt+b'data'+struct.pack('<I',len(raw))+raw)
  
  def read_wav(path):
      path=Path(path)
      with path.open('rb') as f:
          h=f.read(12)
          if h[:4]!=b'RIFF' or h[8:12]!=b'WAVE':raise ValueError('Only RIFF WAV supported')
          fmt=None;payload=None
          while True:
              header=f.read(8)
              if not header:break
              if len(header)!=8:raise ValueError('Truncated WAV chunk')
              name,size=struct.unpack('<4sI',header)
              if size>path.stat().st_size:raise ValueError('Invalid WAV chunk size')
              if name==b'fmt ':fmt=f.read(size)
              elif name==b'data':payload=(f.tell(),size);f.seek(size,1)
              else:f.seek(size,1)
              if size%2:f.seek(1,1)
      if fmt is None or payload is None:raise ValueError('Missing WAV format/data')
      tag,channels,rate,_,align,bits=struct.unpack_from('<HHIIHH',fmt)
      if tag==65534:
          if len(fmt)<40:raise ValueError('Short extensible WAV')
          tag=struct.unpack_from('<H',fmt,24)[0]
      if channels not in (1,2) or rate!=RATE or payload[1]%align:raise ValueError('Unsupported WAV dimensions')
      offset,size=payload
      if tag==3 and bits==32:
          data=np.memmap(path,dtype='<f4',mode='r',offset=offset,shape=(size//4//channels,channels))
      elif tag==1 and bits==24:
          b=np.memmap(path,dtype='u1',mode='r',offset=offset,shape=(size//3,3))
          ints=b[:,0].astype(np.int32)|(b[:,1].astype(np.int32)<<8)|(b[:,2].astype(np.int32)<<16)
          ints=(ints^0x800000)-0x800000
          data=(ints.astype(np.float32)/8388608).reshape(-1,channels)
      elif tag==1 and bits==16:
          data=np.memmap(path,dtype='<i2',mode='r',offset=offset,shape=(size//2//channels,channels)).astype(np.float32)/32768
      else:raise ValueError(f'Unsupported WAV encoding {tag}/{bits}')
      return data,{'frames':len(data),'sample_rate':rate,'channels':channels,'bits':bits,'format':tag,'seconds':len(data)/rate}
  
  def audio_hash(path):
      """SHA-256 of a WAV's format and data chunks only: the same audio gives the same hash whatever else the writer
      stored (FluidSynth's PEAK chunk records the time it wrote the file)."""
      h=hashlib.sha256();found=set()
      with Path(path).open('rb') as f:
          head=f.read(12)
          if head[:4]!=b'RIFF' or head[8:12]!=b'WAVE':raise ValueError('Only RIFF WAV supported')
          while True:
              header=f.read(8)
              if len(header)<8:break
              name,size=struct.unpack('<4sI',header)
              if name in (b'fmt ',b'data'):
                  h.update(name+header[4:]);left=size;found.add(name)
                  while left:
                      block=f.read(min(left,1<<20))
                      if not block:raise ValueError('Truncated WAV chunk')
                      h.update(block);left-=len(block)
              else:f.seek(size,1)
              if size%2:f.seek(1,1)
      if found!={b'fmt ',b'data'}:raise ValueError('Missing WAV format/data')
      return h.hexdigest()
  
  def rms(data):return float(np.sqrt(np.mean(np.asarray(data,dtype=np.float64)**2)))
  
  def reference_level(path,mode='active_rms'):
      a,_=read_wav(path)
      if not np.isfinite(a).all():raise ValueError('Nonfinite stem')
      peak=float(np.max(np.abs(a)))
      if peak<1e-9:raise ValueError('Silent stem')
      if mode=='peak':return 20*math.log10(peak)
      blocks=[rms(a[i:i+RATE//2]) for i in range(0,len(a),RATE//2)]
      active=[x for x in blocks if x>max(1e-6,peak*.01)]
      return 20*math.log10(float(np.percentile(active,70)))
  
  def measure(config,path,logs,label):
      log=ffmpeg(config,['-i',path,'-af','loudnorm=I=-16:TP=-1.6:LRA=20:print_format=json','-f','null',os.devnull],logs,label)
      matches=re.findall(r'\{\s*"input_i"[\s\S]*?\}',log)
      if not matches:raise ValueError('No loudness measurement')
      values=json.loads(matches[-1]);a,info=read_wav(path)
      if not np.isfinite(a).all():raise ValueError('Nonfinite master')
      info.update(integrated_lufs=float(values['input_i']),true_peak_dbtp=float(values['input_tp']),lra_lu=float(values['input_lra']),sample_peak_dbfs=20*math.log10(max(1e-15,float(np.max(np.abs(a))))),clipped_samples=int(np.count_nonzero(np.abs(a)>=1)),rms=rms(a),sha256=file_hash(path),meter='FFmpeg loudnorm input analysis; BS.1770/EBU R128')
      if not all(math.isfinite(info[x]) for x in ('integrated_lufs','true_peak_dbtp','lra_lu')):raise ValueError('Silent/invalid loudness input')
      return info
  
  def automation(points,clock):
      if not points:return 'anull'
      pairs=[(clock.seconds(b),v) for b,v in points]
      expr=str(pairs[-1][1])
      for (a,x),(b,y) in reversed(list(zip(pairs,pairs[1:]))):
          expr=f'if(lt(t,{b:.8f}),{x}+({y-x})*(t-{a:.8f})/{b-a:.8f},{expr})'
      return f"volume='pow(10,({expr})/20)':eval=frame"
  
  def control(track,cc,times,clock,default):
      points=sorted((clock.seconds(b),v) for b,c,v in track['performance'].get('cc',[]) if c==cc)
      if not points:return np.full(len(times),default,dtype=np.float32)
      xs=np.array([x[0] for x in points]);ys=np.array([x[1] for x in points],dtype=np.float64)
      if track['performance'].get('cc_mode','linear')=='step':
          idx=np.searchsorted(xs,times,side='right')-1
          return np.where(idx>=0,ys[np.clip(idx,0,None)],default)
      return np.interp(times,xs,ys,left=default)
  
  def bend_curve(track,times,clock):
      """Pitch bend in semitones at `times` (seconds), linear between the part's bend points; zero without any."""
      pts=track['performance'].get('bend',[])
      if not pts:return np.zeros(len(times))
      return np.interp(times,[clock.seconds(b) for b,_ in pts],[v for _,v in pts])
  
  def synthesize(project,track,path):
      clock=Clock(project);frames=round((clock.end+clock.tail)*RATE)
      out=np.zeros((frames,2),dtype=np.float32)
      cfg=track.get('instrument',{})
      notes=performed(track,clock);prior_end=-1;prior_pitch=69;phase_end=0.0
      for ni,n in enumerate(notes):
          start=round(n['start_seconds']*RATE);stop=round(n['end_seconds']*RATE)
          release=round(cfg.get('release_ms',120)/1000*RATE)
          end=min(frames,stop+release);count=end-start
          if count<1:continue
          t=np.arange(count,dtype=np.float64)/RATE
          connected=n['start_seconds']<=prior_end+.012
          pitch=np.full(count,float(n['pitch']))
          glide=min(count,round(cfg.get('glide_ms',28)/1000*RATE)) if connected else 0
          if glide:pitch[:glide]=np.linspace(prior_pitch,n['pitch'],glide)
          pitch+=bend_curve(track,t+n['start_seconds'],clock)
          expr=control(track,11,t+n['start_seconds'],clock,100)/127
          brightness=control(track,74,t+n['start_seconds'],clock,66)
          vib=control(track,1,t+n['start_seconds'],clock,32)
          # Default depth .015+CC1*.001 semitones, unless the instrument sets a CC1=127 depth in cents.
          depth=cfg['vibrato_depth_cents']/100*vib/127 if 'vibrato_depth_cents' in cfg else .015+vib*.001
          pitch+=depth*np.clip((t-cfg.get('vibrato_delay_ms',180)/1000)/.25,0,1)*np.sin(2*math.pi*cfg.get('vibrato_rate_hz',5.1)*t)
          f=440*np.exp2((pitch-69)/12)
          phase=phase_end+2*math.pi*np.cumsum(f)/RATE
          gate=max(1,min(count,stop-start))
          env=np.ones(count,dtype=np.float64)
          attack=min(gate,round((.008 if connected else cfg.get('attack_ms',22)/1000)*RATE))
          if attack:env[:attack]=np.sin(np.linspace(0,math.pi/2,attack))**2
          if count>gate:env[gate:]=np.cos(np.linspace(0,math.pi/2,count-gate))**2
          env*=1-.1*np.minimum(t/.3,1)
          level=(n['velocity']/110)**1.4*.23
          detunes=cfg.get('detune_cents',[-5,0,5]);harmonics=cfg.get('harmonics',12)
          for vi,cents in enumerate(detunes):
              ratio=2**(cents/1200);wave=np.zeros(count)
              cutoff=cfg.get('cutoff_hz',1900)+brightness*18
              for h in range(1,harmonics+1):
                  weight=1/(h**cfg.get('spectral_tilt',1.25)*(1+(f*h/cutoff)**4))
                  if cfg.get('odd_only') and h%2==0:weight*=.12
                  weight=np.where(f*h*ratio<RATE*.45,weight,0)
                  wave+=np.sin(h*(phase*ratio+vi*.43))*weight
              pan=.5 if len(detunes)==1 else .35+.3*vi/(len(detunes)-1)
              signal=wave*env*expr*level/len(detunes)
              out[start:end,0]+=signal*np.sqrt(1-pan)
              out[start:end,1]+=signal*np.sqrt(pan)
          prior_end=n['end_seconds'];prior_pitch=n['pitch'];phase_end=float(phase[gate-1]%(2*math.pi))
      write_wav(path,out)
  
  def noise_fx(project,track,path):
      clock=Clock(project);out=np.zeros((round((clock.end+clock.tail)*RATE),2),dtype=np.float32)
      by_id={n['id']:n for n in track['notes']}
      for event in performed(track,clock):
          n=by_id[event['id']]
          start=round(event['start_seconds']*RATE);end=round(event['end_seconds']*RATE);count=end-start
          if count<1:continue
          rng=np.random.default_rng(int(hashlib.sha256(n['id'].encode()).hexdigest()[:8],16))
          t=np.arange(count)/RATE;progress=np.linspace(0,1,count)
          kind=n.get('articulation','rise');velocity=event['velocity']/127
          if kind=='impact':
              freq=45+70*np.exp(-t*16);sig=np.sin(2*math.pi*np.cumsum(freq)/RATE)*np.exp(-t*6)*np.minimum(t/.008,1)
              wave=np.column_stack([sig,sig])*.2
          elif kind=='vinyl':
              from .production import vinyl
              wave=vinyl(count,rng)*.13;fade=min(count//2,round(.05*RATE))
              wave[:fade]*=np.linspace(0,1,fade)[:,None];wave[count-fade:]*=np.linspace(1,0,fade)[:,None]
          else:
              white=rng.standard_normal((count,2));freq=np.fft.rfftfreq(count,1/RATE)
              band=(freq/1200)**2/(1+(freq/1200)**2)/(1+(freq/7500)**4)
              wave=np.fft.irfft(np.fft.rfft(white,axis=0)*band[:,None],n=count,axis=0)
              wave*=((progress**1.8 if kind=='rise' else np.exp(-t*4))*.13)[:,None]
              fade=min(count,round(.04*RATE));wave[-fade:]*=np.linspace(1,0,fade)[:,None]
          out[start:end]+=wave.astype(np.float32)*velocity
      write_wav(path,out)
  
  def room_ir(path,seconds):
      frames=round((seconds*1.3+.08)*RATE);rng=np.random.default_rng(64019)
      noise=rng.standard_normal((frames,2));frequency=np.fft.rfftfreq(frames,1/RATE)
      noise=np.fft.irfft(np.fft.rfft(noise,axis=0)/(1+(frequency[:,None]/3600)**2),n=frames,axis=0)
      t=np.arange(frames)/RATE;envelope=np.exp(-math.log(1000)*np.maximum(0,t-.024)/seconds);envelope[t<.024]=0
      noise*=envelope[:,None];noise/=np.sqrt(np.sum(noise**2,axis=0))[None,:]
      noise*=.40
      for delay,level in [(.029,.30),(.047,.24),(.071,.18),(.101,.12)]:
          noise[round(delay*RATE),0]+=level;noise[round((delay+.009)*RATE),1]+=level
      write_wav(path,noise)
  

• skills/music-production/scripts/musiclib/audition.py12 KB89f479897a15upstream ↗

  """Audition one part on several sounds: the same notes, performance and mix, rendered in its passage with the other
  parts (or alone), each brought to one loudness and played in turn, with the part's measured brightness for each.
  
  Choosing a sound by ear on the music it will play, rather than by its name or on a test scale: an A/B on identical
  notes.  The part plays first as written, then once per sound asked for:
  - `surge:<preset>`, a curated Surge XT patch (references/surge-presets.json); what its mod wheel and macros move is
    reported from references/surge-controls.json;
  - `synth:<preset>`, a built-in synth preset (references/synth-presets.json);
  - `soundfont:<bank>:<bank number>:<program>`, a SoundFont preset of the configuration.
  
  Each version is a passage project (excerpt.extract: held notes re-attack at the cut) rendered with the render cache, so
  the other parts are synthesized once.  The part's stem (before its bus and the shared reverbs) is measured for level,
  spectral centroid and the share of its power above 2 kHz and 5 kHz; the masters are measured by FFmpeg and brought to
  the quietest one's integrated loudness for the MP3.  The figures describe the sounds; they do not rank them."""
  import copy
  import json
  import math
  import shutil
  from pathlib import Path
  
  import numpy as np
  
  from . import audio, surge, synth
  from .model import Clock, load_project, save_json, validate
  
  MAX_BARS = 16
  GAP_S = .8
  TAIL_S = 1.0  # of each version's ring-out kept in the MP3
  CONTROLS_FILE = Path(__file__).resolve().parents[2] / 'references' / 'surge-controls.json'
  
  
  def parse_sound(text):
      """('surge', preset) | ('synth', preset) | ('soundfont', bank, bank number, program), checked against the lists."""
      kind, _, rest = text.strip().partition(':')
      if kind == 'surge':
          if surge.entry(rest) is None:
              raise ValueError(f'No Surge patch {rest!r}: name a curated one (references/surge-presets.json) or a catalog id '
                               '(studio.py sounds searches references/surge-catalog.json)')
          return ('surge', rest)
      if kind == 'synth':
          if rest not in synth.presets():
              raise ValueError(f'No built-in synth preset {rest!r}; they are listed in references/synth-presets.json')
          return ('synth', rest)
      if kind == 'soundfont':
          parts = rest.split(':')
          if len(parts) != 3 or not all(p.isdigit() for p in parts[1:]):
              raise ValueError(f'A SoundFont sound is soundfont:<bank>:<bank number>:<program>, not {text!r}')
          return ('soundfont', parts[0], int(parts[1]), int(parts[2]))
      raise ValueError(f'A sound is surge:<preset>, synth:<preset> or soundfont:<bank>:<bank number>:<program>, not {text!r}')
  
  
  def label(sound):
      """A version's name, also its folder: a catalog id's slashes become dots."""
      return 'as_written' if sound is None else '_'.join(str(x) for x in sound).replace('/', '.')
  
  
  SOUNDFONT_ONLY = ('legato', 'dynamics', 'dynamics_mode', 'round_robin', 'bow_change_s', 'bow_ms')
  
  
  def with_sound(track, sound):
      """The track playing `sound`: its engine and the engine's fields, its notes, mix and performance kept, except the
      performance settings only a SoundFont part takes (legato, dynamics, round robin, bow changes), which another engine
      leaves out.  Returns the names of the settings left out."""
      for key in ('synth', 'surge', 'instrument', 'soundfont', 'patch', 'key_map'):
          track.pop(key, None)
      if sound[0] == 'surge':
          track['engine'] = 'surge'
          track['surge'] = {'preset': sound[1], 'fallback': surge.entry(sound[1])['fallback']}
      elif sound[0] == 'synth':
          track['engine'] = 'synth'
          track['synth'] = {'preset': sound[1]}
      else:
          track['engine'] = 'soundfont'
          track['soundfont'] = sound[1]
          track['patch'] = {'bank': sound[2], 'program': sound[3]}
      left = [k for k in SOUNDFONT_ONLY if k in track['performance']] if track['engine'] != 'soundfont' else []
      for k in left:
          track['performance'].pop(k)
      return left
  
  
  def brightness(x, rate):
      """Spectral centroid (Hz) and the shares (%) of power above 2 kHz and 5 kHz, of the sounding frames, from 50 Hz."""
      mono = np.asarray(x, dtype=np.float64).mean(axis=1) if np.ndim(x) == 2 else np.asarray(x, dtype=np.float64)
      n = 8192
      if len(mono) < n:
          return None
      window = np.hanning(n)
      frames = [mono[i:i + n] for i in range(0, len(mono) - n, n // 2)]
      level = np.array([np.sqrt(np.mean(f ** 2)) for f in frames])
      keep = level > level.max() * 10 ** (-40 / 20) if level.max() > 0 else np.zeros(len(frames), bool)
      if not keep.any():
          return None
      power = sum(np.abs(np.fft.rfft(f * window)) ** 2 for f, k in zip(frames, keep) if k)
      f = np.fft.rfftfreq(n, 1 / rate)
      band = (f >= 50) & (f <= 16000)
      total = power[band].sum()
      return {'centroid_hz': round(float((f[band] * power[band]).sum() / total)),
              'above_2k_pct': round(float(100 * power[(f >= 2000) & band].sum() / total), 2),
              'above_5k_pct': round(float(100 * power[(f >= 5000) & band].sum() / total), 2)}
  
  
  def audition(project_file, track_id, sounds, first_bar, last_bar, output, config_file, cache=None, solo=False):
      from excerpt import extract  # the scripts folder, as studio.py runs it
      output = Path(output)
      if output.exists():
          raise ValueError(f'{output} exists; outputs never overwrite, so choose a new path')
      if not 1 <= first_bar <= last_bar or last_bar - first_bar + 1 > MAX_BARS:
          raise ValueError(f'--bars is FIRST-LAST, 1-based, at most {MAX_BARS} bars')
      p = load_project(project_file)
      track = next((t for t in p['tracks'] if t['id'] == track_id), None)
      if track is None:
          raise ValueError(f'No track {track_id!r}; the tracks are ' + ', '.join(t['id'] for t in p['tracks']))
      if track.get('drums') or track['engine'] in ('drum_synth', 'noise'):
          raise ValueError(f'{track_id} is a drum or noise part; audition plays pitched parts')
      if not sounds:
          raise ValueError('Name at least one sound to audition against the part as written')
      passage = extract(p, first_bar, last_bar - first_bar + 1)
      if not any(n for t in passage['tracks'] if t['id'] == track_id for n in t['notes']):
          raise ValueError(f'{track_id} plays no note in bars {first_bar}-{last_bar}')
      cfg = audio.load_config(config_file)
      cache = Path(cache) if cache else output / 'cache'
      controls = json.loads(CONTROLS_FILE.read_text(encoding='utf-8'))['presets'] if CONTROLS_FILE.is_file() else {}
      output.mkdir(parents=True)
      rows, cuts = [], []
      from .render import render  # imported here: render pulls in every engine
      versions = []
      for i, sound in enumerate([None] + list(sounds)):
          q = copy.deepcopy(passage)
          if solo:
              q['tracks'] = [t for t in q['tracks'] if t['id'] == track_id]
          part = next(t for t in q['tracks'] if t['id'] == track_id)
          left_out = with_sound(part, sound) if sound is not None else []
          q['clock']['tail_seconds'] = max(q['clock'].get('tail_seconds', 0), 2)
          try:
              validate(q)  # every version is checked before any is rendered
          except ValueError as e:
              raise ValueError(f'{label(sound)}: {e}') from e
          versions.append((i, sound, q, part, left_out))
      for i, sound, q, part, left_out in versions:
          folder = output / f'{i:02d}_{label(sound)}'
          folder.mkdir()
          # A patch with its own long reverb or delay may still ring when the project's tail ends, which fails the
          # render's quiet-tail check; an audition only plays the passage and a short ring-out, so it renders again
          # with a longer tail.
          for tail in (q['clock']['tail_seconds'], 10, 30):
              q['clock']['tail_seconds'] = tail
              validate(q)
              save_json(folder / 'song.json', q)
              try:
                  render(folder / 'song.json', folder / 'render', config_file, cache)
                  break
              except ValueError:
                  report = folder / 'render' / 'report.json'
                  checks = json.loads(report.read_text(encoding='utf-8')).get('checks', {}) if report.is_file() else {}
                  if tail == 30 or [k for k, ok in checks.items() if not ok] != ['tail_quiet']:
                      raise
                  shutil.rmtree(folder / 'render')
          stem, rate = audio.read_wav(folder / 'render' / 'stems' / f'{track_id}.wav')[0], audio.RATE
          master = audio.read_wav(folder / 'render' / 'master.wav')[0]
          lufs = audio.measure(cfg, folder / 'render' / 'master.wav', folder, 'measure')['integrated_lufs']
          played = sound or (('surge', part['surge']['preset']) if part['engine'] == 'surge' else None)
          receipts = json.loads((folder / 'render' / 'stem_receipts.json').read_text(encoding='utf-8'))
          heard = next((r.get('surge') for r in receipts if r.get('track') == track_id), None) or {}
          row = {'order': i + 1, 'sound': label(sound), 'engine': part['engine'], 'master_lufs': round(lufs, 2),
                 'part_rms_dbfs': round(20 * math.log10(max(1e-12, audio.rms(stem))), 1),
                 'part_brightness': brightness(stem, rate)}
          if left_out:
              row['performance_left_out'] = left_out
          if heard:
              # What actually played: a Surge part falls back to a built-in preset where Surge or its patch is missing.
              row['rendered_with'] = heard.get('rendered_with')
              if heard.get('rendered_with') != 'Surge XT':
                  row['fallback'], row['reason'] = heard.get('fallback'), heard.get('reason')
                  played = None
          if played and played[0] == 'surge':
              found = controls.get(played[1]) or surge.catalog().get(played[1])
              if found:
                  row['surge_controls'] = {k: found[k] for k in ('wheel', 'macros') if k in found}
              described = surge.presets()['presets'].get(played[1]) or surge.catalog().get(played[1]) or {}
              if described.get('character'):
                  row['surge_character'] = described['character']
          rows.append(row)
          cuts.append((master, lufs))
      # The passage's length as played, plus a short ring-out; the quietest version's loudness for all.
      clock = Clock(passage)
      n = int((clock.end + TAIL_S) * audio.RATE)
      target = min(l for _, l in cuts)
      pieces = []
      fade = int(.03 * audio.RATE)
      for master, lufs in cuts:
          cut = np.zeros((n, master.shape[1]))
          part = np.asarray(master[:n], dtype=np.float64)
          cut[:len(part)] = part
          cut[n - fade:] *= np.linspace(1, 0, fade)[:, None]
          pieces += [cut * 10 ** ((target - lufs) / 20), np.zeros((int(GAP_S * audio.RATE), cut.shape[1]))]
      both = np.concatenate(pieces[:-1])
      peak = float(np.max(np.abs(both)))
      if peak > .89:
          both *= .89 / peak
      audio.write_wav(output / 'audition.wav', both.astype(np.float32))
      audio.ffmpeg(cfg, ['-i', output / 'audition.wav', '-c:a', 'libmp3lame', '-b:a', '256k', output / 'audition.mp3'], output,
                   'audition_mp3')
      report = {'project': str(project_file), 'track': track_id, 'bars': [first_bar, last_bar], 'solo': solo,
                'each_seconds': round(n / audio.RATE, 2), 'gap_seconds': GAP_S, 'loudness_lufs': round(target, 2),
                'versions': rows,
                'note': 'Played in this order at one loudness. The figures describe the part\'s sound (its stem, before its '
                        'bus and the shared reverbs); they do not rank the sounds: listen.'}
      fell = [r['sound'] for r in rows if r.get('fallback')]
      if fell:
          report['fallbacks'] = fell
          report['note'] += (' ' + ', '.join(fell) + ' played the fallback preset named in each row, not the Surge patch '
                             'asked for, so they are not an A/B of those patches.')
      save_json(output / 'audition.json', report)
      return report
  

• skills/music-production/scripts/musiclib/compare.py32 KB539886ff1b2aupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/drums.py8.5 KBff11679c7bb0upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/expression.py12 KB55053beffc0fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/harmony.py2.8 KB51069c348d9bupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/layered.py35 KB973cd5ecacb0upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/loop.py19 KB869d2bea4465upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/measure.py2.4 KB46f5289c85d8upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/midi.py11 KBdf44f51396afupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/mixbus.py26 KBc66a44cc1251upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/model.py34 KBcd4dd427dd7aupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/phrasing.py3.7 KB4ace750c78c2upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/production.py15 KB5c001b4a51c4upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/profile.py10 KB5105e7616943upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/render.py19 KB2c101e0fd4c9upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/review.py44 KB220ece4e5629upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/sf2.py13 KBa6abed6c13f1upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/sounds.py2.8 KB4f104219543dupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/stage.py8.0 KB590d8c9be221upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/surge.py17 KB8bcbd7cc66b9upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/synth.py22 KB69a8855c4860upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/variation.py7.5 KB82477bb00492upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/voicing.py12 KB1802cd9deb31upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/musiclib/wavfile.py3.5 KB25f487f81b60upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/selfcheck.py187 KBf268e7464abeupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/soundbank.py91 KB3c162e95e0f4upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/studio.py15 KBba0548bacfd3upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/scripts/surge_host.py4.4 KBb7c3b8f9693cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/anime-opening.md18 KB711e6efd6493upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/arrangement.md17 KBac763f3681dcupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/baroque.md14 KBfa4f821cecf6upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/beats.md16 KBb332e6bbcb0eupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/climax-and-melody.md15 KB11c1f944e33cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/development-and-groove.md16 KB44c8c91f2bc8upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/electronic.md32 KB915b30e286b8upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/film-score.md12 KBf8f8138884caupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/game-music.md15 KB4523bfa1055aupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/harmony.md8.7 KB2ad6744bce38upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/jazz-and-funk.md18 KB72fcfcb64801upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/noise.md15 KB04aaa22d3195upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/orchestration.md21 KB2b7ab92d98d3upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/organ.md6.1 KBb888eee759ceupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/performance-and-mix.md29 KB2e550d1ae485upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/piano-and-chamber.md9.1 KB2518f71ec46eupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/pop-and-ballad.md14 KBa6260983501aupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/rock-and-metal.md22 KBc0aa7fdcd499upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/runtime-lock.json20 KBdb276f4b4dc5upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/scene-music.md13 KBe211ebe6d978upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/schema.md26 KB8cd0213dfd8fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/setup.md10 KB62c580460006upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/sound-selection.md15 KB2104c67d1010upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbank-lock.json189 KB5659762e5befupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks.md71 KBc2416c90018dupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/bass_sneaky.json2.9 KB1febde35badfupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/big_rusty_drums.json22 KB5c02dcc6ccd8upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/bigcat_cello.json4.4 KB226570133252upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/black_and_blue_bass.json2.8 KB1cc876207663upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/erhu.json2.6 KB984f5f06d4ecupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/guitar_shiny.json2.2 KB59cb3bfce61fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_brass.json7.2 KB587572c60ce6upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_brass.notes.json75 KB195c93342b0eupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_guitar.json10 KB020efcbeb69cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_guitar.notes.json58 KB8390d416a1d5upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_perc.json35 KBb83374b10f73upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_perc.notes.json102 KBe03015588e66upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_strings.json63 KBd7b7841a1e44upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_strings.notes.json518 KB3a708327340eupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_winds.json13 KB88919ceafe25upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/iowa_winds.notes.json164 KBd2b1c5f576a3upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/osiris_piano.json14 KB34598e0bdd97upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/sax_bear.json2.2 KB06f7c96cd282upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/steel_drum.json2.8 KB609ac21bb370upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/tuba_war.json2.7 KB6d1168b5011fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vcsl.json21 KBd27f5db6991fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vcsl_keys.json8.3 KB2971392b6e5dupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vcsl_organ.json3.9 KB231064c20b6aupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vcsl_winds.json22 KBa29ce14fc58bupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vcsl_world.json35 KB6d58821b1b3cupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vocal_ah.json1.4 KB59c06f37fa2fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vsco2ce_orchestra.json24 KBfe554fe768b1upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vsco2ce_strings.json9.1 KB68972f726810upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vsco_colors.json17 KBfea778f2e5efupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/soundbanks/vsco_extra.json11 KB43464ac80a49upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/sources-and-rights.md6.2 KBb94e93292bd9upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/surge-catalog.json2.0 MB68e6a9b21d80upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/surge-controls.json14 KBd72b40be4379upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/surge-presets.json27 KBe493d6e96120upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/synth-presets.json12 KBb8f9496175e7upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/references/tonal-references.json9.0 KB069ed5300a5bupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/assets/icon-large.png15 KB7333c87ad41fupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/assets/icon-small.svg429 Bc457ee82388eupstream ↗

  <svg xmlns="http://www.w3.org/2000/svg" width="220" height="220" viewBox="0 0 220 220"><rect width="220" height="220" fill="#f4efe6"/><g>
    <rect x="4" y="140" width="78" height="54" rx="14" fill="#f4efe6" stroke="#1f1c1a" stroke-width="10"/>
    <rect x="44" y="102" width="78" height="54" rx="14" fill="#c8442f"/>
    <rect x="84" y="64" width="68" height="54" rx="14" fill="#1f1c1a" stroke="#1f1c1a" stroke-width="0"/>
  </g></svg>
  

• skills/music-production/agents/openai.yaml448 Be6b7b933b025upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/LICENSE1.1 KB6cba269f86bfupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/LICENSES/GPL-3.0-or-later.txt34 KBf288702d2fa1upstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

• skills/music-production/requirements.txt292 Ba481f4ac5d0aupstream ↗

  # NumPy 2.1.2 is the version the sound banks were built with: fetch_soundbanks.py --from-source reproduces their
  # hashes with it.  It has no wheels for Python 3.14 and newer, which take a current NumPy 2 instead.
  numpy==2.1.2; python_version < "3.14"
  numpy>=2.3,<3; python_version >= "3.14"
  

• README.mdupstream ↗

  Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/owoevan-dev-music-skill-skills-music-production.md