⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
overlay-visual-debugging
ALWAYS USE when debugging, tuning, or fixing how the simple-ptt overlay looks or moves — Liquid Glass (`NSGlassEffectView`), halo blur, pop and correction animations (Emerge, Unfold, Pop, Slide, Expand), text that shifts, jumps, jitters, reflows, or stutters, or `GlassTuning` values — in `src/overlay/` (`glass.rs`, `mod.rs`, `private_effects.rs`, `text_effects.rs`, `dev/`) or the meter (`src/ui_meter.rs`, `src/ui_meter/`, including the spectrum pills), or when using `simple-ptt --debug` or `--ov
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
Not yet listed on ClawHub or SkillsMP
// README
simple-ptt
https://github.com/user-attachments/assets/bee52a5c-f207-4414-be92-f4aae5e65231
A fast, minimal push-to-talk app for macOS with live Deepgram transcription and optional LLM cleanup before paste.
simple-ptt is intentionally small: a menu bar app, a global hotkey, a live on-screen transcript, and a fast paste into the app you are using. The goal is not to be feature-rich. The goal is to stay fast, understandable, and out of the way.
What it does
- Streams your speech to Deepgram while you hold or toggle a global hotkey, and shows the transcript live in an overlay you can edit.
- Pastes the text into the focused app when you stop recording.
- Optionally sends the transcript through an LLM first, to remove filler words, correct punctuation, and format technical terms.
- With an LLM configured, applies spoken corrections ("make that five attempts") to the text you have dictated so far.
- Boosts recognition of product names, jargon, and acronyms you list as Deepgram
keyterms.
How it works
Data flow
- Audio is streamed to Deepgram for transcription.
- If transformation is enabled, buffered transcript text is sent to your configured LLM provider.
Default workflow
- Press the record hotkey (
F5by default) to start listening. - Speak and watch the live transcript overlay update in real time. Words Deepgram may still revise appear dimmed and turn to full contrast once final.
- Stop recording to paste the buffered text into the focused app.
- If transformation is configured and enabled, the app can clean up the transcript before pasting.
Additional controls
- Tap vs hold: short press behaves like toggle; holding past
mic.hold_msturns the same hotkey into hold-to-talk. - Editable overlay: You can click into the overlay at any time to manually type, fix, or delete words before pasting.
- Overlay look: the overlay is dark-tinted clear Liquid Glass on a soft blur of the screen around it, and it pops in when it opens and out when it closes. The text switches between light and dark to stay readable against the glass. Below the transcript, a centred row of bars shows the live spectrum of your voice, low pitches on the left and high on the right, each bar growing up and down from the middle as its pitches get louder and gliding between heights (Meter style in Settings > General,
ui.meter_stylein the config). Each pitch band learns the room's background noise, so the room stays quiet and the row is empty while you are silent, and the bars scale to how loudly you speak above that noise, so soft speech in a quiet office fills them as well as loud speech. While the transformation model rewrites the text, a light sweeps across it; errors show in red. A correction request appears above the transcript as the glass grows upward to hold it, and the glass shrinks back once the correction is applied. With Reduce Motion on, the overlay fades in and out, the correction appears and disappears in place, the sweep is skipped, and the bars jump to their heights instead of gliding. - Correction shortcut (Option+Command by default): hold both keys during dictation or while a buffered annotation is visible, speak a correction request, then release either key to apply that correction to the current annotation. Releasing it without saying anything keeps the narration and resumes dictation if you were recording. During dictation, recording carries on while the correction is applied, so you can keep talking and the meter keeps running; what you say joins the corrected text.
- Transform hotkey (
F6by default): transform the current transcript without auto-pasting it. If you pressF6while dictating, you can keep talking — recording carries on and the meter keeps running while the LLM works, and what you say is appended to the transformed text once it finishes. If the text stays unchanged after a successful manual transform,F5pastes it without another transformation. Added speech or edits still trigger the configured automatic transformation; pressingF6explicitly requests another transform. - Resume dictation: If you have transformed text (or manually stopped recording), pressing
F5again will seamlessly resume dictating onto the end of your existing text. Escape: abort recording, cancel background work, or discard a ready buffer.Cmd+,while the overlay is visible: discard the dictation and open Settings without transforming or pasting it.Cmd+Vwhile recording: splice the current plain-text clipboard contents into the active transcript.
Microphones
simple-ptt keeps the microphone stream open between recordings, so capture starts the moment you press the hotkey instead of after macOS starts the microphone, which can cut off the start of what you say. Keeping the stream open uses a little CPU while the app is idle. To give that up in exchange for the startup delay, clear Keep microphone connection open in Settings > Microphone (mic.always_on = false); the microphone stream is then paused except while you record or while the Settings window is open.
When you plug in or remove a microphone, or switch the system default (for example by connecting AirPods), macOS notifies the app right away. While idle, it reopens the right input on the next status check (within about 75 ms), plus the time macOS takes to start a microphone; during a recording, it switches once the recording ends.
If the microphone in use disconnects, simple-ptt switches to the Mac's built-in microphone and stays on it until the system default input changes again (for example, when the disconnected device reconnects). A microphone named in mic.audio_device is used whenever it is connected, with the built-in microphone standing in while it is not. If it disconnects mid-recording, the overlay says so, and the record shortcut continues on the new microphone. With no microphone connected at all, the record shortcut shows the overlay with a message saying so instead of recording.
If audio capture silently stops, the app detects the missing audio callbacks after 1.5 seconds and rebuilds the microphone connection. During recording, the overlay asks you to press the record shortcut again. An intentionally paused microphone does not trigger this recovery.
Installation
simple-ptt requires macOS 26 or later on Apple Silicon.
- Download the
.dmgfrom the latest release. - Open it and drag
simple-ptt.appinto Applications. - Launch
simple-pttfrom Applications. It runs in the menu bar and has no Dock icon.
[!IMPORTANT] Releases up to v1.6.0 are ad-hoc signed and not notarized, so macOS blocks them on first launch. To open one, run:
xattr -dr com.apple.quarantine /Applications/simple-ptt.app
Quick start
-
Grant permissions. On first launch the Application Permissions window asks for three grants:
- Accessibility, for the global shortcuts and the paste keystroke.
- Input Monitoring, for the global shortcuts. The app requests it on its own.
- Microphone, to capture audio.
Use the window's buttons to request each one, then click Re-check. When the window shows Quit and Reopen, click it: the global shortcuts start working only after the app restarts with the new grants. You can reopen the window from the menu bar with Application Permissions….
-
Add your Deepgram API key. Settings opens when the app has no Deepgram key. Select the Deepgram pane, enter the key, and click Save. See Deepgram costs for what usage costs.
-
Dictate. Click into any text field, tap
F5, speak, and tapF5again. The transcript is pasted where you clicked. On a Mac laptop keyboard, holdfnwhile pressingF5unless the function keys are set to act as standard function keys.
Configuration
simple-ptt looks for config in thi
// HOW IT'S BUILT
// FILES
6 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.
.agents/skills/overlay-visual-debugging/SKILL.md7.4 KB16611f78df97upstream ↗
--- name: overlay-visual-debugging description: ALWAYS USE when debugging, tuning, or fixing how the simple-ptt overlay looks or moves — Liquid Glass (`NSGlassEffectView`), halo blur, pop and correction animations (Emerge, Unfold, Pop, Slide, Expand), text that shifts, jumps, jitters, reflows, or stutters, or `GlassTuning` values — in `src/overlay/` (`glass.rs`, `mod.rs`, `private_effects.rs`, `text_effects.rs`, `dev/`) or the meter (`src/ui_meter.rs`, `src/ui_meter/`, including the spectrum pills), or when using `simple-ptt --debug` or `--overlay-snapshot`. macOS 26 glass and AppKit animation behave differently from what training data says; MUST READ before changing overlay code. Do NOT use for the settings window (`src/settings_window/`). author: alexgorbatchev metadata: created_on: 2026-09-27 11:30 last_modified: 2026-10-07 15:36 status: current --- ## Tools - `simple-ptt --debug` (`just debug-overlay`, `src/overlay/dev/tuner.rs`): the overlay pinned near the top of the screen, plus a window with a control for every `GlassTuning` and `PillTuning` value, a meter style picker (the `meter_style` values Settings offers), and a voice picker for the meter: a synthetic voice that alternates about 8 seconds soft and 8 seconds 15 dB louder (`voice_level_db`), with a spectrum whose two formants drift (`voice_spectrum_db`), or the microphone set in the user's `[mic]` config, captured through the app's own audio path so the room's real noise and the real spectrum drive the pills (the terminal needs Microphone permission). A readout shows the pills' average noise floor and gate, the top, and the latest loudest band, in dB. The synthetic voice falls silent for 1.5 s every 6 s, so the pills can be watched resting. Then come a correction toggle, a "Loop narration" checkbox (narrates the transcript a word at a time, or the correction while it is shown), "Pop again", and "Copy values", which copies the values for the user to paste. Controls that have no effect under the current values are disabled. - `simple-ptt --overlay-snapshot <dir>` (`src/overlay/dev/snapshot.rs`): captures every overlay state over light and dark backdrops as `<dir>/<theme>-<step>.png`, then exits. Use `.tmp/snapshots/` as `<dir>`. It needs Screen Recording permission for the terminal. Captures that show only the desktop wallpaper or solid black mean the display is locked or asleep, or that permission changed; stop and tell the user rather than measuring them. - `assets/view_probe.rs`: a module for scripted, self-exiting runs that samples view geometry and prints who changes a frame. Read [references/probes.md](references/probes.md) before using it. - `scripts/contrast.ts`: WCAG contrast of the text against the glass behind it, measured on captures. It is for text colour and legibility bugs; see "Measuring legibility on captures" in [references/probes.md](references/probes.md). - lldb and the SDK headers, for private AppKit behavior. The recipes are in [references/probes.md](references/probes.md). The tuner-only `OverlayWindow` methods (`pin_to_top`, `glass_tuning`, `set_glass_tuning`, `halo_is_progressive`, `glass_internals_now`, `pill_tuning`, `set_pill_tuning`, `pill_range_now`) exist for `src/overlay/dev/`; do not call them from product code. Values the user settles on in the tuner become the `GlassTuning` defaults in `src/overlay/glass.rs`, with the tests there pinning them, and the `PillTuning` defaults in `src/ui_meter/pill_tuning.rs`. ## Rules - Every `simple-ptt` command you run must contain `--overlay-snapshot` or `VIEW_PROBE=1` (with a `run_script` end time). Any other launch (plain, or `--debug` without a probe) blocks the session and is prohibited. For interactive checks, ask the user to run `just debug-overlay`, and to paste the "Copy values" output or describe what they see. - Measure before you change code. A claim such as "the text moves" or "the animation is cut short" needs samples: `t=` timestamps with screen rects, frames, or presentation heights, printed by a probe run. Do not diagnose from reasoning, code reading, or screenshots alone. - Treat every nonzero deviation as a bug: a 0.2pt drift or a single-frame blip is 1px on a Retina display. Prohibited dismissals: "invisible", "one frame only", "rounding", "AppKit quirk, can't fix". - Before editing any overlay code, read [references/findings.md](references/findings.md). It lists measured macOS 26.6 behaviors that contradict the obvious fix. Re-verify a finding with a probe before relying on it after a macOS update. - Fix the cause the probe shows. Prohibited: - delays or extra timers that race AppKit; - re-setting a frame to hide a jump; - reading a glass view's `frame()` as its target; - "close enough" tolerances in comparisons that hide a sub-point drift. - Every private API goes in `src/overlay/private_effects.rs`, looked up at run time (`respondsToSelector` or `AnyClass::get`). Its doc comment names the measured behavior and the macOS version. Tell the user about each new private API in your final message. Before using one, check the SDK headers for a public replacement. - Keep probes out of commits. Remove `view_probe.rs`, its `mod` line, and the `VIEW_PROBE` block before committing, and confirm with `rg -n "view_probe|VIEW_PROBE" src/` returning nothing. ## Workflow 1. **Reproduce.** Get the tuner values from the user ("Copy values") and the exact steps, such as the effect, the scenario, and show or hide. Put the values into the probe run as a `GlassTuning`. 2. **Write the invariant as numbers,** for example "the transcript's screen y stays at 798.0 through expand and collapse", or "the glass height falls monotonically from 282 to 180 over `correction_seconds`". 3. **Sample.** Install `view_probe`, script the timeline (show at 2.0s, hide at 3.5s, exit at 4.6s), and sample every 0.012s: `screen_rect` of the views that must stay still, `presentation_height` of the glass that moves, and `ancestry` of the view that misbehaves. Collapse repeated lines with `awk` so only changes print. Run 2–3 times; one-frame blips vary between runs. 4. **Find the writer.** Call `watch_frame` on the view that changes, with a predicate that matches only the bad frame. Match the printed stack against the table in [references/probes.md](references/probes.md). When the stack ends in private AppKit, disassemble it (lldb recipes) to see which selectors it reads, then look for a switch or a public API. 5. **Fix and prove it.** Keep the "before" samples. Re-run after the fix and show that the invariant holds in every sample of every run. Then disable the fix (for example, comment out the call), re-run, and show that the failure returns; if it does not, the fix is not the cause. Pure geometry goes in a `glass.rs` function with a unit test written first, and that test must fail first. 6. **Clean up and verify.** Remove the probe wiring, then run `cargo test --locked`, `cargo check --message-format=short`, and `cargo build --locked --release`, each with zero warnings. For changes the user can see, take `--overlay-snapshot` captures and look at them. ## Report End with these five items. If any one is missing, the fix is unverified; say so instead of claiming it works. 1. The invariant from step 2. 2. A "before" sample excerpt that breaks it. 3. An "after" excerpt from each run, showing that it holds. 4. The excerpt from the run with the fix disabled, showing the failure again. 5. The findings.md entries that applied, and the new private APIs (or "none"). Add every newly measured behavior to findings.md in the same change..agents/skills/overlay-visual-debugging/scripts/contrast.ts4.2 KB98120c565c0eupstream ↗
// WCAG contrast between the overlay's text and what is behind it, measured // on `--overlay-snapshot` captures of the recording state (no correction). // Usage: bun <skill>/scripts/contrast.ts <dir> [prefix] [halo_margin] // `halo_margin` is the `GlassTuning::halo_margin` the captures were taken // with (default: the current default, 90.71). // Needs ImageMagick (`magick`). A value under 4.5 (WCAG AA) is marked "!". // // Two methods: // - Paired (preferred): for `<name>.png` with a `<name>-notext.png` taken the // same way with an empty transcript, the pixels that differ are the // transcript's glyphs, and each is compared with the same pixel without // the text: what is behind that glyph, backdrop text included. The body // reports the 90th percentile (glyph cores) and the median of those // per-pixel ratios. // - Median (flat backdrops only): the background is the region's median // luminance and the text the 2% tail farther from it. Backdrop text behind // the glass breaks this, so it is marked "~" and used for the footer, which // the paired captures keep. import { $ } from "bun"; const [dir, prefix = "tint-", marginArg = "90.71"] = process.argv.slice(2); // Captures are 2x and padded by 40pt around the window, and the window adds // the halo margin around the glass, so the glass's top-left corner is at // (40 + margin) * 2 pixels. The regions are measured from that corner. const glass = Math.round((40 + Number(marginArg)) * 2); const region = (width: number, height: number, x: number, y: number) => `${width}x${height}+${glass + x}+${glass + y}`; const regions = { body: region(980, 90, 40, 35), footer: region(1048, 40, 36, 312), }; // A glyph pixel differs from the same pixel without text by more than this // luminance. const GLYPH_DELTA = 0.03; function luminance(r: number, g: number, b: number): number { const channel = (value: number) => { const c = value / 255; return c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4; }; return 0.2126 * channel(r) + 0.7152 * channel(g) + 0.0722 * channel(b); } const ratio = (a: number, b: number) => (Math.max(a, b) + 0.05) / (Math.min(a, b) + 0.05); async function luminances(path: string, crop: string): Promise<number[]> { const raw = new Uint8Array(await $`magick ${path} -crop ${crop} +repage -depth 8 rgb:-`.arrayBuffer()); const values: number[] = []; for (let i = 0; i + 2 < raw.length; i += 3) values.push(luminance(raw[i], raw[i + 1], raw[i + 2])); return values; } const quantile = (sorted: number[], q: number) => sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))]; async function paired(withText: string, withoutText: string, crop: string): Promise<string> { const [text, behind] = await Promise.all([luminances(withText, crop), luminances(withoutText, crop)]); const ratios: number[] = []; for (let i = 0; i < text.length; i++) { if (Math.abs(text[i] - behind[i]) > GLYPH_DELTA) ratios.push(ratio(text[i], behind[i])); } if (ratios.length === 0) return "no glyphs found"; ratios.sort((a, b) => a - b); const cores = quantile(ratios, 0.9); return `${cores.toFixed(2)}${cores < 4.5 ? "!" : " "}(median ${quantile(ratios, 0.5).toFixed(2)}, ${ratios.length} px)`; } async function median(path: string, crop: string): Promise<string> { const sorted = (await luminances(path, crop)).sort((a, b) => a - b); const bg = quantile(sorted, 0.5); const low = quantile(sorted, 0.02); const high = quantile(sorted, 0.98); const text = Math.abs(high - bg) > Math.abs(bg - low) ? high : low; const value = ratio(bg, text); return `${value.toFixed(2)}${value < 4.5 ? "!" : " "}~(bg ${bg.toFixed(3)} text ${text.toFixed(3)})`; } const files = (await $`ls ${dir}`.text()) .split("\n") .filter((f) => f.startsWith(prefix) && f.endsWith(".png") && !f.endsWith("-notext.png")) .sort(); for (const file of files) { const path = `${dir}/${file}`; const blank = `${dir}/${file.replace(".png", "-notext.png")}`; const hasPair = await Bun.file(blank).exists(); const body = hasPair ? await paired(path, blank, regions.body) : await median(path, regions.body); const footer = await median(path, regions.footer); console.log(`${file.replace(".png", "").padEnd(30)} body=${body} footer=${footer}`); }.agents/skills/overlay-visual-debugging/references/findings.md26 KB7710f8bdfc78upstream ↗
--- created_on: 2026-09-27 11:29 last_modified: 2026-10-09 13:23 status: current --- These behaviors were measured on macOS 26.6 with probes, with later versions noted below. Each gives the behavior, then where the code handles it. Do not undo a handling without re-measuring. ## Liquid Glass (`NSGlassEffectView`) - **Glass ignores `alphaValue` and layer opacity** inside an `NSGlassEffectContainerView`: it cannot fade. The correction effects move or scale the glass and fade only its content (`CorrectionEffect` in `glass.rs`); the whole-window pop fades the `NSPanel` instead. - **A new corner radius takes effect only when the glass is shown again.** That is why the tuner pops the overlay again about 300ms after a change. - **An inactive window draws inactive, frosted glass.** `OverlayPanel` overrides the private `-_hasActiveAppearance` to return YES. - **The container renders all its glass together.** Its layers, not each glass view's, hold the `glassBackground` filter; `apply_glass_internals` and `read_glass_internals` take the container. - **Which tuning changes need a re-pop.** Captured live and after a re-pop, with the overlay updated every 75ms as in the tuner and the app: - `corner_radius` differed (the corners keep the old radius) and so did the glass `internals`; `needs_repop` in `dev/tuner.rs` re-pops for those. - Every other value applied live (0 pixels different), including `style` and `appearance`. The `pop_*` values show only in a pop. - **An appearance change drops the progressive blur.** The halo shows as a hard-edged box until the next overlay update re-applies `variableBlur` (`refresh_progressive_halo`). The app updates the overlay only on state changes, or every 20 status ticks (about 1.5s), so while the overlay is idle a system light/dark switch can show the box that long. This is not fixed. ## AppKit animation of glass frames - **`NSAnimationContext`'s completion handler never fires** for a group that animates an `NSGlassEffectView`. Motion ends use a one-shot `NSTimer` (`schedule_after_motion`). - **`animator().setFrame(_:)` on glass steps the model frame** on each display-link tick, so `frame()` returns in-between values. - Setting any frame mid-motion cancels the animation. This was the cause of the "abrupt" Expand collapse (274 → 180pt in one frame). - Compare with and start motions from `main_resting_frame`, never from `main_glass.frame()`. - **A glass frame animation stops at its last displayed step, not its target.** - A pop to origin (40, 40) ended at (39.9, 40.2). Later animations then jittered ±0.2pt around it, and the transcript with them. - `settle_main_glass_after` sets the exact resting frame when a motion ends. - Expand animates only `setFrameSize:`, because its origin does not change. - **The same cut-short bug affects the correction glass.** `update_correction`'s `Place` branch compares `correction_glass.frame()` with the target every tick. In Emerge its width jumped from 317 to 560pt mid-show. This is not yet fixed. ## Scroll views and text - **Keyboard edits must keep their rendered baseline separate from the speech source** (macOS 26.6.2). - A native `insertText:replacementRange:` edit changed `hello` to `Hi` while `still coming in` remained interim. The following speech update displayed `Hi still coming in still coming in and working` at `t=2.113`; all 59 checked samples contained duplication. - The editor now publishes its edit relative to its last rendered text, and the session merges revisions against its preceding speech source under the text lock. Two native runs using the `similar` implementation each checked 59 samples with zero failures: `Hi still coming in and working` appeared at `t=2.113` and survived finalization. The transcript rectangle stayed unchanged across the keyboard and speech updates in each run. - Restoring the old prefix rebase reproduced the duplicated text at `t=2.113` and 59 failures out of 59 checks. The three session regressions failed again. Disabling the stale-editor merge separately lost a word received between UI ticks and failed its state regression. - The original duplication-fix probe also exposed unconditional caret movement from `{ location: 2, length: 0 }` after typing to `{ location: 31, length: 0 }` after the next result. The conditional following rule below now handles that behavior. No private API was added. - **Narration follows only an idle caret already at the end** (macOS 26.6.2). - Before the change, a caret at offset 2 jumped to 15 on incoming narration at `t=2.303`. After a native `insertText:replacementRange:` at `t=3.001`, another result moved offset 20 to 26 at `t=3.204`. The probe failed 165 of 181 checked samples, including a selected range and a second keystroke restarting the pause. - `replace_working_text` captures the selection before replacing storage. It follows the new end only for a zero-length selection at the previous end and at least two seconds since the last `textDidChange:`. Otherwise it restores the existing selection, clamped to valid UTF-16 boundaries when a revision shortens or changes the text. Scrolling to the new end happens only when following. - Two native runs each checked 210 samples with zero failures. In both, selection stayed at offset 2 through the `t=2.101` narration update, at offset 20 through the `t=3.204` update after typing, and at offset 27 through the `t=5.004` update after another keystroke. Moving the caret to the end without typing allowed the `t=6.109` update to follow to offset 37, over two seconds after the last keystroke. Moving it back into the text kept offset 2 through the next update. The transcript screen rectangle stayed `(1000.0, 674.8, 560.0, 120.19999999999999)` across these events in each run. - Disabling the two-second guard moved offset 20 to 26 again at `t=3.205`, failed 92 of 210 native checks, and failed the timing regression. Disabling the end-position guard separately failed the middle-caret and shortened-selection regressions. No private API was added. - **Waiting feedback must update before the finishing layout hold** (macOS 26.6.2). - Updating the footer after the finishing early return left the shortcut hint displayed at `t=5.605` with `waiting=true, finishing=true`. The footer now changes before that return, while the hold still retains text, meter, and layout. - Two runs sampled every 12 ms and checked 542 samples each after selecting seven characters at `t=1.9`: transcript screen rect stayed `(584.0, 732.8, 560.0, 120.19999999999999)`, selection stayed `{ location: 0, length: 7 }`, glass height stayed `180.0`, and finishing scale stayed `0.8` after the transition. Both live and finishing notices appeared and cleared within the next UI update. Restoring the earlier return reproduced the missing finishing notice at `t=5.605` with the same geometry and selection. - Light and dark captures show the waiting footer at full recording size and uniformly scaled with finishing narration. The footer keeps `secondaryLabelColor`; no private API is added. - **An unchanged overlay refresh must preserve text selection and scrolling** (macOS 26.6.2). - With seven error-text characters selected, the old `OverlayWindow::update` moved the selection from `{ location: 0, length: 7 }` to `{ location: 37, length: 0 }`. A subsequent native Command-C event left the pasteboard unchanged; the same event copied successfully before the refresh. - Changed working text now preserves the selection or follows the end according to the idle-caret rule above. `update` leaves selection and scrolling alone when the text is unchanged. In two probe runs, selection remained `{ location: 0, length: 7 }` through the refresh and Command-C wrote the selected text. Restoring the unconditional caret move reproduced the failure. - **Transformation feedback needs readable text through both processing states** (macOS 26.6.2). - The old record/transform handoff set text opacity to `0.02`; after twelve UI updates the text view's measured alpha was `0.0196078431372549`. The shimmer ran only in `STATE_TRANSFORMING`, leaving the final transcription wait without it. - Both transform handoffs in `hotkey.rs` retain full text opacity. `OverlayWindow::update` runs the existing shimmer in `STATE_PROCESSING` and `STATE_TRANSFORMING`. Two probe runs measured text alpha `1.0` in both states and advancing presentation-layer gradient locations; the mask was absent in recording and error states. Disabling the changes reproduced the low alpha and missing processing shimmer. Reduce Motion still suppresses the shimmer through `Shimmer::start`. - **The scroll view pulls its content in near a container's rounded corner.** - On macOS 26 an `NSScrollView` insets its clip view (`contentInsets`) while a rounded ancestor corner is near its edge (`_NSScrollViewLayoutHelper updateLayoutWithMinimumDocumentFrameSize:` → `_applyContentAreaLayout:`). The text view shrinks and scrolls by the inset. The Expand collapse moved the transcript 6pt for about 50ms. - `automaticallyAdjustsContentInsets` does **not** control this. - The switch is the private `-[NSScrollView _setAllowsAdditionalContentInsetsForCornerRadii:]`. `-[NSTextView viewDidMoveToSuperview]` sets it back to YES, so set it after `setDocumentView:` (`set_text_document` in `overlay/mod.rs`). - The public replacement, `NSView.cornerConfiguration`, is macOS 27. - **A vertically resizable `NSTextView` shrinks between updates.** Text views use `setVerticallyResizable(false)`, and `resize_text_view` sets their height and scrolls back to the top when the text fits. - **The Expand correction area rides the glass's top edge.** - Place it for the glass's *current* height (`expansion_frame(resting, margin, glass_height)`); a frame computed for the grown glass sits up to 130pt off mid-motion. - The correction scroll view's autoresizing mask depends on its host: `ViewMaxYMargin` in the expansion area (above the divider), `ViewMinYMargin` in the correction glass. ## Text colour on tinted glass - **Glass does nothing for the legibility of its content.** `NSGlassEffectView.tintColor` tints the glass, but the content keeps the window's appearance. The SDK header and docs promise nothing more. - Measured on captures before the fix: black or blue at alpha 1 with Aqua text read 1.15–1.95:1, and white or yellow at alpha 1 with Dark Aqua text 1.07–1.22:1. - `update_text_appearance` (see `legibility.rs`) sets Aqua or Dark Aqua on the glass content views, whichever `labelColor` contrasts more with the tint over the glass appearance's `windowBackgroundColor`, and `nil` when that is the glass's own appearance. It runs from `apply_look` and from `viewDidChangeEffectiveAppearance` of the container's stack view (`AppearanceTrackingView`), which covers system light/dark switches. - After the fix, body text measured at least 4.29:1 across 32 tint, appearance, and backdrop cases. - **On a mid-luminance background no label colour reaches 4.5:1.** Grey at alpha 1, or black at alpha 0.5 over light glass, measured 4.29–4.57. `labelColor` is 85% black or 85% white. The footer uses `secondaryLabelColor`, which measured 2.50–6.23 (lowest on mid-grey). - **Clear glass does not adapt its content to what is behind it; Regular does.** - In dark mode over a light backdrop, Regular glass rendered dark (0.092 luminance) and its white text read at 5.86:1. - Clear glass stayed see-through (0.924), and its white text read at 1.06:1, the same with or without `update_text_appearance`. - Apple's `Glass.clear` docs: "ensure content remains legible by adding a dimming layer or other treatment beneath the glass", such as transparent black. - **The dimming layer (`GlassTuning::dim`) keeps Clear glass legible.** The defaults now use no dim (0) and a black tint at 69% instead, which darkens the glass itself; the figures below were measured for the dim, and the tinted default has not been measured. - It is an `NSBox` under each glass, filled with `windowBackgroundColor` in the text's appearance. `sync_dims` keeps it on its glass; a probe found frame and visibility equal to the glass in all 863 samples through pops, Emerge, and Expand. - Measured with the paired method over busy backdrop text, the hardest case is dark mode (white text) over a light backdrop: 1.89:1 with no dim, 3.67 at 0.5, 4.32 at 0.6, and 4.71 at 0.65. Every other case was already at 4.26 or more with no dim. - The tint and footer figures above came from the old full-screen backdrop, whose text did not reach under the glass, so they measure a flat backdrop only. - **AppKit's resolved values on macOS 26.6:** `windowBackgroundColor` is 1.0 (Aqua) and 0.118 (Dark Aqua); `labelColor` is black or white at alpha 0.847. - **Only text and the pills use semantic colours.** The pills are `labelColor` resolved in their view's appearance (`cg_color_in` in `ui_meter.rs`); the other meter styles' bars and the shimmer mask are fixed sRGB and do not adapt. ## Window and halo - **A key nonactivating overlay can report AppKit activation without process activation** (macOS 26.6.2). - While recording, the native probe measured `NSApplication::isActive=true` and `NSRunningApplication::currentApplication().isActive=false` with the overlay as key window. Opening Settings from that state produced `active=false, frontmost=false, settings=true, hotkeys_blocked=true, key=None` at `t=2.110`: the Settings window was inaccessible and recording hotkeys were blocked. - The default-mode turn requires both activation flags. `NSRunningApplication::activateWithOptions(empty())` rejects the request even when Settings is ordered front. `NSApplication::activate()` also leaves the process inactive. Keeping Settings hidden while awaiting activation leaves Cmd+, at `pending=AwaitingActivation, settings=false, overlay=false`; waiting for the overlay fade to end does not resolve it. - The delegate orders Settings front without making it key, then calls the public `NSApplication::activateIgnoringOtherApps(true)` for the explicit Settings request. `applicationDidBecomeActive:` makes Settings key and enables its keyboard handling. With no overlay, the explicit activation call alone also leaves Settings hidden; ordering the window first supplies the activation target. If activation is declined, the window stays visible with recording hotkeys enabled until activation or closure. - Native probes sampled every 12ms, forced the overlay to be key immediately before Cmd+, and required actual process activation and a key Settings window after the overlay hid. Replacing the activation call with `NSRunningApplication::activateWithOptions(empty())` left `active=false, frontmost=false, settings=true, pending=AwaitingActivation, overlay=false, key=None` at `t=1.827` and failed. A separate no-overlay probe failed with the window-ordering step disabled and passed with it restored. - Two restored runs reached `active=true, frontmost=true, settings=true, pending=Idle, overlay=false, key=Some("Settings")` at `t=1.824` in each run. Both exited successfully with Settings visible and the overlay hidden. Moving focus to another application afterward is allowed; the invariant requires that Settings actually acquired foreground key focus after the overlay hid. - The successful-activation path needs a native probe: pure state-transition tests cannot validate macOS granting foreground focus. `activateIgnoringOtherApps` is deprecated, but its documented explicit foreground behavior is required by these measurements; it is a direct call for this user action, not a fallback. No private API is added. - **F5 fades the meter without releasing its layout space** (macOS 26.6.2). - The finishing early return previously retained meter opacity at `1.0` throughout processing and transformation. `fade_finishing_meter` now animates the meter container's public `alphaValue` to zero over 0.16 s, alongside the finishing shrink. Its target is set once so status ticks do not restart the fade; the fast-result dismissal path uses it too. - Two native runs sampled every 12 ms checked 125 finishing samples each with zero failures: meter opacity decreased in 9 and 10 sampled steps, reached exactly `0.0`, and stayed there through transformation. The transcript's screen rectangle stayed `(1000.0, 1058.8, 560.0, 120.19999999999999)`; root scale held exactly `0.8` after the transition. The next recording restored meter opacity to `1.0`. - Disabling the fade restored opacity `1.0`, with 100 failing samples out of 125. No private API is added. - **An empty F5 handoff dismisses instead of holding a placeholder** (macOS 26.6.2). - Before the change, empty narration held at scale `0.8`, panel alpha `1.0`, with shimmer throughout processing and transformation. The finishing handoff now marks only nonempty narration for that hold; empty or whitespace-only narration sets the existing dismissal flag before processing starts. - Two native runs sampled every 12ms, each checking 204 samples with zero failures: empty F5 faded immediately through the normal 0.24s dismissal, remained hidden through late final text, and nonempty F5 still held at `0.8` with shimmer. Disabling the handoff change restored 154 failures in 203 samples and the failing hotkey regression test. - The recording hotkey restores visibility when starting the next recording. The queued transcription start does not restore it, so a delayed start cannot undo a subsequent empty F5 dismissal. Errors still restore the overlay through `report_error`. No private API was added. - **Uniform overlay scaling belongs on the root rendering layer** (macOS 26.6.2). - Animating the main glass's frame changed its width during dismissal (560 → 559.5pt in the first step) while the text view stayed 560pt wide. It resized the glass rather than scaling the whole overlay. - `OverlayMotionView` uses public Core Animation transforms from its AppKit `updateLayer` override. The window, glass, root bounds, and text layout stay fixed. Two repeated runs each checked 125 samples with no invariant failures: F5 held root scale `0.8` through processing and transformation, and F6 held `1.0`; both shimmered. Captures show the text, footer, glass, and halo scaling together. - Entrance keyframes go through `1.1` and settle at `1.0`. Sampled entrance peaks were `1.1` and `1.0998241941560991`; the display sampling can miss the exact peak between keyframes. Reduce Motion uses an unscaled fade. - `NSView::convertRect_toView` continues to report the unscaled layout rectangle through the root layer's transform. For composited motion, measure the root presentation transform as well as layout bounds and inspect captures. - Exercise the real capture flag when probing F5. Removing the meter on the recording-to-processing transition changed the text view's height from `120.2` to `151.0` and its screen y from `511.8` to `481.0`, even with the correct root scale. F5's finishing update retains the displayed text and layout instead of rebuilding them; the meter now fades within its retained space as measured above. With the real capture transition, two runs again checked 125 samples each with zero failures; F6 continues its live meter updates. - **Dismissal must retain its displayed text and report actual window visibility** (macOS 26.6.2). - The old `hide()` cleared 84 characters to zero while the panel was still visible and fading. Keeping the view's content until its next visible update retains narration throughout the fade; transformation's `PreserveOverlay` mode collects F5's result without replacing that narration. F6 keeps `ReplaceOverlay`. - Marking the overlay hidden at the start of its fade gave `visible=true, reported_visible=false` at `t=2.017`, allowing the paste handoff to proceed early. `pop_out` now calls the owner's visibility callback after `orderOut`; repeated probes keep both visibility values equal through dismissal. - A result can finish between UI ticks. The finishing marker survives `STATE_IDLE` until the actual hidden callback clears it, and `hide` applies the finishing scale even when no processing tick was rendered. The fast-result probe reached scale `0.8` during its fade; disabling marker retention restores the failing lifecycle test and an 8% dismissal shrink. - Disabling the rendering transforms, narration retention, and finishing marker produced 91 failures in 125 samples and restored the two failing finishing regression tests. No private API was added. - **The glass content must sit on whole points.** The glass sits `margin()` in from the window edge. With a fractional margin (90.71), AppKit snapped the text view to the pixel grid but not the meter's layers, which ended up 0.21 pt apart. `margin()` rounds to whole points, and every margin use goes through it. - **The halo fades out at the edge of its own views** (blur, dim, fallback glass), not at the window's edge. `layout_halo` sizes them. - With Expand they end a margin above the main glass's current top, so the blur grows with the glass. Before, it took the grown size the moment the correction showed. - `layout_halo` runs on every `NSViewFrameDidChangeNotification` of the main glass. The notification fires on each animation step, in the same display tick. - **Window frames are whole points; views are not.** Moving the window's top edge with the glass put the halo 0.5pt off the glass on about 1 step in 10. That is why the window takes its grown height at once (it is transparent there) and only the halo views follow. - **Drawing the progressive-blur mask ring by ring at full size takes 14–22ms** (release, 1280×520–724px), longer than a display frame. `blur_mask_image` stretches a small `blur_mask_cap` in about 1.7ms, and a test proves the result is byte-identical. The first image draw in a process costs about 100ms of one-time warm-up. - **`NSImageResizingModeStretch` is 0 on macOS** (1 is the iOS value; see `NSImage.h`). objc2-app-kit declares the type without its constants. ## The pills (Core Animation) - **Narrower pills and tighter gaps retain the centred animated row** (macOS 26.6.2). - `PillCluster` uses 2.8 pt widths and 1.5 pt gaps, keeping its 1.4 pt corner radius. The 40-pill span falls from 211 pt to 170.5 pt. - Two native 12ms runs checked 247 and 283 samples with zero failures: model and presentation widths stayed at 2.8 pt, the row's centre offset stayed exactly zero, and x positions stayed fixed while presentation heights changed in 187 and 193 samples. The capture shows the compact pills in the overlay. - Restoring the previous widths and gaps makes the native compact-row check fail again. No private API was added. - **Frequency bands spread from the centre for a balanced speech silhouette** (macOS 26.6.2). - With ascending frequencies placed from left to right, the synthetic voice's 284.7 Hz peak appeared 87.8625 pt left of centre; its height-weighted centre moved as far as 68.701156 pt from the middle of the 211 pt cluster. - `PillCluster::pill_x` now gives the lowest pair the two centre positions, and successive pairs alternate towards the edges. The peak above appeared 7.9875 pt right of centre. Two 12ms probe runs checked 342 and 448 samples without failure: every sampled peak was in the middle half, x positions stayed identical through the runs, and the height-weighted centre stayed within 3.421995 and 3.167551 pt respectively. This is a fixed visual arrangement of the bands; their input data remains in ascending frequency order. - Disabling that column arrangement restored the left-hand peak, all 448 native checks failed, and the new frequency-placement unit test failed while both spacing tests still passed. No private API was added. The screenshot contained only wallpaper, so capture attempts stopped under the skill's rule; this change's screen appearance is not visually verified. - **Implicit animations within each update glide the spectrum pills between updates.** `PillCluster::render` (`ui_meter/pill_cluster.rs`) sets every pill's frame and opacity in a `CATransaction` whose duration is the time since the last update (the overlay updates every 75 ms) with linear timing. Sampling the presentation layers every 20 ms over the tuner's synthetic voice, each band took 30 to 72 distinct heights in 2.4 s of speech (32 updates), and no pill showed in 50 samples of its silence. The requirements behind the pills are in `docs/internal/references/pill-meter.md`. The scrolling pill strip that came before the spectrum measured these; no code relies on them now, but they hold for any long Core Animation motion: - **A timing function wrecks a long linear animation, even `kCAMediaTimingFunctionLinear`.** A `transform.translation.x` animation lasting 24 hours is at about 1e-5 of its progress a few seconds in. - With the linear `CAMediaTimingFunction`, the presentation layer's translation crawled at 0.4 pt/s, then jumped 42 and 21 pt, then ran 13% slow. It sat 13–28 pt off the expected line in the first 5 seconds. - A timing function is a cubic Bézier that Core Animation solves numerically; near 0 the linear one is x ≈ 3t², so the solver's tolerance shows as tens of points over a 2.9 million-point travel. - With no timing function (`nil` is linear pacing), the translation stayed within one display frame of the expected line (0–0.56 pt at 33.75 pt/s) in all 875 samples. Leave `timingFunction` unset on long animations. - **Pausing a layer's timing stops and resumes its animations in place** (Apple's Technical Q&A QA1673: `speed` 0 with the paused local time in `timeOffset`, then `speed` 1 with `beginTime` taking up the pause). Over a 1.5 s pause the translation held still, and moved at 53.22 and 53.25 pt/s (53.25 expected) before and after it, with no jump. - **The presentation layer advances once per display frame,** so a sample of it lags the ideal value by up to one frame of motion. Judge smoothness by the step between frames, not by an exact match with `CACurrentMediaTime`. Sampled every 4 ms, a per-sample speed comes out about double, since the pairs that show motion carry a whole frame's travel; measure speed over a second or more. - **A layer's new model position shows only after the transaction commits.** A layer sampled a few milliseconds after its model moved was still drawn at its old position. Compare model and presentation only after the run loop has turned. - **A fade does not hide a layer changing size.** When strip pills took their height while inside a 25 pt fade mask, the user saw them grow from dots. ## Private Core Animation (halo) - **Changes to an installed `variableBlur` filter must go through key paths**: `filters.variableBlur.inputMaskImage` / `.inputRadius`. Assigning `filters` again does not re-render. The filter's layer `scale` must be the display scale. - **Glass internals are set through key paths too**: `filters.glassBackground.<input>`. The rim highlight is a `CASDFKeyFillHighlightEffect`; assign the layer's `effect` again to redraw it..agents/skills/overlay-visual-debugging/references/probes.md8.7 KBea09068cf84fupstream ↗
## Contents - Installing `view_probe` - A scripted run - Reading `watch_frame` stacks - Finding the overlay's views - Measuring legibility on captures - lldb and SDK recipes ## Installing `view_probe` 1. `cp .agents/skills/overlay-visual-debugging/assets/view_probe.rs src/overlay/dev/view_probe.rs` 2. Append `pub mod view_probe;` to `src/overlay/dev/mod.rs`. 3. Add a `VIEW_PROBE` block to `src/overlay/dev/tuner.rs` in `run()`, just before `let _keep = (tuner, window, timer);`, as shown below. 4. Run `cargo build --message-format=short`, then `VIEW_PROBE=1 ./target/debug/simple-ptt --debug > .tmp/probe.txt 2>&1`. The run exits by itself at the script's end time. The module provides: | Function | Returns / does | | --- | --- | | `overlay_panel()` | The `OverlayPanel` window. | | `find_views(root, class)` | Views of that exact class, depth first. | | `screen_rect(view)` | Where the view is on screen. Use this, not `frame()`, to judge "did it move". | | `ancestry(view)` | Frame and bounds of the view and every ancestor. It shows which level moved: a clip view's bounds origin means scrolling or insets; a glass origin means glass motion. | | `presentation_height(view)` | The layer height as drawn now. | | `watch_frame(view, label, predicate)` | Prints the call stack of each frame change the predicate accepts. | | `run_script(events, every, end, sample)` | Runs the timed events, samples every `every` seconds, and exits at `end`. | | `elapsed()` | Seconds since the script started. | ## A scripted run This is a tested example (macOS 26.6): Expand, then show and hide the correction, sample the transcript, and report any transcript shrink. ```rust if std::env::var("VIEW_PROBE").is_ok() { use super::view_probe::{ancestry, find_views, overlay_panel, presentation_height, run_script, screen_rect, watch_frame}; let mut tuning = state.overlay.glass_tuning(); tuning.correction_effect = CorrectionEffect::Expand; state.show_tuning(&tuning); state.overlay.set_glass_tuning(tuning); let (show, hide) = (state.clone(), state.clone()); let events: Vec<(f64, Box<dyn Fn()>)> = vec![ (1.9, Box::new(|| { let Some(root) = overlay_panel().and_then(|panel| panel.contentView()) else { return }; if let Some(text) = find_views(&root, "NSTextView").first() { watch_frame(text, "transcript", |frame| frame.size.height < 120.0); } })), (2.0, Box::new(move || show.correction_shown.set(true))), (3.5, Box::new(move || hide.correction_shown.set(false))), ]; run_script(events, 0.012, 4.6, |t| { let Some(root) = overlay_panel().and_then(|panel| panel.contentView()) else { return }; let glass_height = find_views(&root, "NSGlassEffectView").first().and_then(|glass| presentation_height(glass)); let texts = find_views(&root, "NSTextView"); let Some(transcript) = texts.first() else { return }; eprintln!("t={t:.3} glass_h={glass_height:?} transcript={:?}", screen_rect(transcript)); eprintln!(" {}", ancestry(transcript)); }); } ``` - The tuner redraws the overlay every 0.075s from `TunerState::tick`. `correction_shown` and `show_tuning` are `TunerState` members, reachable because the block sits in `tuner.rs`. - Before 2.0s the overlay is still popping in. Keep events at or after about 1.9s, or the samples mix in the pop. - Print only lines that differ from the previous one, for example `awk '{k=$0; sub(/^t=[0-9.]+ /,"",k); if (k!=prev) print; prev=k}'`. ## Reading `watch_frame` stacks | Frames under `_postFrameChangeNotification` | Meaning | | --- | --- | | `setFrameSize:`/`setFrameOrigin:` ← `setValue:forKeyPath:` ← `CABasicAnimation applyForTime:` ← `NSAnimationManager` | An `animator()` animation is stepping the model frame. | | `… NSAnimationManagerStopAnimation` | That animation's end. Compare its last value with the target: glass stops short. | | `setContentInsets:` ← `-[NSScrollView _applyContentAreaLayout:]` ← `tile` ← `layout` | The scroll view changed its clip insets (the corner insets in findings.md). | | `setPostsFrameChangedNotifications:` ← KVO `didChangeValueForKey:` | The frame changed indirectly; read further down the stack for the source. | | `simple_ptt::…` frames | Our code; the symbol names the function. | ## Finding the overlay's views - The main glass is the first `NSGlassEffectView`. The halo and correction glass are `SimplePttPassiveGlassEffectView`, so they do not match that class name. - `NSTextView`, depth first: the transcript first, then the correction text. - The transcript's ancestry is: text view < clip view < scroll view < `main_content_view` < the glass's content wrapper < a private `ContentHolderView` < main glass < stack view < `NSGlassEffectContainerView` < root view. ## Measuring legibility on captures For text colour or contrast bugs, measure pixels rather than reasoning about colours. The snapshot backdrop is centred on the overlay, with rows of text running under the glass, so captures include busy content behind the text. Temporarily add a matrix loop at the top of the `for dark in [false, true]` loop in `src/overlay/dev/snapshot.rs`, after `pump(0.3);`. It sets tunings, redraws like the tuner, captures each case with and without the transcript (the `-notext` pair the paired method needs), then continues. This is the tested dim matrix; vary other `GlassTuning` fields the same way: ```rust if std::env::var("DIM_MATRIX").is_ok() { use crate::overlay::glass::OverlayAppearance; let final_text = text(&full, None); for dim in [0.0, 0.35, 0.65] { for (appearance_name, appearance) in [("aqua", OverlayAppearance::Light), ("darkaqua", OverlayAppearance::Dark)] { let mut tuning = overlay.glass_tuning(); tuning.dim = dim; tuning.appearance = appearance; overlay.set_glass_tuning(tuning); for (suffix, transcript) in [("", &final_text), ("-notext", &empty)] { for _ in 0..8 { update(STATE_RECORDING, transcript, "", &empty, false, meter); pump(0.075); } capture(mtm, dir, &format!("dim-{dim:.2}-{appearance_name}-{theme}{suffix}")); } } } backdrop.orderOut(None); pump(0.3); continue; } ``` Run it with `cargo build --release && DIM_MATRIX=1 ./target/release/simple-ptt --overlay-snapshot .tmp/dim`, then `bun .agents/skills/overlay-visual-debugging/scripts/contrast.ts .tmp/dim dim-`. Pass the capture's `halo_margin` as the third argument when it is not the default. - The body result compares each transcript glyph pixel with the same pixel in the `-notext` capture, which is what is behind that glyph. Without a pair the script falls back to the region median, marked `~`; that method is unreliable with backdrop text behind the glass, and the footer (which the pairs keep) is always measured that way. - The regions in `scripts/contrast.ts` fit the recording state on a 2x display. Before trusting any numbers, draw them on one capture (the body region starts 40,35 pixels from the glass corner at `(40 + halo_margin) * 2`) and look at it. - Remove the matrix block before committing (`rg -n _MATRIX src/` must print nothing). ## lldb and SDK recipes Build first. Every command works on `./target/debug/simple-ptt` without a running process, except the breakpoint run. ```sh # Which classes implement a selector (regex over AppKit symbols): lldb --batch -o "image lookup -r -s 'contentViewInsets|_allowsAdditional'" ./target/debug/simple-ptt 2>&1 | rg -o '[-+]\[[^]]+\]' | sort -u # What a private method reads and calls (selectors it sends): lldb --batch -o "dis -n '-[NSScrollView _applyContentAreaLayout:]'" ./target/debug/simple-ptt 2>&1 | rg -o 'objc_msgSend\$[A-Za-z_:]+' | awk '!seen[$0]++' # Who calls a setter, with its receiver and first argument (x2), in a self-exiting probe run: VIEW_PROBE=1 lldb --batch -o "breakpoint set -n '-[NSScrollView _setAllowsAdditionalContentInsetsForCornerRadii:]' -C 'register read x0 x2' -C 'bt 6' -G true" -o "run --debug" ./target/debug/simple-ptt > .tmp/lldb.txt 2>&1 # Public API check (macOS 27 SDK headers name APIs not on 26): rg -n -i "<term>" "$(xcrun --sdk macosx --show-sdk-path)/System/Library/Frameworks/AppKit.framework/Headers/" ``` - A 4-instruction getter or setter (`ldrb`/`strb` at an ivar offset) is a plain flag. Look for the code that writes it back, as `viewDidMoveToSuperview` does in findings.md. - Symbolic breakpoints on `setFrame:` miss frame changes made through `setFrameOrigin:`/`setFrameSize:` or KVC. Use `watch_frame` instead. - Apple's documentation pages render empty through WebFetch. Fetch the JSON instead: `https://developer.apple.com/tutorials/data/documentation/appkit/<class>/<member>.json` (lowercase path)..agents/skills/overlay-visual-debugging/assets/view_probe.rs6.4 KB3ad77b0146e0upstream ↗
//! Temporary probes for overlay motion and layout bugs. Copy to //! `src/overlay/dev/view_probe.rs`, add `pub mod view_probe;` to //! `src/overlay/dev/mod.rs`, and call it from `tuner::run()`. Delete the file, //! the `mod` line, and the call before committing. use std::cell::Cell; use std::ptr::NonNull; use std::time::Instant; use block2::RcBlock; use objc2::rc::Retained; use objc2::runtime::AnyObject; use objc2::{msg_send, MainThreadMarker, Message}; use objc2_app_kit::{NSApplication, NSView, NSViewFrameDidChangeNotification, NSWindow}; use objc2_foundation::{ NSNotification, NSNotificationCenter, NSRect, NSRunLoop, NSRunLoopCommonModes, NSString, NSThread, NSTimer, }; use objc2_quartz_core::CALayer; thread_local! { static START: Cell<Option<Instant>> = const { Cell::new(None) }; } /// Seconds since `run_script` started (0 before). pub fn elapsed() -> f64 { START.with(|start| start.get().map_or(0.0, |start| start.elapsed().as_secs_f64())) } pub fn class_name(object: &AnyObject) -> String { // SAFETY: every Objective-C object answers `className`. let name: Retained<NSString> = unsafe { msg_send![object, className] }; name.to_string() } /// The overlay's panel (`OverlayPanel` in `src/overlay/glass.rs`). pub fn overlay_panel() -> Option<Retained<NSWindow>> { let app = NSApplication::sharedApplication(MainThreadMarker::new()?); app.windows().iter().find(|window| class_name(window) == "OverlayPanel") } /// Views of class `class` under `root`, depth first: in the overlay the /// transcript's views come before the correction's. pub fn find_views(root: &NSView, class: &str) -> Vec<Retained<NSView>> { let mut found = Vec::new(); fn walk(view: &NSView, class: &str, found: &mut Vec<Retained<NSView>>) { if class_name(view) == class { found.push(view.retain()); } for subview in view.subviews().iter() { walk(&subview, class, found); } } walk(root, class, &mut found); found } /// `view`'s bounds in screen coordinates: where it really is on screen. pub fn screen_rect(view: &NSView) -> NSRect { let in_window = view.convertRect_toView(view.bounds(), None); view.window().map_or(in_window, |window| window.convertRectToScreen(in_window)) } /// Frame and bounds of `view` and each of its ancestors, innermost first. pub fn ancestry(view: &NSView) -> String { let mut parts = Vec::new(); let mut cursor = Some(view.retain()); while let Some(current) = cursor { let (f, b) = (current.frame(), current.bounds()); parts.push(format!( "{}[f={:.2},{:.2} {:.1}x{:.1} b={:.2},{:.2}]", class_name(¤t), f.origin.x, f.origin.y, f.size.width, f.size.height, b.origin.x, b.origin.y, )); // SAFETY: reading the superview of a live view on the main thread. cursor = unsafe { current.superview() }; } parts.join(" < ") } /// Height of `view`'s layer as drawn now. It differs from `frame()` while a /// Core Animation animation runs; for AppKit `animator()` frame animations /// the two move together. pub fn presentation_height(view: &NSView) -> Option<f64> { let layer = view.layer()?; // SAFETY: `presentationLayer` returns nil or a copy of the layer. let presentation: Option<Retained<CALayer>> = unsafe { msg_send![&*layer, presentationLayer] }; presentation.map(|layer| layer.bounds().size.height) } /// Prints the AppKit call stack of every frame change of `view` that /// `report` accepts. It finds who moves or resizes a view when breakpoints /// on `setFrame:` catch nothing (AppKit's animation manager and layout /// helpers call `setFrameOrigin:` and `setFrameSize:` directly). pub fn watch_frame(view: &NSView, label: &'static str, report: impl Fn(NSRect) -> bool + 'static) { let observed = view.retain(); let block = RcBlock::new(move |_notification: NonNull<NSNotification>| { let frame = observed.frame(); if !report(frame) { return; } eprintln!("FRAME {label} t={:.3} {frame:?}", elapsed()); let symbols = NSThread::callStackSymbols(); let symbols: Vec<String> = symbols.iter().map(|symbol| symbol.to_string()).collect(); // Skip the probe's own frames and the notification center's. let first = symbols .iter() .position(|symbol| symbol.contains("_postFrameChangeNotification")) .unwrap_or(0); for symbol in symbols.iter().skip(first).take(24) { eprintln!(" {symbol}"); } }); let center = NSNotificationCenter::defaultCenter(); // SAFETY: the block only reads a main-thread view, and the notification // is posted on the main thread that changes the frame. let token = unsafe { center.addObserverForName_object_queue_usingBlock( Some(NSViewFrameDidChangeNotification), Some(view), None, &block, ) }; // The probe lives as long as the process. std::mem::forget(token); } /// Runs `events` (seconds from now, action) once each, calls `sample` every /// `sample_every` seconds with the elapsed time, and exits the process at /// `end` seconds, so a scripted run needs no one to close the app. pub fn run_script( events: Vec<(f64, Box<dyn Fn()>)>, sample_every: f64, end: f64, sample: impl Fn(f64) + 'static, ) { START.with(|start| start.set(Some(Instant::now()))); let pending = Cell::new(events); let block = RcBlock::new(move |_timer: NonNull<NSTimer>| { let now = elapsed(); let mut events = pending.take(); events.retain(|(at, action)| { if *at <= now { eprintln!("-- t={now:.3} event scheduled at {at:.3}"); action(); false } else { true } }); pending.set(events); sample(now); if now >= end { std::process::exit(0); } }); // SAFETY: the timer is added to the main run loop, so the block runs on // the main thread. let timer = unsafe { NSTimer::timerWithTimeInterval_repeats_block(sample_every, true, &block) }; // SAFETY: main thread; `NSRunLoopCommonModes` is a Foundation constant. unsafe { NSRunLoop::mainRunLoop().addTimer_forMode(&timer, NSRunLoopCommonModes) }; std::mem::forget(timer); }README.mdupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
Plain-text copy, no login: /skill-bundles/alexgorbatchev-simple-ptt-agents-skills-overlay-visual-debugging.md