⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
audio-flyover-geolocation
>-
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
// README
Audio Fly-over Geolocation
English | 简体中文
An agent skill that works out where an audio or video recording was made from the faint propeller aircraft in its background. It ships a complete, tested Python pipeline and comes in English and Simplified Chinese.
Small aircraft give off a steady tone. As one flies past, the Doppler effect bends that tone into a falling curve, and the shape of the curve gives the aircraft's speed, its closest distance to the microphone and the second it was closest. Public ADS-B archives record where every transponder-equipped aircraft was at every second. If you search those tracks for the place where the predicted curve matches what was recorded, you find the microphone. Two fly-overs in one recording are enough, even when the recording time is unknown, because their time difference is fixed by the recording itself.
The method comes from colsto's video I discovered a sound that leaks your exact location (2026-11-02). colsto did not release code; this repository is an independent implementation with no affiliation.
Synthetic blind test (seed 23): each fly-over alone gives a strip of possible locations. Scored together on one shared clock, they give a point 46 m from the true recording location, inside the 0.9-score region the pipeline reports (radius 182 m).
Contents
- How it works
- Accuracy on synthetic blind tests
- Install
- Use
- Check your own videos
- Limitations
- Responsible use
- Repository layout and development
- Credits and sources
How it works
flowchart LR
A[Audio or video] --> B[Whitened spectrogram]
B --> C[Trace each Doppler ridge]
C --> D["Fit v, d, f0, tCPA<br/>k = v²/d"]
E[Historical ADS-B tracks] --> F
D --> F["Search funnel<br/>corridor → CPA match → ΔT → silence"]
F --> G["Exact emission-time scoring<br/>both events on one clock"]
G --> H["Location + error region<br/>+ recording start time"]
1. Read the curve. The heard frequency is f = f0 / (1 + Ṙ/c), where Ṙ is the aircraft's range rate at
the moment the sound was emitted. The two flat ends of the curve give the speed
(v = c·(f_hi − f_lo)/(f_hi + f_lo)). The steepness at closest approach gives the distance
(d = f0·v²/(c·|slope|)).
2. Trace it out of the noise. The spectrogram is whitened against a running median background, and
frames with speech or knocks are gated out. A monotone Viterbi path is then accepted only if it beats a null
path computed on time-shuffled frames, so the tracer does not hallucinate curves in pure noise. When the flat
ends are faint, v and d trade off against each other, but k = v²/d stays well determined. The search
therefore checks every candidate aircraft against k computed from its own ADS-B ground speed.
3. Search the ADS-B archive. The search runs on a 200 m grid with 3D slant distances in ECEF and DEM ground elevation, and narrows the candidates in stages:
- corridors around tracks that fit the aircraft class and speed;
- a closest-point-of-approach match that includes the sound's travel time;
- the time offset between the two fly-overs;
- a silence check: no other aircraft may have passed close while the recording was quiet.
4. Score against the exact physics. For each surviving candidate, the curve is predicted from the aircraft's real track by emission time, and the spectrogram is sampled along it. Both events share one clock offset, which keeps the strongest constraint: the purely physical time difference between them. A shrinking hill-climb refines the best points to 25 m. A fine heat map gives the reported error region. Degenerate geometry is always reported, never hidden:
- straight single passes have a mirror twin;
- a loose clock leaves an along-track strip;
- near-parallel tracks leave a strip.
Accuracy on synthetic blind tests
selftest simulates a 50–70 km area over 4 hours with about 70 flights (45 propeller, 25 jet). It adds a
~24-minute recording with two fly-overs plus pink noise, mains hum, speech and jet noise. Event 2's aircraft
accelerates before its closest approach and climbs after it, on purpose, so the simple fit is biased. The
pipeline then runs blind, and the result is compared with the hidden truth. CI runs all six seeds on every push.
| Seed | Two events, clock unknown | One event, clock ±2 s, turning pass | One event, clock ±2 s, straight pass | One event, clock ±30 s |
|---|---|---|---|---|
| 7 | 24 m | 23 m | mirror; truth is the flagged alternative | truth inside the reported strip |
| 11 | 477 m, flagged (tracks cross at 3°); truth inside the reported 849 m region | 35 m | mirror; truth is the flagged alternative | truth inside the reported strip |
| 23 | 46 m | 79 m | 162 m | truth inside the reported strip |
| 42 | 31 m | 86 m | 206 m | truth inside the reported strip |
| 101 | 53 m | 48 m | 213 m | truth inside the reported strip |
| 2026 | 25 m | 96 m | 6 m | truth inside the reported strip |
With a ±30 s clock the strips are 2.1–4.8 km long. Each run takes 45–55 s on one CPU core.
These numbers are for idealised data: no ADS-B gaps, no wind, no multipath. Expect larger errors on real footage. What the test does establish is that the pipeline is correct and that its stated uncertainty is honest.
Install
The two language editions are the same skill (audio-flyover-geolocation) with the same scripts; install one.
It needs Python 3 with numpy, scipy and matplotlib (CI tests Python 3.13). ffmpeg is needed for video and compressed
audio, and rasterio (optional) for DEM GeoTIFFs.
Claude Code
git clone https://github.com/AlanBacker/audio-flyover-geolocation
cp -r audio-flyover-geolocation/skills/en/audio-flyover-geolocation ~/.claude/skills/
pip install numpy scipy matplotlib
For the Chinese edition, copy skills/zh-CN/audio-flyover-geolocation instead.
Claude apps. Download audio-flyover-geolocation-en.skill or audio-flyover-geolocation-zh-CN.skill from
Releases, then upload it in the Skills
section of the app's settings. Code execution must be turned on.
Without an agent. The pipeline is a plain CLI:
python skills/en/audio-flyover-geolocation/scripts/afg_cli.py -h
Use
Ask in plain language, for example:
- "Where was this video recorded?
clip.mp4, filmed sometime on 12 October in eastern Kansas." - "Can you ve
// HOW IT'S BUILT
// FILES
25 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.
skills/en/audio-flyover-geolocation/SKILL.md23 KBe754b6dee279upstream ↗
--- name: audio-flyover-geolocation description: >- Acoustic fly-over geolocation: locate where an audio or video recording was made from the near-inaudible Doppler curves of propeller aircraft and helicopters in the background, matched against historical ADS-B tracks (about 100 m in good conditions), with an honest uncertainty region. Also covers thunder plus lightning-data localisation and a pre-publication acoustic privacy self-audit. Use this skill whenever the user wants to locate, verify or time a video, recording, livestream or call from its sound, background noise, aircraft noise, Doppler shift, spectrogram, ADS-B data or thunder; mentions acoustic geolocation, audio OSINT, "where was this recorded", verifying a video's location by sound, or colsto's method; or wants to check whether their own video leaks its location through sound, even if they never say Doppler or ADS-B. For OSINT verification, journalism, CTFs, research and self-audits; ships a tested Python pipeline. --- # Acoustic fly-over geolocation (Doppler × ADS-B) This skill reproduces and productionises the method from colsto's video *"I discovered a sound that leaks your exact location (and it's everywhere)"* (2026-11-02, <https://www.youtube.com/watch?v=UytYrhYLOKI>): > A propeller's or rotor's **blade-passing frequency (BPF)** is a stable narrow-band tone. Even when no one can > hear it, it leaves a smooth descending **Doppler curve** in a spectrogram. The curve's shape gives the > aircraft's **speed v**, its **closest-approach slant distance d** and the **CPA time tCPA**. Comparing it point > by point on a ground grid against historical ADS-B tracks (3D slant range + DEM + sound travel time + > emission-time radial velocity) narrows the candidates. The **time gap between two fly-overs** (920 s in the > video) plus a **silence constraint** (no other aircraft heard at other times) then reduces them to a single > point. Finally, a "locked two-event score" (shared time offset, per-event f0 fine-tuning) rates exact model > curves against the spectrogram. `scripts/afg_cli.py` is the full implementation (numpy/scipy/matplotlib). It ships with `selftest`, a blind test on synthetic scenarios. --- ## 0. Scope of use (do this first) **In scope**: verifying where and when public or news media was recorded (OSINT, fact-checking); privacy self-audits of the user's own recordings or videos; CTFs, teaching and research; investigations with lawful authority or consent (e.g. assisting law enforcement, with the subject's agreement). **Out of scope**: locating a **private individual's** home or movements. Examples: "where does this streamer / my ex / this person online live", or tracing someone from a private call or home video. Decline these, even when framed as a "test", and offer **self-audit mode** (section 7) instead. That mode helps people assess and remove the leak from their own content. Footage of public places and news events, the user's own material and anonymised research data are fine. If the purpose is unclear, ask what it is for before deciding. Do not add personal inferences unrelated to the location to the report. --- ## 1. Feasibility triage (decides whether to run and which mode) | Condition | What is needed | If not met | |---|---|---| | A **clear monotonically descending narrow-band curve** in the spectrogram (propellers commonly 40–400 Hz; helicopter main-rotor BPF 10–25 Hz, tail rotor 50–120 Hz) | At least 1; ideally 2 with a known time gap | No curve → method not applicable (consider the thunder method or visual methods) | | **Date** | Must be known (or bounded to a few days) | ADS-B data volume becomes unmanageable | | **Region** | Country or state level is enough for two events; a city-level prior is faster | One event alone needs a tight clock **and** a limited area: over a large area every aircraft with a matching speed and distance explains the curve about equally well (`score` says `DEGENERATE`); the self-test area is 50–70 km across | | **Recording clock** (start time) | Known to ±2 s → one event can give a point; ±30 s → one event gives only a strip along the track | Use two events; their time gap replaces absolute time | | **ADS-B coverage** | The target aircraft must have ADS-B Out and be received by ground stations | US 14 CFR 91.225: GA aircraft below 10,000 ft outside Class B/C airspace and the Mode C veil need not carry ADS-B Out → may be absent from the data | | Audio processing chain | Original or lightly compressed is best; phone-call codecs usually keep the tone | Heavy noise reduction or editing can cut the curve or destroy the gap between events | Tell the user the triage outcome and expected precision before starting the work. ## 2. Choosing a mode ``` How many usable curves? ├─ ≥2 (same continuous recording) → Mode A: two events, unknown absolute time (the video's main method; most robust) │ If the two tracks cross at < ~25°, the result degrades to a strip along the tracks (the script warns) ├─ 1 │ ├─ Recording time known to a few seconds → Mode B: one event, known time (the video's second half, ranked first │ │ US-wide there; in busier airspace many aircraft can fit equally, so restrict the area) │ │ A straight constant-speed pass is mirror-symmetric → there is always a "mirror candidate"; │ │ a second clue (visual, terrain) is needed to pick between them │ └─ Time known only to ±30 s or worse → Mode B′: only a strip of length ≈ 2·v·Δt is possible; report it as such ├─ No aircraft curve, but thunder + known date → Mode T: thunder envelope × lightning data (km level, see references/thunder-method.md) └─ The user wants to protect their own content → Mode S: self-audit (section 7) ``` --- ## 3. Workflow (CLI) Convention: `AFG="python <skill dir>/scripts/afg_cli.py"`, working directory `work/`. Every step writes PNGs. **Look at the PNGs yourself** with your image-reading ability (spectrograms, fit overlays, heat maps) rather than relying on the numbers alone. Check the environment first: `pip install numpy scipy matplotlib` (the DEM needs `rasterio`; non-WAV input needs `ffmpeg`). On first use, run `$AFG selftest --seed 7` (about 40–60 s; exit 0 means the pipeline works). ### 3.1 Get the audio and metadata - `ffprobe -v quiet -show_entries format_tags=creation_time:stream_tags=creation_time -of json input.mp4` gives the capture time. Mind the time zone. Social platforms often strip or rewrite metadata on re-encode, so **treat it only as a lead and cross-check it**. - Pass the original sample rate to the script; it resamples internally to 4 kHz mono (`--channel` selects left or right). ### 3.2 Spectrogram → detect → trace → fit ```bash $AFG spectrogram input.mp4 --out work/rec # whitened narrow-band spectrogram + work/rec.png $AFG detect work/rec.spec.npz --out work/boxes.json # candidate boxes (t0, t1, fmin, fmax, score_z) + boxes.diag.json $AFG plot work/rec.spec.npz --t0 160 --t1 320 --fmin 60 --fmax 380 --out work/zoom1.png $AFG trace work/rec.spec.npz --t0 166 --t1 315 --fmin 98 --fmax 364 --out work/e1.csv $AFG fit work/e1.csv --temp-c 18 --out work/e1.json # v, d, tCPA, f0, search windows; overlay in e1.fit.png ``` Key points: - `boxes.json` is the event list (a JSON array); the detection status is in `boxes.diag.json`. Stdout prints both in one object (`detection`, `events`, `events_file`). - An empty `boxes.json` means "no fly-over" only when `boxes.diag.json` says `status: ok`. `limited` (short clip judged against the calibrated noise model, or mostly gated audio) and `insufficient` (clip shorter than `--min-dur`, or almost everything gated) come with `reasons`: read them, inspect `rec.png`, and lower `--min-dur` for short clips. - Each event carries `trend` (`descending` or `flat`) and `drop_pct`. A ridge that rises in frequency is not a fly-over: it is listed under `rejected` in `boxes.diag.json`, not in `boxes.json`. `warnings` on an event mean its ridge does what one pass cannot (rises again, or falls by more than 45 %): it probably joins two aircraft or two harmonics. Inspect it with `plot` and trace each part in its own narrow box. - `--win`: use 0.5–1 s for close, steep curves and 2–4 s for distant or faint ones. For a faint curve, zoom in with `plot` and confirm it by eye first. Then give `trace` a narrow box containing **only one harmonic**, and if needed add a few manual points with `--anchors "t:f,t:f,..."`. - `trace` reports `box_coverage`; below 50 % it adds a `hint`. A curve that rises before it falls (a turning or accelerating aircraft) needs `--no-monotone`. - Check the `fit` output: - `implausible` non-empty = the trace ran onto noise, jumped harmonics or kept only a short piece. **Do not** go on to the search (the script refuses too). - `notes`: left/right residual asymmetry = a turn or speed change; different approach/recede speeds = a climb or acceleration. - `sigma` already allows for correlated residuals (`resid_corr_time`, `sigma_inflation`, `n_eff`), but it is statistical only: a speed change during the pass biases v and d by more than sigma. `corr_v_d` near 1 means v and d trade off along k = v²/d. The search therefore uses wide speed windows and the k window. - `constraints.d_min/d_max` is the search's distance window, which follows from the speed and k windows. `d_fit_min/d_fit_max` is what the fit itself supports (3 sigma); a note says when the two differ a lot. - If you know the air temperature, pass `--temp-c` (c = 20.05·√(T+273.15)). v and d both scale with c; without a temperature, keep the default ±4 %. - f0 is just the rest frequency of whichever harmonic you traced, not necessarily the fundamental; v and d do not depend on the harmonic number. - BPF = blades × RPM / 60 can **help** guess the type (C172, 2 blades × 2400 rpm ≈ 80 Hz; DHC-8-400, 6 blades × 1020 rpm ≈ 102 Hz). Do not use it as a hard candidate filter. ### 3.3 ADS-B data (details in `references/adsb-data.md`) - Sources: - adsb.lol `globe_history_YYYY`: one global release per day, free, ODbL. - ADS-B Exchange free samples: **1st of each month only**. - OpenSky Trino: needs a research account; table `state_vectors_data4`. - Your own readsb/tar1090 receiver. - Extract only the target date, the target window ± 1 h, and the target area padded by ~0.5°. ```bash $AFG adsb-ingest --src traces_dir/ --start 2026-09-08T13:00Z --end 2026-09-08T18:00Z \ --bbox 38.5,39.7,-95.4,-93.8 --qnh 1018 --geoid-n -29 --out work/tracks.npz ``` - **The altitude datum decides slant-range accuracy.** - `alt_baro` is standard pressure altitude: correct it with local QNH (≈ 8.3 m/hPa; see METAR). - `alt_geom` is WGS84 ellipsoidal height (HAE); MSL = HAE − N. - `--alt-source auto` prefers geometric altitude. ### 3.4 DEM and search ```bash $AFG dem-fetch --bbox 38.5,39.7,-95.4,-93.8 --out work/dem # Copernicus GLO-30 (AWS public bucket) $AFG search --tracks work/tracks.npz --events work/e1.json work/e2.json \ --bbox 38.5,39.7,-95.4,-93.8 --dem work/dem/*.tif --rec-duration 1450 --out work/cands.npy ``` The funnel has three stages: 1. **Event-1 corridor**: speed window, k = v²/d consistency, slant-range window, and the CPA must not fall at a track endpoint. 2. **Inter-event offset**: tCPA2 − tCPA1, including each event's d/c travel delay. 3. **Silence filter**: in the remaining quiet spans, no other propeller or rotor aircraft may pass within `--d-quiet`. The search stats (`cands.stats.json`) also report: - `crossing_angle_deg` (two events): if every pair crosses at < 25°, the result can only be a strip; - `flight_pairs`: with one event, how many different aircraft can explain it; - `ground_elev_m`: the elevation range used. With `--ground-elev` it is a constant; in relief use `dem-fetch` (300 m of elevation error at 1.3 km slant range shifts the modelled distance by ~75 m). Options: - Mode B: add `--rec-start <UTC> --rec-start-tol 2` (or 30). - `--quiet "a-b,c-d"` names recording spans where you are sure nothing was heard (default: everything outside the events). Give fewer spans when the recording has a lot of wind or speech. **Silence is signal, but false silence kills the true answer.** In sparse airspace (night, remote areas) it removes little: most candidates had no aircraft nearby anyway. It only gains power with traffic and with recording length. - Default grid is 200 m. If there are too many candidates, shrink the bbox rather than enlarging the grid. ### 3.5 Scoring and refinement ```bash $AFG score --tracks work/tracks.npz --spec work/rec.spec.npz --events work/e1.json work/e2.json \ --cands work/cands.npy --dem work/dem/*.tif --out work/result.json ``` - **Model curves**: for each candidate, generate an exact model curve at **emission time** from the interpolated ADS-B positions and velocities (gs/track/vrate, not differenced positions), and compare it with the spectrogram. One time offset δ is shared, which locks in the physical time gap between the two events. Each event's f0 is fine-tuned within ±0.6 %. - **Refinement**: every kept node gets a shrinking 3×3 hill-climb (100 → 25 m). - **Grouping**: results are grouped by **flight pair**. Each pair keeps up to 3 hypotheses more than 500 m apart, so both mirror sides survive. - **Trace check**: every hypothesis' model curve is compared with the traced points within ±60 s of each CPA (`trace_check`). A bright texture under a wrong curve (speech, music, wind) can lift a brightness score; it cannot make the model reproduce the trace. With two events, hypotheses that fail go below all others. - **Outputs**: - `result.json`: best, `n_consistent`, `margin_vs_2nd`, `robust_z`, `ambiguous_alternatives`, `coarse_plateau`, `geometry`, `scoring` (how many candidates were scored and why others were not), `verdict`, `diagnosis`. - The verdict starts with `NO CONSISTENT SITE` (two events, nothing reproduces both traces; `diagnosis` lists the likely causes), `STRIP ONLY` (tracks cross at < 25°), `DEGENERATE` (one event, three or more other aircraft within 2 %), `AMBIGUOUS` or `unique within scored set`. - `result.rankN.*.png`: model-curve overlays. - `result.heatmap.png/csv`: radius of the 0.9 region. - `result.plateau.csv`. - With a known recording clock, add `--rec-start ...`. Scoring then also forbids time shifts outside the clock; without `--rec-start-tol` it uses the search's tolerance. The time shift spans at least event 1's CPA-time tolerance `t_tol`, the same window the funnel used. ### 3.6 Verification (not optional; see `references/verification.md`) 0. **Trace check**: `best.trace_check.consistent` must be true. `NO CONSISTENT SITE` means there is no answer, whatever the score. 1. **Overlays**: the model curve must fit the observation on **both asymptotes and through the CPA**, not just one segment. 2. **Heat map**: the 0.9-region radius is the error to report. Report a strip when either holds: - two events: `geometry.warning` is set (small crossing angle); - one event: `coarse_plateau.length_m` > 600 m. 3. **Mirror candidates and alternatives**: rule them out with visual clues (horizon, buildings, vegetation, road noise, echoes), DEM viewsheds and sun azimuth. 4. **Falsification**: should other fly-overs near the answer also have been heard? - Heard but not detected = a detection-threshold problem. - Present when they should not have been audible = the location is wrong. ## 4. Precision rules (and why) 1. **Emission time**: f_obs(t) = f0 / (1 + Ṙ(τ)/c), with t = τ + R(τ)/c. Using Ṙ at reception time gives a curve of the **wrong shape**, not just a shifted one. 2. **3D slant range + DEM**: observer elevation plus microphone height (default 2 m). Treating 500 m of terrain relief as 0 makes both d and tCPA wrong. 3. **Altitude datum**: uncorrected pressure altitude with a 30 hPa QNH offset ≈ 250 m of altitude error, which goes straight into d. 4. **Delayed CPA**: the moment f = f0 is observed is tCPA_emit + d/c (1 km ≈ 2.9 s). Event offsets and the clock must both be converted with this. 5. **k = v²/d is more reliable than v or d alone.** - The mid-curve slope, −f0·v²/(c·d), is pinned down tightly. The asymptotes are often polluted by noise, gating, or the aircraft accelerating or climbing. - So the search uses a wide speed window plus a narrow k window, and each candidate aircraft's own ground speed predicts its d. 6. **Locked scoring**: both events share one δ. Otherwise each event aligns its own time, and wrong locations score well too. 7. **Refine to ≤100 m**: the 200 m grid is only a coarse screen; the final point comes from sub-grid hill-climbing and the heat map. 8. **Silence is signal**, but keep the thresholds generous (defaults: propeller 3 km, rotor 2.5 km). Distant or low-power aircraft may genuinely be inaudible. 9. **Loose windows, strict scoring**: in the funnel, err wide, because a true answer filtered out can never be recovered. Leave discrimination to exact-model scoring. 10. **Group by flight pair and keep the mirror**: a straight constant-speed pass is symmetric about its track. Single-event results should report at least both sides. 11. **ADS-B gaps**: no match ≠ not in the area. Low-level GA, military aircraft and some helicopters may lack ADS-B or go unreceived (terrain masking, low altitude). 12. **Sound speed and wind**: v and d scale with c. With wind, c_eff = c + w·n; in strong wind (>8 m/s), widen the d window. ## 5. Report Output using `assets/report_template.md`, in the user's language. It must include: - the answer coordinate plus an error radius or strip; - the matched flights (hex / registration / type) and the recording start time in UTC; - the key evidence (fit parameters, overlays, heat map); - ambiguities and alternatives, and data-coverage gaps; - a confidence grade (see the rubric in verification.md) and what cannot be ruled out. **Do not** present a single high score as "certain". When margin < 10 % or ambiguous_alternatives exist, say explicitly that the result is "ambiguous". ## 6. Typical failures and fixes | Symptom | Cause | Fix | |---|---|---| | fit gives v≈280 m/s or d=60 km | The trace ran onto noise, jumped between harmonics, or kept only a short piece (`trace` `hint`) | Narrow to a single-harmonic box; use `--anchors`; `--no-monotone` if the curve rises before it falls; inspect with `plot` before tracing | | `detect` finds nothing in a short clip | `status` is `insufficient` (clip shorter than `--min-dur`) or the curve is weak | Read `boxes.diag.json`; lower `--min-dur`; look at `rec.png` and trace by hand with `--anchors` | | An event has `warnings`, or one box spans two curves | Two aircraft audible at the same time, or the ridge jumped between harmonics | `plot` the box; trace each curve in its own box; `detect --window 20 --step 5` separates closer passes | | A visible tone is missing from `boxes.json` | Its ridge rises, so it is listed under `rejected` in `boxes.diag.json` | A straight pass never rises; if it is a turning aircraft, trace it by hand with `--no-monotone`, not as a primary event | | Empty funnel | Windows too narrow / wrong altitude datum / missing ADS-B / wrong time window | First widen `--v-margin/--k-margin/--c-uncert`; check QNH and `--start/--end`; check `--event-classes` | | Many candidates with similar scores, `DEGENERATE` | One event over a large area or with no clock, near-parallel tracks, or a curve that is too short | Add a second event, pin down the clock, shrink the area, report a strip | | `NO CONSISTENT SITE` | No candidate's model reproduces both traced curves: recordings from different places or a wrong offset between files, an aircraft missing from ADS-B, the site outside the bbox, or a bad trace | Read `diagnosis`; check each fit overlay; widen bbox and time window; say in the report that a missing aircraft cannot be ruled out | | `score` says "nothing scored", or `scoring.rejected_clock` > 0 | Score's clock tolerance is tighter than the search's | Leave out `--rec-start-tol` in `score` (it then uses the search's) or pass the same value | | The true speed lies far outside fit `sigma` | The aircraft changed speed or climb during the pass; sigma is statistical only | Expected; the search's speed and k windows cover it. Compare `fit_split` | | Verdict says the f0 scale sits at the limit | The fitted f0 is off, often from a speed change in the pass | Re-run `score` with `--f-rel 0.012`; do not treat it as a reason to drop the answer | | Best point's rec_start differs from metadata by tens of seconds | The metadata is the export/upload time | Trust the acoustic result and explain it in the report | | Jet interference | Jets are broadband noise plus a few tones | By default only prop/rotor/unknown classes are events; jets only count for the silence check | ## 7. Self-audit mode (protecting your own content) ```bash $AFG audit my_video.mp4 --out work/audit ``` This outputs candidate fly-overs, the detection status and a risk level. Only ridges that fall in frequency count as fly-overs: rising tones are set aside (`detection.rejected`, mentioned in `notes`), and a candidate with `warnings` counts as two. `undetermined` means the automatic check could not decide (for example a clip shorter than 12 s, a tonal ridge too short or too flat to judge, or a rising tone that may hide a fly-over); it is not a pass. A file shorter than the 1 s analysis window is `undetermined` with no PNG. Even `low` only covers sustained aircraft tones this detector can see, so always look at the PNG. **What works**: - **Cut** at least one fly-over, or break the time gap between two fly-overs. - Replace or remove the original audio track (re-record the narration). - Also scrub GPS/EXIF/QuickTime metadata and visual clues. **What doesn't**: - High-pass filtering: it overlaps the voice band, and the residual can still be recovered. - "Drowning it out": a narrow-band tone still stands out in the spectrogram. - Call noise suppression: it only gates during silence. See `references/countermeasures.md`. ## 8. Reference files (read as needed) | File | When to read | |---|---| | `references/physics.md` | Derivations, identities, error propagation, wind/temperature, helicopters/multi-engine | | `references/spectrogram.md` | Parameter choice, whitening/gating, diagnosing trace failures, identifying harmonics | | `references/adsb-data.md` | How to download each data source, formats, fields, altitude datums, coverage gaps, type classification | | `references/search-and-scoring.md` | Funnel and scoring algorithm details, every parameter, tuning advice | | `references/verification.md` | Visual verification, confidence rubric, stereo-bearing extension, synthetic test baseline | | `references/thunder-method.md` | Thunder × lightning-data localisation when there is no aircraft | | `references/countermeasures.md` | How effective each privacy countermeasure is | | `references/sources.md` | Video, paper, data and regulation sources |skills/en/audio-flyover-geolocation/scripts/afg/__init__.py981 B8abfcc87ad19upstream ↗
"""afg - Acoustic Flyover Geolocation toolkit. Locate where an audio/video recording was made by matching faint aircraft Doppler curves (and optionally thunder) in its soundtrack against public flight-tracking (ADS-B) and lightning data. Modules ------- common constants, time parsing, JSON helpers geo WGS84 geodesy, ENU frames, implicit search grid dem elevation model sampling (GeoTIFF via rasterio, or constant) physics sound speed, Doppler models, curve fitting audio audio loading, chunked STFT, whitening, ridge tracking, detection adsb readsb / ADS-B Exchange / adsb.lol / OpenSky ingestion, classification tracks per-flight resampling to a uniform 1 s ECEF time base search candidate funnel: CPA events, offset filter, silence filter scoring emission-time Doppler modelling and locked two-event scoring thunder thunder-envelope geolocation (lightning data) simulate synthetic scenario generator used by the self-test """ __version__ = "1.2.0"skills/en/audio-flyover-geolocation/scripts/afg/adsb.py13 KB8bd7eed42fe4upstream ↗
"""ADS-B ingestion and aircraft classification. Supported inputs ---------------- readsb trace files (trace_full_<hex>.json, gzip or plain) as published by - ADS-B Exchange free samples: samples.adsbexchange.com/traces/YYYY/MM/DD/xx/ - adsb.lol globe_history daily releases (github.com/adsblol/globe_history_YYYY) - any tar1090/readsb install (/var/globe_history/YYYY/MM/DD/traces/) OpenSky state_vectors_data4 CSV exports (Trino / pyopensky) Generic CSV: hex,t,lat,lon,alt_m,gs_ms,track_deg[,vr_ms,type] readsb trace entry layout (wiedehopf/readsb README-json.md): [dt, lat, lon, alt_baro_ft|"ground"|null, gs_kt, track_deg, flags, vrate_fpm, details|null, source, alt_geom_ft, geom_rate_fpm, ias_kt, roll_deg] flags: 1 stale, 2 new leg, 4 vrate geometric, 8 altitude geometric Altitude datum (matters for slant distance!) alt_baro = pressure altitude (29.92 inHg / 1013.25 hPa), NOT true altitude. true ~= baro + (QNH - 1013.25) * 8.3 m (+ temperature error) alt_geom = GNSS height, WGS84 ellipsoid (HAE). MSL = HAE - N (geoid undulation; N ~ -20..-35 m over CONUS, so MSL = HAE + 20..35 m). """ import csv import glob import gzip import json import os import urllib.request import numpy as np from .common import FPM, FT, HPA_TO_M, KT, log TYPES_URL = "https://raw.githubusercontent.com/wiedehopf/tar1090-db/master/db/icao_aircraft_types.js" # Minimal fallback (ICAO Doc 8643 description codes) if the full table is unavailable. _FALLBACK_TYPES = { "C150": "L1P", "C152": "L1P", "C172": "L1P", "C182": "L1P", "C206": "L1P", "C210": "L1P", "P28A": "L1P", "P28R": "L1P", "PA32": "L1P", "PA34": "L2P", "PA44": "L2P", "PA46": "L1P", "SR20": "L1P", "SR22": "L1P", "BE35": "L1P", "BE36": "L1P", "BE58": "L2P", "M20P": "L1P", "DA40": "L1P", "DA42": "L2P", "C208": "L1T", "PC12": "L1T", "TBM7": "L1T", "TBM9": "L1T", "BE20": "L2T", "BE9L": "L2T", "B350": "L2T", "DH8A": "L2T", "DH8B": "L2T", "DH8C": "L2T", "DH8D": "L2T", "AT43": "L2T", "AT45": "L2T", "AT72": "L2T", "AT75": "L2T", "AT76": "L2T", "SF34": "L2T", "C130": "L4T", "DHC6": "L2T", "E120": "L2T", "R22": "H1P", "R44": "H1P", "R66": "H1T", "EC35": "H2T", "EC45": "H2T", "AS50": "H1T", "B06": "H1T", "B407": "H1T", "S76": "H2T", "A139": "H2T", "H60": "H2T", "A319": "L2J", "A320": "L2J", "A321": "L2J", "B737": "L2J", "B738": "L2J", "B739": "L2J", "B38M": "L2J", "B39M": "L2J", "E170": "L2J", "E75L": "L2J", "CRJ2": "L2J", "CRJ7": "L2J", "CRJ9": "L2J", "B752": "L2J", "B763": "L2J", "B772": "L2J", "B77W": "L2J", "B788": "L2J", "B789": "L2J", "A332": "L2J", "A333": "L2J", "A359": "L2J", "C25A": "L2J", "C56X": "L2J", "CL30": "L2J", "GLF4": "L2J", "GLF5": "L2J", "GLEX": "L2J", "E55P": "L2J", "PC24": "L2J", } def _read_maybe_gz(path): with open(path, "rb") as fh: b = fh.read() if b[:2] == b"\x1f\x8b": b = gzip.decompress(b) return b def load_type_table(path=None, cache_dir=None, allow_download=True): """ICAO type designator -> Doc 8643 description code (e.g. C172 -> L1P). desc code: [L/S/A/H/G/T = land/sea/amphibian/helicopter/gyro/tilt-wing] [engine count][J jet / T turboprop / P piston / E electric]""" cand = [] if path: cand.append(path) cache_dir = cache_dir or os.path.join(os.path.expanduser("~"), ".cache", "afg") cache = os.path.join(cache_dir, "icao_aircraft_types.js") cand.append(cache) for p in cand: if p and os.path.exists(p): try: return _parse_types(json.loads(_read_maybe_gz(p))) except Exception as e: log("type table unreadable:", p, e) if allow_download: try: os.makedirs(cache_dir, exist_ok=True) log("downloading ICAO type table", TYPES_URL) urllib.request.urlretrieve(TYPES_URL, cache) return _parse_types(json.loads(_read_maybe_gz(cache))) except Exception as e: log("type table download failed, using built-in fallback:", e) return dict(_FALLBACK_TYPES) def _parse_types(obj): out = {} for k, v in obj.items(): if isinstance(v, dict): out[k.upper()] = v.get("desc", "") elif isinstance(v, (list, tuple)) and len(v) >= 2: out[k.upper()] = v[1] return out def classify(type_code, types, category=None): """-> 'prop' | 'rotor' | 'jet' | 'glider' | 'balloon' | 'unknown'""" desc = types.get((type_code or "").upper(), "") if type_code else "" if desc and len(desc) >= 3: c0, eng = desc[0], desc[2] if c0 in "HG": return "rotor" if eng == "J": return "jet" if eng in "PTE": return "prop" cat = (category or "").upper() if cat == "A7": return "rotor" if cat in ("A3", "A4", "A5", "A6"): return "jet" if cat == "B1": return "glider" if cat == "B2": return "balloon" if (type_code or "").upper() in ("GLID",): return "glider" if (type_code or "").upper() in ("BALL",): return "balloon" return "unknown" class _Acc: """Accumulates per-sample rows and per-flight metadata.""" def __init__(self): self.cols = {k: [] for k in ("fid", "t", "lat", "lon", "alt_baro", "alt_geom", "gs", "trk", "vr")} self.flights = [] def add_segment(self, meta, arr): if arr["t"].size < 3: return fid = len(self.flights) self.flights.append(meta) self.cols["fid"].append(np.full(arr["t"].size, fid, np.int32)) for k in ("t", "lat", "lon", "alt_baro", "alt_geom", "gs", "trk", "vr"): self.cols[k].append(arr[k]) def finish(self): if not self.flights: raise RuntimeError("no ADS-B samples inside the requested window/bbox") return {k: np.concatenate(v) for k, v in self.cols.items()}, self.flights def _segments(t, breaks, max_gap): """Split index range at explicit breaks and time gaps.""" cut = np.zeros(t.size, bool) cut[1:] = (np.diff(t) > max_gap) | breaks[1:] starts = np.flatnonzero(cut | (np.arange(t.size) == 0)) ends = np.append(starts[1:], t.size) return list(zip(starts, ends)) def _f(v): return np.nan if v is None or isinstance(v, str) else float(v) def ingest_readsb(paths, t0, t1, bbox=None, buffer_deg=0.15, max_gap=45.0, types=None, acc=None): acc = acc or _Acc() files = [] for p in paths: if os.path.isdir(p): files += glob.glob(os.path.join(p, "**", "trace_full_*"), recursive=True) else: files += glob.glob(p) log(f"readsb: {len(files)} trace files") if bbox: la0, la1, lo0, lo1 = bbox[0] - buffer_deg, bbox[1] + buffer_deg, bbox[2] - buffer_deg, bbox[3] + buffer_deg nseg = 0 for i, fn in enumerate(files): if i and i % 20000 == 0: log(f" {i}/{len(files)} files, {nseg} segments kept") try: obj = json.loads(_read_maybe_gz(fn)) except Exception: continue tr = obj.get("trace") or [] if not tr: continue base = float(obj.get("timestamp", 0)) tt = base + np.array([e[0] for e in tr], float) if tt[-1] < t0 or tt[0] > t1: continue lat = np.array([_f(e[1]) for e in tr]) lon = np.array([_f(e[2]) for e in tr]) m = (tt >= t0) & (tt <= t1) & np.isfinite(lat) if bbox: m &= (lat >= la0) & (lat <= la1) & (lon >= lo0) & (lon <= lo1) if not m.any(): continue idx = np.flatnonzero(m) alt_raw = [tr[k][3] for k in idx] ground = np.array([a == "ground" for a in alt_raw]) alt_b = np.array([_f(a) for a in alt_raw]) * FT gs = np.array([_f(tr[k][4]) for k in idx]) * KT trk = np.array([_f(tr[k][5]) for k in idx]) flags = np.array([int(tr[k][6] or 0) if len(tr[k]) > 6 else 0 for k in idx]) vr = np.array([_f(tr[k][7]) if len(tr[k]) > 7 else np.nan for k in idx]) * FPM alt_g = np.array([_f(tr[k][10]) if len(tr[k]) > 10 else np.nan for k in idx]) * FT gr = np.array([_f(tr[k][11]) if len(tr[k]) > 11 else np.nan for k in idx]) * FPM vr = np.where(np.isfinite(gr), gr, vr) cat = None for k in idx: if len(tr[k]) > 8 and isinstance(tr[k][8], dict) and tr[k][8].get("category"): cat = tr[k][8]["category"] break typ = obj.get("t") or "" meta = {"hex": obj.get("icao", os.path.basename(fn)[11:].split(".")[0]), "type": typ, "reg": obj.get("r") or "", "category": cat or "", "desc": (types or {}).get(typ.upper(), "") if types else "", "cls": classify(typ, types or {}, cat), "src": "readsb"} keep = ~ground breaks = ((flags & 2) > 0) | ((flags & 1) > 0) sub = {"t": tt[idx], "lat": lat[idx], "lon": lon[idx], "alt_baro": alt_b, "alt_geom": alt_g, "gs": gs, "trk": trk, "vr": vr} # remove ground points, then split sub = {k: v[keep] for k, v in sub.items()} brk = breaks[keep] if sub["t"].size < 3: continue for s, e in _segments(sub["t"], brk, max_gap): seg = {k: v[s:e] for k, v in sub.items()} acc.add_segment(dict(meta), seg) nseg += 1 return acc def ingest_csv(path, t0, t1, bbox=None, fmt="generic", max_gap=45.0, types=None, acc=None): """fmt='opensky' expects state_vectors_data4 columns (time, icao24, lat, lon, velocity, heading, vertrate, baroaltitude, geoaltitude, onground). fmt='generic': hex,t,lat,lon,alt_m,gs_ms,track_deg[,vr_ms,type,alt_kind] (alt_m MSL).""" acc = acc or _Acc() rows = {} with open(path, newline="", encoding="utf-8") as fh: for r in csv.DictReader(fh): if fmt == "opensky": if str(r.get("onground", "")).lower() in ("true", "1", "t"): continue key = r["icao24"] rec = (float(r["time"]), _num(r["lat"]), _num(r["lon"]), _num(r.get("baroaltitude")), _num(r.get("geoaltitude")), _num(r.get("velocity")), _num(r.get("heading")), _num(r.get("vertrate")), "") else: key = r["hex"] rec = (float(r["t"]), _num(r["lat"]), _num(r["lon"]), np.nan, _num(r["alt_m"]), _num(r.get("gs_ms")), _num(r.get("track_deg")), _num(r.get("vr_ms")), r.get("type", "")) if not (t0 <= rec[0] <= t1) or not np.isfinite(rec[1]): continue if bbox and not (bbox[0] - 0.15 <= rec[1] <= bbox[1] + 0.15 and bbox[2] - 0.15 <= rec[2] <= bbox[3] + 0.15): continue rows.setdefault(key, []).append(rec) for key, recs in rows.items(): a = np.array([x[:8] for x in recs], float) a = a[np.argsort(a[:, 0])] typ = recs[0][8] or "" meta = {"hex": key, "type": typ, "reg": "", "category": "", "desc": (types or {}).get(typ.upper(), ""), "cls": classify(typ, types or {}), "src": fmt} sub = {"t": a[:, 0], "lat": a[:, 1], "lon": a[:, 2], "alt_baro": a[:, 3], "alt_geom": a[:, 4], "gs": a[:, 5], "trk": a[:, 6], "vr": a[:, 7]} if fmt == "generic": # generic alt_m is already MSL: store as geom with N=0 handled by caller sub["alt_baro"] = np.full(a.shape[0], np.nan) for s, e in _segments(sub["t"], np.zeros(a.shape[0], bool), max_gap): acc.add_segment(dict(meta), {k: v[s:e] for k, v in sub.items()}) return acc def _num(x): try: return float(x) except (TypeError, ValueError): return np.nan def finalize(acc, alt_source="auto", qnh_hpa=1013.25, geoid_n=0.0): """Choose the altitude used for geometry: h_msl (m). auto: geometric (HAE - N) where present, else baro corrected by QNH.""" cols, flights = acc.finish() h_geom = cols["alt_geom"] - geoid_n h_baro = cols["alt_baro"] + (qnh_hpa - 1013.25) * HPA_TO_M if alt_source == "geom": h = h_geom elif alt_source == "baro": h = h_baro else: h = np.where(np.isfinite(h_geom), h_geom, h_baro) cols["h"] = h ok = np.isfinite(h) & np.isfinite(cols["lat"]) & np.isfinite(cols["lon"]) cols = {k: v[ok] for k, v in cols.items()} return cols, flights def save_tracks(path, cols, flights, meta): np.savez_compressed( path, fid=cols["fid"], t=cols["t"], lat=cols["lat"], lon=cols["lon"], h=cols["h"].astype(np.float32), alt_baro=cols["alt_baro"].astype(np.float32), alt_geom=cols["alt_geom"].astype(np.float32), gs=cols["gs"].astype(np.float32), trk=cols["trk"].astype(np.float32), vr=cols["vr"].astype(np.float32), f_hex=np.array([f["hex"] for f in flights]), f_type=np.array([f["type"] for f in flights]), f_reg=np.array([f["reg"] for f in flights]), f_cls=np.array([f["cls"] for f in flights]), f_desc=np.array([f["desc"] for f in flights]), f_cat=np.array([f["category"] for f in flights]), meta=json.dumps(meta), )skills/en/audio-flyover-geolocation/scripts/afg/audio.py37 KBb095a620fffaupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/scripts/afg/common.py2.1 KB7060fd66552aupstream ↗
"""Shared constants and small helpers.""" import datetime as _dt import json import sys import numpy as np FT = 0.3048 # m per foot KT = 0.514444 # m/s per knot FPM = 0.00508 # m/s per foot-per-minute HPA_TO_M = 8.3 # metres of pressure-altitude per hPa near sea level (~27 ft/hPa) def log(*args): print("[afg]", *args, file=sys.stderr, flush=True) def parse_time(value): """Parse ISO-8601 (assumed UTC if naive) or unix seconds -> float unix seconds.""" if value is None: return None if isinstance(value, (int, float, np.floating, np.integer)): return float(value) s = str(value).strip() try: return float(s) except ValueError: pass s = s.replace("Z", "+00:00") t = _dt.datetime.fromisoformat(s) if t.tzinfo is None: t = t.replace(tzinfo=_dt.timezone.utc) return t.timestamp() def iso(t): if t is None or not np.isfinite(t): return None return _dt.datetime.fromtimestamp(float(t), _dt.timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z") class _Enc(json.JSONEncoder): def default(self, o): if isinstance(o, np.integer): return int(o) if isinstance(o, np.floating): return None if not np.isfinite(o) else float(o) if isinstance(o, np.ndarray): return o.tolist() if isinstance(o, np.bool_): return bool(o) return super().default(o) def dump_json(obj, path): with open(path, "w", encoding="utf-8") as fh: json.dump(obj, fh, cls=_Enc, indent=2, ensure_ascii=False) def load_json(path): with open(path, "r", encoding="utf-8") as fh: return json.load(fh) def parse_bbox(s): """'lat_min,lat_max,lon_min,lon_max' -> tuple of floats (validated).""" if s is None: return None v = [float(x) for x in str(s).split(",")] if len(v) != 4: raise ValueError("bbox must be lat_min,lat_max,lon_min,lon_max") la0, la1, lo0, lo1 = v if la0 > la1: la0, la1 = la1, la0 if lo0 > lo1: lo0, lo1 = lo1, lo0 return (la0, la1, lo0, lo1)skills/en/audio-flyover-geolocation/scripts/afg/dem.py4.5 KB753c28d31775upstream ↗
"""Ground elevation for candidate observer points. The aircraft altitude in ADS-B is above sea level, the observer is on the ground: slant distance needs ground elevation at every grid point. Use a real DEM (Copernicus GLO-30 / USGS 3DEP / SRTM). A constant elevation is accepted for flat areas or tests but is reported as a precision downgrade. """ import math import os import urllib.error import urllib.request from collections import OrderedDict import numpy as np from .common import log COP30_URL = "https://copernicus-dem-30m.s3.amazonaws.com/{name}/{name}.tif" def cop30_tile_name(lat_floor, lon_floor): ns = "N" if lat_floor >= 0 else "S" ew = "E" if lon_floor >= 0 else "W" return f"Copernicus_DSM_COG_10_{ns}{abs(lat_floor):02d}_00_{ew}{abs(lon_floor):03d}_00_DEM" def fetch_copernicus(bbox, outdir): """Download Copernicus GLO-30 tiles (public AWS open data, no account) covering bbox. Ocean-only tiles do not exist (HTTP 404) and are skipped (treated as 0 m).""" la0, la1, lo0, lo1 = bbox os.makedirs(outdir, exist_ok=True) paths = [] for la in range(math.floor(la0), math.floor(la1) + 1): for lo in range(math.floor(lo0), math.floor(lo1) + 1): name = cop30_tile_name(la, lo) dst = os.path.join(outdir, name + ".tif") if not os.path.exists(dst): url = COP30_URL.format(name=name) try: log("downloading", url) urllib.request.urlretrieve(url, dst + ".part") os.replace(dst + ".part", dst) except urllib.error.HTTPError as e: if e.code in (403, 404): log(f" no tile {name} (ocean?) - treated as sea level") continue raise paths.append(dst) return paths class ElevationModel: """Bilinear sampling over one or more GeoTIFF tiles (lazy, LRU-cached), or a constant. Points outside every tile get `fill` (default 0 = sea).""" def __init__(self, paths=None, constant=None, fill=0.0, cache_tiles=6): self.constant = constant self.fill = float(fill) self.tiles = [] self._cache = OrderedDict() self._cache_n = cache_tiles if constant is None: if not paths: raise ValueError("ElevationModel needs DEM paths or a constant") try: import rasterio # noqa: F401 except ImportError as e: raise ImportError("Reading a DEM needs `pip install rasterio` (or pass --ground-elev)") from e import rasterio for p in paths: with rasterio.open(p) as ds: b = ds.bounds self.tiles.append((p, b.left, b.bottom, b.right, b.top)) log(f"DEM: {len(self.tiles)} tile(s)") @property def is_constant(self): return self.constant is not None def _load(self, path): if path in self._cache: self._cache.move_to_end(path) return self._cache[path] import rasterio with rasterio.open(path) as ds: arr = ds.read(1).astype(np.float32) nod = ds.nodata tr = ds.transform if nod is not None: arr[arr == nod] = np.nan item = (arr, tr) self._cache[path] = item if len(self._cache) > self._cache_n: self._cache.popitem(last=False) return item def sample(self, lat, lon): lat = np.asarray(lat, dtype=np.float64) lon = np.asarray(lon, dtype=np.float64) if self.constant is not None: return np.full(lat.shape, float(self.constant)) out = np.full(lat.shape, np.nan) for path, l, b, r, t in self.tiles: m = np.isnan(out) & (lon >= l) & (lon <= r) & (lat >= b) & (lat <= t) if not m.any(): continue arr, tr = self._load(path) inv = ~tr col, row = inv * (lon[m], lat[m]) col = np.asarray(col) - 0.5 row = np.asarray(row) - 0.5 c0 = np.clip(np.floor(col).astype(int), 0, arr.shape[1] - 2) r0 = np.clip(np.floor(row).astype(int), 0, arr.shape[0] - 2) fc = np.clip(col - c0, 0, 1) fr = np.clip(row - r0, 0, 1) v = (arr[r0, c0] * (1 - fc) * (1 - fr) + arr[r0, c0 + 1] * fc * (1 - fr) + arr[r0 + 1, c0] * (1 - fc) * fr + arr[r0 + 1, c0 + 1] * fc * fr) out[m] = v out[np.isnan(out)] = self.fill return outskills/en/audio-flyover-geolocation/scripts/afg/geo.py4.6 KBf9afcd9dbf23upstream ↗
"""WGS84 geodesy, local ENU frames and the implicit near-uniform search grid. All slant distances in this toolkit are computed as straight-line ECEF distances between the aircraft (lat, lon, height) and the observer (lat, lon, ground elevation + mic height). Heights only need a *consistent* vertical datum (we use orthometric/MSL for both); the geoid offset cancels to < 1 mm over the ranges involved. """ import numpy as np WGS84_A = 6378137.0 WGS84_F = 1.0 / 298.257223563 WGS84_E2 = WGS84_F * (2.0 - WGS84_F) M_PER_DEG = 111320.0 # only used to lay out grid rows/cols, never for distances def geodetic_to_ecef(lat, lon, h): lat = np.radians(np.asarray(lat, dtype=np.float64)) lon = np.radians(np.asarray(lon, dtype=np.float64)) h = np.asarray(h, dtype=np.float64) s, c = np.sin(lat), np.cos(lat) n = WGS84_A / np.sqrt(1.0 - WGS84_E2 * s * s) x = (n + h) * c * np.cos(lon) y = (n + h) * c * np.sin(lon) z = (n * (1.0 - WGS84_E2) + h) * s return np.stack([x, y, z], axis=-1) def enu_rotation(lat0, lon0): """3x3 matrix whose rows are the E, N, U unit vectors expressed in ECEF.""" la, lo = np.radians(lat0), np.radians(lon0) return np.array([ [-np.sin(lo), np.cos(lo), 0.0], [-np.sin(la) * np.cos(lo), -np.sin(la) * np.sin(lo), np.cos(la)], [np.cos(la) * np.cos(lo), np.cos(la) * np.sin(lo), np.sin(la)], ]) def enu_vectors_to_ecef(lat, lon, ve, vn, vu): """Rotate local ENU vectors (arrays) at (lat, lon) into ECEF components.""" la, lo = np.radians(lat), np.radians(lon) sla, cla, slo, clo = np.sin(la), np.cos(la), np.sin(lo), np.cos(lo) vx = -slo * ve - sla * clo * vn + cla * clo * vu vy = clo * ve - sla * slo * vn + cla * slo * vu vz = cla * vn + sla * vu return np.stack([vx, vy, vz], axis=-1) def ecef_to_enu(xyz, lat0, lon0, h0=0.0): ref = geodetic_to_ecef(lat0, lon0, h0) return (np.asarray(xyz) - ref) @ enu_rotation(lat0, lon0).T def radii(lat): """Meridional (M) and prime-vertical (N) radii of curvature at lat (deg).""" s = np.sin(np.radians(lat)) w = np.sqrt(1.0 - WGS84_E2 * s * s) return WGS84_A * (1.0 - WGS84_E2) / w**3, WGS84_A / w def offset_latlon(lat0, lon0, east_m, north_m): """Small-offset (<~20 km) ENU -> lat/lon using local radii of curvature.""" m, n = radii(lat0) lat = lat0 + np.degrees(np.asarray(north_m) / m) lon = lon0 + np.degrees(np.asarray(east_m) / (n * np.cos(np.radians(lat0)))) return lat, lon def latlon_to_local_en(lat, lon, lat0, lon0): m, n = radii(lat0) north = np.radians(np.asarray(lat) - lat0) * m east = np.radians(np.asarray(lon) - lon0) * n * np.cos(np.radians(lat0)) return east, north def haversine_m(lat1, lon1, lat2, lon2): r = 6371008.8 p1, p2 = np.radians(lat1), np.radians(lat2) dp, dl = p2 - p1, np.radians(np.asarray(lon2) - np.asarray(lon1)) a = np.sin(dp / 2) ** 2 + np.cos(p1) * np.cos(p2) * np.sin(dl / 2) ** 2 return 2 * r * np.arcsin(np.sqrt(np.clip(a, 0, 1))) class Grid: """Global implicit grid: rows of constant latitude spaced `spacing_m`, per-row longitude spacing chosen so east-west spacing is also ~`spacing_m`. Node ids are stable int64s (row << 32 | col), so runs can be merged.""" LAT0 = -90.0 LON0 = -180.0 def __init__(self, spacing_m): self.g = float(spacing_m) self.dlat = self.g / M_PER_DEG def _dlon(self, lat): return self.g / (M_PER_DEG * np.maximum(np.cos(np.radians(lat)), 1e-6)) def nodes_in_bbox(self, lat_min, lat_max, lon_min, lon_max): k0 = int(np.floor((lat_min - self.LAT0) / self.dlat)) k1 = int(np.ceil((lat_max - self.LAT0) / self.dlat)) ids, lats, lons = [], [], [] for k in range(k0, k1 + 1): lat = self.LAT0 + k * self.dlat if lat < lat_min - self.dlat or lat > lat_max + self.dlat: continue dl = self._dlon(lat) m0 = int(np.floor((lon_min - self.LON0) / dl)) m1 = int(np.ceil((lon_max - self.LON0) / dl)) m = np.arange(m0, m1 + 1, dtype=np.int64) ids.append((np.int64(k) << 32) | m) lats.append(np.full(m.size, lat)) lons.append(self.LON0 + m * dl) if not ids: return np.zeros(0, np.int64), np.zeros(0), np.zeros(0) return np.concatenate(ids), np.concatenate(lats), np.concatenate(lons) def node_latlon(self, ids): ids = np.asarray(ids, dtype=np.int64) k = ids >> 32 m = ids & 0xFFFFFFFF lat = self.LAT0 + k * self.dlat return lat, self.LON0 + m * self._dlon(lat)skills/en/audio-flyover-geolocation/scripts/afg/physics.py15 KBb702a8bfb1afupstream ↗
"""Acoustic Doppler physics for an aircraft passing a fixed microphone. Conventions ----------- tau emission time (when the aircraft produced the sound) t reception time (when the microphone heard it): t = tau + R(tau)/c R observer-aircraft slant range; Rdot = dR/dtau (>0 receding) f0 rest (source) frequency of the tone being tracked (any harmonic) Exact relation for a stationary observer in still air, evaluated at the EMISSION time (this is the point the video stresses - using the radial velocity at reception time produces a wrong curve, not just a delayed one): f_obs(t) = f0 / (1 + Rdot(tau)/c) ( = f0 * dtau/dt ) With a uniform wind w the ground-frame propagation speed along the ray is c_eff = c + w.n (n = unit vector source->observer); replace c by c_eff. Linear constant-velocity pass (speed v, CPA slant distance d) closed form, with tc = reception time of the sound emitted AT the CPA (where f = f0): T = t - (tc - d/c) s = [c^2 T - sqrt(c^2 v^2 T^2 + d^2 (c^2 - v^2))] / (c^2 - v^2) (tau - tau_cpa) R = c (T - s), Rdot = v^2 s / R, f = f0 / (1 + Rdot/c) Useful identities: f(-inf) = f0/(1 - v/c), f(+inf) = f0/(1 + v/c) v = c (f_hi - f_lo)/(f_hi + f_lo), f0 = 2 f_hi f_lo/(f_hi + f_lo) df/dt at tc = -f0 v^2 / (c d) -> d = f0 v^2 / (c |slope|) v sets the size of the frequency drop, d sets how fast the drop happens. """ import numpy as np from scipy.optimize import least_squares def sound_speed(temp_c=15.0): """Speed of sound in dry air (m/s); humidity adds < 0.5 %.""" return 20.0468 * np.sqrt(np.asarray(temp_c, dtype=float) + 273.15) def doppler_linear(t, f0, v, d, tc, c=343.0): t = np.asarray(t, dtype=float) tau_c = tc - d / c T = t - tau_c c2, v2 = c * c, v * v s = (c2 * T - np.sqrt(c2 * v2 * T * T + d * d * (c2 - v2))) / (c2 - v2) R = c * (T - s) rdot = v2 * s / np.maximum(R, 1e-9) return f0 / (1.0 + rdot / c) def doppler_split(t, f0, v_in, v_out, d, tc, c=343.0): """Linear pass with a different speed before (v_in) and after (v_out) the CPA. Cheap stand-in for speed/climb changes during the pass (departures, arrivals, power changes): each side of the curve gets its own asymptote while f0, d and tc stay shared.""" t = np.asarray(t, dtype=float) return np.where(t < tc, doppler_linear(t, f0, v_in, d, tc, c), doppler_linear(t, f0, v_out, d, tc, c)) def cpa_slope(f0, v, d, c=343.0): return -f0 * v * v / (c * d) def doppler_from_track(tau, pos, vel, obs, f0=1.0, c=343.0, wind=None): """Exact emission-time model along an arbitrary sampled trajectory. tau: (N,) emission times; pos/vel: (N,3) aircraft position/velocity in any Cartesian frame (ECEF or ENU); obs: (3,) observer position (same frame). wind: optional (3,) wind vector in the same frame (m/s). Returns reception times t (N,), observed frequency (N,), slant range (N,). """ D = pos - obs R = np.linalg.norm(D, axis=1) rdot = np.einsum("ij,ij->i", D, vel) / np.maximum(R, 1e-9) if wind is None: ceff = np.full(R.shape, c) else: n = -D / np.maximum(R, 1e-9)[:, None] ceff = c + n @ np.asarray(wind, dtype=float) t = tau + R / ceff return t, f0 / (1.0 + rdot / ceff), R # ---------------------------------------------------------------------------- # Fitting a traced curve # ---------------------------------------------------------------------------- def _initial_guess(t, f, c): o = np.argsort(t) t, f = t[o], f[o] q = max(3, len(t) // 6) f_hi, f_lo = np.median(f[:q]), np.median(f[-q:]) if f_hi <= f_lo: raise ValueError("traced curve does not decrease in frequency - not a single fly-by " "(check the box/anchors, or the event may be two overlapping aircraft)") f0 = 2 * f_hi * f_lo / (f_hi + f_lo) v = c * (f_hi - f_lo) / (f_hi + f_lo) below = np.where(f <= f0)[0] tc = t[below[0]] if below.size else t[len(t) // 2] sel = np.abs(f - f0) < 0.25 * (f_hi - f_lo) slope = None if sel.sum() >= 4: slope = np.polyfit(t[sel], f[sel], 1)[0] if slope is None or slope >= 0: slope = -(f_hi - f_lo) / max(t[-1] - t[0], 1.0) * 3 d = f0 * v * v / (c * abs(slope)) return np.array([f0, v, np.clip(d, 50, 30000), tc]) def corr_time(res): """Integrated autocorrelation time of time-ordered residuals (1 = independent). 1 + 2 * sum of the autocorrelation over lags, summed until it first drops below 0.05. n / corr_time is the effective number of independent points.""" r = np.asarray(res, float) r = r - r.mean() if r.size else r v = float(np.dot(r, r)) if r.size < 10 or v <= 0: return 1.0 tau = 1.0 for k in range(1, r.size // 4): rho = float(np.dot(r[:-k], r[k:])) / v if rho < 0.05: break tau += 2.0 * rho return tau def _inflation(res_sorted, corr_factor): """Variance inflation for correlated residuals: the larger of the frame-overlap factor (window/hop) and the measured residual correlation time. The two describe the same correlation, so they are not multiplied.""" tau = corr_time(res_sorted) return max(corr_factor**2, tau), tau def fit_linear(t, f, c=343.0, weights=None, corr_factor=1.0, robust=True): """Fit the closed-form linear-pass model to traced (t, f) points. corr_factor: sqrt(window/hop) of the spectrogram - adjacent frames are not independent, so formal sigmas are inflated by this factor. When the residuals stay correlated for longer than the window (a turn, a speed or power change, or a tracer that follows a smooth wrong path), the measured correlation time is used instead (see `corr_time`). Returns dict with params, 1-sigma errors, residual diagnostics. """ t = np.asarray(t, float) f = np.asarray(f, float) w = np.ones_like(t) if weights is None else np.asarray(weights, float) x0 = _initial_guess(t, f, c) def resid(x, tt, ff, ww): return (doppler_linear(tt, x[0], x[1], x[2], x[3], c) - ff) * ww lo = [0.5 * f.min(), 1.0, 5.0, t.min() - 600] hi = [2.0 * f.max(), 0.85 * c, 60000.0, t.max() + 600] best = None for vs in (1.0, 1.25, 1.6, 0.8): for ds in (1.0, 0.5, 2.0): xi = x0.copy() xi[1] = np.clip(xi[1] * vs, lo[1] + 1e-3, hi[1] - 1e-3) xi[2] = np.clip(xi[2] * ds, lo[2] + 1e-3, hi[2] - 1e-3) xi[0] = np.clip(xi[0], lo[0] + 1e-6, hi[0] - 1e-6) xi[3] = np.clip(xi[3], lo[3] + 1e-3, hi[3] - 1e-3) try: r = least_squares(resid, xi, args=(t, f, w), bounds=(lo, hi), loss="soft_l1" if robust else "linear", f_scale=max(0.2, 0.01 * np.median(f) * 0.05), x_scale=[max(0.01 * x0[0], 0.05), 5.0, 100.0, 1.0]) except Exception: continue if best is None or r.cost < best.cost: best = r if best is None: raise RuntimeError("fit failed") x = best.x res = doppler_linear(t, *x, c) - f mad = 1.4826 * np.median(np.abs(res - np.median(res))) + 1e-9 inl = np.abs(res) < 4 * mad # refit on inliers with linear loss for honest covariance r2 = least_squares(resid, x, args=(t[inl], f[inl], w[inl]), bounds=(lo, hi), loss="linear", x_scale=[max(0.01 * x0[0], 0.05), 5.0, 100.0, 1.0]) x = r2.x res_in = doppler_linear(t[inl], *x, c) - f[inl] n, p = inl.sum(), 4 s2 = float(np.sum((res_in * w[inl]) ** 2) / max(n - p, 1)) infl, tau = _inflation(res_in[np.argsort(t[inl], kind="stable")], corr_factor) J = r2.jac try: cov = np.linalg.inv(J.T @ J) * s2 * infl sig = np.sqrt(np.clip(np.diag(cov), 0, None)) except np.linalg.LinAlgError: cov = np.full((4, 4), np.nan) sig = np.full(4, np.nan) rho_vd = float(cov[1, 2] / np.sqrt(cov[1, 1] * cov[2, 2])) if np.all(np.isfinite(cov)) and cov[1, 1] > 0 and cov[2, 2] > 0 else np.nan res_all = doppler_linear(t, *x, c) - f left = t < x[3] rms_l = float(np.sqrt(np.mean(res_all[left & inl] ** 2))) if (left & inl).any() else np.nan rms_r = float(np.sqrt(np.mean(res_all[~left & inl] ** 2))) if (~left & inl).any() else np.nan return { "f0": x[0], "v": x[1], "d": x[2], "t_cpa_rx": x[3], "sigma": {"f0": sig[0], "v": sig[1], "d": sig[2], "t_cpa_rx": sig[3]}, "cov": cov, "rms_hz": float(np.sqrt(np.mean(res_in**2))), "rms_left_hz": rms_l, "rms_right_hz": rms_r, "n_points": int(len(t)), "n_inliers": int(n), "resid_corr_time": float(tau), "sigma_inflation": float(np.sqrt(infl)), "n_eff": float(n / infl), "corr_v_d": rho_vd, "f_hi_asym": x[0] / (1 - x[1] / c), "f_lo_asym": x[0] / (1 + x[1] / c), "cpa_slope_hz_s": cpa_slope(x[0], x[1], x[2], c), "c": c, } def fit_split(t, f, lin, c=343.0, corr_factor=1.0): """Refit with separate approach/recede speeds, starting from the linear fit. Returns a fit-like dict (v = mean of the two, v_in/v_out, d, t_cpa_rx, ...). A large |v_out - v_in| means the aircraft changed speed or climb rate across the pass, which biases the constant-speed v (and d ~ v^2) - the search windows should then cover both fits.""" t = np.asarray(t, float) f = np.asarray(f, float) res_l = doppler_linear(t, lin["f0"], lin["v"], lin["d"], lin["t_cpa_rx"], c) - f mad = 1.4826 * np.median(np.abs(res_l - np.median(res_l))) + 1e-9 inl = np.abs(res_l) < 5 * mad tt, ff = t[inl], f[inl] def resid(x): return doppler_split(tt, *x, c=c) - ff lo = [0.5 * f.min(), 1.0, 1.0, 5.0, t.min() - 600] hi = [2.0 * f.max(), 0.85 * c, 0.85 * c, 60000.0, t.max() + 600] x0 = np.clip([lin["f0"], lin["v"], lin["v"], lin["d"], lin["t_cpa_rx"]], np.add(lo, 1e-6), np.subtract(hi, 1e-6)) try: r = least_squares(resid, x0, bounds=(lo, hi), x_scale=[max(0.01 * lin["f0"], 0.05), 5.0, 5.0, 100.0, 1.0]) except Exception: return None x = r.x n, p = inl.sum(), 5 rr = resid(x) s2 = float(np.sum(rr**2) / max(n - p, 1)) infl, _ = _inflation(rr[np.argsort(tt, kind="stable")], corr_factor) try: cov = np.linalg.inv(r.jac.T @ r.jac) * s2 * infl sig = np.sqrt(np.clip(np.diag(cov), 0, None)) except np.linalg.LinAlgError: sig = np.full(5, np.nan) v_lo, v_hi = sorted((x[1], x[2])) return {"f0": x[0], "v": 0.5 * (x[1] + x[2]), "v_in": x[1], "v_out": x[2], "d": x[3], "t_cpa_rx": x[4], "v_span": (v_lo, v_hi), "sigma": {"f0": sig[0], "v": float(np.hypot(sig[1], sig[2]) / 2 + 0.5 * (v_hi - v_lo)), "v_in": sig[1], "v_out": sig[2], "d": sig[3], "t_cpa_rx": sig[4]}, "rms_hz": float(np.sqrt(np.mean(rr**2))), "n_inliers": int(n), "c": c} def search_constraints(fit, v_rel_margin=0.20, d_rel_margin=0.15, t_margin_s=4.0, c_rel_uncert=0.04, nsig=3.0, k_rel_margin=0.08, alt=None): """Turn a fit into deliberately generous search windows. Key point: v and d are individually ill-determined when the curve's tails (asymptotes) are faint or the aircraft changed speed/climb during the pass, but their combination k = v^2/d = c |slope_at_CPA| / f0 is pinned by the steep middle of the curve. So the windows are: a WIDE speed window, a tight k window, and the d window implied by both. In the search each ADS-B candidate's own ground speed then predicts its d = gs^2 / k. Sound-speed error scales v, d and k by the same factor (c_rel_uncert). alt: optional second fit (e.g. fit_split); the windows become the union. """ if alt is not None: a = search_constraints(fit, v_rel_margin, d_rel_margin, t_margin_s, c_rel_uncert, nsig, k_rel_margin) b = search_constraints(alt, v_rel_margin, d_rel_margin, t_margin_s, c_rel_uncert, nsig, k_rel_margin) lo_t = min(a["t_rec"] - a["t_tol"], b["t_rec"] - b["t_tol"]) hi_t = max(a["t_rec"] + a["t_tol"], b["t_rec"] + b["t_tol"]) for key in ("v_min", "d_min", "k_min"): a[key] = min(a[key], b[key]) for key in ("v_max", "d_max", "k_max"): a[key] = max(a[key], b[key]) a.update(t_rec=0.5 * (lo_t + hi_t), t_tol=0.5 * (hi_t - lo_t)) return a v, d, s = fit["v"], fit["d"], fit["sigma"] dv = max(nsig * np.nan_to_num(s["v"], nan=v * 0.1), v * v_rel_margin) + v * c_rel_uncert k = v * v / d cov = fit.get("cov") sk = np.nan if cov is not None and np.all(np.isfinite(cov[1:3, 1:3])): J = np.array([2 * v / d, -v * v / (d * d)]) sk = float(np.sqrt(max(J @ cov[1:3, 1:3] @ J, 0.0))) dk = max(nsig * np.nan_to_num(sk, nan=0.1 * k), k * k_rel_margin) + k * c_rel_uncert v_min, v_max = max(v - dv, 5.0), v + dv k_min, k_max = max(k - dk, 1e-3), k + dk dd = max(nsig * np.nan_to_num(s["d"], nan=d * 0.3), d * d_rel_margin) + d * c_rel_uncert dt = max(nsig * np.nan_to_num(s["t_cpa_rx"], nan=t_margin_s), t_margin_s) return { "v_min": v_min, "v_max": v_max, "k_min": k_min, "k_max": k_max, "d_min": max(min(d - dd, v_min**2 / k_max), 30.0), "d_max": max(d + dd, v_max**2 / k_min), "t_rec": fit["t_cpa_rx"], "t_tol": dt, # the distance the fit itself supports (+-nsig sigma), for comparison with the search window above, # which follows from the speed and k windows and is deliberately much wider "d_fit_min": max(d - nsig * np.nan_to_num(s["d"], nan=d * 0.3), 0.0), "d_fit_max": d + nsig * np.nan_to_num(s["d"], nan=d * 0.3), "f0": fit["f0"], "f0_rel_tol": max(0.004, 3 * np.nan_to_num(s["f0"], nan=0) / fit["f0"]), } def plausibility(fit, v_max=150.0, d_max=15000.0, rel_sigma_max=0.5): """Reasons a fit is NOT a usable low-level aircraft pass (empty list = ok). A tracer that wandered through noise or hopped harmonics typically yields v near c or d at the bound; feeding that to the search makes a corridor tens of km wide (slow and meaningless). Props/helicopters audible on the ground fly < ~110 m/s and pass within a few km. """ why = [] if not np.isfinite(fit["v"]) or fit["v"] > v_max: why.append(f"v={fit['v']:.0f} m/s > {v_max:.0f} (trace probably followed noise or switched harmonics, or " "covers only a short piece of the curve: if it rises before it falls, re-trace with --no-monotone)") if not np.isfinite(fit["d"]) or fit["d"] > d_max: why.append(f"d={fit['d']:.0f} m > {d_max:.0f} (no usable curvature)") for k in ("v", "d"): s = fit["sigma"][k] if np.isfinite(s) and s > rel_sigma_max * fit[k]: why.append(f"sigma_{k}/{k}={s / fit[k]:.2f} (poorly constrained)") if fit.get("n_inliers", 99) < 20: why.append(f"only {fit.get('n_inliers')} inlier points") return whyskills/en/audio-flyover-geolocation/scripts/afg/scoring.py22 KBb7601a2fd07dupstream ↗
"""Trajectory-exact Doppler modelling and locked multi-event scoring. For each candidate observer point the *exact* curve it would have heard is modelled from the real ADS-B trajectory (turns, speed and altitude changes included) using emission-time radial velocity, then laid on the measured spectrogram: * event 1 is anchored at its measured delayed-CPA recording time; a small time shift (delta) and a frequency scale (f0 measurement error / RPM) are swept to absorb measurement error; * event 2 uses the SAME delta - its placement relative to event 1 is fixed purely by flight geometry ('locked' scoring), only its own f0 scale is swept. A wrong location therefore cannot fix a wrong inter-event offset. Brightness score = mean of clipped whitened dB under the modelled curve (and selected harmonics) over a fixed window; 'trace' score = Gaussian closeness to hand/auto-traced points (better for very faint curves). Bright texture under a wrong curve (speech, music, wind) can lift a brightness score, so `trace_check` then tests whether each hypothesis' model curve actually reproduces the traced points. """ import numpy as np from .audio import sample_spec from .common import log from .geo import geodetic_to_ecef, offset_latlon from .physics import doppler_from_track, doppler_linear class CurveModel: def __init__(self, R, c=343.0, wind_ecef=None): self.R, self.c, self.wind = R, c, wind_ecef def curve(self, fid, obs, t_emit_center, half_s=100.0): R = self.R a, b = R.flight_slice(fid, t_emit_center - half_s - 40, t_emit_center + half_s + 40) if b - a < 5: return None tau = R.t[a:b] t_rx, ratio, rng = doppler_from_track(tau, R.ecef[a:b], R.vel[a:b], obs, 1.0, self.c, self.wind) k = int(np.argmin(rng)) if k == 0 or k == len(rng) - 1: t_cpa_rx = t_rx[k] edge = True else: y = rng[k - 1:k + 2] ** 2 den = y[0] - 2 * y[1] + y[2] dl = 0.5 * (y[0] - y[2]) / den if den > 0 else 0.0 t_cpa_rx = np.interp(k + dl, np.arange(len(t_rx)), t_rx) edge = False return {"t_rx": t_rx, "ratio": ratio, "rng": rng, "t_cpa_rx": t_cpa_rx, "d_cpa": float(rng.min()), "edge": edge} def _event_scores(spec, cur, off, rec_center, f0, half_w, step, deltas, scales, harmonics, clip=(0.0, 30.0)): rec = rec_center + np.arange(-half_w, half_w + step / 2, step) utc = rec[None, :] + off + deltas[:, None] r = np.interp(utc, cur["t_rx"], cur["ratio"], left=np.nan, right=np.nan) valid = np.isfinite(r) r = np.where(valid, r, 0.0) h = np.asarray(harmonics, float) F = f0 * scales[:, None, None, None] * h[None, None, None, :] * r[None, :, :, None] T = np.broadcast_to(rec[None, None, :, None], F.shape) v = sample_spec(spec, T, F, clip_db=clip) v = np.where(valid[None, :, :, None], v, 0.0) return v.mean(axis=(2, 3)) # (n_scales, n_deltas) def _trace_scores(trace, cur, off, f0, deltas, scales, sigma_hz): tr_t, tr_f = trace utc = tr_t[None, :] + off + deltas[:, None] r = np.interp(utc, cur["t_rx"], cur["ratio"], left=np.nan, right=np.nan) valid = np.isfinite(r) fm = f0 * scales[:, None, None] * np.where(valid, r, 0)[None] g = np.exp(-0.5 * ((fm - tr_f[None, None, :]) / sigma_hz) ** 2) g = np.where(valid[None], g, 0.0) return g.mean(axis=2) def score_point(model, spec, events, obs, fids, t_cpas, cfg, traces=None, why=None): """Score one observer point for one assignment of flights to events. Returns dict (total, per-event scores, delta, scales, rec_start_utc) or None. why: optional dict; the reason for a None is counted in it ('no_track': too little ADS-B around the CPA, 'clock': every time shift lies outside the known recording clock).""" curves = [] for fid, tc in zip(fids, t_cpas): cu = model.curve(int(fid), obs, float(tc), cfg["half_w"] + 20) if cu is None: if why is not None: why["no_track"] = why.get("no_track", 0) + 1 return None curves.append(cu) off0 = curves[0]["t_cpa_rx"] - events[0]["t_rec"] # rec -> utc offset anchoring event 1 deltas, scales = cfg["deltas"], cfg["scales"] per = [] for k, (ev, cu) in enumerate(zip(events, curves)): rec_center = ev["t_rec"] if k == 0 else cu["t_cpa_rx"] - off0 if traces is not None and traces[k] is not None: S = _trace_scores(traces[k], cu, off0, ev["f0"], deltas, scales, cfg["trace_sigma_hz"]) else: S = _event_scores(spec, cu, off0, rec_center, ev["f0"], cfg["half_w"], cfg["step"], deltas, scales, cfg["harmonics"]) per.append(S) best_s = [S.max(axis=0) for S in per] # per delta, best scale for each event if cfg.get("clock") is not None: # known recording clock: forbid shifts outside it rs, tol = cfg["clock"] bad = np.abs(off0 + deltas - rs) > tol if bad.all(): if why is not None: why["clock"] = why.get("clock", 0) + 1 return None best_s = [np.where(bad, -np.inf, b) for b in best_s] if cfg.get("lock_mode", "joint") == "e1": di = int(np.argmax(best_s[0])) else: di = int(np.argmax(np.sum(best_s, axis=0))) ev_scores = [float(b[di]) for b in best_s] sc = [float(scales[int(np.argmax(S[:, di]))]) for S in per] return {"total": float(np.sum(ev_scores)), "events": ev_scores, "delta": float(deltas[di]), "scales": sc, "rec_start_utc": float(off0 + deltas[di]), "d_cpa": [cu["d_cpa"] for cu in curves], "t_cpa_rx": [cu["t_cpa_rx"] for cu in curves]} def make_cfg(fine=True, half_w=60.0, delta_max=4.0, f_rel=0.006, harmonics=(1, 2), lock_mode="joint", trace_sigma_hz=0.6): if fine: return {"half_w": half_w, "step": 0.5, "deltas": np.arange(-delta_max, delta_max + 1e-9, 0.25), "scales": 1 + np.linspace(-f_rel, f_rel, 25), "harmonics": list(harmonics), "lock_mode": lock_mode, "trace_sigma_hz": trace_sigma_hz} return {"half_w": half_w, "step": 1.0, "deltas": np.arange(-delta_max, delta_max + 1e-9, 1.0), "scales": 1 + np.linspace(-f_rel, f_rel, 9), "harmonics": list(harmonics)[:1], "lock_mode": lock_mode, "trace_sigma_hz": trace_sigma_hz} def subgrid(lat, lon, spacing, n): o = np.arange(-n, n + 1) * spacing E, N = np.meshgrid(o, o) la, lo = offset_latlon(lat, lon, E.ravel(), N.ravel()) return la, lo def effective_delta_max(events, delta_max=4.0, cap=30.0): """Time-shift range for scoring: at least event 1's CPA-time tolerance t_tol (the funnel admits candidates that far off), in whole seconds, at most cap.""" t_tol = float(events[0].get("t_tol", 0.0)) if events else 0.0 return float(min(max(delta_max, np.ceil(t_tol - 1e-9)), max(cap, delta_max))) def score_candidates(cand, R, spec, events, dem, c=343.0, mic_height=2.0, top_pairs=40, nodes_per_pair=4, sub_m=100.0, sub_n=1, traces=None, harmonics=(1, 2), lock_mode="joint", half_w=60.0, delta_max=4.0, f_rel=0.006, clock=None, min_step_m=25.0, max_iter=8, info=None): """clock=(rec_start_utc, tol_s) restricts the time shift to the known recording clock. The time shift spans +-max(delta_max, event 1's t_tol), the same CPA-time tolerance the funnel used. Each kept node is refined by a shrinking 3x3 hill-climb (sub_m -> min_step_m). info: optional dict filled with counts (unique candidates, scored, and why the others were not).""" model = CurveModel(R, c) nev = len(events) dm = effective_delta_max(events, delta_max) if dm > delta_max: log(f"scoring: time shift widened to +-{dm:.0f} s to cover event 1's CPA-time tolerance " f"({events[0]['t_tol']:.1f} s)") delta_max = dm coarse = make_cfg(False, half_w, delta_max, f_rel, harmonics, lock_mode) fine = make_cfg(True, half_w, delta_max, f_rel, harmonics, lock_mode) coarse["clock"] = fine["clock"] = clock # unique (node, fid1, t1, fid2, t2) key = np.c_[cand["node_id"], cand["fid1"], np.round(cand["t1_cpa"]), cand["fid2"], np.round(np.nan_to_num(cand["t2_cpa"]))] _, ui = np.unique(key, axis=0, return_index=True) cand = cand[np.sort(ui)] log(f"scoring: {len(cand)} unique candidates (coarse pass)") cs = np.full(len(cand), -np.inf) why = {} for i, cd in enumerate(cand): obs = geodetic_to_ecef(cd["lat"], cd["lon"], cd["elev"] + mic_height) fids = [cd["fid1"]] + ([cd["fid2"]] if nev > 1 else []) tcs = [cd["t1_cpa"]] + ([cd["t2_cpa"]] if nev > 1 else []) r = score_point(model, spec, events, obs, fids, tcs, coarse, traces, why) if r is not None: cs[i] = r["total"] if i and i % 2000 == 0: log(f" coarse {i}/{len(cand)}") if info is not None: info.update(unique_candidates=int(len(cand)), scored=int(np.isfinite(cs).sum()), rejected_clock=int(why.get("clock", 0)), no_track=int(why.get("no_track", 0)), delta_max_s=float(delta_max), clock=None if clock is None else {"rec_start": float(clock[0]), "tol_s": float(clock[1])}) if why.get("clock"): log(f"scoring: {why['clock']} of {len(cand)} candidates dropped by the recording clock: their implied start " f"is more than {clock[1]:.0f} s (+-{delta_max:.0f} s shift) from --rec-start - give score the same " "--rec-start-tol as search, or widen it") if why.get("no_track"): log(f"scoring: {why['no_track']} candidates without enough ADS-B samples around the CPA") # keep the best node per flight pair plus spatially distinct runners-up # (a straight pass is mirror-symmetric about its track: both sides must survive) pair = cand["fid1"].astype(np.int64) * 10_000_000 + (cand["fid2"].astype(np.int64) + 1) keep = [] for p in np.unique(pair): idx = np.flatnonzero((pair == p) & np.isfinite(cs)) idx = idx[np.argsort(-cs[idx])] keep += _diverse(idx, cand["lat"], cand["lon"], nodes_per_pair, 400.0) keep = np.array(keep, int) if keep.size == 0: return [], cand, cs pbest = {} for i in keep: pbest[pair[i]] = max(pbest.get(pair[i], -np.inf), cs[i]) top = sorted(pbest, key=lambda p: -pbest[p])[:top_pairs] sel = [i for i in keep if pair[i] in set(top)] log(f"scoring: fine pass on {len(sel)} nodes from {len(top)} flight pairs") results = [] for i in sel: cd = cand[i] fids = [cd["fid1"]] + ([cd["fid2"]] if nev > 1 else []) tcs = [cd["t1_cpa"]] + ([cd["t2_cpa"]] if nev > 1 else []) best, step, seen = None, sub_m, {} clat, clon = float(cd["lat"]), float(cd["lon"]) for _ in range(max_iter): la, lo = subgrid(clat, clon, step, sub_n) el = dem.sample(la, lo) moved = False for a, b, e in zip(la, lo, el): kk = (round(a, 6), round(b, 6)) if kk not in seen: seen[kk] = score_point(model, spec, events, geodetic_to_ecef(a, b, e + mic_height), fids, tcs, fine, traces) r = seen[kk] if r is not None and (best is None or r["total"] > best["total"] + 1e-9): best, moved = dict(r, lat=float(a), lon=float(b), elev=float(e)), (a, b) != (clat, clon) if best is None: break clat, clon = best["lat"], best["lon"] if not moved: if step <= min_step_m: break step /= 2.0 if best: best.update(fid1=int(cd["fid1"]), fid2=int(cd["fid2"]), t1_cpa=float(cd["t1_cpa"]), t2_cpa=float(cd["t2_cpa"]) if nev > 1 else None, coarse=float(cs[i])) results.append(best) # hypotheses = spatially distinct maxima per flight pair (>= 500 m apart) hyps = [] for k in {(r["fid1"], r["fid2"]) for r in results}: rr = sorted([r for r in results if (r["fid1"], r["fid2"]) == k], key=lambda r: -r["total"]) la = np.array([r["lat"] for r in rr]) lo = np.array([r["lon"] for r in rr]) for j in _diverse(np.arange(len(rr)), la, lo, 3, 500.0): hyps.append(rr[j]) ranked = sorted(hyps, key=lambda r: -r["total"]) return ranked, cand, cs def trace_check(R, hyps, events, traces, fits, c=343.0, mic_height=2.0, half_w=60.0, f_rel=0.006, ratio_max=4.0, slack_hz=1.0): """Does each hypothesis' model curve reproduce the traced points? Adds hyp["trace_check"] and hyp["scale_at_bound"] in place. The model curve uses the hypothesis' own position, clock and f0 scale (as scored). It is compared with the event's traced points within +-half_w s of the fitted CPA (the stretch the brightness score sees; the tracer's far tails lean towards a straight pass when the aircraft turns), using the points the event's linear fit kept as inliers. Statistic: robust RMS, the square root of the median squared residual. The reference is the same statistic for the linear fit; an event passes when rms <= max(ratio_max * ref, ref + slack_hz). On the synthetic self-tests the true two-event answer stays within 1.9 x ref, and wrong aircraft pairs exceed 15 x ref on at least one event. The true pair at a site 2-6 km away reaches 2.7-15 x, so some of those pass; the brightness score still ranks them below the true site. One event alone is weaker: other aircraft with a similar speed and distance reproduce the curve too.""" model = CurveModel(R, c) nev = len(events) for r in hyps: fids = [r["fid1"]] + ([r["fid2"]] if nev > 1 else []) tcs = [r["t1_cpa"]] + ([r["t2_cpa"]] if nev > 1 else []) obs = geodetic_to_ecef(r["lat"], r["lon"], r["elev"] + mic_height) per = [] for k, (fid, tc, ev) in enumerate(zip(fids, tcs, events)): tr = traces[k] if traces is not None and k < len(traces) else None ft = fits[k] if fits is not None and k < len(fits) else None if tr is None or ft is None or len(tr[0]) < 10: per.append(None) continue t, f = np.asarray(tr[0], float), np.asarray(tr[1], float) res_l = doppler_linear(t, ft["f0"], ft["v"], ft["d"], ft["t_cpa_rx"], ft.get("c", c)) - f mad = 1.4826 * np.median(np.abs(res_l - np.median(res_l))) + 1e-9 use = (np.abs(res_l) < 4 * mad) & (np.abs(t - ft["t_cpa_rx"]) <= half_w) if use.sum() < 10: per.append(None) continue ref = max(float(np.sqrt(np.median(res_l[use] ** 2))), 0.05) cu = model.curve(int(fid), obs, float(tc), half_w + 60.0) fm = np.full(t.size, np.nan) if cu is not None: fm = np.interp(t, cu["t_rx"] - r["rec_start_utc"], ev["f0"] * r["scales"][k] * cu["ratio"], left=np.nan, right=np.nan) ok = use & np.isfinite(fm) cover = float(ok.sum() / use.sum()) if ok.sum() < 10 or cover < 0.5: # the model does not reach the traced CPA: wrong time or no track per.append({"rms_hz": None, "ref_hz": ref, "ratio": None, "n": int(ok.sum()), "coverage": cover, "ok": False}) continue rms = float(np.sqrt(np.median((fm[ok] - f[ok]) ** 2))) per.append({"rms_hz": rms, "ref_hz": ref, "ratio": rms / ref, "n": int(ok.sum()), "coverage": cover, "ok": bool(rms <= max(ratio_max * ref, ref + slack_hz))}) known = [p["ok"] for p in per if p is not None] r["trace_check"] = {"events": per, "consistent": (all(known) if known else None), "rule": f"robust rms <= max({ratio_max:g} x fit rms, fit rms + {slack_hz:g} Hz) " f"within +-{half_w:g} s of CPA"} r["scale_at_bound"] = [bool(abs(sc - 1.0) >= f_rel * (1 - 1e-6)) for sc in r["scales"]] return hyps def rank_consistent(hyps, nev): """Two events: hypotheses whose curves fail `trace_check` go below all the others (score order is kept inside each group). One event: unchanged - a single curve is reproduced by too many wrong aircraft for the check to rank.""" if nev < 2: return list(hyps) return sorted(hyps, key=lambda r: ((r.get("trace_check") or {}).get("consistent") is False, -r["total"])) def pass_geometry(R, best, nev, min_cross_deg=25.0): """Ground-track headings at CPA and, for two events, their crossing angle. Each pass pins the observer across its own track (curve shape) and, with a clock, along it. Two passes share one unknown clock, so the along-track direction is only broken if the tracks cross at a decent angle: near- parallel passes leave a strip along the tracks.""" out = {"headings_deg": []} for fid, tc in [(best["fid1"], best["t1_cpa"])] + ([(best["fid2"], best["t2_cpa"])] if nev > 1 else []): a, b = R.flight_slice(fid, tc - 1.5 * R.dt, tc + 1.5 * R.dt) if b <= a: out["headings_deg"].append(None) continue i = a + int(np.argmin(np.abs(R.t[a:b] - tc))) out["headings_deg"].append(float(np.degrees(np.arctan2(R.ve[i], R.vn[i])) % 360)) h = out["headings_deg"] if nev > 1 and None not in h: x = abs(h[0] - h[1]) % 180.0 out["crossing_angle_deg"] = float(min(x, 180.0 - x)) if out["crossing_angle_deg"] < min_cross_deg: out["warning"] = (f"tracks cross at only {out['crossing_angle_deg']:.0f} deg: position along the tracks is " "weakly constrained - report the heat-map strip, not a point") return out def score_region(cand, cs, frac=0.9, best=None): """Coarse-score plateau: every candidate node scoring >= frac * best. One straight, steady pass constrains distance-to-track and crossing time, but NOT where along the track you stood unless the recording clock is known: the plateau is then a strip ~2 * v * clock_tolerance long (plus the mirror strip on the other side). Report this extent instead of a point. """ from .geo import latlon_to_local_en sel = np.isfinite(cs) if best is not None: # only the best hypothesis' flight pair: other aircraft are separate hypotheses sel &= (cand["fid1"] == best["fid1"]) & (cand["fid2"] == best["fid2"]) ok = sel & (cs >= frac * np.max(cs[sel])) if sel.any() else np.zeros(len(cs), bool) la, lo = cand["lat"][ok], cand["lon"][ok] out = {"frac": frac, "n_nodes": int(ok.sum()), "lat": la, "lon": lo, "length_m": 0.0, "width_m": 0.0, "axis_deg": None} if la.size >= 2: e, n = latlon_to_local_en(la, lo, float(la.mean()), float(lo.mean())) X = np.c_[e - e.mean(), n - n.mean()] w, V = np.linalg.eigh(X.T @ X) p = X @ V # columns: minor, major axis out["length_m"] = float(np.ptp(p[:, 1])) out["width_m"] = float(np.ptp(p[:, 0])) out["axis_deg"] = float(np.degrees(np.arctan2(V[0, 1], V[1, 1])) % 180) # bearing of major axis return out def _diverse(idx_sorted, lat, lon, k, min_sep_m): from .geo import haversine_m picks = [] for i in idx_sorted: if all(haversine_m(lat[i], lon[i], lat[p], lon[p]) > min_sep_m for p in picks): picks.append(int(i)) if len(picks) >= k: break return picks def ambiguity(ranked, frac=0.9, min_sep_m=500.0): """Competing hypotheses scoring >= frac*best that are >= min_sep_m away (mirror side of a straight pass, or a different flight pair).""" from .geo import haversine_m if not ranked: return [] b = ranked[0] out = [] for r in ranked[1:]: dist = float(haversine_m(b["lat"], b["lon"], r["lat"], r["lon"])) if r["total"] >= frac * b["total"] and dist >= min_sep_m: out.append({"lat": r["lat"], "lon": r["lon"], "score_ratio": r["total"] / b["total"], "dist_m": dist, "same_flights": (r["fid1"], r["fid2"]) == (b["fid1"], b["fid2"])}) return out def heatmap(R, spec, events, dem, best, c=343.0, mic_height=2.0, radius_m=1500.0, step_m=30.0, traces=None, harmonics=(1, 2), lock_mode="joint", half_w=60.0, delta_max=4.0, f_rel=0.006, clock=None): """Dense local score map around the best candidate for its flight pair, with the same time-shift range and recording clock as `score_candidates`. Returns lat, lon grids and total / per-event score grids.""" model = CurveModel(R, c) cfg = make_cfg(True, half_w, effective_delta_max(events, delta_max), f_rel, harmonics, lock_mode) cfg["clock"] = clock cfg["scales"] = 1 + np.linspace(-f_rel, f_rel, 13) n = int(round(radius_m / step_m)) o = np.arange(-n, n + 1) * step_m E, N = np.meshgrid(o, o) la, lo = offset_latlon(best["lat"], best["lon"], E.ravel(), N.ravel()) el = dem.sample(la, lo) nev = len(events) fids = [best["fid1"]] + ([best["fid2"]] if nev > 1 else []) tcs = [best["t1_cpa"]] + ([best["t2_cpa"]] if nev > 1 else []) tot = np.full(la.size, np.nan) per = np.full((nev, la.size), np.nan) for i, (a, b, e) in enumerate(zip(la, lo, el)): r = score_point(model, spec, events, geodetic_to_ecef(a, b, e + mic_height), fids, tcs, cfg, traces) if r is not None: tot[i] = r["total"] per[:, i] = r["events"] shp = E.shape return la.reshape(shp), lo.reshape(shp), tot.reshape(shp), per.reshape((nev,) + shp), E, N def model_overlay(R, best, events, dem, c=343.0, mic_height=2.0, half_w=90.0): """Modelled curves (recording time, Hz) for plotting over the spectrogram.""" model = CurveModel(R, c) obs = geodetic_to_ecef(best["lat"], best["lon"], best["elev"] + mic_height) out = [] fids = [best["fid1"]] + ([best["fid2"]] if len(events) > 1 else []) tcs = [best["t1_cpa"]] + ([best["t2_cpa"]] if len(events) > 1 else []) for k, (fid, tc, ev) in enumerate(zip(fids, tcs, events)): cu = model.curve(int(fid), obs, float(tc), half_w) if cu is None: continue rec = cu["t_rx"] - best["rec_start_utc"] out.append({"t": rec, "f": ev["f0"] * best["scales"][k] * cu["ratio"], "label": f"event {k + 1} model"}) return outskills/en/audio-flyover-geolocation/scripts/afg/search.py17 KB8da11021225dupstream ↗
"""Candidate funnel (coarse grid, typically 200 m): A event-1 corridor: grid nodes near speed-matching flights whose 3D CPA slant distance and ground speed match event 1 (and its absolute time, if known) B offset filter: the same node must hear an event-2-matching flight whose delayed-CPA time is (t2_rec - t1_rec) +- tol after event 1 C silence filter: no other flight may pass close enough to be heard during the parts of the recording where no fly-by is visible 'Delayed CPA time' = t_cpa(emission) + d_cpa / c : the moment the observer hears the sound emitted at closest approach - exactly where the observed curve crosses f0 (zero radial velocity). That is what gets matched to the recording clock. """ import numpy as np from scipy.spatial import cKDTree from .common import log from .geo import Grid, geodetic_to_ecef, latlon_to_local_en EV_DTYPE = [("node", np.int64), ("j", np.int64), ("fid", np.int32), ("t_cpa", np.float64), ("t_rx", np.float64), ("d", np.float32), ("gs", np.float32), ("edge", np.bool_)] CAND_DTYPE = [("node_id", np.int64), ("lat", np.float64), ("lon", np.float64), ("elev", np.float32), ("fid1", np.int32), ("t1_cpa", np.float64), ("t1_rx", np.float64), ("d1", np.float32), ("v1", np.float32), ("fid2", np.int32), ("t2_cpa", np.float64), ("t2_rx", np.float64), ("d2", np.float32), ("v2", np.float32)] def cpa_events(node_ecef, R, sample_idx, radius, c=343.0, chunk=3000, tree=None): """All closest-point-of-approach events between nodes and the flights whose samples are listed in sample_idx (indices into R). A CPA = local minimum of slant range along a flight (refined with a parabola on R^2, exact for straight flight). Range minima at a track end are flagged edge=True. Returns structured array EV_DTYPE (node = row index into node_ecef).""" sample_idx = np.asarray(sample_idx, np.int64) if sample_idx.size == 0 or len(node_ecef) == 0: return np.zeros(0, EV_DTYPE) if tree is None: tree = cKDTree(R.ecef[sample_idx]) outs = [] for s in range(0, len(node_ecef), chunk): nodes = node_ecef[s:s + chunk] tn = cKDTree(nodes) M = tn.sparse_distance_matrix(tree, radius, output_type="ndarray") if M.size == 0: continue i = M["i"].astype(np.int64) j = sample_idx[M["j"]] d0 = M["v"] hp, hn = R.has_prev[j], R.has_next[j] jp = np.where(hp, j - 1, j) jn = np.where(hn, j + 1, j) dp = np.where(hp, np.linalg.norm(R.ecef[jp] - nodes[i], axis=1), np.inf) dn = np.where(hn, np.linalg.norm(R.ecef[jn] - nodes[i], axis=1), np.inf) is_min = (d0 <= dp) & (d0 < dn) if not is_min.any(): continue i, j, d0, dp, dn, hp, hn, jp, jn = (a[is_min] for a in (i, j, d0, dp, dn, hp, hn, jp, jn)) edge = ~(hp & hn) # parabola on squared range through (-1, 0, +1) y0, ym, yp = d0**2, np.where(hp, dp**2, d0**2), np.where(hn, dn**2, d0**2) den = ym - 2 * y0 + yp delta = np.where((den > 0) & ~edge, 0.5 * (ym - yp) / np.where(den > 0, den, 1), 0.0) delta = np.clip(delta, -0.5, 0.5) r2 = y0 - 0.25 * (ym - yp) * delta d = np.sqrt(np.clip(np.minimum(r2, y0), 0, None)) t_cpa = R.t[j] + delta * R.dt gs = R.gs[j] + delta * np.where(delta > 0, R.gs[jn] - R.gs[j], R.gs[j] - R.gs[jp]) ev = np.zeros(i.size, EV_DTYPE) ev["node"], ev["j"], ev["fid"] = i + s, j, R.fid[j] ev["t_cpa"], ev["d"], ev["gs"], ev["edge"] = t_cpa, d, gs, edge ev["t_rx"] = t_cpa + d / c outs.append(ev) return np.concatenate(outs) if outs else np.zeros(0, EV_DTYPE) def corridor_nodes(R, sample_idx, grid, half_width, chunk_s=180.0, step_s=4.0): """Grid node ids within horizontal distance `half_width` of the listed samples.""" sample_idx = np.asarray(sample_idx, np.int64) if sample_idx.size == 0: return np.zeros(0, np.int64) ids_all = [] fids = R.fid[sample_idx] order = np.lexsort((R.t[sample_idx], fids)) si = sample_idx[order] f_s = R.fid[si] t_s = R.t[si] # chunk boundaries: new flight or > chunk_s elapsed newc = np.r_[True, (f_s[1:] != f_s[:-1]) | (np.floor(t_s[1:] / chunk_s) != np.floor(t_s[:-1] / chunk_s))] starts = np.flatnonzero(newc) ends = np.r_[starts[1:], si.size] stride = max(1, int(round(step_s / R.dt))) for a, b in zip(starts, ends): js = si[a:b:stride] if js.size == 0: continue lat, lon = R.lat[js], R.lon[js] pad = (half_width + grid.g) / 111320.0 padlon = pad / max(np.cos(np.radians(lat.mean())), 0.2) ids, nlat, nlon = grid.nodes_in_bbox(lat.min() - pad, lat.max() + pad, lon.min() - padlon, lon.max() + padlon) if ids.size == 0: continue lat0, lon0 = lat.mean(), lon.mean() e, n = latlon_to_local_en(nlat, nlon, lat0, lon0) te, tn = latlon_to_local_en(lat, lon, lat0, lon0) dist, _ = cKDTree(np.c_[te, tn]).query(np.c_[e, n], k=1) slack = 0.5 * stride * R.dt * float(np.nanmax(R.gs[js])) + 0.75 * grid.g # along-track sampling + cell size ids_all.append(ids[dist <= half_width + slack]) if not ids_all: return np.zeros(0, np.int64) return np.unique(np.concatenate(ids_all)) def _in_window(x, lo, hi): return (x >= lo) & (x <= hi) def _k_ok(ev, e): """v^2/d consistency: the candidate's own ADS-B ground speed must predict its CPA distance through the curve steepness k (see search_constraints).""" if "k_min" not in e: return np.ones(len(ev), bool) k = ev["gs"] ** 2 / np.maximum(ev["d"], 1.0) return _in_window(k, e["k_min"], e["k_max"]) def run_search(R, flights, events, dem, bbox=None, grid_m=200.0, c=343.0, rec_start=None, rec_start_tol=None, rec_duration=None, quiet_intervals=None, event_classes=("prop", "rotor", "unknown"), d_quiet=None, guard_s=25.0, mic_height=2.0, offset_tol=None, allow_same_aircraft=True, max_nodes=3_000_000): """events: list of 1 or 2 search-constraint dicts (physics.search_constraints). rec_start: absolute UTC (unix) of recording t=0 if known (+-rec_start_tol s). quiet_intervals: list of (t0, t1) recording-time spans that show NO fly-by; default = whole recording minus +-guard around observed events. d_quiet: {class: slant-distance threshold in m} - any such flight with CPA closer than this inside a quiet interval disqualifies the candidate. Returns (candidates structured array, funnel stats dict).""" grid = Grid(grid_m) stats = {} d_quiet = d_quiet or {"prop": 3000.0, "rotor": 2500.0, "unknown": 3000.0, "jet": 2500.0} cls_arr = np.asarray(flights.cls) samp_cls = cls_arr[R.fid] in_bbox = np.ones(R.n, bool) if bbox: in_bbox = (R.lat >= bbox[0]) & (R.lat <= bbox[1]) & (R.lon >= bbox[2]) & (R.lon <= bbox[3]) e1 = events[0] m1 = np.isin(samp_cls, list(event_classes)) & in_bbox m1 &= _in_window(R.gs, e1["v_min"] - 3, e1["v_max"] + 3) if rec_start is not None: tol = (rec_start_tol or 0) + e1["t_tol"] + e1["d_max"] / c + 60 t1_abs = rec_start + e1["t_rec"] m1 &= _in_window(R.t, t1_abs - tol, t1_abs + tol) s1 = np.flatnonzero(m1) stats["flights_total"] = int(flights.nflights) stats["flights_event1_speed_class"] = int(np.unique(R.fid[s1]).size) log(f"A: {stats['flights_event1_speed_class']} flights match event-1 class/speed") node_ids = corridor_nodes(R, s1, grid, e1["d_max"]) if bbox: nl, no = grid.node_latlon(node_ids) node_ids = node_ids[(nl >= bbox[0]) & (nl <= bbox[1]) & (no >= bbox[2]) & (no <= bbox[3])] if node_ids.size > max_nodes: raise RuntimeError(f"{node_ids.size} corridor nodes - narrow bbox/time window or tile the search") stats["corridor_nodes"] = int(node_ids.size) log(f"A: {node_ids.size} corridor nodes") nlat, nlon = grid.node_latlon(node_ids) nelev = dem.sample(nlat, nlon) necef = geodetic_to_ecef(nlat, nlon, nelev + mic_height) if rec_start is not None: stats["clock"] = {"rec_start": float(rec_start), "rec_start_tol": float(rec_start_tol or 0)} stats["ground_elev_m"] = {"source": "constant" if dem.is_constant else "dem", "min": float(nelev.min()) if nelev.size else None, "max": float(nelev.max()) if nelev.size else None} ev1 = cpa_events(necef, R, s1, e1["d_max"] * 1.02, c) ok = (~ev1["edge"]) & _in_window(ev1["d"], e1["d_min"], e1["d_max"]) & _in_window(ev1["gs"], e1["v_min"], e1["v_max"]) ok &= _k_ok(ev1, e1) if rec_start is not None: ok &= np.abs(ev1["t_rx"] - (rec_start + e1["t_rec"])) <= (rec_start_tol or 0) + e1["t_tol"] ev1 = ev1[ok] stats["event1_matches"] = int(ev1.size) stats["nodes_after_event1"] = int(np.unique(ev1["node"]).size) log(f"A: {ev1.size} event-1 CPA matches at {stats['nodes_after_event1']} nodes") if ev1.size == 0: log("no node hears a flight matching event 1 - widen v/d windows, bbox or time window, or check data coverage") stats["candidates"] = 0 return np.zeros(0, CAND_DTYPE), stats # ---- B: offset filter -------------------------------------------------- if len(events) >= 2: e2 = events[1] dT = e2["t_rec"] - e1["t_rec"] tol = offset_tol if offset_tol is not None else np.hypot(e1["t_tol"], e2["t_tol"]) cand_nodes = np.unique(ev1["node"]) m2 = np.isin(samp_cls, list(event_classes)) & _in_window(R.gs, e2["v_min"] - 3, e2["v_max"] + 3) t_lo, t_hi = ev1["t_rx"].min() + dT - tol - e2["d_max"] / c - 30, ev1["t_rx"].max() + dT + tol + 30 m2 &= _in_window(R.t, t_lo, t_hi) s2 = np.flatnonzero(m2) ev2 = cpa_events(necef[cand_nodes], R, s2, e2["d_max"] * 1.02, c) ev2["node"] = cand_nodes[ev2["node"]] ok2 = (~ev2["edge"]) & _in_window(ev2["d"], e2["d_min"], e2["d_max"]) & _in_window(ev2["gs"], e2["v_min"], e2["v_max"]) ok2 &= _k_ok(ev2, e2) ev2 = ev2[ok2] pairs = _pair(ev1, ev2, dT, tol, allow_same_aircraft) if ev2.size else np.zeros((0, 2), np.int64) stats["offset_pairs"] = int(len(pairs)) stats["nodes_after_offset"] = int(np.unique(ev1["node"][pairs[:, 0]]).size) if len(pairs) else 0 log(f"B: {len(pairs)} (event1, event2) pairs at {stats['nodes_after_offset']} nodes (dT={dT:.1f}+-{tol:.1f}s)") else: ev2 = np.zeros(0, EV_DTYPE) pairs = np.c_[np.arange(ev1.size), np.full(ev1.size, -1)] # ---- C: silence filter ------------------------------------------------- if rec_duration is not None and len(pairs): if quiet_intervals is None: busy = [(e["t_rec"] - guard_s - e.get("half_span_s", 60), e["t_rec"] + guard_s + e.get("half_span_s", 60)) for e in events] quiet_intervals = _complement(busy, guard_s, rec_duration - guard_s) stats["quiet_intervals"] = quiet_intervals pnodes = ev1["node"][pairs[:, 0]] un = np.unique(pnodes) dq_max = max(d_quiet.values()) t_anchor = ev1["t_rx"][pairs[:, 0]] - e1["t_rec"] # absolute UTC of recording t=0 per pair # every flight counts here, even outside the bbox: it may still pass near a node mall = _in_window(R.t, t_anchor.min() - dq_max / c - 120, t_anchor.max() + rec_duration + 120) mall &= np.isin(samp_cls, list(d_quiet.keys())) sall = np.flatnonzero(mall) evq = cpa_events(necef[un], R, sall, dq_max, c) evq["node"] = un[evq["node"]] thr = np.array([d_quiet.get(k, 0.0) for k in cls_arr])[evq["fid"]] evq = evq[evq["d"] <= thr] bad = _silence_violations(pairs, ev1, ev2, evq, t_anchor, quiet_intervals) stats["pairs_failing_silence"] = int(bad.sum()) pairs = pairs[~bad] stats["pairs_after_silence"] = int(len(pairs)) stats["nodes_after_silence"] = int(np.unique(ev1["node"][pairs[:, 0]]).size) if len(pairs) else 0 log(f"C: {len(pairs)} pairs survive silence filter at {stats['nodes_after_silence']} nodes") if len(events) >= 2 and len(pairs): stats["crossing_angle_deg"] = _crossing_stats(R, ev1["j"][pairs[:, 0]], ev2["j"][pairs[:, 1]]) if stats["crossing_angle_deg"]["pairs_below_25"] == len(pairs): log("B: every remaining pair of tracks crosses at < 25 deg - two events then only give a strip " "along the tracks, not a point") # ---- assemble ---------------------------------------------------------- cand = np.zeros(len(pairs), CAND_DTYPE) if len(pairs): a = ev1[pairs[:, 0]] k = a["node"] cand["node_id"], cand["lat"], cand["lon"], cand["elev"] = node_ids[k], nlat[k], nlon[k], nelev[k] cand["fid1"], cand["t1_cpa"], cand["t1_rx"], cand["d1"], cand["v1"] = a["fid"], a["t_cpa"], a["t_rx"], a["d"], a["gs"] has2 = pairs[:, 1] >= 0 cand["fid2"] = -1 if has2.any(): b = ev2[pairs[has2, 1]] cand["fid2"][has2], cand["t2_cpa"][has2], cand["t2_rx"][has2] = b["fid"], b["t_cpa"], b["t_rx"] cand["d2"][has2], cand["v2"][has2] = b["d"], b["gs"] stats["candidates"] = int(len(cand)) stats["flight_pairs"] = int(np.unique(np.c_[cand["fid1"], cand["fid2"]], axis=0).shape[0]) if len(cand) else 0 if len(events) == 1 and stats["flight_pairs"] > 1: log(f"{stats['flight_pairs']} different aircraft can explain the event - one event fits every aircraft with a " "matching speed and distance; a second event, a tight clock or a small area is needed for a point") return cand, stats def _crossing_stats(R, j1, j2, min_deg=25.0): """Crossing angle (0-90 deg) between the two tracks at their CPAs, over all pairs.""" h1 = np.degrees(np.arctan2(R.ve[j1], R.vn[j1])) h2 = np.degrees(np.arctan2(R.ve[j2], R.vn[j2])) x = np.abs(h1 - h2) % 180.0 a = np.minimum(x, 180.0 - x) return {"min": float(a.min()), "median": float(np.median(a)), "max": float(a.max()), "pairs_below_25": int((a < min_deg).sum())} def _pair(ev1, ev2, dT, tol, allow_same_aircraft=True): """All (i1, i2) with the same node and |t_rx2 - t_rx1 - dT| <= tol. Node and time go into one int64 key per event: the node's dense rank times a slot wide enough for every time involved, plus the time in milliseconds. Ranking first keeps the key small and exact for any node ids (also the global Grid ids, ~1e14), and integer keys cannot lose the time term.""" if ev1.size == 0 or ev2.size == 0: return np.zeros((0, 2), np.int64) nodes, inv = np.unique(np.r_[ev2["node"], ev1["node"]], return_inverse=True) r2, r1 = inv[:ev2.size].astype(np.int64), inv[ev2.size:].astype(np.int64) pad = abs(dT) + tol + 1.0 t0 = min(ev1["t_rx"].min(), ev2["t_rx"].min()) - pad slot = int(np.ceil((max(ev1["t_rx"].max(), ev2["t_rx"].max()) + pad - t0) * 1000.0)) + 1 if nodes.size * float(slot) >= 2.0**62: raise RuntimeError("too many nodes x time span for exact pairing - split the time window") key2 = r2 * slot + np.round((ev2["t_rx"] - t0) * 1000.0).astype(np.int64) o2 = np.argsort(key2, kind="stable") key2 = key2[o2] k_lo = r1 * slot + np.floor((ev1["t_rx"] + dT - tol - t0) * 1000.0).astype(np.int64) k_hi = r1 * slot + np.ceil((ev1["t_rx"] + dT + tol - t0) * 1000.0).astype(np.int64) a = np.searchsorted(key2, k_lo, "left") b = np.searchsorted(key2, k_hi, "right") cnt = b - a i1 = np.repeat(np.arange(ev1.size), cnt) i2 = o2[np.concatenate([np.arange(x, y) for x, y in zip(a, b)])] if cnt.sum() else np.zeros(0, np.int64) p = np.c_[i1, i2].astype(np.int64) if len(p) and not allow_same_aircraft: p = p[ev1["fid"][p[:, 0]] != ev2["fid"][p[:, 1]]] return p def _complement(busy, lo, hi): busy = sorted((max(lo, a), min(hi, b)) for a, b in busy if b > lo and a < hi) out, cur = [], lo for a, b in busy: if a > cur: out.append((cur, a)) cur = max(cur, b) if cur < hi: out.append((cur, hi)) return out def _silence_violations(pairs, ev1, ev2, evq, t_anchor, quiet): """True for pairs where some flight (not one of the pair's own events) reaches its CPA within a quiet interval.""" bad = np.zeros(len(pairs), bool) if evq.size == 0 or not quiet: return bad pn = ev1["node"][pairs[:, 0]] o = np.argsort(pn, kind="stable") pn_s = pn[o] a = np.searchsorted(pn_s, evq["node"], "left") b = np.searchsorted(pn_s, evq["node"], "right") cnt = b - a if cnt.sum() == 0: return bad qi = np.repeat(np.arange(evq.size), cnt) pi = o[np.concatenate([np.arange(x, y) for x, y in zip(a, b)])] rec_t = evq["t_rx"][qi] - t_anchor[pi] inq = np.zeros(qi.size, bool) for q0, q1 in quiet: inq |= (rec_t >= q0) & (rec_t <= q1) own1 = (evq["fid"][qi] == ev1["fid"][pairs[pi, 0]]) & (np.abs(evq["t_cpa"][qi] - ev1["t_cpa"][pairs[pi, 0]]) < 30) has2 = pairs[pi, 1] >= 0 j2 = np.where(has2, pairs[pi, 1], 0) own2 = has2 & (evq["fid"][qi] == ev2["fid"][j2] if ev2.size else False) & \ (np.abs(evq["t_cpa"][qi] - (ev2["t_cpa"][j2] if ev2.size else 0)) < 30) viol = inq & ~own1 & ~own2 bad[np.unique(pi[viol])] = True return badskills/en/audio-flyover-geolocation/scripts/afg/simulate.py11 KB494cbb50aa75upstream ↗
"""Synthetic ground-truth scenario: realistic ADS-B traces (readsb format) and a physically exact recording (emission-time Doppler, 1/R spreading, voice, hum, wind, pink noise) at a hidden observer. Used by `selftest` to prove the pipeline end to end, and handy for rehearsing a case before touching real data. """ import gzip import json import os import numpy as np from scipy.io import wavfile from .common import FPM, FT, KT, dump_json, iso, parse_time from .geo import geodetic_to_ecef, offset_latlon PROPS = [ # type, desc, blades, rpm -> BPF = blades*rpm/60 ("C172", "L1P", 2, 2400), ("SR22", "L1P", 3, 2600), ("PC12", "L1T", 4, 1700), ("BE20", "L2T", 4, 2000), ("C208", "L1T", 3, 1900), ("DH8D", "L2T", 6, 1020), ("P28A", "L1P", 2, 2500), ] JETS = ["B738", "CRJ9", "E75L", "A320", "C56X"] def _fly(rng, t0, dur, x0, y0, hdg, v, z, maneuvers=(), dt=0.1): n = int(dur / dt) + 1 t = t0 + np.arange(n) * dt turn = np.zeros(n) acc = np.zeros(n) climb = np.zeros(n) for m in maneuvers: s = (t >= t0 + m["t"]) & (t < t0 + m["t"] + m["dur"]) turn[s] += m.get("turn", 0.0) acc[s] += m.get("acc", 0.0) climb[s] += m.get("climb", 0.0) h = np.radians(hdg + np.cumsum(turn) * dt) spd = np.clip(v + np.cumsum(acc) * dt, 20, 300) vx, vy = spd * np.sin(h), spd * np.cos(h) x = x0 + np.cumsum(vx) * dt - vx[0] * dt y = y0 + np.cumsum(vy) * dt - vy[0] * dt zz = z + np.cumsum(climb) * dt return {"t": t, "x": x, "y": y, "z": zz, "vx": vx, "vy": vy, "vz": climb, "spd": spd, "hdg": np.degrees(h) % 360} def _random_maneuvers(rng, dur, k=3): out = [] for _ in range(rng.integers(0, k + 1)): out.append({"t": rng.uniform(0, dur), "dur": rng.uniform(10, 60), "turn": rng.uniform(-2.5, 2.5), "acc": rng.uniform(-0.3, 0.3), "climb": rng.uniform(-3, 3)}) return out def _ecef_track(tr, lat0, lon0): la, lo = offset_latlon(lat0, lon0, tr["x"], tr["y"]) return la, lo, geodetic_to_ecef(la, lo, tr["z"]) def simulate(outdir, lat0=39.10, lon0=-94.60, ground=250.0, seed=7, n_prop=45, n_jet=25, area_km=70.0, day="2026-09-08T14:00:00Z", span_h=4.0, c=343.0, fs=4000, t1_rec=240.0, gap=920.0, tail=300.0, tone_amp=0.005, noise_rms=0.03): rng = np.random.default_rng(seed) os.makedirs(outdir, exist_ok=True) t_day = parse_time(day) half = area_km * 500.0 obs_x, obs_y = rng.uniform(-6000, 6000, 2) obs_lat, obs_lon = offset_latlon(lat0, lon0, obs_x, obs_y) obs_ecef = geodetic_to_ecef(obs_lat, obs_lon, ground + 2.0) T1 = t_day + rng.uniform(0.35, 0.55) * span_h * 3600 flights = [] def target(Tc, r_h, agl, v, man, typ): hdg = rng.uniform(0, 360) hr = np.radians(hdg) side = rng.choice([-1, 1]) px, py = obs_x + side * r_h * np.cos(hr), obs_y - side * r_h * np.sin(hr) pre = rng.uniform(600, 1200) dur = pre + rng.uniform(600, 1200) x0, y0 = px - v * pre * np.sin(hr), py - v * pre * np.cos(hr) man = [dict(m, t=m["t"] + pre) for m in man] tr = _fly(rng, Tc - pre, dur, x0, y0, hdg, v, ground + agl, man) return {"tr": tr, "type": typ, "cls": "prop"} p1 = PROPS[rng.integers(len(PROPS))] p2 = PROPS[rng.integers(len(PROPS))] flights.append(target(T1, rng.uniform(600, 1300), rng.uniform(500, 900), rng.uniform(75, 100), [{"t": 25, "dur": 35, "turn": 1.5}], p1)) flights.append(target(T1 + gap, rng.uniform(700, 1500), rng.uniform(500, 900), rng.uniform(55, 75), [{"t": -60, "dur": 50, "acc": 0.25}, {"t": 10, "dur": 40, "climb": 2.0}], p2)) rec_start = T1 + 2.0 - t1_rec # approx (exact reception offsets come from physics) rec_dur = t1_rec + gap + tail win = (rec_start - 90, rec_start + rec_dur + 30) def min_dist_in_window(tr): s = (tr["t"] >= win[0]) & (tr["t"] <= win[1]) if not s.any(): return np.inf _, _, e = _ecef_track({k: v[s] for k, v in tr.items()}, lat0, lon0) return float(np.linalg.norm(e - obs_ecef, axis=1).min()) def rand_flight(kind): for _ in range(200): t0 = t_day + rng.uniform(-0.5, span_h) * 3600 if kind == "prop": typ = PROPS[rng.integers(len(PROPS))] v, z, dur = rng.uniform(45, 110), ground + rng.uniform(350, 3000), rng.uniform(900, 3600) man = _random_maneuvers(rng, dur) else: typ = JETS[rng.integers(len(JETS))] v, z, dur = rng.uniform(150, 240), ground + rng.uniform(3000, 11000), rng.uniform(600, 1500) man = [] x0, y0 = rng.uniform(-half, half, 2) tr = _fly(rng, t0, dur, x0, y0, rng.uniform(0, 360), v, z, man) if min_dist_in_window(tr) > (5000 if kind == "prop" else 4500): return {"tr": tr, "type": typ, "cls": kind} raise RuntimeError("could not place distractor") # a few distractors deliberately pass close to the observer OUTSIDE the recording for _ in range(4): Tc = T1 + rng.choice([-1, 1]) * rng.uniform(2500, 6000) f = target(Tc, rng.uniform(500, 2000), rng.uniform(400, 1200), rng.uniform(55, 100), [], PROPS[rng.integers(len(PROPS))]) if min_dist_in_window(f["tr"]) > 5000: flights.append(f) flights += [rand_flight("prop") for _ in range(n_prop)] flights += [rand_flight("jet") for _ in range(n_jet)] # ---------------- ADS-B traces ------------------------------------------------ tdir = os.path.join(outdir, "traces") truth_f = [] for i, f in enumerate(flights): tr = f["tr"] hexid = f"{0xA00000 + i * 7919:06x}" f["hex"] = hexid typ = f["type"][0] if f["cls"] == "prop" else f["type"] ts = [tr["t"][0]] while ts[-1] < tr["t"][-1]: step = rng.choice([1, 1, 2, 2, 3, 4, 5]) if rng.random() < 0.02: step = rng.uniform(8, 25) ts.append(ts[-1] + step + rng.uniform(-0.3, 0.3)) ts = np.array(ts[:-1]) g = lambda k: np.interp(ts, tr["t"], tr[k]) # noqa: E731 x, y = g("x") + rng.normal(0, 2.5, ts.size), g("y") + rng.normal(0, 2.5, ts.size) la, lo = offset_latlon(lat0, lon0, x, y) z, spd, vz = g("z"), g("spd"), g("vz") hdg = np.interp(ts, tr["t"], np.degrees(np.unwrap(np.radians(tr["hdg"])))) % 360 entries = [] for k in range(ts.size): vr_fpm = int(round(vz[k] / FPM / 64) * 64) entries.append([round(ts[k] - ts[0], 2), round(la[k], 6), round(lo[k], 6), int(round((z[k] + rng.normal(0, 3)) / FT / 25) * 25), # baro (QNH = std here) round(spd[k] / KT + rng.normal(0, 0.25), 1), round((hdg[k] + rng.normal(0, 0.15)) % 360, 1), 2 if k == 0 else 0, vr_fpm, None, "adsb_icao", int(round((z[k] + rng.normal(0, 4)) / FT / 25) * 25), # geometric (N = 0 here) vr_fpm, None, None]) obj = {"icao": hexid, "r": f"N{100 + i}SIM", "t": typ, "dbFlags": 0, "desc": "SIMULATED", "timestamp": round(ts[0], 3), "trace": entries} d = os.path.join(tdir, hexid[-2:]) os.makedirs(d, exist_ok=True) with open(os.path.join(d, f"trace_full_{hexid}.json"), "wb") as fh: fh.write(gzip.compress(json.dumps(obj).encode())) truth_f.append({"hex": hexid, "type": typ, "cls": f["cls"]}) # ---------------- audio --------------------------------------------------- n = int(rec_dur * fs) ta = rec_start + np.arange(n) / fs audio = np.zeros(n) ev_truth = [] for i, f in enumerate(flights): tr = f["tr"] s = (tr["t"] >= win[0] - 60) & (tr["t"] <= win[1]) if s.sum() < 10: continue sub = {k: v[s] for k, v in tr.items()} _, _, e = _ecef_track(sub, lat0, lon0) R = np.linalg.norm(e - obs_ecef, axis=1) if R.min() > 12000: continue trx = sub["t"] + R / c m = (ta >= trx[0]) & (ta <= trx[-1]) if not m.any(): continue tau = np.interp(ta[m], trx, sub["t"]) Rt = np.interp(ta[m], trx, R) if f["cls"] == "prop": typ, _, blades, rpm = f["type"] bpf = blades * rpm / 60.0 * (1 + rng.uniform(-0.01, 0.01)) drift = np.cumsum(rng.normal(0, 2e-5, sub["t"].size)) phase = 2 * np.pi * bpf * np.cumsum(np.r_[0, np.diff(sub["t"])] * (1 + drift)) ph = np.interp(tau, sub["t"], phase) sig = np.zeros(m.sum()) for k, a in enumerate([1.0, 0.7, 0.5, 0.35, 0.25, 0.18], start=1): sig += a * np.sin(k * ph + rng.uniform(0, 2 * np.pi)) audio[m] += tone_amp * 1000.0 / Rt * sig if i < 2: j = int(np.argmin(R)) ev_truth.append({"event": i + 1, "hex": f["hex"], "type": typ, "bpf_hz": bpf, "t_cpa_emit": iso(sub["t"][j]), "d_cpa_m": float(R[j]), "t_cpa_rx_unix": float(sub["t"][j] + R[j] / c), "t_cpa_rec_s": float(sub["t"][j] + R[j] / c - rec_start), "speed_ms": float(sub["spd"][j])}) else: noise = rng.normal(0, 1, m.sum()) from scipy import signal as _s noise = _s.sosfilt(_s.butter(2, 600, fs=fs, output="sos"), noise) audio[m] += 0.02 * 3000.0 / Rt * noise # ambient: pink noise + hum + voice bursts + wind w = rng.normal(0, 1, n) W = np.fft.rfft(w) fr = np.fft.rfftfreq(n, 1 / fs) W[1:] /= np.sqrt(fr[1:] / 50.0) W[0] = 0 pink = np.fft.irfft(W, n) audio += noise_rms * pink / pink.std() tt = np.arange(n) / fs enf = 60 + 0.02 * np.sin(2 * np.pi * tt / 97.0) audio += 0.002 * np.sin(2 * np.pi * np.cumsum(enf) / fs) + 0.001 * np.sin(2 * np.pi * 3 * np.cumsum(enf) / fs) t_v = 0.0 while True: t_v += rng.exponential(35.0) if t_v > rec_dur - 6: break L = rng.uniform(1.5, 5.0) a, b = int(t_v * fs), int((t_v + L) * fs) tv = np.arange(b - a) / fs pitch = rng.uniform(100, 220) * (1 + 0.15 * np.sin(2 * np.pi * rng.uniform(0.5, 2) * tv)) ph = 2 * np.pi * np.cumsum(pitch) / fs v = sum((1.0 / k) * np.sin(k * ph) * (1 + np.cos(2 * np.pi * k / 7)) for k in range(1, 13)) env = np.sin(np.pi * tv / L) ** 0.5 * (0.5 + 0.5 * np.abs(np.sin(2 * np.pi * 3.5 * tv))) audio[a:b] += rng.uniform(0.05, 0.12) * v * env peak = np.max(np.abs(audio)) audio = 0.9 * audio / peak wav = os.path.join(outdir, "recording.wav") wavfile.write(wav, fs, (audio * 32767).astype(np.int16)) truth = {"observer": {"lat": float(obs_lat), "lon": float(obs_lon), "ground_elev_m": ground}, "rec_start_utc": iso(rec_start), "rec_start_unix": rec_start, "rec_duration_s": rec_dur, "events": ev_truth, "region_center": [lat0, lon0], "area_km": area_km, "bbox": [lat0 - area_km / 222.0, lat0 + area_km / 222.0, lon0 - area_km / (222.0 * np.cos(np.radians(lat0))), lon0 + area_km / (222.0 * np.cos(np.radians(lat0)))], "day_window": [iso(t_day - 3600), iso(t_day + (span_h + 1) * 3600)], "c": c, "fs": fs, "n_flights": len(flights)} dump_json(truth, os.path.join(outdir, "truth.json")) return truth, wav, tdirskills/en/audio-flyover-geolocation/scripts/afg/thunder.py3.5 KBb588858b2814upstream ↗
"""Thunder-envelope geolocation (the author's earlier, coarser method). Observed: band-pass 20-250 Hz -> RMS envelope -> light smoothing. Model at each candidate point: each strike contributes a pulse at t_strike + slant/c (acoustic source ~2 km up for CG strikes) with amplitude ~ 1/r^p (p=1 for an RMS *amplitude* envelope, p=2 if you compare energy), capped near the source, cut beyond audibility range (~20-25 km: thunder is refracted upward and rarely heard further). Match: cosine similarity between observed envelope and the model window, evaluated at every time offset with FFT cross-correlation; the model is normalised *within each window* (the clip knows nothing about strikes outside it). Precision is limited by strike-location error (free networks ~1-5 km; national/commercial networks ~100-500 m) and by strike geometry (strikes from several directions are needed to triangulate). """ import numpy as np from scipy import signal from scipy.ndimage import gaussian_filter1d from .geo import geodetic_to_ecef def observed_envelope(x, fs, band=(20.0, 250.0), rate=2.0, smooth_s=1.5): sos = signal.butter(4, band, btype="bandpass", fs=fs, output="sos") y = signal.sosfiltfilt(sos, x) hop = int(round(fs / rate)) n = len(y) // hop env = np.sqrt(np.mean(y[: n * hop].reshape(n, hop) ** 2, axis=1)) env = gaussian_filter1d(env, smooth_s * rate) env = env - np.median(env) return np.clip(env, 0, None) def model_envelope(node_ecef, strikes_ecef, strike_t, t0, n, rate, c=343.0, p=1.0, r_cap=1500.0, r_max=25000.0, smooth_s=1.5, decay_s=2.5, weights=None): r = np.linalg.norm(strikes_ecef - node_ecef, axis=1) m = r <= r_max tarr = strike_t[m] + r[m] / c amp = 1.0 / np.maximum(r[m], r_cap) ** p if weights is not None: amp = amp * weights[m] idx = np.round((tarr - t0) * rate).astype(int) ok = (idx >= 0) & (idx < n) env = np.zeros(n) np.add.at(env, idx[ok], amp[ok]) k = np.exp(-np.arange(int(decay_s * rate * 5)) / (decay_s * rate)) env = signal.fftconvolve(env, k)[:n] return gaussian_filter1d(env, smooth_s * rate) def sliding_cosine(model, obs): """cos(model[k:k+m], obs) for every k (FFT); returns array of len(model)-m+1.""" m = len(obs) if len(model) < m: return np.zeros(0) num = signal.fftconvolve(model, obs[::-1], mode="valid") csum = np.r_[0, np.cumsum(model**2)] den = np.sqrt(csum[m:] - csum[:-m]) * np.linalg.norm(obs) return np.where(den > 1e-30, num / np.maximum(den, 1e-30), 0.0) def locate(obs_env, rate, strikes, nodes_lat, nodes_lon, nodes_elev, t_start, t_end, c=343.0, source_h=2000.0, p=1.0, min_strikes=5, weights=None): """Grid scan. strikes: dict t (unix), lat, lon. Returns score, best offset per node.""" s_ecef = geodetic_to_ecef(strikes["lat"], strikes["lon"], np.full(len(strikes["lat"]), source_h)) n = int((t_end - t_start) * rate) + len(obs_env) score = np.zeros(len(nodes_lat)) best_t = np.full(len(nodes_lat), np.nan) necef = geodetic_to_ecef(nodes_lat, nodes_lon, nodes_elev) for i in range(len(nodes_lat)): r = np.linalg.norm(s_ecef - necef[i], axis=1) if (r < 25000).sum() < min_strikes: continue me = model_envelope(necef[i], s_ecef, strikes["t"], t_start, n, rate, c, p, weights=weights) cs = sliding_cosine(me, obs_env) if cs.size: k = int(np.argmax(cs)) score[i], best_t[i] = cs[k], t_start + k / rate return score, best_tskills/en/audio-flyover-geolocation/scripts/afg/tracks.py5.6 KBc9f554d779fcupstream ↗
"""Uniform-time resampling of ADS-B flights into ECEF position + velocity. Velocity comes from the broadcast ground speed / track / vertical rate when available (ADS-B velocity messages are GNSS-derived, ~0.5 m/s), because differentiating noisy positions would inject several m/s of error - and the Doppler shift is directly proportional to radial velocity (1 m/s ~ 0.3 %). """ import json import numpy as np from scipy.interpolate import PchipInterpolator from scipy.signal import savgol_filter from .geo import enu_vectors_to_ecef, geodetic_to_ecef class Flights: """Raw per-sample table + per-flight metadata (from tracks.npz).""" def __init__(self, path): z = np.load(path, allow_pickle=False) self.fid, self.t = z["fid"], z["t"] self.lat, self.lon, self.h = z["lat"], z["lon"], z["h"].astype(np.float64) self.gs, self.trk, self.vr = z["gs"].astype(np.float64), z["trk"].astype(np.float64), z["vr"].astype(np.float64) self.hex, self.type, self.reg = z["f_hex"], z["f_type"], z["f_reg"] self.cls, self.desc = z["f_cls"], z["f_desc"] self.meta = json.loads(str(z["meta"])) self.nflights = len(self.hex) def label(self, fid): return f"{self.hex[fid]}/{self.type[fid] or '?'}/{self.reg[fid] or '-'}#{fid}" class Resampled: """Concatenated uniform samples: arrays aligned by sample index. Samples of one flight are contiguous and time-ordered; prev/next validity is encoded in has_prev / has_next.""" def __init__(self, fl, dt=1.0, t0=None, t1=None, fids=None): self.dt = float(dt) out = {k: [] for k in ("fid", "t", "lat", "lon", "h", "gs", "ve", "vn", "vu")} order = np.argsort(fl.fid, kind="stable") fid_sorted = fl.fid[order] bounds = np.flatnonzero(np.diff(fid_sorted)) + 1 starts = np.r_[0, bounds] ends = np.r_[bounds, fid_sorted.size] allowed = None if fids is None else set(int(x) for x in fids) for s, e in zip(starts, ends): ii = order[s:e] f = int(fl.fid[ii[0]]) if allowed is not None and f not in allowed: continue ii = ii[np.argsort(fl.t[ii])] t = fl.t[ii] t, uniq = np.unique(t, return_index=True) ii = ii[uniq] if t.size < 3: continue a = np.ceil(max(t[0], t0 if t0 is not None else -np.inf) / dt) * dt b = np.floor(min(t[-1], t1 if t1 is not None else np.inf) / dt) * dt if b - a < 2 * dt: continue tg = np.arange(a, b + dt / 2, dt) lat = PchipInterpolator(t, fl.lat[ii])(tg) lon = PchipInterpolator(t, np.degrees(np.unwrap(np.radians(fl.lon[ii]))))(tg) hh = fl.h[ii] okh = np.isfinite(hh) if okh.sum() < 2: continue h = np.interp(tg, t[okh], hh[okh]) gs, trk, vr = fl.gs[ii], fl.trk[ii], fl.vr[ii] okv = np.isfinite(gs) & np.isfinite(trk) if okv.sum() >= 2: ve_s = gs[okv] * np.sin(np.radians(trk[okv])) vn_s = gs[okv] * np.cos(np.radians(trk[okv])) ve = np.interp(tg, t[okv], ve_s) vn = np.interp(tg, t[okv], vn_s) else: # fall back to smoothed position derivative m_lat = 111132.954 - 559.822 * np.cos(2 * np.radians(lat)) m_lon = 111412.84 * np.cos(np.radians(lat)) win = min(len(tg) - (1 - len(tg) % 2), 21) if win >= 5: ve = savgol_filter(lon, win, 2, deriv=1, delta=dt) * m_lon vn = savgol_filter(lat, win, 2, deriv=1, delta=dt) * m_lat else: ve = np.gradient(lon, dt) * m_lon vn = np.gradient(lat, dt) * m_lat okr = np.isfinite(vr) vu = np.interp(tg, t[okr], vr[okr]) if okr.sum() >= 2 else np.gradient(h, dt) out["fid"].append(np.full(tg.size, f, np.int32)) out["t"].append(tg) out["lat"].append(lat) out["lon"].append(((lon + 180) % 360) - 180) out["h"].append(h) out["ve"].append(ve) out["vn"].append(vn) out["vu"].append(vu) out["gs"].append(np.hypot(ve, vn)) if not out["t"]: raise RuntimeError("no flight samples after resampling (check time window)") for k in out: setattr(self, k, np.concatenate(out[k])) self.ecef = geodetic_to_ecef(self.lat, self.lon, self.h) self.vel = enu_vectors_to_ecef(self.lat, self.lon, self.ve, self.vn, self.vu) same = (self.fid[1:] == self.fid[:-1]) & (np.abs(np.diff(self.t) - dt) < 1e-6) self.has_prev = np.r_[False, same] self.has_next = np.r_[same, False] self.n = self.t.size # flights are contiguous (input grouped by fid): first/last index per flight fids, first = np.unique(self.fid, return_index=True) cnt = np.bincount(self.fid) self._first = dict(zip(fids.tolist(), first.tolist())) self._last = {f: a + int(cnt[f]) - 1 for f, a in zip(fids.tolist(), first.tolist())} def flight_slice(self, fid, ta=None, tb=None): """Index range [i0, i1) of flight fid restricted to [ta, tb].""" a, b = self._first[int(fid)], self._last[int(fid)] + 1 if ta is not None: a = max(a, a + int(np.ceil((ta - self.t[a]) / self.dt))) if tb is not None: b = min(b, self._first[int(fid)] + int(np.floor((tb - self.t[self._first[int(fid)]]) / self.dt)) + 1) return a, max(a, b)skills/en/audio-flyover-geolocation/scripts/afg_cli.py48 KB1b5d3de6c664upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/scripts/requirements.txt186 B4642f608d01eupstream ↗
# core numpy>=1.24 scipy>=1.10 matplotlib>=3.7 # optional: DEM GeoTIFF sampling (dem-fetch / --dem) rasterio>=1.3 # external (not pip): ffmpeg on PATH for video / compressed audio inputskills/en/audio-flyover-geolocation/references/adsb-data.md6.4 KB767cbd276b29upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/countermeasures.md3.0 KB7007ca0d19ffupstream ↗
# Privacy countermeasures: what works and what doesn't The end of the video is devoted to this. Conclusion: **only destroying the information itself (cutting it, replacing it) is reliable; signal-processing "masking" mostly fails.** | Measure | Effect | Why | |---|---|---| | **Cut out the fly-over** | ✅ Works | No curve means no v/d/tCPA. Cut at least one fly-over, or change the time interval between two fly-overs (the two-event method depends on it) | | **Replace / remove the original audio** (re-dub, narration only) | ✅ Most reliable | The background sound field disappears entirely | | Publish only very short clips (< 20 s, without the CPA) | ⚠️ Partial | With only one asymptote v and d are undetermined, but if the steep CPA section is included, k = v²/d still works | | **High-pass filter** (e.g. > 200 Hz) | ❌ Doesn't work | Propeller BPF harmonics extend to several hundred Hz and overlap the voice band. Filtering only attenuates; narrow-band tones can still be recovered in the whitened spectrogram | | **"Drowning it out"** (music, white noise, talking) | ❌ Mostly doesn't work | Broadband masking only raises the noise floor; a long narrow-band tone still has enough integrated SNR. Narrow-band parts of music are discontinuous and easy to tell apart | | **Phone-call codecs / noise suppression** | ❌ Unreliable | Speech codecs keep narrow-band tones between 100 and 4000 Hz. Noise suppression usually gates only when nobody is speaking; while you talk, the background goes through with your voice | | Speed / pitch change | ⚠️ Partial | A uniform speed change scales f0 and ΔT together; it can be undone from the 50/60 Hz hum or a known duration | | Adding a fake Doppler curve | ⚠️ Risky | Could mislead, but only "looks real" if it is consistent with real ADS-B, and it doesn't erase the real curve | ## Pre-publication checklist 1. `$AFG audit my.mp4 --out work/audit` → look at `risk` and `work/audit.png` (look for smooth descending curves). 2. Also check: - EXIF/GPS and QuickTime `©xyz` location tags, plus creation time; - in the picture: road signs, skyline, sun shadows, reflections; - other ambient sounds (bells, PA announcements, trains); - ENF (mains frequency), which can also be used for time forensics. 3. If risk is medium/high: cut the fly-over or replace the audio, then audit again. 4. If risk is `undetermined` or `low, limited check`, the automatic check did not clear the file: read `detection.reasons` and decide from the PNG. 5. Only falling ridges count as fly-overs. A candidate with `warnings` (its ridge rises again, or falls further than one pass can) counts as two. Rising tones are listed in `detection.rejected` and in `notes`; check in the PNG that no falling curve hides under them. ## Points to explain to the user - Inaudible to the ear ≠ absent from the spectrogram. - Two fly-overs + their time difference are more dangerous than one fly-over + exact time, because the recording time isn't needed. - Risk grows with nearby GA airfields / flight schools and helicopter route density.skills/en/audio-flyover-geolocation/references/physics.md6.5 KB75ed8dc66d38upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/search-and-scoring.md11 KBaa287967003fupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/sources.md2.8 KB9aae752499c9upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/spectrogram.md8.1 KB5dbaca0d92efupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/thunder-method.md2.6 KB88cae7bc7464upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/references/verification.md6.9 KBf9e78515ba63upstream ↗
Contents are not inlined in the catalog. Open the upstream file.
skills/en/audio-flyover-geolocation/assets/report_template.md1.8 KB7d2cea566affupstream ↗
# Acoustic geolocation report: <footage name> ## Conclusion - **Location**: <lat, lon> (<place name / landmark, if confirmed>) - **Error**: about ±<R> m (0.9 score region) | or: strip <end A> — <end B>, width <W> m, bearing <deg>° - **Recording start time (inferred)**: <UTC ISO> (metadata: <value / none>, difference <Δ>) - **Confidence**: high / medium / low / unusable — <one-sentence reason> ## Evidence | Event | tCPA in recording | f0 (Hz) | v (m/s) | d (m) | k=v²/d | Matched flight (hex / registration / type) | |---|---|---|---|---|---|---| | 1 | | | | | | | | 2 | | | | | | | - Search scope: <bbox / date / time window>; ADS-B source: <adsb.lol / ADSBx / OpenSky>; altitude datum: <geom / baro+QNH xxxx> - Funnel: flights <N> → corridor nodes <N> → event 1 <N> → offset <N> → silence <N> → candidates <N> - Scoring: margin_vs_2nd = <x>, robust_z = <x>, crossing angle = <x>° - Figures: spectrogram/fit `e1.fit.png`, `e2.fit.png`; model overlay `result.rank1.*.png`; heatmap `result.heatmap.png` ## Ambiguity and alternatives - <ambiguous_alternatives list: coordinates, score ratio, distance, same flight or not> - <mirror / strip explanation and the information needed to resolve it> ## Independent corroboration - <street view / terrain / sun / ambient sound etc., or "not done"> ## Limitations - ADS-B coverage: <low-level GA may be missing / what was checked this time> - Sound speed / wind: <temperature source, wind speed> - Footage processing: <signs of editing, transcoding, speed change> - Other cases that cannot be excluded: <...> ## Method Doppler curves (emission-time model) × historical ADS-B tracks, 3D slant distance + DEM, inter-event offset and silence constraints, locked scoring and local refinement (audio-flyover-geolocation skill; method after colsto's 2026-11-02 video).README.mdupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
Plain-text copy, no login: /skill-bundles/alanbacker-audio-flyover-geolocation-skills-en-audio-flyover-geolocation.md