⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
gap-trap
Use when asked to set up, install, or refine an agent quality framework in a repository (rules with gates, architecture contracts, proven-red tests, ratchets, playbooks), or when the user says "gap-trap", "gap-trap setup", "gap-trap refine", "gate the rules", or "make the agents' code match the rules".
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
Not yet listed on ClawHub or SkillsMP
// README
gap-trap
Turns vibe coding into high quality code. Complements SDD if you are using it (think of it as the inner loop within)
The problem
An agent writes most of your code. It is fast, so you stop reading every diff. Quality drifts, and you find out when something breaks.
The drift looks the same in every codebase. The agent writes a helper that already exists. It crosses a layer boundary because the shortcut compiled. It writes a test that asserts the code ran, not that it did the right thing. It follows a rule in your instructions file for a week and then forgets it.
Nothing stops any of this. The rules are text, and the only thing that reads them is the agent that breaks them.
What gap-trap does
Run it once in a repository. It reads the code, writes the rules that fit that codebase, and adds a gate for each rule. A gate is a check that fails the commit or the CI run when the rule is broken. The agent cannot skip a gate.
It sets up four things:
| What it is | |
|---|---|
| Contracts | One entry per part of the code that has one right way to do things (HTTP, logging, settings, auth). Each says what to use, what never to use, and which check finds bypasses. |
| Proven red | A CI job that runs every new test against the old code and fails when the test passes there. A test that cannot fail proves nothing. |
| Ratchets | Counts of known problems, such as the lint backlog or files over 400 lines. They may go down, never up. |
| Playbooks | The facts the project learned the hard way, written down so the next session does not learn them again. |
It also installs slop-mop, a sibling skill, so the agent's docs, commit messages, and PR bodies read like a person wrote them. A harness loads its skills when the session starts, so slop-mop applies from the session after the one that installs it.
Works with any language. Node and Python repos get gates inside their test suite; everything else (Go, Rust, Java, Ruby, .NET, PHP, Swift, C++) gets shell versions that need only git, grep, awk, and your test command.
IMPORTANT: Validate the rules/contracts that are generated. This is where it is important to review thoroughly so you start with a strong foundation.
Install
With the skills CLI:
npx skills add pliablepixels/gap-trap
Or by hand. Claude Code reads ~/.claude/skills, Codex reads
~/.agents/skills:
git clone https://github.com/pliablepixels/gap-trap.git
mkdir -p ~/.claude/skills # Codex: mkdir -p ~/.agents/skills
cp -r gap-trap/gap-trap ~/.claude/skills/
The skill runs on the top coding model of the harness (Opus or better in Claude Code, the frontier coding model at high reasoning in Codex) and stops on anything smaller. Choosing the wrong contracts costs every later session.
Update
Every push to main is a new version. With the skills CLI:
npx skills update gap-trap
For a hand install, pull the clone and replace the copy, so files removed upstream do not linger:
git -C gap-trap pull
rm -rf ~/.claude/skills/gap-trap # Codex: ~/.agents/skills/gap-trap
cp -r gap-trap/gap-trap ~/.claude/skills/
A repository you already set up keeps the files setup wrote. Updating
the skill changes what the next setup or refine run does.
Run
In the repository you want to set up:
/gap-trap setup
Codex invokes the same skill as $gap-trap setup.
The skill reads the repo, shows you a short plan (contracts, gates, commands), asks about anything it could not settle, then writes the files, proves each gate red, and commits. It does not push. At the end it asks whether to check the repo against the new framework. On yes, it lists the existing violations by contract and rule and changes nothing.
Later, after the first incidents or about once a month:
/gap-trap refine
This looks for rules without gates, contracts that name code that no longer exists, and lessons in the commit history nobody wrote down. It proposes one PR.
Where it comes from
The framework is the one zmNinjaNg runs on. Agents write most of its code and no one reads the diffs line by line. Chapter 14 of its developer guide, Agent development model, is the inspiration for gap-trap. It explains rules, contracts, and gates, and follows one feature from spec to merge.
What it costs
The origin repo's full gate set (4,400 unit tests, a build, three lints) runs in about a minute locally. The instruction gate adds about a second. Proven red runs only in CI.
Release
For the maintainer. scripts/release.sh bumps the version tag, writes
the notes from the commits since the last tag, and publishes the release
with gh:
scripts/release.sh # patch bump
scripts/release.sh minor # feature bump
scripts/release.sh v1.0.0 # that exact version
scripts/release.sh --dry-run # print the version and notes, change nothing
It stops if you are not on main, the tree has uncommitted changes, or
main and origin/main differ. The skills CLI installs from main, so
a tag marks a version rather than gating one.
Layout
gap-trap/
SKILL.md the skill: model gate, setup, refine
reference/ the framework, discovery, gates, refine
templates/ AGENTS.md, AGENTS.project.md, playbooks, gate scripts
License
MIT.
// HOW IT'S BUILT
// FILES
25 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.
gap-trap/SKILL.md9.0 KB262b7d9b653dupstream ↗
--- name: gap-trap description: Use when asked to set up, install, or refine an agent quality framework in a repository (rules with gates, architecture contracts, proven-red tests, ratchets, playbooks), or when the user says "gap-trap", "gap-trap setup", "gap-trap refine", "gate the rules", or "make the agents' code match the rules". --- # gap-trap Sets up a quality framework for repositories where agents write most of the code. Every rule the agents follow has a gate that fails a commit, a push, or a CI run when the rule breaks. A rule a script could check but that names no gate is a defect. The framework is the one described in `reference/framework.md`; read it before either mode. Two modes: `setup` builds the framework in a repo that has none. `refine` audits a repo that has it and proposes the next rules and gates from what broke. Bare `gap-trap` with no keyword means `setup` when the repo has no `AGENTS.project.md`, otherwise `refine`. A hand-written `AGENTS.md` or `CLAUDE.md` on its own is input to setup, not a sign the framework exists. ## Model gate This skill runs on the top coding model of your harness: in Claude Code, Opus or a more capable model (Opus 4.8 or newer, Opus 5, Fable, Mythos); in Codex, the frontier coding model at high or xhigh reasoning effort. Discovery decides which parts of a codebase get a contract, and a wrong contract costs every later session. Your system prompt names your model. If it is a cheaper or faster tier (Sonnet, Haiku, a mini model, or low reasoning effort), stop. Read nothing, write nothing. Say exactly this and end the turn: > gap-trap runs on the top coding model of this harness. You are on > <model name>. Switch with `/model`, then run gap-trap again. Do not dispatch stronger subagents from a weaker orchestrator, start "the easy parts", or offer a lighter version. ## setup 1. **Discover.** Follow `reference/discovery.md`. It produces a plan file in your scratch directory: stack, commands, CI provider, contract candidates with real symbol names, the first domain facts from history (the revert probe runs at any commit count), and one row per rule found in an existing instruction file with where it goes and what gates it. 2. **Confirm once.** Show the plan as a short list: the contracts you will write (at most six), the gates you will add, the test and lint commands you found, the migrated-rules table, and where slop-mop goes. A build, type check, or lint that is already red is the first item: setup fixes only what its own gate files need (a missing type package, a lockfile) and records the rest as a ratchet count or an out-of-scope entry. Name the cause of each red before recording it: reproduce it, read the failure, and say what is broken. A failure that names an environment variable can still be a code defect. In one trial, seven tests failing with `Environment variable not found: DATABASE_URL` were a Prisma mock loaded after the module under test, not a missing `.env`; recording that as an environment gap would have buried the bug and the contract that prevents it. Ask one round of questions, only about what discovery could not settle. Then proceed. 3. **Install slop-mop.** `git clone https://github.com/pliablepixels/slop-mop.git` into your scratch directory and copy `slop-mop/slop-mop` to the skills directory of the harness you are running in (Claude Code: `~/.claude/skills/`; Codex: `~/.agents/skills/`), skipping it if present. It is not vendored into the repo; the project rule in `AGENTS.project.md` requires it. A harness loads its skill list when the session starts, so a skill installed here is not available to the run that installed it: write this run's prose from P10 in `AGENTS.md`, and tell the user in the report that slop-mop applies from their next session. 4. **Write the instruction files** from `templates/`. `AGENTS.md` and `agents/generic/agent-workflows.md` are copied unchanged, with one exception: a rule whose gate cannot exist in this repo (I3's accessibility lint and C3's locales in a repo with no UI) gets its `Gate:` clause replaced by `Gate: n/a, no <thing> in this repo`. Nothing else in `AGENTS.md` changes. `AGENTS.project.md`, `CLAUDE.md`, and `agents/project/` are filled from the plan; drop the UI and end-to-end lines and `{{E2E_CMD}}` when the repo has no UI, and delete every placeholder you cannot fill with a verified value. Contracts name real symbols only. Existing instruction files (`AGENTS.md`, `CLAUDE.md`, `.cursorrules`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`) are migrated from the plan's migrated-rules table, one row at a time: a contract row becomes a contract block, a rule row becomes a project rule with its gate name or `Gate: review`, a playbook row goes verbatim into the named `agents/project/` file, a command row fills the verification block, and a duplicate or dropped row is left out. No rule is dropped silently; the confirm step showed the table. A file whose name a template owns (`AGENTS.md`, `CLAUDE.md`) is replaced by that template. Any other instruction file (`.cursorrules`, `.github/copilot-instructions.md`) becomes a one-line pointer to `AGENTS.md`, so the agent that reads it lands on the rules instead of a stale copy. `CONTRIBUTING.md` stays where it is. The user sees the diff. 5. **Write the gates** from `reference/gates.md`, in the repo's own test runner and CI provider: the instruction gate, proven red, the PR body check, the ratchet (always, seeded with the counters the repo supports: files over the C2 length, lint problems per rule when a linter runs, each contract Never clause that has violations today), and the mutation smoke when discovery named a risky module (auth, a parser, the API client); otherwise refine adds it after the first fix chain. The stack table there picks the reference: native ports for Node and Python, shell references that need only git, grep, and awk for every other language. Set the instruction gate's word budget at the current count plus half, rounded up to the next 500. 6. **Prove each gate red** with a scratch violation, remove the violation, run the repo's full test command, and commit one logical change per commit, in this order: the instruction files with the gate that checks them; one commit per gate after that (ratchet, proven red, mutation smoke, PR body check); the combined gate command and the pre-commit hook; CI last. One commit holding the whole framework hides which gate was proven red against what, and leaves nothing to revert on its own. Do not push unless asked. 7. **Report** what was written, the instruction word count against the budget, which contracts say `Gate: review` and why, whether this run installed slop-mop and so it applies from the next session, any always-loaded file outside the repo whose prose style conflicts with the repo's (`~/.codex/AGENTS.md`, `~/.claude/CLAUDE.md`, a terse or compressed house style): name the file, and say that the repo's rule governs repo prose. End with the one next step: run `gap-trap refine` after the first incident or in a month. 8. **Offer a check.** End the report with one question: check the repo against the new framework now? On no, stop. On yes, change no files and list, grouped by contract or rule ID: - each ratchet counter, its count, and the file and line behind each unit of it; - each `Gate: review` contract: read the code its `Path:` and `Never:` lines cover and list suspected violations with file and line, marked unverified because no gate checks them; - the red build, type check, or lint items recorded in step 2. Fix nothing in this pass. A fix the user picks is its own commit, and a fix that lowers a count reruns the ratchet with `--update`. Instruction files are read by agents. Write them as terse statements with IDs and gate names. Human-voice prose belongs in the repo's docs, not in `AGENTS*.md` or `agents/`. ## refine Follow `reference/refine.md`. It audits rules without gates, contract names that no longer exist, and the commit history since the last refine, and it proposes edits through the self-improvement protocol as one PR. It removes rules with no incident behind them as readily as it adds gates. ## Common mistakes | Mistake | Fix | |---|---| | A contract with a guessed symbol name | Every name in a `Path:` line is greppable in the tree; the instruction gate fails otherwise | | Writing ten contracts on day one | Start with the places that have one sanctioned path and a violation today; add the rest through refine | | A gate that passed on first run | Prove it red with a scratch violation first; a gate that never failed proves nothing | | Vendoring slop-mop into the repo | It installs per agent from its repo; the project rule requires it | | Instruction prose polished for people | Agents read it. Statement, one-clause why, gate name | | Skipping the model gate because the repo is small | Discovery quality is the deliverable; the gate is first for a reason |gap-trap/reference/discovery.md7.0 KB1bf674647cbcupstream ↗
# Discovery Output: a plan file in your scratch directory with the sections below. Every claim in it is verified by a command you ran. Do not guess a symbol name; grep it. ## 1. Stack and commands Record, with the file that proves each: - Languages and the primary one (by file count under the source dirs). - Package manager and the working directory the commands run from (a monorepo may have several; pick the one agents will edit). The manifest names the stack: | Manifest | Stack | Test | Lint / vet | Type or build | |---|---|---|---|---| | `package.json` | Node | `npm test` (vitest, jest) | `eslint .` | `tsc --noEmit` | | `pyproject.toml`, `setup.py`, `requirements.txt` | Python | `pytest` | `ruff check .` | `mypy` | | `go.mod` | Go | `go test ./...` | `go vet ./...`, `golangci-lint run` | `go build ./...` | | `Cargo.toml` | Rust | `cargo test` | `cargo clippy` | `cargo build` | | `pom.xml`, `build.gradle*` | Java, Kotlin | `mvn test`, `gradle test` | checkstyle, detekt | compile step | | `Gemfile` | Ruby | `rspec` | `rubocop` | none | | `*.csproj`, `*.sln` | .NET | `dotnet test` | analyzers | `dotnet build` | | `composer.json` | PHP | `phpunit` | `phpstan` | none | | `Package.swift` | Swift | `swift test` | swiftlint | `swift build` | | `CMakeLists.txt`, `Makefile` | C, C++ | `ctest` | clang-tidy | build | Confirm each command by running it once; the plan records the output line that proves it. - Test runner and its single-run command. Lint command(s). Type check. Build. One combined "gates" command exists? If not, you will add one. Run each once and record whether it is green today; a red build or lint goes to the top of the confirm step. - Node: a committed lockfile (`npm ci` in CI needs one) and `@types/node` when the gate file will import `node:*` under a strict type check. - CI provider and workflow files. Branch protection (`gh api repos/{owner}/{repo}/branches/{default}/protection` when `gh` works). Pre-commit hooks (husky, pre-commit, lefthook). - Existing instruction files: `AGENTS.md`, `CLAUDE.md`, `.cursorrules`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`. Record each path; section 4 sorts their rules. - Always-loaded files outside the repo: `~/.codex/AGENTS.md` and `~/.claude/CLAUDE.md`. They are not migrated and not edited. Record a prose-style rule there that conflicts with P10, such as a terse or compressed house style, because setup names it in the report and the prose this run writes will otherwise follow it. - Docs: user docs, developer docs, ADRs, and where they live. ## 2. History probes Count with `git rev-list --count HEAD`, never a piped `wc`. The revert probe runs at any commit count; the fix-chain and churn statistics need about 50 commits to mean anything. - Reverts: `git log --grep=revert -i --oneline`. Each is a paid experiment; the lesson is what not to retry, and its subject line is the first `domain-context.md` entry even in a six-commit repo. - Fix chains: `git log --format=%s | grep -oE '^fix\(([^)]+)\)' | sort | uniq -c | sort -rn`. A scope with many fixes is one misunderstanding coming back. - Churn files: files touched most by fix commits. Read the fix bodies. - Same-day fix pairs: a PR followed by a fix within a day is a review gap. Each probe yields candidate `domain-context.md` entries (one fact, the hashes behind it) and candidate contracts (a recurring class). ## 3. Contract candidates A contract exists where the code has one sanctioned path. Find them: - **Wrappers with bypasses.** A module that wraps a platform API and raw uses of the same API elsewhere. Grep the raw API, list the files. The wrapper is the Path, the raw uses are today's violations, and the grep is the gate. What to grep for by concern: | Concern | Raw API to grep | Wrapper looks like | |---|---|---| | HTTP | `fetch(`, `axios`, `requests.`, `httpx.`, `http.Get`, `http.NewRequest`, `reqwest::`, `HttpClient` | `lib/http`, `httpx/client.go`, `net/client.rs` | | Logging | `console.`, `print(`, `fmt.Print`, `log.Print`, `println!`, `System.out` | `logger`, `logx`, `tracing` setup | | Settings, env | `localStorage`, `os.environ`, `os.Getenv`, `env::var`, `System.getenv` | `settings`, `config` | | Database | driver calls (`cursor.execute`, `sql.Open`, `db.Query`, `sqlx::query`) outside a repository layer | `repo/`, `store/`, `dao/` | | Auth, secrets | token strings in URLs or logs, `Authorization:` built by hand | one token helper | | Time, sleep | `time.sleep`, `time.Sleep`, `setTimeout` in tests | a clock or an awaited condition | - **Fan-in modules.** Modules imported from many places (count importers). Each is a concern something owns. - **Security paths.** Token storage, auth refresh, secrets in logs or URLs. These get a contract even without a violation. - **State.** The store or context library and its subscription rules. - **Recurring fix scopes** from the history probes. For each candidate write the four lines with real names, then grep every name in `Path:` to confirm it exists. Rank by violations today and by blast radius. Take at most six for setup. Note which Never clauses a text search can settle (those get a grep gate) and which cannot (`Gate: review`). ## 4. Migrated rules Skip when section 1 found no instruction file. Otherwise read every file found and split it into single rules: one statement, one imperative, one fact. A rule that is already a template rule (a G, P, C, I, or M line in `templates/AGENTS.md`) is a duplicate. Sort the rest by what the codebase confirms: | Rule says | Disposition | Where it lands | |---|---|---| | Use X, never Y, for one subsystem, and X exists in the tree | contract | section 3, ranked with the other candidates | | A do or do-not with no single owner | project rule | `AGENTS.project.md` project rules, with a gate | | A fact about an API, a quirk, a decision, a term | playbook | `agents/project/domain-context.md` or `glossary.md`, verbatim | | A test, lint, or build command | command | section 1, if it matches what runs today | | Prose about the project, tone, or the reader | dropped | nowhere; list it so the user sees it go | Every contract and project rule gets a gate the same way refine does: grep the Never clause or the rule. Clean today: a zero-tolerance grep in the instruction gate. Violations today: a ratchet counter at that count. No text search settles it: `Gate: review`. A rule that names a path or symbol that no longer exists is reported, not migrated. Write the table into the plan with the source file and line for each row. It goes on the confirm list next to the contracts. ## 5. Gate plan For each gate in `reference/gates.md`, record: applies here (yes/no and why), the file it will live in, the command that runs it, and the scratch violation that will prove it red. ## 6. Questions for the user Only what discovery could not settle: which of two test commands is canonical, whether a candidate contract's bypasses are deliberate, whether CI secrets exist for integration tests. If nothing is open, say so and proceed after showing the plan.gap-trap/reference/framework.md4.8 KBe0d656829c47upstream ↗
# The framework What gap-trap installs, in agent terms. The origin is zmNinjaNg (`docs/developer-guide/14-agent-development-model.rst` there), where one maintainer landed 2,313 commits in eight months (fourteen reverts) with agents writing the code and no one reading diffs. This file is the model; the templates are its files. ## Four kinds of instruction | Kind | Where | Form | Wins on conflict | |---|---|---|---| | Rule | `AGENTS.md` | ID, statement, one-clause why, gate | rule | | Contract | `AGENTS.project.md` | Owns, Path, Never, Gate for one subsystem | rule | | Practice | `agents/**` playbooks | advice with evidence (commit hashes), no ID | loses to rules | | Fact | `agents/project/domain-context.md` | API quirk, platform behavior, failed approach, with hashes | loses to rules | Rule IDs carry a tier letter: I invariants (never traded away), P process, C code, M meta (govern the instruction files). Docs cite IDs, never copied text, so a rule changes in one place. ## Contracts A contract is four lines for one subsystem with one sanctioned path: ``` ### HTTP Owns: all network requests, including TLS handling. Path: helpers in `src/lib/http.ts`. Never: raw `fetch` or `axios`. Gate: `tests/instruction-gate.test.ts` (no raw `fetch`/`axios`). ``` An agent reads it and learns, without opening source, who owns the concern, the one way to use it, that a working bypass is still a bug, and what will fail if it bypasses. `Gate: review` is allowed only when no text search settles the Never clause. ## Gates A gate fails a commit, a push, or a CI run. Kinds, cheapest first: - **Grep gate**: a test that scans source for a forbidden pattern (`console.`, raw `fetch`, a static plugin import). Milliseconds. - **Instruction gate**: a test over the instruction files themselves. Contract names exist in the tree, `AGENTS.md` has no project names, the always-loaded files stay under a word budget, cited commit hashes exist, no emails or IPs in knowledge files. - **Ratchet**: a stored count that may fall or hold, never grow. For lint backlogs, mocks of internal modules, existence-only assertions, files over a length. Raising a number by hand needs a reason in the commit message. - **Proven red**: CI runs the tests a range changed against the code from before the range, in a worktree at the fork point, and fails when they pass there. It says whether the red was an assertion failure or only a missing symbol. - **Mutation smoke**: flip one branch in each of a few risky modules and require the module's tests to fail. The one check that proves existing tests can fail. - **PR body check**: the body quotes the issue's acceptance lines; a `feat` PR names its spec or says why it has none. Rule M2: read what a gate measured, not only its exit code. A job that skips itself is green. ## Process rules that matter most - P1 issue before work; PR quotes acceptance lines. - P2 failing test before implementation; proven red enforces it. - P3 run the gates covering a change before commit, all before push. - P6 gates run as bare commands; output wrappers and compressors hid failures. - P10 docs move with behavior. - M1 a checkable rule needs a gate in the same change. - M3 instruction files change only through the self-improvement protocol. - M5 durable facts go to the domain playbook, not agent memory. ## Self-improvement protocol Trigger: a breakage, a review finding, or a wasted session that a rule, contract, or fact would have prevented. Action: the PR that fixes it also proposes the instruction edit, with the gate M1 requires. The maintainer merges or rejects it like any diff. Rules are removed the same way when they cost more than they catch. The `refine` mode is the periodic sweep for lessons nobody recorded at the time. ## Context budget `AGENTS.md`, `AGENTS.project.md`, and `CLAUDE.md` load into every session, so the instruction gate caps their combined words (start at current count plus room; zmNinjaNg sits near 2,200 under a 4,000 cap). How they load differs by harness. Claude Code reads `CLAUDE.md`, which imports the other two. Codex reads `AGENTS.md` from the repo root down and has no import syntax, so it reaches `AGENTS.project.md` only through the load-order line at the top of `AGENTS.md`; keep that line. Detail goes to playbooks, which load only for work in their area. Instruction-following degrades with rule count, so every addition answers: could a script check this instead? ## Review Mechanical tasks rely on their gates. Judgment work gets a second agent against the brief before the next task. Before a PR, one whole-branch review on two axes in two contexts: Standards (the diff against contracts and playbooks, skipping what a gate enforces) and Spec (the diff against the acceptance lines). Findings stay under separate headings; a change can pass one axis and fail the other.gap-trap/reference/gates.md12 KB120ea3862b7dupstream ↗
# Gates Each gate below has a spec and reference implementations under `templates/gates/`. Pick by stack: | Stack | Instruction gate | Proven red | Ratchet | |---|---|---|---| | Node (vitest/jest) | `instruction-gate.test.ts` | `proven-red.mjs` | `ratchet.mjs` | | Python (pytest) | `instruction_gate_test.py` | `proven-red.sh` with `python -m pytest "$@"` | `ratchet.sh` | | Go, Rust, Java, Ruby, .NET, PHP, Swift, C++, anything else | `instruction-gate.sh` as a CI step and pre-commit hook, or a port into the runner | `proven-red.sh` with the runner in `run_tests` | `ratchet.sh` | The `.sh` references need only git, grep, awk, and the repo's test command, and were run against a Go repo and the origin TypeScript repo. `github-ci.yml` is written for Node; on another stack replace the setup-node and `npm` steps with the toolchain's (`actions/setup-go`, `actions/setup-python`, `dtolnay/rust-toolchain`) and call the `.sh` gates through the Makefile. A native port into the runner is better when one is cheap (the gate then runs with `npm test` or `pytest` and nobody forgets it); the shell gate is the fallback that always works. Keep the same assertions and failure messages whichever you pick. Every gate is proven red with a scratch violation before it lands. Unit test file patterns by stack, for the proven-red classifier and the test-dir exemptions: | Stack | Unit test files | Runs a subset by | |---|---|---| | Node | `*.test.ts`, `*.spec.ts`, `__tests__/` | file paths | | Python | `test_*.py`, `*_test.py`, `tests/` | file paths | | Go | `*_test.go` beside the source | package dirs (`go test ./pkg`) | | Rust | `#[cfg(test)]` in source, `tests/*.rs` | `cargo test <name>`; whole crate is fine | | Java/Kotlin | `*Test.java` under `src/test` | `-Dtest=Class` (Maven), `--tests` (Gradle) | | Ruby | `*_spec.rb`, `spec/` | file paths | | .NET | `*Tests.cs` | `--filter` | ## Instruction gate Spec (one test file, runs with the unit tests). In `Path:` and `Gate:` lines, backtick only what the gate can look up: a file path (contains `/`) or a bare symbol. A symbol on a `Path:` line is looked for inside the paths that line names, because that is the claim the line makes; a symbol anywhere else is looked for across the non-test source. A counter name, a dotted call such as `log.Print`, or a flag goes unquoted. Lines may wrap: a field runs on until the next field or a blank line. The minimum contract count is the honest number, two is fine; never pad. Cite commit hashes with at least 7 characters (`git log --oneline` prints 7) and the gate resolves them. 1. `AGENTS.project.md` has at least N contracts and each has `Owns:`, `Path:`, `Never:`, `Gate:` lines. 2. Every backticked token in a `Path:` or `Gate:` line exists: a token with `/` is a file path; any other is a symbol, resolved against the paths its own `Path:` line names and otherwise against the non-test source. A symbol that survives only in a test is not a symbol the code uses. Failure names the contract and the token. A contract whose `Path:` and `Gate:` lines give the gate nothing to look up fails too. 3. `AGENTS.md` contains none of a list of project-specific tokens (product name, framework names, file paths). Fill the list from discovery. 4. `AGENTS.md` + `AGENTS.project.md` + `CLAUDE.md` all exist and stay under a word budget. Set it at current count plus room; raising it needs a reason in the commit message. Summing only the files that happen to be present means deleting one comes in under budget. 5. Every 8-hex commit hash cited in `domain-context.md` exists (`git cat-file -e <hash>^{commit}`; CI needs full history). 6. Knowledge files (`domain-context.md`, `glossary.md`, `out-of-scope.md`, `agent-workflows.md`) contain no email or IP address. 7. Every `rule <ID>` cited in the developer docs exists in `AGENTS.md`. 8. Grep gates for each contract Never clause a text search settles, comments stripped by a scanner that knows a string from a comment, with the sanctioned file(s) exempt. The instruction gate holds only the clauses that are clean today (zero is the number). A clause with violations today goes into the ratchet as a count instead, so the backlog can fall but not grow and the instruction gate never carries an allowed number. 9. Every check states its own denominator: tokens resolved, files scanned, contracts read. A check that quietly measured nothing reports the same green as one that measured everything, which is how a gate dies without anyone noticing (M2). References: `templates/gates/instruction-gate.test.ts` (vitest), `templates/gates/instruction_gate_test.py` (pytest), `templates/gates/instruction-gate.sh` (any stack; config through `GT_*` environment variables or the block at the top). The vitest port imports `node:*` and uses `__dirname`; a repo whose type check covers the test directory needs `@types/node` installed or the gate itself fails `tsc`. ## Proven red Spec: for a PR range, take the unit test files changed since the fork point, copy them into a worktree at the fork point, run each one there, and fail when it passes. Judge each file on its own result: batched into one run and one exit status, a file that passed on the old code rides out on a failing sibling. The same holds for tests inside one file where the runner reports each test: a test whose name the fork-point copy of the file lacks is new, and a new test that passes on the old code fails the gate even when another test in its file went red. The Node port gets the old names by running the fork-point copy first. The shell port only sees a file's exit status, so there a new test can still ride out on a failing test in the same file. A test deleted at head is not run, because the worktree still holds its older copy, which passes. Skip and say why when no source changed, when the only source changes are the gate scripts (proven red by scratch violation), when the title type is `docs|chore|ci|refactor|build|style|test` and no unit test changed, or when the only changed tests are end-to-end and need a live server. The last two are bypasses rather than proofs — a title is unverified text — so they are announced as CI warnings. A test-support file is not a test and excuses nothing. Fail when source changed and no test did. Read the runner's report: count assertion failures and missing references (`is not a function`, `Cannot find module`, `undefined:`) apart per file, print both, and warn for each file whose red is only a missing reference. Match assertion evidence against the runner's own verdict lines, not the whole output: pytest echoes the failing source line, so a bare `assert` over everything marks every missing-symbol red as an assertion red. A non-zero exit with no report, no sign the runner started, or a status of 126, 127 or a signal is a gate that could not run: exit 2, never a pass. References: `templates/gates/proven-red.mjs` (Node, reads the vitest JSON report) and `templates/gates/proven-red.sh` (any stack, reads the runner's text output with a cross-language missing-symbol pattern). Adapt the three classifiers (unit test, test support, non-code) and the `run_tests` command. CI job: `templates/gates/github-ci.yml`. ## Ratchet Spec: a JSON baseline of named counts. The check fails when a count grows, prints improvements when one falls, and fails when the baseline sits more than a small slack above the real count (a raised number nobody lowered back). Every baseline name is checked, not only the configured counters, so deleting a counter cannot retire the number it held. A counter command that fails, or prints anything but one whole number, is an error and exits 2: counted as zero it reads as a cleared backlog. `--update` only lowers. A count that rose, or a baseline name whose counter has gone, needs a hand edit, so the raise arrives as a reviewable diff with a reason in the commit message rather than as a flag the agent can reach for. The ratchet is always installed, because C2, C6, and C7 in `AGENTS.md` name it as their gate. Seed it with what the repo supports: files over the C2 length limit (always), lint problems per rule when a linter runs (from its JSON output; ESLint's `max-lines` rule then covers C2 with no new code), each contract Never clause that has violations today, test files that mock the repo's own modules, assertions that only prove existence, fixed sleeps in end-to-end steps. References: `templates/gates/ratchet.mjs` (Node) and `templates/gates/ratchet.sh` (any stack: counters are shell one-liners in `.ratchet-counters`, the baseline is a text file). ## PR body check Spec: a CI job on same-repo, non-bot PRs. Fails when `## Acceptance` has no content after stripping HTML comments. On a `feat` title, fails when `## Spec` has no content. The template is `templates/pull_request_template.md`. The check lives in a script the job calls, so it can be proven red locally with a body on stdin. The title and body reach the script through environment variables, never interpolated into the `run:` text (a PR title is attacker-controlled). Reference: `templates/gates/pr-body-check.sh`, called by the `pr-acceptance` job in `templates/gates/github-ci.yml`. On GitLab the same shell reads `CI_MERGE_REQUEST_DESCRIPTION` and `CI_MERGE_REQUEST_TITLE` in a `rules: - if: $CI_MERGE_REQUEST_IID` job. The three CI jobs (gates, proven red, PR body) map one to one onto any provider; only the checkout depth (full history for the hash check) and the toolchain setup step change. ## Mutation smoke Spec: a list of `{file, from, to, tests}`; for each, replace `from` with `to`, run `tests`, require failure, restore the file whether or not the run failed. Start with three or four risky modules (auth, the API client, a parser). Runs in CI, about two seconds per target. Reference: the pattern is short enough to write from the spec in any language (`sed -i` the mutation, run the tests, `git checkout` the file in a trap); see `app/scripts/mutation-smoke.mjs` in zmNinjaNg for one. ## Combined gate command and hooks Add one command that runs the unit tests, the build or type check, the blocking lints, the instruction gate, and the ratchet, so P3 has a name to cite: `npm run gates` on Node, a `gates` target in a `Makefile` everywhere else (Python, Go, Rust, Java all read a Makefile). Cost it: the origin repo's runs in about a minute locally. Pre-commit: Node repos use husky; every other stack gets a versioned `.githooks/pre-commit` that runs the instruction gate and the ratchet, enabled once per clone with `git config core.hooksPath .githooks`. The project rules name that command so an agent runs it on a fresh clone. CI re-runs the same gates, so a skipped hook changes nothing that merges. No CI provider yet: write the GitHub Actions workflow when the remote is GitHub or there is no remote (it is the reference and costs nothing unused), and say in the report that no CI runs until the repo is pushed there. A GitLab remote gets the three jobs as `.gitlab-ci.yml`. ## What not to add - Coverage thresholds gate nothing on a PR run. A ratchet on what tests catch beats a percentage. - A second run of the same linter (an advisory pass beside a ratchet). - Any gate whose input you have not read once. A gate that scans nothing is green (M2); assert the file count it scanned. - A pass-branch for a known red. A gate that reads the failure text and succeeds on a match (`if output includes "Environment variable not found: DATABASE_URL" then pass`) exempts every future failure that prints the same string, including the one it was built to catch. A red that stays for now is a ratchet count or an out-of-scope entry, recorded outside the gate, and the gate keeps failing on it.gap-trap/reference/refine.md2.5 KB9bc86305de80upstream ↗
# Refine Periodic sweep for a repo that already has the framework. Output is one PR through the self-improvement protocol (M3) plus a report in the session. Every proposal carries evidence: a commit hash, a grep count, or a gate's output. No evidence, no proposal. ## 1. Read what exists `AGENTS.md`, `AGENTS.project.md`, every file under `agents/`. Only new lessons get proposed; re-reporting a covered fact is noise. ## 2. Rules without gates (M1) For each rule and each contract `Gate:` line, classify: gated by a script, `review`, or nothing. For every `review` or nothing, ask whether a text search settles it. Grep it. Three outcomes: - Grep is clean today: add a zero-tolerance gate. Cost is milliseconds. - Grep finds violations: add a ratchet at today's count and list the sites. - No text search settles it: leave `Gate: review`, and check the rule has an incident behind it (step 4). A review-only rule with no incident is a deletion candidate. ## 3. Contract truth Run the instruction gate. Then, for each contract, grep the `Path:` symbols and confirm the sanctioned path is what the code does. A code/contract mismatch is a finding either way: the code drifted, or the contract is wrong. ## 4. History since last refine Window: since the last commit whose message contains `history-mined`, or the full history the first time. Run the probes in `reference/discovery.md` section 2. For each candidate: | Signal | Verdict | |---|---| | Reverted approach with a working alternative | domain entry: what failed, what works, do not retry | | 3+ fixes, one misunderstanding | domain entry stating the stable fact | | Recurring class across dozens of commits | contract candidate with Owns/Path/Never/Gate | | Security-shaped fix | contract Never-line candidate | | One-off fix, no reusable fact | skip | | Fix predates the rule that now covers it | already handled; list it | ## 5. Cost Rules load into every session. For each rule, find the incident it prevented (domain-context, history). None found: propose deletion. Duplicate facts across files: keep one home. Gates that overlap (two tests on one invariant): keep one. Report the words removed against the words added; the PR should not grow the always-loaded files without a gate to show for it. ## 6. Propose One PR: instruction edits, new gates proven red, baseline updates, domain entries verbatim with hashes. Commit message includes `history-mined`. Report in the session: proposals with evidence, the checked-and-already-covered list, and the word count before and after.gap-trap/templates/AGENTS.md5.2 KB949823178c9bupstream ↗
# Development Guidelines Portable core. This file contains no project-specific names and copies verbatim into any project. Load order: this file, then `AGENTS.project.md` (architecture contracts, project rules, verification commands, playbooks), then the playbook it lists for your work area. Adopting this core in another project: copy this file unchanged, then write your own `AGENTS.project.md` and the gates it names. Project facts never belong in this file. ## Rule format Every rule is a statement, a one-clause why, and a gate. A rule a script could check but names no gate is a defect here. Rules carry stable IDs by tier; docs reference IDs, never copied text. ## Invariants (never simplified away) - I1. Validate at trust boundaries: server responses, user input, IPC. Malformed input is routine, not rare. - I2. Destructive operations need error handling and a recovery path. Lost user data cannot be patched later. - I3. Security and accessibility are never traded for simplicity or speed. Gate: blocking accessibility and correctness lints. ## Process - P1. Create or use an issue before feature or bug work, after checking the out-of-scope ledger; land through an issue-linked PR whose body quotes the issue's acceptance lines. Commits reference the issue; closing keywords only after the user confirms. If instructed to push directly to the default branch, do so and verify the issue timeline. Typo-level fixes and doc corrections with no behavior change need no issue. - P2. Test first: a failing test precedes the implementation of every feature and bugfix. A test that has never failed does not demonstrate it can catch the bug. Changes an existing gate already covers fully rely on that gate instead of a bespoke new test. Gate: the proven-red CI job runs each change's tests against the pre-change code, fails when they pass, and says whether the red was an assertion or only a missing symbol. - P3. Run the gates covering the change before every commit; run the full suite before push or PR. Never commit after a failed or unrun gate. - P4. Read failures and fix the cause. Never blindly retry. - P5. One logical change per conventional commit. - P6. Verification runs direct commands. Tooling that transforms output, such as wrappers, compressors, or summarizers, is untrusted until validated once against raw output. - P7. Finish the requested behavior. Materially different UX options need approval before choosing. - P8. Never merge the default branch without approval. - P10. Docs move with behavior: user docs for changed behavior, developer docs and call flows for new APIs, components, hooks, and utilities. All prose reads like a developer explaining to a colleague: no marketing language, filler, headline headings, aphorisms, or news cadence. ## Code - C1. Reuse ladder: existing codebase helper, then stdlib, then platform feature, then installed dependency, then new code. A new dependency is a last resort. - C2. Keep files under 400 lines of code. No dead code, commented-out replacements, or speculative abstractions. Gate: the lint ratchet holds the count of over-long files. - C3. Never hardcode user-facing text; every locale updates together. - C4. Never inline semantic values; constants live in their dedicated modules. - C5. New modules live in domain folders. - C6. Test assertions must be able to fail: assert fetched values or user-visible outcomes, never element existence or child count. Gate: the quality ratchet. - C7. The lint ratchet baseline shrinks or holds, never grows. Raising a number by hand needs a reason in the commit message. ## Meta (governs this file) - M1. A rule a script can check needs a gate, added in the same change. Ungated rules drift; an audit of ungated rules found every one violated while every gated rule held. - M2. A gate's input needs checking, not just its exit code. Confirm the number a gate reports describes what it claims to measure. - M3. Instruction files change only through the self-improvement protocol below. One-off facts go to the project file or a playbook, never here. - M4. This file owns process rules; other docs link to rule IDs and never copy the text. - M5. Project facts (API quirks, platform behavior, failed approaches) go to the domain playbook, proven workflow practices to the generic playbooks, through the protocol, never only into agent memory, which no other agent sees. Repo files carry no personal or private data; only such specifics (names, hosts, credentials) and unproven taste stay in agent memory. ## Self-improvement protocol Trigger: a breakage, review finding, or wasted session an instruction or contract would have prevented, or an instruction that itself caused harm. Action: the PR fixing the problem also proposes the instruction edit, with the gate M1 requires. The maintainer merges or rejects it like any diff. Agents never edit instruction files outside this protocol. ## Project knowledge Architecture contracts, project rules, verification commands, and the playbook table live in `AGENTS.project.md`. Each contract states what it owns, the sanctioned path, forbidden bypasses, and the gate. Trust the contract over rediscovering the invariant from code; a code/contract mismatch is a finding for the self-improvement protocol.gap-trap/templates/AGENTS.project.md2.5 KB229ccd3e6409upstream ↗
# {{PROJECT}} Project Instructions Read `AGENTS.md` first. Contracts, project rules, verification, playbooks. The sanctioned path is the only path; a bypass is a bug even when it works. ## Architecture contracts <!-- One block per subsystem with one sanctioned path. Real symbol names only; the instruction gate greps every backticked token in Path and Gate. `Gate: review` only when no text search settles the Never clause. --> ### {{CONTRACT_NAME}} Owns: {{what this subsystem owns}}. Path: `{{sanctionedSymbol}}` (`{{path/to/module}}`). Never: {{the bypasses that count as bugs even when they work}}. Gate: `{{path/to/instruction-gate}}` ({{what it greps}}); the ratchet holds {{counter name}}; review for the rest. ## Project rules - Run commands from `{{WORKDIR}}`. Run `{{HOOK_INSTALL}}` once per clone so the pre-commit gates exist. - Prose people read (docs, commit bodies, PR and issue bodies, review comments) is written with the slop-mop skill; prose reviews run its detect mode on the harness's top coding model. - UI changes that alter behavior or navigation need an outcome-based end-to-end test; cosmetic changes rely on existing gates. - A `feat` PR links its spec or says in its `## Spec` section why it needs none. Gate: the `pr-acceptance` CI job. - GitHub comments by an agent end with `Posted by <agent>, assisting @<login>.` where `<login>` comes from `gh api user --jq .login`. ## Verification ``` {{GATES_CMD}} # unit tests, build or type check, blocking lints {{TEST_CMD}} {{LINT_CMD}} ``` Per commit, run what the change touches; the full set before push or PR. Ratchet baselines lower with `{{RATCHET_CMD}} --update`, which refuses to write a rise; raising one is a hand edit to the baseline file with a reason in the commit message (C7). CI also runs proven red (P2) and `pr-acceptance` (PR body needs `## Acceptance` content, and `## Spec` on a `feat`). State completed checks in handoff. ## Playbooks Read each listed playbook before work in that area. | Work | Read first | |---|---| | Multi-agent or long-running work | `agents/generic/agent-workflows.md` | | Naming, briefs, docs, proposing work | `agents/project/glossary.md`, `agents/project/out-of-scope.md` | | Tests, UI, or platform checks | `agents/project/testing.md` | | Developer or user documentation | `agents/project/documentation.md` | | {{DOMAIN}} APIs and quirks | `agents/project/domain-context.md` | Portable playbooks live in `agents/generic/`; project ones in `agents/project/`.gap-trap/templates/agents/generic/agent-workflows.md8.0 KBedeb7433f9f0upstream ↗
# Agent workflow playbook Advisory, not binding. Last validated 2026-09 against Claude Code with the Claude 5 family, in the repo gap-trap was extracted from. Codex has the same pieces under other names (subagents, `/model`, `~/.codex/config.toml`); those lines come from its docs and have not been through a run here. The instruction files (`AGENTS.md`, `AGENTS.project.md`) state what must hold; this playbook records what has worked. When the two disagree, the instruction files win. Harness features churn; check the date above before trusting specifics. ## Scale ceremony to risk What scales with risk is review ceremony: a one-file fix with a covering test needs no independent reviewer, while multi-task plans, refactors with blast radius, and contract changes do. Delegation has a floor too; see Orchestration. Evidence from the origin repo: reviews of mechanical transcription tasks found zero defects; reviews of judgment work (doc remapping, whole-branch review) found every real one. Review where judgment lives; skip where the gate already proves the result. The final whole-branch review before a PR is the one step never skipped, and it runs on two axes in two separate contexts: a Standards agent judging the diff against the contracts and playbooks (skipping anything a gate enforces), and a Spec agent judging the diff against the issue's acceptance lines (missing, unrequested, looks-implemented-but-wrong, each quoting the issue). Findings stay under their own heading, never merged or reranked: a change can pass one axis and fail the other; in the origin repo two fix chains passed every gate while missing what the issue asked. Agent count is a cost, never a quality signal: prefer the fewest dispatches that produce the evidence, and let the harness parallelize instead of choreographing a named fleet. ## Orchestration - The orchestrator (the main session, or a team lead coordinating others) runs on the harness's top coding model; never a smaller one. Coordination quality bounds everything downstream: a weak orchestrator writes weak briefs, misreads reports, and wastes every strong agent under it. An instruction file cannot switch a running session's model, so pin the default in the harness config (Claude Code: `"model": "opus"` in `.claude/settings.json`; Codex: `model` in `~/.codex/config.toml`); if your session runs on something smaller anyway, say so and suggest `/model`. - Implementation delegates to subagents by default: the orchestrator's context stays clean for briefs, report verdicts, and review decisions, and an orchestrator that edits files skips its own review structure. Exception: a trivial gate-covered edit (single file, no judgment, a gate proves the result) costs more to brief than to make; edit it directly and run its gate. - Let the harness decide how many agents run and when they parallelize. Choose what each dispatch is for; do not choreograph counts. ## Multi-agent execution - One subagent per task, fresh context each time. Hand each agent a brief file (its requirements) and a report-file path; it returns only status, commits, and a one-line test summary. Never paste session history into a dispatch. - Model tiering: cheapest model when the task text contains the complete content to write (transcription plus testing); mid tier for code reviews; the most capable model only for the final whole-branch review. Tasks whose output is prose (docs, reports, prose review with slop-mop) run on the harness's top coding model. Smaller models follow slop-mop's word lists but skip the checks that need judgment. - Tasks involving judgment get an independent review against their brief before the next task starts; purely mechanical tasks rely on their gates. Fixes get a scoped re-review that verdicts each finding ADDRESSED or NOT ADDRESSED and looks only at the fix diff. - After the final whole-branch review, dispatch one fix wave with the whole findings list, then one scoped re-review. Never one fixer per finding. - Stop every agent as soon as its report is verified. Idle agents keep burning context and can wake with stale intent. - Long-running work keeps a ledger file (task, commits, review outcome, one line each) outside the repo tree. Context compaction loses memory; the ledger plus `git log` is the recovery map. ## Trust boundaries for tooling - A reviewer only re-runs tests when the code changed after the last run; otherwise the implementer's report carries the evidence. - Repo-wide greps run from the repo root or with absolute paths; a cwd of a subdirectory silently misses everything above it. - Never gate a commit on a piped command (`npm test | grep ...`): the pipeline exits with the last command's status and a failing suite reads as success. Run the gate bare, check its exit, then filter output separately. This shipped a red commit here once. - A CI job that skips its own steps still reports green. Before trusting a required check, read what it ran: two required checks here passed on every PR for months, one whose workflow had been deleted and one that skipped itself for a missing secret (M2). - Merging waits for green CI, but never by polling: queue it with `gh pr merge --auto` at PR creation and GitHub merges when checks pass. Caution: if the repo's auto-merge setting is off, `--auto` silently falls back to an immediate merge; that landed one PR here before its checks finished. Confirm `allow_auto_merge` is true before relying on it. ## Where knowledge goes (M5 in practice) - Project fact (API quirk, platform behavior, failed approach): the domain playbook in the project playbook directory, via the protocol PR. Check whether an entry already covers it; update rather than duplicate. - Workflow practice that proved out (a review pattern, a model-tier choice, a tooling guard): this playbook or a sibling generic playbook, via the same protocol. Personal preference becomes shared practice exactly when it has evidence behind it. - Agent private memory holds only what the repo must not: personal and private specifics (names, hosts, credentials, machine paths) and habits with no evidence yet. Memory is a staging area, not an archive; when a memory keeps proving true, promote it and delete the private copy. ## Known failure modes this playbook exists to prevent - A compression wrapper garbled test output and hid a failure; the direct command showed it immediately. - Reviewers approving their own implementation. Separate agents, always. - Doc references that drift silently from renumbered rules; the instruction gate catches the repo cases, but dispatches into agents should cite rule IDs, not copied text, for the same reason. ## Multi-agent lessons (origin repo, 2026-08) - The task brief is the reviewer's contract. Anything added only in the dispatch prose is invisible to the reviewer; fold every dispatch-time addition into the brief or expect false scope-creep findings. - Match the reviewer to RISK, not size. Concurrency, token handling, and native-adjacent diffs get the top-tier reviewer on the first pass. - Keep a task's reviewer alive until its scoped re-review completes. - Fix waves on concurrency-heavy subsystems create defects (three consecutive waves each introduced one). Budget a scoped re-review per wave; a fresh Critical inside a wave is fixed, never parked. - Subagent final plain text often never reaches the controller. Every dispatch states where the report goes: the report-file path in the brief, which works in any harness, plus the harness's own channel (Claude Code: SendMessage to the controller). - Briefs cite symbols and anchors, never line numbers; line refs rot within hours on an active branch. - A one-line fix the reviewer itself specified needs no re-review round. - Full-gate runs happen once per wave, at push time. Fix rounds run the scoped suites plus the gates their change class implicates. - One checkout, many agents: the branch can change under you between edit and commit. Verify `git branch --show-current` before every commit; make out-of-band commits from a temporary `git worktree`.gap-trap/templates/agents/project/documentation.md1.1 KBe0bda4a81234upstream ↗
# Documentation playbook Read before editing user or developer documentation. ## Scope - User-visible behavior changes update `{{USER_DOCS}}`. - New APIs, components, services, or traced paths update `{{DEV_DOCS}}`. - A changed user-visible flow updates its call-flow trace. Add a trace only for a main journey not already covered. - Developer docs teach a competent programmer who does not know this stack. Explain a framework mechanism where it first affects behavior. ## Style - Write and edit with the slop-mop skill. The rules below add what it does not cover. - Write like a developer explaining to a colleague. No headline-style headings ("The X", "A deep dive into Y") and no news-article cadence. - Examples must grep-hit the codebase unless marked simplified. Cite symbols, never line numbers; line numbers rot within hours. - Developer docs cite AGENTS.md rule IDs or AGENTS.project.md contract names instead of copying process rules. - Reports and retrospectives are documentation; every rule above applies. Claims cite their commit, verified to exist before citing.gap-trap/templates/agents/project/domain-context.md616 Bd649a896536aupstream ↗
# {{PROJECT}} domain context Verified project facts for writing code: API quirks, platform behavior, and approaches that already failed. Read before working on the subsystem it covers. New entries arrive through the self-improvement protocol (AGENTS.md M5) when a session learns a durable fact the hard way. Entries carry no personal data, hostnames, or addresses. Each entry cites the commit hash behind it; the instruction gate checks the hash exists. ## {{SECTION, e.g. the external API}} - {{One fact. What is true, what was tried, what to do instead. (abcd1234)}} ## Platform quirks ## Libraries and stategap-trap/templates/agents/project/glossary.md1.1 KB7d145306be6cupstream ↗
# Glossary One name per concept, for code identifiers, briefs, docs, and commit messages. Each entry says what the thing is, not what it does, and lists the words to stop using for it. User-facing copy follows the product wording and is exempt. A quality ratchet may count avoided terms in agent and developer prose. ## Process **Contract**: An architecture entry in `AGENTS.project.md`: what it owns, the sanctioned path, forbidden bypasses, and its gate. _Avoid_: architecture rule, architecture convention **Gate**: A command that fails a commit, push, or CI run when a rule breaks. _Avoid_: blocking check, guard script **Brief**: The requirements file handed to a subagent for one task. _Avoid_: task prompt, instructions file **Spec**: A design document approved before a plan is written. _Avoid_: design doc **Seam**: The interface a test exercises; the highest one that reaches the behavior. _Avoid_: test boundary, layer under test **Proven red**: A new test that was run against the pre-change code and shown to fail there. _Avoid_: verified failing, shown failing ## {{DOMAIN}} **{{Term}}**: {{What it is.}} _Avoid_: {{synonyms to stop using}}gap-trap/templates/agents/project/out-of-scope.md531 B9aba4a099c3dupstream ↗
# Out of scope Requests the maintainer has declined, with the reason, so nobody re-opens or re-proposes them without new evidence. Check this before opening an issue or suggesting a feature (P1). One entry per decision: the gist, why, and the issue that records it. An entry is reopened only by a new issue that cites the old one and says what changed. - Feature suggestions made without first checking the existing surface. Grep the code and the user docs before proposing; an inventory from a subagent alone misses things.gap-trap/templates/agents/project/testing.md1.5 KB2ae015245c13upstream ↗
# Testing playbook Read before tests, UI work, or platform checks. ## Test design - Test outcomes a person sees: changed data, navigation, rendered real data, persistence, errors, and edge cases. Never element existence or child count (C6). - Name the seams under test in the issue or brief before writing a test: the highest interface that reaches the behavior, ideally one. Mock only the system boundary (`{{API_BOUNDARY}}`); a mock of the repo's own modules is counted by the quality ratchet and may not multiply. - Prove red before green: run the new test against the pre-change code and show it fail (`{{PROVEN_RED_CMD}} <base> <head>`; CI runs it on every PR). A bug fix starts with that red command, shown, before any code is read for a theory. - A new gate assertion (anything under `{{GATE_DIR}}`) is proven red against a scratch violation, then the scratch is removed in the same commit. - Route tests by tier: pure logic gets unit tests beside the source; user-visible behavior gets an end-to-end scenario plus units for the logic beneath. End-to-end asserts the journey, units the edge cases; never the same assertion in both. - No fixed sleeps. Use auto-retrying assertions. - New interactive UI needs a stable test id. Repeated elements suffix the entity id; variants suffix kind or role. ## Commands ```bash {{TEST_CMD}} {{E2E_CMD}} ``` ## Traps that have burned agents - {{Add one per incident, with the commit hash. Delete this line.}}gap-trap/templates/CLAUDE.md139 Bf798f1a8cf2eupstream ↗
Use @AGENTS.md and @AGENTS.project.md as your instructions Multi-agent or long-running work? Read agents/generic/agent-workflows.md first.gap-trap/templates/gates/github-ci.yml2.6 KBf582a8a0f745upstream ↗
# Reference CI for the gap-trap gates (GitHub Actions). ADAPT: the install # and test commands, the working directory, and the Node setup if the repo # is not Node. Make `unit-tests`, `gates`, and `proven-red` required checks # in branch protection; a check nobody requires is advice. name: CI on: pull_request: push: branches: [main] # ADAPT: the default branch concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} jobs: # P1: the body quotes the issue's acceptance lines, which the Spec review # axis judges the diff against. A feat PR also names its spec. Fork PRs # and bots are exempt. pr-acceptance: runs-on: ubuntu-latest if: >- github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.user.type != 'Bot' steps: - uses: actions/checkout@v4 - name: PR body has acceptance lines, and a spec on a feat env: PR_BODY: ${{ github.event.pull_request.body }} PR_TITLE: ${{ github.event.pull_request.title }} run: sh scripts/pr-body-check.sh gates: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: # The instruction gate checks that cited commit hashes exist. fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: '22' cache: 'npm' - run: npm ci # ADAPT - run: npm run gates # ADAPT: unit tests, build/type check, blocking lints proven-red: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: '22' cache: 'npm' - run: npm ci # ADAPT # P2 by machine: the tests this range changed must fail on the code # the range replaces. Then the mutation smoke proves existing tests # can fail at all. # The title goes through env, never into the script text: it is # attacker-controlled on a fork PR. - name: Changed tests go red on the pre-change code env: PR_TITLE: ${{ github.event.pull_request.title }} BASE: ${{ github.event.pull_request.base.sha || github.event.before }} HEAD: ${{ github.event.pull_request.head.sha || github.event.after }} run: node scripts/proven-red.mjs "$BASE" "$HEAD" --title "$PR_TITLE" # ADAPT: sh scripts/proven-red.sh for the shell port - run: npm run test:mutation # ADAPT, or drop until mutation targets existgap-trap/templates/gates/instruction-gate.sh9.9 KBe7bd6231f953upstream ↗
#!/bin/sh # Instruction gate, language-neutral reference from gap-trap. Needs git, # grep, awk, wc. Run from the repo root as a CI step and a pre-commit hook # when the repo's test runner has no native port (see gates.md). # # Checks: contracts parse and every name they cite exists where the contract # says it lives; AGENTS.md holds no project tokens; the always-loaded files # all exist and stay under the word budget; cited commit hashes exist; # knowledge files hold no email or IP; docs cite rule IDs that exist; and the # grep gates below (contract Never clauses a text search settles). Exit 1 # with every failure listed. # # Every check also asserts its own denominator: how many tokens it resolved, # how many files it scanned. A check that quietly measures nothing reports # the same green as one that measured everything (M2). set -u # ---- config --------------------------------------------------------------- SRC_DIR=${GT_SRC_DIR:-src} # ADAPT: source tree contracts name SOURCE_EXT=${GT_SOURCE_EXT:-'\.(go|py|rs|ts|tsx|js|java|kt|rb|swift|cs|php)$'} TEST_PATH_RE=${GT_TEST_PATH_RE:-'(^|/)(__tests__|tests?|spec)(/|$)|_test\.|\.test\.|^test_|Test\.'} # ADAPT. SRC_DIR is in the list because the portable core must not name the # project's own source tree. FORBIDDEN_IN_CORE=${GT_FORBIDDEN_IN_CORE:-"{{PRODUCT}} {{FRAMEWORK}} agents/project $SRC_DIR"} WORD_BUDGET=${GT_WORD_BUDGET:-4000} # ADAPT: current count plus room (C7) MIN_CONTRACTS=${GT_MIN_CONTRACTS:-2} # the honest count; never pad ALWAYS_LOADED=${GT_ALWAYS_LOADED:-'AGENTS.md AGENTS.project.md CLAUDE.md'} KNOWLEDGE_FILES='agents/project/domain-context.md agents/project/glossary.md agents/project/out-of-scope.md agents/generic/agent-workflows.md' DOCS_DIR=${GT_DOCS_DIR:-docs} # ADAPT: '' skips the rule-ID check # ADAPT: one grep gate per printf row: name, ERE over source lines, ERE over # relative paths to exempt. Fields are tab-separated so patterns may use `|`. # Only clauses that are clean today belong here; a clause with violations # goes into .ratchet-counters as a count (gates.md, instruction gate item 8). TAB=$(printf '\t') GREP_GATES=$(printf '%s\t%s\t%s\n' \ 'Logging: no raw print or console outside the logger' '\bconsole\.[a-z]+\(|\bfmt\.Print|\blog\.Print|\bprint\(' '^lib/log|^logx/' \ 'HTTP: no raw HTTP client outside the wrapper' '[^.A-Za-z_]fetch\(|\bhttp\.(Get|Post|NewRequest|DefaultClient)\b|\brequests\.(get|post)\(' '^lib/http|^httpx/' \ ) # ---- end config ----------------------------------------------------------- fail=0 bad() { echo "FAIL: $*"; fail=1; } # Fold a contract body's wrapped lines back into the field they continue. # Markdown prose wraps, and a Path: line long enough to wrap used to have # everything past the wrap silently dropped from the check. fold_fields() { awk ' /^[[:space:]]*(Owns|Path|Never|Gate):/ { if (buf != "") print buf buf = $0; sub(/^[[:space:]]+/, "", buf); next } /^[[:space:]]*$/ { if (buf != "") { print buf; buf = "" } next } { if (buf != "") { line = $0; sub(/^[[:space:]]+/, "", line); buf = buf " " line } } END { if (buf != "") print buf } ' } src_files=$(git ls-files "$SRC_DIR" 2>/dev/null | grep -E -- "$SOURCE_EXT" || true) nontest_files=$(printf '%s\n' "$src_files" | grep -vE -- "$TEST_PATH_RE" || true) # A contract symbol found only in a test file is not a symbol the code uses, # so the fallback haystack is the non-test source only. # ponytail: word-split file list, so paths with spaces are missed. Switch to # `grep -r` over a directory if a repo has them. symbol_in_nontest_source() { [ -n "$nontest_files" ] || return 1 # shellcheck disable=SC2086 grep -qw -- "$1" $nontest_files 2>/dev/null } symbol_in_paths() { token=$1; shift for p in "$@"; do [ -e "$p" ] || continue grep -rqw --exclude-dir=node_modules --exclude-dir=.git -- "$token" "$p" 2>/dev/null && return 0 done return 1 } # 1. contracts parse contracts=$(awk '/^## Architecture contracts/{f=1;next} /^## /{f=0} f' AGENTS.project.md) names=$(printf '%s\n' "$contracts" | grep -E '^### ' | sed 's/^### //') count=$(printf '%s\n' "$names" | grep -c .) [ "$count" -ge "$MIN_CONTRACTS" ] || bad "only $count contracts, need $MIN_CONTRACTS" # 2. every backticked token in Path/Gate lines exists where the contract says resolve_symbol() { # $1 = token, $2 = contract name, $3 = the folded line, $4 = path tokens token=$1; cname=$2; cline=$3; cpaths=$4 # A Path line names where the subsystem lives, so its symbols are looked for # there. Resolving them anywhere in the tree let a symbol that had moved out # of its own module keep the contract green. if [ -n "$cpaths" ]; then case "$cline" in Path:*) # shellcheck disable=SC2086 symbol_in_paths "$token" $cpaths && return 0 echo "FAIL: $cname: symbol $token not found in $(printf '%s' "$cpaths" | tr '\n' ' ')" return 0 ;; esac fi symbol_in_nontest_source "$token" || echo "FAIL: $cname: symbol $token not found in $SRC_DIR (non-test files)" } check_contracts() { printf '%s\n' "$names" | while IFS= read -r name; do [ -n "$name" ] || continue block=$(printf '%s\n' "$contracts" | awk -v n="### $name" '$0==n{f=1;next} /^### /{f=0} f' | fold_fields) for field in Owns: Path: Never: Gate:; do printf '%s\n' "$block" | grep -q "^$field" || echo "FAIL: contract $name missing $field" done printf '%s\n' "$block" | grep -E '^(Path|Gate):' | while IFS= read -r line; do tokens=$(printf '%s\n' "$line" | grep -o '`[^`]*`' | tr -d '`' | sed 's/()$//' | sort -u) paths=$(printf '%s\n' "$tokens" | grep '/' || true) symbols=$(printf '%s\n' "$tokens" | grep -v '/' | grep -v '^$' || true) for token in $paths; do [ -e "$token" ] || echo "FAIL: $name: path $token missing" echo "CHECKED $name" done for token in $symbols; do resolve_symbol "$token" "$name" "$line" "$paths" echo "CHECKED $name" done done done } contract_report=$(check_contracts) printf '%s\n' "$contract_report" | grep -v '^CHECKED ' | grep -v '^$' || true tokens_checked=$(printf '%s\n' "$contract_report" | grep -c '^CHECKED ') # Per contract as well as overall: one contract that cites real names must not # cover for a neighbour that cites nothing the gate can look up. denominator_report=$(printf '%s\n' "$names" | while IFS= read -r name; do [ -n "$name" ] || continue printf '%s\n' "$contract_report" | grep -qx -- "CHECKED $name" || echo "FAIL: contract $name: its Path/Gate lines name nothing the gate can check" done) [ -z "$denominator_report" ] || { printf '%s\n' "$denominator_report"; fail=1; } # No overall "did anything get checked" line here: a contracts section this # script cannot parse yields no names, which fails the MIN_CONTRACTS check # above before it could matter. The count still goes in the summary (M2). case "$contract_report" in *FAIL*) fail=1 ;; esac # 3. portable core for token in $FORBIDDEN_IN_CORE; do grep -qi -- "$token" AGENTS.md && bad "AGENTS.md contains \"$token\"" done # 4. word budget. Every always-loaded file has to be there: summing whatever # happens to exist means deleting one reads as coming in under budget. missing='' for f in $ALWAYS_LOADED; do [ -f "$f" ] || missing="$missing $f" done if [ -n "$missing" ]; then bad "always-loaded file(s) missing:$missing; the budget would fall for free" words=0 else # shellcheck disable=SC2086 words=$(cat $ALWAYS_LOADED | wc -w | tr -d ' ') [ "$words" -le "$WORD_BUDGET" ] || bad "always-loaded files are $words words, budget $WORD_BUDGET" fi # 5. cited hashes exist if [ -f agents/project/domain-context.md ]; then # 7 to 40 hex chars with at least one digit: --oneline prints 7, and the # digit keeps English words made of a-f (acceded, defaced) out. for h in $(grep -oE '\b[0-9a-f]{7,40}\b' agents/project/domain-context.md | grep '[0-9]' | sort -u); do git cat-file -e "$h^{commit}" 2>/dev/null || bad "cited commit $h not found in history" done fi # 6. no private data in knowledge files. The domain needs a letter TLD, or a # pinned version such as `[email protected]` reads as an email and the gate cries wolf. for f in $KNOWLEDGE_FILES; do [ -f "$f" ] || continue grep -qE '\b[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\b' "$f" && bad "$f contains an IP address" grep -qE '\b[A-Za-z0-9._+-]+@[A-Za-z0-9-]+\.[A-Za-z]{2,}\b' "$f" && bad "$f contains an email address" done # 7. docs cite rule IDs that exist if [ -n "$DOCS_DIR" ] && [ -d "$DOCS_DIR" ]; then valid=$(grep -oE '^- [IPCM][0-9]+\.' AGENTS.md | sed 's/[-. ]//g') rules_report=$(grep -rhoiE '\brules? [IPCM][0-9]+\b' "$DOCS_DIR" | awk '{print toupper($2)}' | sort -u | while IFS= read -r id; do [ -n "$id" ] || continue printf '%s\n' "$valid" | grep -qx -- "$id" || echo "FAIL: docs cite unknown rule $id" done) [ -z "$rules_report" ] || { printf '%s\n' "$rules_report"; fail=1; } fi # 8. grep gates over non-test source, comment lines dropped [ -d "$SRC_DIR" ] || bad "SRC_DIR $SRC_DIR does not exist" scanned=$(printf '%s\n' "$nontest_files" | grep -c .) [ "$scanned" -gt 0 ] || bad "grep gates scanned no source files under $SRC_DIR (M2)" # ponytail: comment lines are dropped line by line, so a violation commented # out inside a /* ... */ block still reads as a violation. Loud, not silent; # the native ports use a real scanner. grep_report=$(printf '%s\n' "$GREP_GATES" | grep -v '^$' | while IFS="$TAB" read -r name pattern exempt; do hits='' for f in $nontest_files; do rel=${f#"$SRC_DIR"/} [ -n "$exempt" ] && printf '%s\n' "$rel" | grep -qE -- "$exempt" && continue grep -vE '^[[:space:]]*(//|#|\*|/\*|--)' "$f" | grep -qE -- "$pattern" && hits="$hits $rel" done [ -z "$hits" ] || echo "FAIL: $name:$hits" done) [ -z "$grep_report" ] || { printf '%s\n' "$grep_report"; fail=1; } [ "$fail" -eq 0 ] && echo "instruction gate: ok ($count contracts, $tokens_checked cited names, $scanned source files, $words words)" exit $failgap-trap/templates/gates/instruction-gate.test.ts12 KB876dffd80884upstream ↗
/** * Instruction gate. Reference implementation from gap-trap (vitest). * * The instruction files are read by every session, so they are checked like * code: every name a contract cites exists where the contract says it lives, * the portable core stays portable, the always-loaded files stay small, cited * commits exist, and no private data leaks into knowledge files. The grep * gates at the bottom are the contract Never clauses a text search settles. * * Every check here also asserts its own denominator: how many tokens it * resolved, how many files it scanned. A check that quietly measures nothing * reports the same green as a check that measured everything, and that is how * a gate dies without anyone noticing (M2). * * ADAPT the constants in the config block; the assertions stay as they are. */ import { describe, expect, it } from 'vitest'; import { execSync } from 'node:child_process'; import fs from 'node:fs'; import path from 'node:path'; // ---- config --------------------------------------------------------------- // ADAPT: repo root relative to this file, and the source tree contracts name. const repoRoot = path.resolve(__dirname, '../../..'); const SRC_DIR = 'app/src'; const SOURCE_EXT = /\.(ts|tsx)$/; /** Directories under SRC_DIR that hold tests or gates, skipped by the grep gates. */ const TEST_DIRS = ['__tests__', 'tests']; /** ADAPT: project-specific tokens that must never appear in the portable core. */ const FORBIDDEN_IN_CORE = ['{{PRODUCT}}', '{{FRAMEWORK}}', SRC_DIR, 'agents/project']; /** ADAPT: current count plus room. Raising it needs a reason in the commit message (C7). */ const WORD_BUDGET = 4000; const MIN_CONTRACTS = 2; // the honest count; never pad /** ADAPT: the files loaded into every session. All of them must exist. */ const ALWAYS_LOADED = ['AGENTS.md', 'AGENTS.project.md', 'CLAUDE.md']; const KNOWLEDGE_FILES = [ 'agents/project/domain-context.md', 'agents/project/glossary.md', 'agents/project/out-of-scope.md', 'agents/generic/agent-workflows.md', ]; /** ADAPT: where developer docs live, or '' to skip the rule-ID check. */ const DOCS_DIR = 'docs/developer-guide'; const DOC_EXT = /\.(rst|md)$/; /** * ADAPT: one entry per contract Never clause a grep settles. `exempt` names * the sanctioned file(s). Comments are stripped before matching. */ const GREP_GATES: { name: string; pattern: RegExp; exempt?: (rel: string) => boolean }[] = [ { name: 'Logging: no console calls outside the logger', pattern: /\bconsole\.\w/, exempt: (f) => f === 'lib/logger.ts' || f.startsWith('lib/log-file/') }, { name: 'HTTP: no raw fetch or axios outside lib/http', pattern: /(^|[^.\w])fetch\s*\(|\baxios\b/, exempt: (f) => f.startsWith('lib/http') }, ]; // ---- end config ----------------------------------------------------------- const appSrc = path.join(repoRoot, SRC_DIR); function walk(dir: string, acc: string[] = []): string[] { for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { const full = path.join(dir, entry.name); if (entry.isDirectory()) { if (entry.name === 'node_modules') continue; walk(full, acc); } else if (SOURCE_EXT.test(entry.name)) { acc.push(full); } } return acc; } const read = (f: string) => fs.readFileSync(f, 'utf8'); const sourceFiles = walk(appSrc); const isTestPath = (rel: string) => TEST_DIRS.some((d) => rel.split(path.sep).includes(d)); /** A contract symbol found only in a test file is not a symbol the code uses. */ const nonTestSourceText = sourceFiles .filter((f) => !isTestPath(path.relative(appSrc, f))) .map(read) .join('\n'); /** The text under a repo-relative path, file or directory, for symbol lookups. */ function textUnder(rel: string): string { const full = path.join(repoRoot, rel); if (!fs.existsSync(full)) return ''; if (fs.statSync(full).isDirectory()) return walk(full).map(read).join('\n'); return read(full); } function parseContracts(md: string): { name: string; body: string }[] { const section = md.split('## Architecture contracts')[1]?.split('\n## ')[0] ?? ''; return section .split('\n### ') .slice(1) .map((block) => { const [name, ...rest] = block.split('\n'); return { name: name.trim(), body: rest.join('\n') }; }); } /** * Fold a contract body's wrapped lines back into the field they continue. * Markdown prose wraps, and a `Path:` line long enough to wrap used to have * everything past the wrap silently dropped from the check. */ export function foldFields(body: string): string[] { const out: string[] = []; let open = false; for (const raw of body.split('\n')) { const line = raw.trim(); if (!line) { open = false; } else if (/^(Owns|Path|Never|Gate):/.test(line)) { out.push(line); open = true; } else if (open) { out[out.length - 1] += ` ${line}`; } } return out; } // Remove comments and leave strings alone. A plain regex for block comments // treats the "/*" inside a glob such as "src/**/*.ts" as the start of one and // deletes every line up to the next "*/", carrying real violations out with // it, after which the gate reports clean. Quoted strings end at the line // break, as JS string literals do, so an apostrophe in JSX text costs one // line rather than the rest of the file. Newlines survive either way. // ponytail: no regex-literal state, so a literal ending in "\/" followed by // "/" reads as a line comment. Add one if a repo trips it. export function stripComments(code: string): string { let out = ''; let i = 0; while (i < code.length) { const c = code[i]; const next = code[i + 1]; if (c === '/' && next === '/') { while (i < code.length && code[i] !== '\n') i += 1; continue; } if (c === '/' && next === '*') { i += 2; while (i < code.length && !(code[i] === '*' && code[i + 1] === '/')) { if (code[i] === '\n') out += '\n'; i += 1; } i += 2; continue; } if (c === '"' || c === "'" || c === '`') { const multiline = c === '`'; out += c; i += 1; while (i < code.length && code[i] !== c && (multiline || code[i] !== '\n')) { if (code[i] === '\\' && i + 1 < code.length) { out += code[i] + code[i + 1]; i += 2; continue; } out += code[i]; i += 1; } if (i < code.length && code[i] === c) { out += c; i += 1; } continue; } out += c; i += 1; } return out; } const backtickTokens = (line: string) => [...line.matchAll(/`([^`]+)`/g)].map((m) => m[1].replace(/\(\)$/, '')); const escape = (s: string) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); describe('AGENTS.project.md architecture contracts', () => { const md = read(path.join(repoRoot, 'AGENTS.project.md')); const contracts = parseContracts(md); it(`holds at least ${MIN_CONTRACTS} contracts with all four lines`, () => { expect(contracts.length).toBeGreaterThanOrEqual(MIN_CONTRACTS); for (const c of contracts) { const fields = foldFields(c.body); for (const field of ['Owns:', 'Path:', 'Never:', 'Gate:']) { expect(fields.some((l) => l.startsWith(field)), `${c.name} missing ${field}`).toBe(true); } } }); it('every symbol and path named in Path/Gate lines exists where the contract says', () => { let checkedOverall = 0; for (const c of contracts) { const fields = foldFields(c.body).filter((l) => l.startsWith('Path:') || l.startsWith('Gate:')); let checked = 0; for (const line of fields) { const tokens = backtickTokens(line); const paths = tokens.filter((t) => t.includes('/')); const symbols = tokens.filter((t) => !t.includes('/')); for (const token of paths) { expect(fs.existsSync(path.join(repoRoot, token)), `${c.name}: path ${token} missing`).toBe(true); checked += 1; } // A Path line names where the subsystem lives, so its symbols are // looked for there. Resolving them anywhere in the tree let a symbol // that had moved out of its own module keep the contract green. const scoped = line.startsWith('Path:') && paths.length > 0; const haystack = scoped ? paths.map(textUnder).join('\n') : nonTestSourceText; const where = scoped ? paths.join(', ') : `${SRC_DIR} (non-test files)`; for (const token of symbols) { expect( new RegExp(`\\b${escape(token)}\\b`).test(haystack), `${c.name}: symbol ${token} not found in ${where}`, ).toBe(true); checked += 1; } } expect(checked, `${c.name}: its Path/Gate lines name nothing the gate can check`).toBeGreaterThan(0); checkedOverall += checked; } expect(checkedOverall, 'no contract token was checked at all').toBeGreaterThan(0); }); }); describe('AGENTS.md stays portable and small', () => { it('contains no project-specific tokens', () => { const core = read(path.join(repoRoot, 'AGENTS.md')).toLowerCase(); for (const token of FORBIDDEN_IN_CORE) { expect(core.includes(token.toLowerCase()), `AGENTS.md contains "${token}"`).toBe(false); } }); it('the always-loaded files stay inside the word budget', () => { // Every file has to be there. Summing whatever happens to exist means // deleting an always-loaded file reads as coming in under budget. for (const f of ALWAYS_LOADED) { expect(fs.existsSync(path.join(repoRoot, f)), `${f} is missing; the budget would fall for free`).toBe(true); } const words = (f: string) => read(path.join(repoRoot, f)).split(/\s+/).filter(Boolean).length; const total = ALWAYS_LOADED.reduce((n, f) => n + words(f), 0); expect(total, `combined ${total} words > budget ${WORD_BUDGET}`).toBeLessThanOrEqual(WORD_BUDGET); }); }); describe('knowledge files stay evidence-backed and private-data-free (M5)', () => { it('every commit hash cited in domain-context exists in this repo', () => { const md = read(path.join(repoRoot, 'agents/project/domain-context.md')); // 7 to 40 hex chars with a digit: --oneline prints 7, and the digit keeps // English words made of a-f (acceded, defaced) out. const hashes = [...new Set([...md.matchAll(/\b[0-9a-f]{7,40}\b/g)].map((m) => m[0]).filter((h) => /\d/.test(h)))]; for (const hash of hashes) { expect( () => execSync(`git cat-file -e ${hash}^{commit}`, { cwd: repoRoot, stdio: 'pipe' }), `cited commit ${hash} not found in history`, ).not.toThrow(); } }); it('knowledge files contain no emails or IP addresses', () => { for (const file of KNOWLEDGE_FILES) { const full = path.join(repoRoot, file); if (!fs.existsSync(full)) continue; const text = read(full); expect(text, `${file} contains an IP address`).not.toMatch(/\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b/); // The domain needs a letter TLD, or a pinned version such as // `[email protected]` reads as an email and the gate cries wolf. expect(text, `${file} contains an email address`).not.toMatch(/\b[\w.+-]+@[\w-]+\.[A-Za-z]{2,}\b/); } }); }); describe('developer docs cite rule IDs that exist', () => { it('every "rule <id>" reference resolves', () => { if (!DOCS_DIR || !fs.existsSync(path.join(repoRoot, DOCS_DIR))) return; const core = read(path.join(repoRoot, 'AGENTS.md')); const valid = new Set([...core.matchAll(/^- ([IPCM][0-9]+)\./gm)].map((m) => m[1])); expect(valid.size).toBeGreaterThan(0); for (const file of fs.readdirSync(path.join(repoRoot, DOCS_DIR)).filter((f) => DOC_EXT.test(f))) { const text = read(path.join(repoRoot, DOCS_DIR, file)); for (const m of text.matchAll(/\brules? ([IPCM][0-9]+)\b/gi)) { expect(valid.has(m[1].toUpperCase()), `${file}: unknown rule id "${m[1]}"`).toBe(true); } } }); }); describe('contract Never clauses a grep can decide', () => { const codeFiles = sourceFiles .map((f) => [path.relative(appSrc, f), f] as const) .filter(([rel]) => !isTestPath(rel)) .map(([rel, f]) => [rel, stripComments(read(f))] as const); it('scans source files at all', () => { // Without this the suite is green after any path move (M2). expect(codeFiles.length).toBeGreaterThan(5); }); for (const gate of GREP_GATES) { it(gate.name, () => { const offenders = codeFiles.filter(([rel, code]) => !(gate.exempt?.(rel) ?? false) && gate.pattern.test(code)).map(([rel]) => rel); expect(offenders).toEqual([]); }); } });gap-trap/templates/gates/instruction_gate_test.py9.4 KB77461f8ae5b6upstream ↗
"""Instruction gate. Reference implementation from gap-trap (pytest). Same assertions as instruction-gate.test.ts: every name a contract cites exists where the contract says it lives, the portable core stays portable, the always-loaded files stay small, cited commits exist, knowledge files hold no private data, docs cite rule IDs that exist, and the grep gates settle the contract Never clauses a text search can. Every check also asserts its own denominator: how many tokens it resolved, how many files it scanned. A check that quietly measures nothing reports the same green as one that measured everything (M2). ADAPT the config block; keep the assertions. """ from __future__ import annotations import io import re import subprocess import tokenize from pathlib import Path import pytest # ---- config --------------------------------------------------------------- REPO = Path(__file__).resolve().parents[1] # ADAPT: repo root relative to this file SRC_DIR = "src" # ADAPT SOURCE_EXT = (".py",) TEST_DIRS = {"tests", "test", "__tests__"} FORBIDDEN_IN_CORE = ["{{PRODUCT}}", "{{FRAMEWORK}}", SRC_DIR, "agents/project"] # ADAPT WORD_BUDGET = 4000 # ADAPT: current count plus room (C7) MIN_CONTRACTS = 2 # the honest count; never pad ALWAYS_LOADED = ["AGENTS.md", "AGENTS.project.md", "CLAUDE.md"] # ADAPT: loaded every session KNOWLEDGE_FILES = [ "agents/project/domain-context.md", "agents/project/glossary.md", "agents/project/out-of-scope.md", "agents/generic/agent-workflows.md", ] DOCS_DIR = "docs" # ADAPT: "" skips the rule-ID check # ADAPT: (name, pattern over comment-stripped source, exempt-path predicate) GREP_GATES = [ ("Logging: no print outside the logger", re.compile(r"\bprint\("), lambda rel: rel == "lib/logger.py"), ("HTTP: no raw requests/httpx outside the wrapper", re.compile(r"\b(requests|httpx)\.(get|post|put|delete|request)\("), lambda rel: rel.startswith("lib/http")), ] # ---- end config ----------------------------------------------------------- SRC = REPO / SRC_DIR FIELD_RE = re.compile(r"^(Owns|Path|Never|Gate):") def read(p: Path) -> str: return p.read_text(encoding="utf-8") def source_files() -> list[Path]: return [p for p in SRC.rglob("*") if p.suffix in SOURCE_EXT and "node_modules" not in p.parts] def is_test_path(p: Path) -> bool: rel = p.relative_to(SRC) return any(part in TEST_DIRS for part in rel.parts) or p.name.startswith("test_") or p.name.endswith("_test.py") def parse_contracts(md: str) -> list[tuple[str, str]]: section = md.split("## Architecture contracts", 1)[1].split("\n## ", 1)[0] if "## Architecture contracts" in md else "" out = [] for block in section.split("\n### ")[1:]: name, _, body = block.partition("\n") out.append((name.strip(), body)) return out def fold_fields(body: str) -> list[str]: """Fold a contract body's wrapped lines back into the field they continue. Markdown prose wraps, and a ``Path:`` line long enough to wrap used to have everything past the wrap silently dropped from the check. """ out: list[str] = [] open_field = False for raw in body.splitlines(): line = raw.strip() if not line: open_field = False elif FIELD_RE.match(line): out.append(line) open_field = True elif open_field: out[-1] += " " + line return out def backtick_tokens(line: str) -> list[str]: return [t.removesuffix("()") for t in re.findall(r"`([^`]+)`", line)] def text_under(rel: str) -> str: """The text under a repo-relative path, file or directory.""" p = REPO / rel if not p.exists(): return "" if p.is_dir(): return "\n".join(read(f) for f in p.rglob("*") if f.suffix in SOURCE_EXT and f.is_file()) return read(p) CONTRACTS = parse_contracts(read(REPO / "AGENTS.project.md")) # A contract symbol found only in a test file is not a symbol the code uses. NON_TEST_SOURCE_TEXT = "\n".join(read(p) for p in source_files() if not is_test_path(p)) def test_contracts_have_all_four_lines(): assert len(CONTRACTS) >= MIN_CONTRACTS for name, body in CONTRACTS: fields = fold_fields(body) for field in ("Owns:", "Path:", "Never:", "Gate:"): assert any(l.startswith(field) for l in fields), f"{name} missing {field}" def test_every_symbol_and_path_in_contracts_exists(): checked_overall = 0 for name, body in CONTRACTS: lines = [l for l in fold_fields(body) if l.startswith(("Path:", "Gate:"))] checked = 0 for line in lines: tokens = backtick_tokens(line) paths = [t for t in tokens if "/" in t] symbols = [t for t in tokens if "/" not in t] for token in paths: assert (REPO / token).exists(), f"{name}: path {token} missing" checked += 1 # A Path line names where the subsystem lives, so its symbols are # looked for there. Resolving them anywhere in the tree let a # symbol that had moved out of its own module keep the contract # green. scoped = line.startswith("Path:") and paths haystack = "\n".join(text_under(p) for p in paths) if scoped else NON_TEST_SOURCE_TEXT where = ", ".join(paths) if scoped else f"{SRC_DIR} (non-test files)" for token in symbols: assert re.search(rf"\b{re.escape(token)}\b", haystack), f"{name}: symbol {token} not found in {where}" checked += 1 assert checked > 0, f"{name}: its Path/Gate lines name nothing the gate can check" checked_overall += checked assert checked_overall > 0, "no contract token was checked at all" def test_core_is_portable(): core = read(REPO / "AGENTS.md").lower() for token in FORBIDDEN_IN_CORE: assert token.lower() not in core, f'AGENTS.md contains "{token}"' def test_always_loaded_files_within_word_budget(): # Every file has to be there. Summing whatever happens to exist means # deleting an always-loaded file reads as coming in under budget. for f in ALWAYS_LOADED: assert (REPO / f).exists(), f"{f} is missing; the budget would fall for free" total = sum(len(read(REPO / f).split()) for f in ALWAYS_LOADED) assert total <= WORD_BUDGET, f"combined {total} words > budget {WORD_BUDGET}" def test_cited_commit_hashes_exist(): md = read(REPO / "agents/project/domain-context.md") # 7 to 40 hex chars with a digit: --oneline prints 7, and the digit keeps # English words made of a-f (acceded, defaced) out. for h in sorted({h for h in re.findall(r"\b[0-9a-f]{7,40}\b", md) if re.search(r"\d", h)}): r = subprocess.run(["git", "cat-file", "-e", f"{h}^{{commit}}"], cwd=REPO, capture_output=True) assert r.returncode == 0, f"cited commit {h} not found in history" def test_knowledge_files_hold_no_private_data(): for f in KNOWLEDGE_FILES: p = REPO / f if not p.exists(): continue text = read(p) assert not re.search(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", text), f"{f} contains an IP address" # The domain needs a letter TLD, or a pinned version such as # `[email protected]` reads as an email and the gate cries wolf. assert not re.search(r"\b[\w.+-]+@[\w-]+\.[A-Za-z]{2,}\b", text), f"{f} contains an email address" def test_docs_cite_rule_ids_that_exist(): if not DOCS_DIR or not (REPO / DOCS_DIR).exists(): pytest.skip("no docs dir") valid = set(re.findall(r"^- ([IPCM]\d+)\.", read(REPO / "AGENTS.md"), re.M)) assert valid for p in (REPO / DOCS_DIR).rglob("*"): if p.suffix not in (".md", ".rst"): continue for m in re.finditer(r"\brules? ([IPCM]\d+)\b", read(p), re.I): assert m.group(1).upper() in valid, f"{p.name}: unknown rule id {m.group(1)}" def _strip_comments(code: str) -> str: """Blank out comments, leaving strings and line positions untouched. A regex over ``#`` cannot tell a comment from a ``#`` inside a string, and dropping only whole-line comments left trailing ones in, so a commented-out ``print(`` failed the gate that exists to find real ones. ``tokenize`` knows the difference. """ lines = code.splitlines(keepends=True) try: tokens = list(tokenize.generate_tokens(io.StringIO(code).readline)) except (tokenize.TokenError, SyntaxError, IndentationError): return re.sub(r"(?m)^\s*#.*$", "", code) for tok in tokens: if tok.type != tokenize.COMMENT: continue row, col = tok.start end_col = tok.end[1] line = lines[row - 1] lines[row - 1] = line[:col] + " " * (end_col - col) + line[end_col:] return "".join(lines) def _code_files() -> list[tuple[str, str]]: return [ (p.relative_to(SRC).as_posix(), _strip_comments(read(p))) for p in source_files() if not is_test_path(p) ] def test_grep_gates_scan_files_at_all(): # Without this the suite is green after any path move (M2). assert len(_code_files()) > 0 @pytest.mark.parametrize("name,pattern,exempt", GREP_GATES, ids=[g[0] for g in GREP_GATES]) def test_grep_gate(name, pattern, exempt): offenders = [rel for rel, code in _code_files() if not exempt(rel) and pattern.search(code)] assert offenders == [], f"{name}: {offenders}"gap-trap/templates/gates/pr-body-check.sh1.6 KBdc9220bc8931upstream ↗
#!/bin/sh # PR body check (AGENTS.md P1), reference from gap-trap. Reads PR_TITLE and # PR_BODY from the environment (CI passes them through `env:`, never by # interpolating into the run script: a title is attacker-controlled text). # # PR_TITLE='feat: x' PR_BODY="$(cat body.md)" sh pr-body-check.sh # # Fails when `## Acceptance` has no content after HTML comments are removed, # and, on a feat title, when `## Spec` has none. # # Exit codes: 0 sections present, 1 a section is empty, 2 the check could not # run. set -u # perl is the one tool here the shell gates do not otherwise need. Without it # the pipeline below yields an empty section and every PR reads as missing its # acceptance lines, so say what is wrong instead of guessing a verdict. command -v perl >/dev/null 2>&1 || { echo "::error title=pr-body-check cannot run::perl is not installed"; exit 2; } : "${PR_BODY:?PR_BODY is required}" : "${PR_TITLE:=}" section() { printf '%s' "$PR_BODY" | awk -v h="$1" '$0 ~ "^##[[:space:]]*"h {f=1;next} /^##[[:space:]]/{f=0} f' | perl -0pe 's/<!--.*?-->//gs' | tr -d '[:space:]' } if [ -z "$(section Acceptance)" ]; then echo "::error title=No acceptance lines::This PR body has no '## Acceptance' section with content (AGENTS.md P1: the Spec review axis judges the diff against the issue's acceptance lines)." exit 1 fi case "$PR_TITLE" in feat*) if [ -z "$(section Spec)" ]; then echo "::error title=No spec::A feat PR needs a '## Spec' section: a link to its spec, or one line saying why this feature needs none." exit 1 fi ;; esac echo "PR body sections present."gap-trap/templates/gates/proven-red.mjs16 KB022f563e3506upstream ↗
#!/usr/bin/env node /** * Proven-red gate (AGENTS.md P2). Reference implementation from gap-trap. * * P2 says a failing test precedes every feature and bugfix. Nothing checked * it in the origin repo: three tests passed on the code they were meant to * catch, each found only because someone stashed the fix and re-ran by * hand. This script does that by machine. * * For a range base..head it takes the test files changed since the two * forked, runs them in a throwaway worktree that holds the fork-point code * plus the head tests, and fails when they pass there. A test that is green * on the code it claims to guard proves nothing. The normal unit-test job * proves the same tests are green on head. * * node scripts/proven-red.mjs <base> <head> [--title "<pr title>"] * * Every changed test file is judged on its own. One file that fails carries * no proof for the others: a run that batched them together let a test which * passed on the old code ride out on a failing sibling's exit code. The same * holds inside a file: a test whose name the fork-point copy of its file does * not have is new, and a new test that passes on the old code fails the gate * even when another test in its file went red (issue #2). * * Skips, and says so: a range that changes no source; a range whose only * source changes are the gate scripts themselves (proven red by scratch * violation); a range whose only changed tests are end-to-end steps, which * need a live server. Two skips are bypasses rather than proofs, and say so * as a CI warning: a skip-type title, which is unverified text, and the * end-to-end case. Fails when a behavior change arrives with no changed * test. A changed unit test is always proved, whatever the title claims. * * A red run is not one proof but two. A test that fails an assertion on the * old code shows the assertion bites. A test that fails only because it * references a symbol the fork point does not have (`x is not a function`, * `Cannot find module`) shows the code is new and nothing about the * assertions; that is the only red a new module can ever give against the * fork point. The job reads each file's failures and names the files whose * red is of the second kind, so a reviewer knows which assertions still need * a look (M2: read what a gate measured). * * Exit codes: 0 proved or skipped, 1 a test proved nothing, 2 the gate could * not run (no report, a test file the runner never opened). */ import { execFileSync } from 'node:child_process'; import { existsSync, mkdirSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; import { mkdtempSync } from 'node:fs'; import { tmpdir } from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; export const SKIP_TYPES = ['docs', 'chore', 'ci', 'refactor', 'build', 'style', 'test']; // ADAPT: the classifiers below are the whole port. Unit tests are what gets // proven. End-to-end tests are real tests that cannot run here, so they skip // with a warning. Test support travels with the tests but is not a test and // never excuses one. Gate files are proven red by scratch violation instead. // Non-code never counts as a behavior change. UNIT_TEST wins where more than // one matches. The origin repo's values are shown. APP_DIR is the directory // the test runner runs from, relative to the repo root ('' for the root). const APP_DIR = 'app'; const UNIT_TEST = /^app\/src\/(?!tests\/).*\.test\.(ts|tsx)$/; const E2E_TEST = /^app\/tests\/steps\//; const TEST_SUPPORT = /^app\/src\/tests\/|\/__tests__\//; const GATE_FILE = /^scripts\/|instruction-gate|proven-red|ratchet/; const NON_CODE = /^(docs\/|agents\/|\.github\/|.*\.md$|.*\.rst$|app\/\.[\w-]+-baseline\.json$)/; // ADAPT: the runner and its JSON report. vitest: `--reporter=json --outputFile=<path>`. // pytest: `--json-report --json-report-file=<path>` (pytest-json-report), and // map its `tests[].nodeid` and `longrepr` into readFileResults. go test: `-json`. const TEST_CMD = ['npx', 'vitest', 'run']; /** * Split a changed-file list into what to run, what to carry along, and what * counts as behavior. `deleted` names the paths gone at head: a deleted test * cannot be proven and must not be run, because the fork-point worktree still * holds its old copy, which passes and reads as a test that proves nothing. */ export function classify(files, deleted = new Set()) { const live = (f) => !deleted.has(f); const unitTests = files.filter((f) => UNIT_TEST.test(f) && live(f)); const e2eTests = files.filter((f) => E2E_TEST.test(f) && !UNIT_TEST.test(f) && live(f)); const testSupport = files.filter( (f) => TEST_SUPPORT.test(f) && !UNIT_TEST.test(f) && !E2E_TEST.test(f) && live(f), ); const source = files.filter( (f) => !UNIT_TEST.test(f) && !E2E_TEST.test(f) && !TEST_SUPPORT.test(f) && !NON_CODE.test(f), ); return { unitTests, e2eTests, testSupport, source }; } /** * Why a range needs no red proof, or null when it does. `warn` marks a skip * that is a bypass rather than a proof, so CI can say so out loud. */ export function skipReason(title, { unitTests, e2eTests, source }) { if (source.length === 0) return { reason: 'no source file changed', warn: false }; // The title is unverified input, so it cannot excuse a changed test from the // proof: a behavior change mislabelled `refactor:` used to skip the gate // entirely. A skip-type title still excuses a source change that brings no // test, which is what a real refactor looks like, and that skip is warned // about because nothing verified the claim. if (unitTests.length > 0) return null; if (source.every((f) => GATE_FILE.test(f))) { return { reason: 'only gate files changed; a gate is proven red by scratch violation', warn: false }; } const type = /^([a-z]+)(\(.+\))?!?:/.exec(title ?? '')?.[1]; if (type && SKIP_TYPES.includes(type)) { return { reason: `title type "${type}" carries no behavior change`, warn: true }; } if (e2eTests.length > 0) { return { reason: 'the only changed tests are end-to-end steps, which need a live server', warn: true }; } return null; } function git(args, cwd) { return execFileSync('git', args, { cwd, encoding: 'utf8' }).trim(); } const lines = (text) => text.split('\n').filter(Boolean); /** A failure that says the symbol or module is missing, not that a value was wrong. */ const MISSING_REFERENCE = /is not a function|is not defined|is not a constructor|Cannot find module|Failed to resolve import|does not provide an export|has no exported member/; /** Sort one file's failure messages into assertion failures and missing references. */ export function classifyFailures(failures) { const missing = failures.filter((m) => MISSING_REFERENCE.test(m)); return { assertion: failures.length - missing.length, missing: missing.length, sample: missing[0] }; } /** Match a report entry to the repo-relative path the diff named. */ function resultFor(fileResults, rel) { return fileResults.find((r) => r.file === rel || r.file.endsWith(`/${rel}`) || rel.endsWith(r.file)); } /** * Judge each changed test file on its own run result. Returns the lines to * print, the lines to raise as CI warnings, and an exit code, so the whole * verdict is testable without git. */ export function judge(relative, { code, fileResults }, oldNames = new Map()) { const out = []; const warnings = []; if (!Array.isArray(fileResults) || fileResults.length === 0) { out.push( `proven-red: the test command exited ${code} without producing a report. ` + 'That is a runner that did not run, not a test that went red. Fix the command or the worktree setup.', ); return { lines: out, warnings, code: 2 }; } const green = []; const newGreen = []; const missingOnly = []; let assertions = 0; let missing = 0; for (const rel of relative) { const result = resultFor(fileResults, rel); if (!result) { out.push(`proven-red: ${rel} is in the range but the runner never opened it; the gate cannot prove it.`); return { lines: out, warnings, code: 2 }; } if (result.status !== 'failed') { green.push(rel); continue; } // A file absent at the fork point has no old names: every test in it is new. // ponytail: an old name whose body changed is not caught; that needs a diff of test bodies. const old = oldNames.get(rel) ?? new Set(); for (const t of result.tests ?? []) { if (t.status === 'passed' && !old.has(t.name)) newGreen.push(`${rel} > ${t.name}`); } const kinds = classifyFailures(result.messages ?? []); assertions += kinds.assertion; missing += kinds.missing; if (kinds.assertion === 0 && kinds.missing > 0) missingOnly.push({ rel, sample: kinds.sample }); } if (green.length > 0) { out.push( `proven-red: ${green.length} changed test file(s) pass on the pre-change code; ` + `they cannot catch the bug they claim to: ${green.join(', ')}`, ); return { lines: out, warnings, code: 1 }; } if (newGreen.length > 0) { out.push( `proven-red: ${newGreen.length} new test(s) pass on the pre-change code, though a sibling in the ` + `same file failed; they cannot catch the bug they claim to: ${newGreen.join(', ')}`, ); return { lines: out, warnings, code: 1 }; } if (missingOnly.length > 0) { const names = missingOnly.map((m) => m.rel).join(', '); warnings.push( 'proven-red: these files fail on the pre-change code only because they reference code it does not ' + `have, e.g. "${(missingOnly[0].sample ?? '').split('\n')[0]}": ${names}. That proves the code is new, ` + 'not that the assertions would catch a wrong value. Check those assertions in review, or run the ' + 'mutation smoke on the module.', ); } out.push( `proven-red: changed tests fail on the pre-change code, as they should ` + `(${assertions} assertion failure(s), ${missing} missing reference(s) across ${relative.length} file(s)).`, ); return { lines: out, warnings, code: 0 }; } /** * Run the head tests against the base code in a worktree. * `runTests(appDir, files)` returns `{ code, fileResults }`; injectable for tests. */ export function proveRed({ base, head, repo, title, runTests = runVitest, log = console.log }) { // The fork point, not `base` itself. CI passes // github.event.pull_request.base.sha, which is the base branch's tip at // event time, so on a branch whose base has moved since it was cut, a // two-dot diff reports the base branch's own newer commits as this range's // changes. Those tests then rightly pass on `base` and the gate fails a PR // that did nothing wrong. The same tip is also the wrong worktree to prove // against: the pre-change code is the fork point, not whatever landed on // the base branch afterwards. const forkPoint = git(['merge-base', base, head], repo); const range = `${forkPoint}..${head}`; const files = lines(git(['diff', '--name-only', range], repo)); const deleted = new Set(lines(git(['diff', '--name-only', '--diff-filter=D', range], repo))); const split = classify(files, deleted); const skip = skipReason(title, split); if (skip) { const message = `proven-red: skipped, ${skip.reason}.`; log(skip.warn && process.env.GITHUB_ACTIONS ? `::warning title=Proven red skipped::${message}` : message); return 0; } if (split.unitTests.length === 0) { log('proven-red: a behavior change arrived with no changed test (P2).'); return 1; } const worktree = mkdtempSync(path.join(tmpdir(), 'proven-red-')); try { git(['worktree', 'add', '--detach', '-q', worktree, forkPoint], repo); // ADAPT: share installed dependencies with the worktree instead of reinstalling. const modules = path.join(repo, APP_DIR, 'node_modules'); if (existsSync(modules)) { mkdirSync(path.join(worktree, APP_DIR), { recursive: true }); symlinkSync(modules, path.join(worktree, APP_DIR, 'node_modules')); } const prefix = APP_DIR ? `${APP_DIR}/` : ''; const toRelative = (f) => (f.startsWith(prefix) ? f.slice(prefix.length) : f); const appDir = path.join(worktree, APP_DIR); // Which tests are new: run the fork-point copies first and take their // names from the same report, so old and new names are spelled alike. const existing = split.unitTests.filter((f) => existsSync(path.join(worktree, f))).map(toRelative); const oldNames = new Map(); if (existing.length > 0) { const { fileResults } = runTests(appDir, existing); for (const rel of existing) { oldNames.set(rel, new Set((resultFor(fileResults, rel)?.tests ?? []).map((t) => t.name))); } } // Read the head versions from git, not the working tree: locally the // checkout may be on another branch, and a test copied from there // proves nothing. CI checks out head, so the two agree there. for (const f of [...split.unitTests, ...split.testSupport]) { const content = execFileSync('git', ['show', `${head}:${f}`], { cwd: repo, encoding: 'utf8' }); mkdirSync(path.dirname(path.join(worktree, f)), { recursive: true }); writeFileSync(path.join(worktree, f), content); } const relative = split.unitTests.map(toRelative); const verdict = judge(relative, runTests(appDir, relative), oldNames); for (const line of verdict.warnings) { log(process.env.GITHUB_ACTIONS ? `::warning title=Red by missing symbol only::${line}` : line); } for (const line of verdict.lines) log(line); return verdict.code; } finally { try { git(['worktree', 'remove', '--force', worktree], repo); } catch { rmSync(worktree, { recursive: true, force: true }); } } } /** Run vitest; the JSON report beside the default one is how the failures get read. */ function runVitest(appDir, files) { const report = path.join(mkdtempSync(path.join(tmpdir(), 'proven-red-report-')), 'vitest.json'); const args = [...TEST_CMD.slice(1), ...files, '--reporter=default', '--reporter=json', `--outputFile=${report}`]; let code = 0; try { execFileSync(TEST_CMD[0], args, { cwd: appDir, stdio: 'inherit' }); } catch (error) { code = error.status ?? 1; } return { code, fileResults: readFileResults(report) }; } /** * One entry per test file in a vitest JSON report: its status and every * failure message, including the file-level one when the file could not even * load, and each test's name and status. Per file, not pooled: a file's own * result is the only thing that says whether that file proved anything. */ export function readFileResults(report) { if (!existsSync(report)) return []; return (JSON.parse(readFileSync(report, 'utf8')).testResults ?? []).map((file) => { const perTest = (file.assertionResults ?? []).flatMap((t) => t.failureMessages ?? []); const messages = perTest.length === 0 && file.status === 'failed' && file.message ? [file.message] : perTest; const tests = (file.assertionResults ?? []).map((t) => ({ name: t.fullName ?? t.title, status: t.status })); return { file: file.name, status: file.status, messages, tests }; }); } // Compare real paths: a checkout reached through a symlink (macOS /tmp and // /var are symlinks) made argv[1] and import.meta.url disagree, and the gate // exited 0 having run nothing at all. const invokedDirectly = () => { if (!process.argv[1]) return false; try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); } catch { return false; } }; if (invokedDirectly()) { const argv = process.argv.slice(2); const titleIndex = argv.indexOf('--title'); const positional = argv.filter((a, i) => !a.startsWith('--') && (titleIndex === -1 || i !== titleIndex + 1)); const [base, head] = positional; const repo = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); if (!base || !head) { console.error('usage: proven-red.mjs <base> <head> [--title "<title>"]'); process.exit(2); } // An empty --title (a push event has no PR title) falls back to the head subject. const title = (titleIndex > -1 && argv[titleIndex + 1]) || git(['log', '-1', '--format=%s', head], repo); process.exit(proveRed({ base, head, repo, title })); }gap-trap/templates/gates/proven-red.sh9.0 KBc8bd32bd8e43upstream ↗
#!/bin/sh # Proven-red gate (AGENTS.md P2), language-neutral reference from gap-trap. # Needs git and the repo's test command. For a range base..head it takes the # unit test files changed since the fork point, runs each one in a worktree # that holds the fork-point code plus the head tests, and fails when it passes # there. A test that is green on the code it claims to guard proves nothing. # # sh proven-red.sh <base> <head> [--title "<pr title>"] # # Each changed test file is run and judged on its own. One file that fails # carries no proof for the others: a run that batched them together let a test # which passed on the old code ride out on a failing sibling's exit status. # Inside one file it still can: the verdict is the file's exit status, and # telling a new test from an old one needs per-test results, which only a # runner-specific port can read (proven-red.mjs does it for vitest). # # It also reads the run's output twice. A red made only of missing symbols # ("undefined:", "ImportError", "is not a function") proves the code is new, # not that the assertions bite, and gets a warning instead of a pass line. # And a non-zero status with no sign the runner ever started is not a red at # all: it exits 2 rather than reporting a proof it did not observe. # # Exit codes: 0 proved or skipped, 1 a test proved nothing, 2 the gate could # not run. set -u # ---- config --------------------------------------------------------------- APP_DIR=${GT_APP_DIR:-.} # ADAPT: where the test command runs # ADAPT: which changed paths are unit tests (proven), end-to-end tests (real # tests that need a live server, so they skip with a warning), test support # (travels along, never excuses a missing test), gate files (proven red by # scratch violation), non-code (never a behavior change). UNIT_TEST_RE=${GT_UNIT_TEST_RE:-'(_test\.go|_test\.py|/test_[^/]+\.py|\.test\.[jt]sx?|_spec\.rb|Test\.java|Tests?\.cs|_test\.rs)$'} E2E_TEST_RE=${GT_E2E_TEST_RE:-'(^|/)(e2e|steps|features)/'} TEST_SUPPORT_RE=${GT_TEST_SUPPORT_RE:-'(^|/)(tests?|__tests__|spec|testdata|fixtures)/|conftest\.py$|/setup\.(ts|js)$'} GATE_FILE_RE=${GT_GATE_FILE_RE:-'^scripts/|instruction.gate|proven-red|ratchet'} NON_CODE_RE=${GT_NON_CODE_RE:-'^(docs/|agents/|\.github/|\.githooks/|Makefile$|\.ratchet-|.*\.(md|rst|txt|yml|yaml)$|.*baseline.*\.json$)'} SKIP_TYPES='docs chore ci refactor build style test' # ADAPT: run one test file. Go tests run by package, so map the file to its dir. run_tests() { # $1 = one changed unit test path relative to APP_DIR go test "$(dirname "./$1")" # npx vitest run "$1" # python -m pytest "$1" # cargo test # rust: cannot select by file; runs all } # Symptoms of a missing symbol rather than a wrong value, across runners. # Matched against the whole run output. MISSING_RE=${GT_MISSING_RE:-'undefined: |undefined \(type|has no field or method|is not a function|Cannot find module|Failed to resolve import|ImportError|ModuleNotFoundError|AttributeError|NameError|cannot find symbol|unresolved import|no required module provides package|cannot find package|undeclared name|is not defined|no member named|has no exported member'} # Lines a runner prints as its own verdict, as opposed to source it echoed # back. ASSERTION_RE is matched against these lines only: every pytest failure # echoes the failing source line, so a bare `assert` over the whole output # marked every missing-symbol red as an assertion red and the warning below # could never fire. VERDICT_RE=${GT_VERDICT_RE:-'^E[[:space:]]|^[[:space:]]*---[[:space:]]*FAIL|^[[:space:]]*FAILED[[:space:]]|^[[:space:]]*AssertionError|^[[:space:]]*AssertionFailedError|^[[:space:]]*Error:|panic:|^[[:space:]]*[0-9]+\)[[:space:]]'} ASSERTION_RE=${GT_ASSERTION_RE:-'AssertionError|AssertionFailedError|assertion failed|--- FAIL|panic:|assert|expected|Expected'} # Evidence the runner started at all. A non-zero status without it is a broken # command or a broken worktree, not a test going red. RAN_RE=${GT_RAN_RE:-'--- FAIL|--- PASS|^ok[[:space:]]|^FAIL|^PASS|build failed|setup failed|Test Files|Tests[[:space:]]|passed|failed|error|no test files|Ran [0-9]+ test'} # ---- end config ----------------------------------------------------------- base=$1; head=$2; shift 2 title='' [ "${1:-}" = "--title" ] && title=${2:-} [ -n "$title" ] || title=$(git log -1 --format=%s "$head") fork=$(git merge-base "$base" "$head") || { echo "proven-red: no merge base for $base..$head"; exit 2; } files=$(git diff --name-only "$fork..$head") # A test deleted at head cannot be proven, and must not be run: the worktree # still holds its old copy, which passes and reads as a test that proves # nothing. Deletions still count as a source change below. live=$(git diff --name-only --diff-filter=d "$fork..$head") unit=$(printf '%s\n' "$live" | grep -E -- "$UNIT_TEST_RE" || true) e2e=$(printf '%s\n' "$live" | grep -E -- "$E2E_TEST_RE" | grep -vE -- "$UNIT_TEST_RE" || true) source_files=$(printf '%s\n' "$files" | grep -vE -- "$UNIT_TEST_RE" | grep -vE -- "$E2E_TEST_RE" | grep -vE -- "$TEST_SUPPORT_RE" | grep -vE -- "$NON_CODE_RE" || true) support=$(printf '%s\n' "$live" | grep -E -- "$TEST_SUPPORT_RE" | grep -vE -- "$UNIT_TEST_RE" | grep -vE -- "$E2E_TEST_RE" || true) warn_skip() { # $1 = reason. A skip that is a bypass rather than a proof. if [ -n "${GITHUB_ACTIONS:-}" ]; then echo "::warning title=Proven red skipped::proven-red: skipped, $1." else echo "proven-red: skipped, $1." fi exit 0 } if [ -z "$source_files" ]; then echo "proven-red: skipped, no source file changed."; exit 0; fi if [ -z "$unit" ]; then # Only gate files changed: those are proven red by scratch violation. if [ -z "$(printf '%s\n' "$source_files" | grep -vE -- "$GATE_FILE_RE" || true)" ]; then echo "proven-red: skipped, only gate files changed; a gate is proven red by scratch violation." exit 0 fi # The title is unverified text, so this skip is announced as a bypass. type=$(printf '%s' "$title" | sed -nE 's/^([a-z]+)(\(.+\))?!?:.*/\1/p') for t in $SKIP_TYPES; do [ "$type" = "$t" ] && warn_skip "title type \"$type\" carries no behavior change" done [ -n "$e2e" ] && warn_skip "the only changed tests are end-to-end, which need a live server" echo "proven-red: a behavior change arrived with no changed test (P2)."; exit 1 fi worktree=$(mktemp -d "${TMPDIR:-/tmp}/proven-red-XXXXXX") || { echo "proven-red: cannot make a worktree dir"; exit 2; } trap 'git worktree remove --force "$worktree" 2>/dev/null || rm -rf "$worktree"' EXIT git worktree add --detach -q "$worktree" "$fork" || { echo "proven-red: git worktree add failed; the gate cannot run."; exit 2; } # Read the head versions from git, not the working tree: locally the checkout # may be on another branch, and a test copied from there proves nothing. for f in $unit $support; do mkdir -p "$worktree/$(dirname "$f")" || { echo "proven-red: cannot write $f into the worktree"; exit 2; } git show "$head:$f" > "$worktree/$f" || { echo "proven-red: cannot read $head:$f"; exit 2; } done # ADAPT: share installed dependencies with the worktree instead of reinstalling. mkdir -p "$worktree/$APP_DIR" for d in node_modules .venv vendor; do [ -e "$APP_DIR/$d" ] && ln -s "$(cd "$APP_DIR" && pwd)/$d" "$worktree/$APP_DIR/$d" done green='' missing_only='' sample='' for f in $unit; do # Prefix removal, not sed: APP_DIR goes into a regex there, and the common # value "." matches any character, so "a/b.test.js" lost its first segment. rel=${f#"$APP_DIR"/} out=$(cd "$worktree/$APP_DIR" && run_tests "$rel" 2>&1); code=$? echo "--- $rel ---" printf '%s\n' "$out" if [ "$code" -eq 0 ]; then green="$green $rel" continue fi case "$code" in 126|127) echo "proven-red: the test command could not be run for $rel (exit $code)."; exit 2 ;; esac if [ "$code" -gt 128 ]; then echo "proven-red: the test command was killed for $rel (exit $code)."; exit 2; fi if ! printf '%s\n' "$out" | grep -qE -- "$RAN_RE"; then echo "proven-red: the test command exited $code for $rel with no sign it ran. That is a broken command or worktree, not a test going red." exit 2 fi verdicts=$(printf '%s\n' "$out" | grep -E -- "$VERDICT_RE" || true) if printf '%s\n' "$out" | grep -qE -- "$MISSING_RE" && ! printf '%s\n' "$verdicts" | grep -qE -- "$ASSERTION_RE"; then missing_only="$missing_only $rel" [ -n "$sample" ] || sample=$(printf '%s\n' "$out" | grep -E -- "$MISSING_RE" | head -1) fi done if [ -n "$green" ]; then echo "proven-red: these changed test files pass on the pre-change code; they cannot catch the bug they claim to:$green" exit 1 fi if [ -n "$missing_only" ]; then msg="proven-red: these files fail on the pre-change code only because they reference code it does not have, e.g. \"$sample\":$missing_only. That proves the code is new, not that the assertions would catch a wrong value. Check those assertions in review, or add the module to the mutation smoke." if [ -n "${GITHUB_ACTIONS:-}" ]; then echo "::warning title=Red by missing symbol only::$msg" else echo "$msg" fi fi echo "proven-red: changed tests fail on the pre-change code, as they should." exit 0gap-trap/templates/gates/ratchet.mjs6.9 KBb02b9408154bupstream ↗
#!/usr/bin/env node /** * Count ratchet (AGENTS.md C7). Reference implementation from gap-trap. * * A rule that says "do not do X" and has a backlog of X cannot be a hard * gate on day one, and an advisory check lets the backlog grow for free. * This records each count in a baseline file and fails when a count grows. * * node ratchet.mjs check against the baseline * node ratchet.mjs --update lower the baseline to the current counts * * `--update` only ever lowers. A count that rose, or a counter that left the * config, needs a hand edit to the baseline file, so the raise arrives as a * reviewable diff with a reason in the commit message instead of as a flag * the agent can reach for. A counter that cannot run is an error, never a * zero: a broken command that counted as 0 used to read as a clean sweep. * * Wire it into the unit suite (import `currentCounts` and compare) or run it * as its own CI step. ADAPT: fill `counters` with the counts this repo needs. * * Exit codes: 0 within baseline, 1 a count grew or drifted, 2 the ratchet * could not run (broken counter, missing baseline). */ import { readFileSync, readdirSync, realpathSync, writeFileSync } from 'node:fs'; import { fileURLToPath } from 'node:url'; import path from 'node:path'; const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); export const baselinePath = path.join(root, '.quality-baseline.json'); // A baseline this far above the real count is a raised number nobody lowered back. const SLACK = 5; function walk(dir, match, acc = []) { for (const entry of readdirSync(dir, { withFileTypes: true })) { if (entry.name === 'node_modules' || entry.name.startsWith('.')) continue; const full = path.join(dir, entry.name); if (entry.isDirectory()) walk(full, match, acc); else if (match.test(entry.name)) acc.push(full); } return acc; } /** Files under `dir` whose text matches `pattern`, as repo-relative paths. */ export function filesMatching(dir, fileMatch, pattern) { return walk(path.join(root, dir), fileMatch) .filter((f) => pattern.test(readFileSync(f, 'utf8'))) .map((f) => path.relative(root, f)); } /** Total occurrences of `pattern` across files under `dir`. */ export function occurrences(dir, fileMatch, pattern) { return walk(path.join(root, dir), fileMatch).reduce( (n, f) => n + (readFileSync(f, 'utf8').match(pattern)?.length ?? 0), 0, ); } // ADAPT: one entry per count. Names are stable keys in the baseline file. // Examples from the origin repo: // internalMockFiles: () => filesMatching('src', /\.test\.tsx?$/, /vi\.mock\(\s*['"]\.{1,2}\/(?:[^'"]*\/)?(stores|hooks|services|components)\//).length, // existenceAssertions: () => occurrences('src', /\.test\.tsx?$/, /(?<!\.not)\.toBeInTheDocument\(\)|\.toBeDefined\(\)/g), // fixedSleeps: () => occurrences('tests/steps', /\.ts$/, /waitForTimeout\(/g), export const counters = {}; /** * Run every counter. A counter that throws, or that returns anything but a * whole number, is a broken counter and stops the run: counting it as 0 would * report the backlog as swept and invite `--update` to lock the zero in. */ export function currentCounts(config = counters) { const out = {}; for (const [name, count] of Object.entries(config)) { let value; try { value = count(); } catch (error) { throw new Error(`counter "${name}" failed to run: ${error.message}`); } if (!Number.isInteger(value) || value < 0) { throw new Error(`counter "${name}" returned ${JSON.stringify(value)}, which is not a count`); } out[name] = value; } return out; } /** * Compare counts to the baseline. Returns the lines to print and an exit code. * Every baseline name is checked too, so deleting a counter from the config * cannot retire the number it was holding. */ export function check(counts, baseline) { const lines = []; let code = 0; for (const name of [...new Set([...Object.keys(baseline), ...Object.keys(counts)])].sort()) { if (!(name in counts)) { lines.push(`${name}: in the baseline but no counter is configured; restore the counter, or delete the baseline entry and say why in the commit message`); code = 1; continue; } const count = counts[name]; const allowed = baseline[name] ?? 0; if (count > allowed) { lines.push(`${name}: ${allowed} allowed, ${count} found (+${count - allowed})`); code = 1; } else if (allowed - count > SLACK) { lines.push(`${name}: baseline ${allowed} is ${allowed - count} above the ${count} found; run --update`); code = 1; } else if (count < allowed) { lines.push(`${name}: ${allowed} -> ${count}, run --update to lock the gain in`); } } return { lines, code }; } /** * What `--update` would do. It writes only when every count holds or falls and * every baseline name still has a counter; anything else has to be a hand edit. */ export function planUpdate(counts, baseline) { const refusals = []; for (const [name, count] of Object.entries(counts)) { const allowed = baseline[name]; if (allowed !== undefined && count > allowed) { refusals.push(`${name}: ${allowed} -> ${count} is a rise`); } } for (const name of Object.keys(baseline)) { if (!(name in counts)) refusals.push(`${name}: in the baseline but no counter is configured`); } return { refusals, next: { ...baseline, ...counts } }; } function run(argv) { let counts; try { counts = currentCounts(); } catch (error) { console.error(`ratchet: ${error.message}`); return 2; } let baseline = {}; try { baseline = JSON.parse(readFileSync(baselinePath, 'utf8')); } catch (error) { if (!argv.includes('--update')) { console.error(`ratchet: cannot read ${baselinePath}: ${error.message}`); return 2; } } if (argv.includes('--update')) { const { refusals, next } = planUpdate(counts, baseline); if (refusals.length > 0) { for (const line of refusals) console.log(line); console.log('--update only lowers. Edit the baseline by hand and say why in the commit message (C7).'); return 1; } writeFileSync(baselinePath, `${JSON.stringify(next, null, 2)}\n`); console.log('Baseline written:', next); return 0; } const { lines, code } = check(counts, baseline); for (const line of lines) console.log(line); if (code) console.log('Fix the new problems, or raise the number deliberately in the baseline file and say why in the commit message.'); else console.log('Counts within baseline:', counts); return code; } // Compare real paths: a checkout reached through a symlink (macOS /tmp and // /var are symlinks) made argv[1] and import.meta.url disagree, and the gate // exited 0 having run nothing at all. const invokedDirectly = () => { if (!process.argv[1]) return false; try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); } catch { return false; } }; if (invokedDirectly()) { process.exit(run(process.argv.slice(2))); }gap-trap/templates/gates/ratchet.sh5.0 KBde7e3423f00dupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
gap-trap/templates/pull_request_template.md430 B34f5c05f7161upstream ↗
Refs # ## Acceptance <!-- Quote the lines from the issue that say what done looks like. The Spec review axis (agents/generic/agent-workflows.md) judges the diff against these lines, so a PR without them cannot be reviewed for spec. --> ## Spec <!-- feat PRs only: link the spec, or one line saying why this feature needs none. --> ## Checks run <!-- The gates this change touched, and the full gate command before push. -->README.mdupstream ↗
Contents are not inlined in the catalog. Open the upstream file.
Plain-text copy, no login: /skill-bundles/pliablepixels-gap-trap-gap-trap.md