# ablate-ai-layer

Adopt this skill: https://proskills.md/skills/other/coleam00-skills-claude-skills-ablate-ai-layer

Page: https://proskills.md/skills/other/coleam00-skills-claude-skills-ablate-ai-layer
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: .claude/skills/ablate-ai-layer/SKILL.md
size: 7654
upstream: https://github.com/coleam00/skills/blob/main/.claude/skills/ablate-ai-layer/SKILL.md

````text
---
name: ablate-ai-layer
description: Measure whether a repository's AI instructions still earn their place, by running the same real task many times with the layer intact and with it stripped, then grading every rule against what actually changed. Runs both arms itself in throwaway git worktrees and never touches the working tree. Agent-agnostic across CLAUDE.md, AGENTS.md, .claude/, .agents/, .cursor/rules, .clinerules, .windsurfrules and copilot-instructions. Use when the user wants to prune, audit, clean up, shrink or "delete" their CLAUDE.md, AGENTS.md, cursor rules, agent instructions or AI layer; when they ask whether their rules are still needed, whether their context is bloated, or what to cut; or when they mention ablating, ablation, or testing their agent without its instructions.
---

# Ablate the AI layer

Model upgrades quietly retire instructions. A rule written to work around a weaker
model becomes dead weight that competes for attention with the rules that still
matter. Reading the file will not tell you which is which. Only an experiment will.

**You run the experiment. The user picks the task and approves the conclusion.**
Do not hand the user a list of commands to run; the script drives both arms.

## What makes the result trustworthy

- **Both arms, many runs each.** A stripped agent does not visibly fail, so a single
  run has nothing to compare against and "seems fine" becomes "delete something
  load-bearing". Two runs of the *same* arm can also differ more than the two arms
  differ, so one pair per arm is the floor, not the target.
- **Nothing is moved aside.** Every run happens in a detached git worktree built
  from HEAD in a temp directory, outside the repo, and deleted afterwards. The
  user's working tree is never modified, so there is no restore step to forget.
- **Only the always-loaded set is stripped by default.** Skills, subagents and
  path-scoped rules cost nothing until they fire, so deleting them buys back no
  context. Hooks and permissions are never touched: they run as code and spend no
  attention.

---

## Step 1. Map the layer

```bash
python <skill>/scripts/map_layer.py [repo_root]
```

Read-only. Sorts every artifact into always-loaded, on-demand, and enforcement, and
prints what the always-loaded set costs on every session before the user types
anything. Show them that number.

If nothing is found, say so and stop. There is nothing to test.

## Step 2. Get the probe task

**This is the one thing you must not decide for the user.** The task determines
whether the experiment can detect anything at all.

A good probe task is real work they would do anyway, touches code where house
conventions plausibly apply, and adds something that has to be wired in: a test, an
endpoint, a migration, a command.

A bad one is a typo, a rename, or any one-line fix. It is fully derivable, both arms
will match, and the user will wrongly conclude their whole layer is worthless. Say
that out loud if they offer one, and ask for something with conventions at stake.

Write the agreed task verbatim to a file. Every run reuses it byte for byte.

## Step 3. Show the plan and get approval

Report before spending anything: how many runs, which model, roughly what it will
cost, and that the working tree will not be touched.

```bash
python <skill>/scripts/run_ablation.py <repo> --task-file <task.md> --dry-run
```

The dry run also surfaces two things worth pausing on:

- **A dirty working tree.** Worktrees are built from HEAD, so uncommitted edits are
  not under test. Offer to commit or stash first.
- **A build-dependency warning.** Some repos import their own AI layer as source. A
  CLI that reads its skill markdown at build time breaks the moment those files go
  missing, and the user will read a compile error as an agent regression. Keep
  `--scope always` if this warns.

## Step 4. Run it

```bash
python <skill>/scripts/run_ablation.py <repo> --task-file <task.md> --runs 2
```

This is the whole experiment. It builds a fresh worktree per run, strips the layer
in the stripped arm, runs the same prompt in each, captures every diff, cleans up
every worktree, and writes results to `.ablation/<timestamp>/` (gitignored).

Useful flags: `--runs 3` when the user intends to act on the result, `--scope all`
to test the harder claim that skills and subagents have expired too, `--model`,
`--jobs` for concurrency, `--runner` for a non-Claude agent that reads a prompt on
stdin.

If an arm produced nothing usable, stop. An empty arm is a broken experiment, not a
finding. Re-run before drawing anything from it.

## Step 5. Grade

Read `references/comparison.md` before analysing. It is the rubric, and it contains
the two things that make the difference between a real result and a confident wrong
one: grade **per rule** rather than diffing the arms against each other, and grade
**blind** to which arm a diff came from.

The short version:

1. Turn the always-loaded files into a numbered checklist of testable claims. Mark
   anything unfalsifiable ("write clean code") as exactly that.
2. Judge every run's diff against every claim: `followed`, `violated`, or `n/a`.
3. Only then join verdicts back to arms and read the pattern.

## Step 6. Report

Give the user a table, one row per rule, sorted so the actionable rows are first:

| Pattern across runs | Verdict | Action |
|---|---|---|
| control follows, stripped violates | load-bearing | keep, rewrite shorter |
| both arms follow | model does this anyway | delete |
| both arms violate | ignored even when loaded | make it a hook or test, or delete |
| never applicable | **untested** | keep, no evidence either way |
| inconsistent within an arm | noise | more runs or a better task |

Keep "untested" visually separate from "no difference". They look identical in the
data and mean opposite things, and merging them is how a rule that protects a case
this task never touched gets deleted.

## Step 7. Apply, with the user's approval

Never edit the rules file unattended. Propose the edit, show the diff, wait.

Re-add or keep one line at a time, only for rules with observed evidence, and prefer
a test, then a hook, then an on-demand instruction, and only then an always-loaded
line. Finish by re-running `map_layer.py` so the new always-loaded total sits next to
the old one.

---

## Honest framing to give the user

- **One probe task is a data point, not a verdict.** Encourage a second task on a
  different part of the codebase before deleting anything large.
- **A null result is a real result.** If the arms match, that part of the layer has
  genuinely expired and can go.
- **The reverse is also true.** Do not let one clean run justify deleting rules for
  cases this task never exercised: security, compliance, release procedure.
- **Existing code substitutes for the rules file.** A stripped run copies
  conventions from neighbouring code when there is a neighbour to copy. The same
  rule can hold in an edited file and break in a new one. Weight new-file evidence
  more heavily and say which kind each verdict rests on.
- **Cheaper and smaller models lean on instructions more than frontier models do.**
  A layer that looks redundant under a frontier model may still be carrying a
  cheaper one. If the team runs a mix, ablate against the weakest model in use.

## Resources

- `scripts/map_layer.py`: read-only inventory of the layer, agent-agnostic.
- `scripts/run_ablation.py`: runs both arms and collects the diffs. `--help` lists
  every flag. Never read either script into context; only their output.
- `references/comparison.md`: the grading rubric. Read it before Step 5.
````

## File: README.md
upstream: https://github.com/coleam00/skills/blob/main/README.md

````text
# Cole's AI Skills

The skills I actually use to build software with coding agents. Straight out of my `.claude/skills/` folder.

## What this is

A skill is a folder with a `SKILL.md` in it: a name, a description of when to use it, and the procedure the agent
should follow. Your agent loads the description at startup and pulls in the full skill only when the work matches.
That's the whole idea, and it's why skills scale where a 2,000-line `CLAUDE.md` doesn't.

These skills build on the AI Layer from my [Agentic Coding course](https://dynamous.ai). They're built around one
loop I run on nearly every ticket:

**prime → plan → implement → validate → review → commit → PR**

Around that loop sit the pieces that feed it (PRD, architecture, epic slicing), the pieces that run it in parallel
(worktrees), and the meta-skills that let you build more of your own AI Layer (rules, hooks, skills, opportunity
scans).

Nothing here is a framework. Each skill is a plain markdown file you can read in two minutes, disagree with, and
edit. That's the point: read them, take the ones that fit how you work, rewrite the rest.

## Bring them in

### As a Claude Code plugin (easiest, stays updated)

Run these two commands inside Claude Code:

```
/plugin marketplace add coleam00/skills
/plugin install skills@cole-medin
```

That's it. The skills are managed and read-only, and `/plugin marketplace update` pulls new ones as I add them.
Plugin skills are namespaced, so you invoke them as `/skills:piv-implement`.

Only the descriptions stay in context; the bodies load when a skill fires. Run `claude plugin details skills`
to see the current per-skill breakdown, and disable the plugin any
time with `/plugin`.

> **If that first command fails with `Permission denied (publickey)`:** the `owner/repo` shorthand prefers SSH,
> and your SSH key isn't authenticating to GitHub. Recent Claude Code versions detect that and fall back to HTTPS
> on their own. If yours doesn't, pass the HTTPS URL directly, which needs no key:
>
> ```
> /plugin marketplace add https://github.com/coleam00/skills.git
> ```
>
> Setting `CLAUDE_CODE_PLUGIN_PREFER_HTTPS=1` in your environment makes the shorthand use HTTPS permanently.

### As editable files, in any agent

```bash
npx skills add coleam00/skills                             # everything
npx skills add coleam00/skills --list                      # see what's here first
npx skills add coleam00/skills --skill piv-plan-implementation piv-implement piv-validate
```

Add `-g` to install globally (`~/.claude/skills/`) instead of into the current project. This route writes real
files into your repo, so you can edit them, which is what I'd actually recommend once you know which ones you keep
reaching for.

### Or just clone and copy

They're only markdown files:

```bash
git clone https://github.com/coleam00/skills.git
cp -r skills/.claude/skills/piv-implement your-project/.claude/skills/
```

### Or ask your agent

This works fine too:

> Clone https://github.com/coleam00/skills, look at the skills in `.claude/skills/`, and copy the ones
> that fit this project into my `.claude/skills/` folder. Tell me which ones you picked and why.

For the last two, restart your session (or run `/skills`) and they'll show up.

## The skills

**Prime: load the right context, and only that**

| Skill | What it does |
|---|---|
| `prime-codebase` | Orients the agent in a codebase before planning or implementing |
| `prime-backend` | Same, scoped to API routes, services, and the data layer |
| `prime-frontend` | Same, scoped to components, routing, state, and styling |

**Plan: intent before implementation**

| Skill | What it does |
|---|---|
| `plan-create-prd` | Interviews you into a problem-first PRD. Intent, never engineering decisions |
| `plan-architecture` | A working session on *how* to build it: stack, data shape, trade-offs, risks |
| `plan-with-priorities` | Elicits what matters before recommending options, with small spikes and explicit tradeoffs |
| `piv-slice-epic` | Slices an epic plus its architecture into PIV-sized tickets with a dependency graph |
| `plan-create-stories` | Turns a PRD into a real backlog in Jira or GitHub |

**The PIV loop: plan, implement, validate**

| Skill | What it does |
|---|---|
| `piv-plan-implementation` | Deep codebase analysis plus research into a one-pass-ready implementation plan |
| `piv-implement` | Executes that plan task-by-task, validating at every step |
| `piv-validate` | Runs the project's full suite and returns one PASS/FAIL verdict |
| `piv-review-changes` | Pre-commit technical review of what changed |
| `piv-fix-review-findings` | Triages review findings. You decide what gets fixed now vs deferred |
| `piv-commit` | One atomic, conventionally-tagged commit |
| `piv-create-pr` | Pushes the branch and opens the PR with a real body |
| `piv-review-pr` | The agentic gate on an open PR: fresh eyes, severity-ranked, posted to GitHub |
| `piv-run-full-loop` | Chains the core loop end-to-end from a single feature description |

**Issues: diagnose before you fix**

| Skill | What it does |
|---|---|
| `piv-investigate-issue` | Parallel investigation of a GitHub issue into an evidence-backed RCA |
| `piv-implement-issue` | Implements the fix from that RCA, with regression tests |

**Parallel work**

| Skill | What it does |
|---|---|
| `worktree-create` | Spins up N git worktrees, each configured, installed, and health-checked |
| `worktree-merge` | Integrates those branches through one safe integration branch |

**Build your own AI Layer: the meta-skills**

| Skill | What it does |
|---|---|
| `rules-create-global` | Derives a lean root `CLAUDE.md` from your codebase or your specs. The customizable `/init` |
| `rules-check-drift` | Checks whether your rules file is still *true* after recent changes |
| `ablate-ai-layer` | Tests whether your rules still earn their place: strips them, reruns the same task, diffs the two |
| `skills-create` | Authors a new skill, or refactors a fat one into `SKILL.md` plus `references/` |
| `hooks-create` | Turns "never let the agent touch my migrations" into a working hook, wired into settings |
| `opportunity-scan` | Reads how you actually work and recommends what to encode next |
| `system-execution-report` | Reflects on a just-finished implementation: what diverged from the plan |
| `system-evolution-review` | Finds the bugs in your *process*, not your code |
| `second-brain-audit` | Finds facts in your notes that quietly stopped being true, and restructures so they stop |
| `second-brain-prd` | Turns a filled-out requirements template into a phased TypeScript build plan for your own AI second brain: memory, hooks, search, integrations, skills, heartbeat, chat, security, deployment |

**Autonomy: hand the whole loop over**

| Skill | What it does |
|---|---|
| `build-dark-factory` | Takes a PRD and builds a repo around it that ships validated code with nobody at the keyboard. All five components, in construction order, plus a deterministic audit of what you built. It encodes the AI coding process you already run rather than replacing it, and it deliberately does not write the PRD: bring one, or make one with `plan-create-prd` first |

**Keeping up: a signal engine**

| Skill | What it does |
|---|---|
| `build-signal-engine` | Interviews you about what you need to keep up with, what you missed last month and where it showed up, then builds a system from scratch that reads every source you care about (changelogs, communities, feeds, videos), makes a quick decision on every item, and hands an LLM only what survives for one daily digest. Built a stage at a time into your repo, each proven on real data |

**Tools**

| Skill | What it does |
|---|---|
| `agent-browser` | Browser automation for the agent: navigate, fill, click, screenshot, extract |
| `ast-grep` | Structural code search by AST pattern instead of text |
| `drive-screen` | Real desktop control: focus a window, type, paste, click, s
````
