skeleton-first

skeleton-first

An agent skill that makes coding agents prove a data / AI workflow end to end on a tiny sample before spending real money on it, then scale in cost-gated steps.

Agents like to build every stage at full size and run it for the first time on the whole dataset. With paid compute (LLM calls, embeddings, warehouse credits) that fails expensively: a wrong prompt runs over the whole table, a bad grain is found after twenty models exist, a budget is blown before anyone looks at the data.

skeleton-first changes the order of work:

1. Define "it works": the thinnest end-to-end path, one pass/fail test, a budget.
2. Build the walking skeleton on 1 record with the cheapest settings, and look at the sample for cheap wins (duplicates, boilerplate, invalid inputs the API still bills).
3. Scale in rungs (1 → 10 → ~100 → all). At each rung: measure cost, project the next rung, and gate on the test and the budget. If the projection is over budget, find a cheaper path that still passes the test before handing the problem back.
4. Add features one at a time, re-running every test.
5. Keep a RUNLOG.md with the cost, time and decision at each rung (example).

It works with any tool that supports the open Agent Skills SKILL.md format, and includes cost levers for Snowflake / Cortex and dbt.

Install

The skill is one folder: skills/skeleton-first/.

git clone https://github.com/danhe6/skeleton-first.git
mkdir -p ~/.claude/skills ~/.agents/skills
cp -r skeleton-first/skills/skeleton-first ~/.claude/skills/   # Claude Code
cp -r skeleton-first/skills/skeleton-first ~/.agents/skills/   # Codex, Cursor

Restart the agent if the skill doesn't show up. It triggers on its own for pipelines, batch LLM jobs, RAG ingestion, dbt builds and similar. You can also ask for it by name: "use skeleton-first".

Does it help? Eval results

We tested it with skill-creator style A/B runs: the same task given to an agent with the skill and an agent without it. Every "paid" API is a local mock that writes each billed call to a ledger, so spend is exact. The tasks contain realistic traps: duplicate records, a boilerplate block that is 70–90% of every document, empty inputs the API bills anyway, and a budget the naive approach would exceed.

Assertion pass rate: 100% with the skill vs 80% without (iteration 2). Cost of using it: about 6.5k extra tokens and ~30 s more wall time per task, mostly from the extra rungs.

Caveats: one run per task per configuration, mock APIs, and assertions written by the skill's authors. Treat these numbers as a sanity check, not a benchmark. The prompts are in evals/evals.json; evals/make_fixtures.py recreates the test data and evals/grade.py grades the outputs.

What iteration 1 taught us (and what changed in the skill):

• The first version stopped and asked as soon as the projection exceeded the budget, and delivered only 10 of 300 docs. Now the budget is a constraint to design around: find a cheaper path that passes the same test, prove it on a small rung, and stop only if none exists.
• Neither version noticed the repeated footer. The skill now says to look at the sample and measure where the cost goes (duplicates, boilerplate, invalid inputs) before scaling.

Repo layout

skills/skeleton-first/SKILL.md   the skill
examples/                        real RUNLOG.md files from eval runs
evals/                           prompts, fixture generator, grader

License

MIT