autoresearch

autoresearch — OpenClaw Skill

Auto-improve any OpenClaw skill using iterative, scored prompt optimization.
Inspired by Karpathy's autoresearch method. Zero manual work required.

What it does

Your Claude skills fail silently. You don't know which ones, or why.

This skill fixes that. It:

1. Runs your skill on test inputs
2. Scores each output against a binary yes/no checklist
3. Proposes one targeted change to the skill prompt
4. Keeps the change if the score improves — reverts if it doesn't
5. Repeats until 95%+ or max rounds hit

Fully autonomous once started. You define the checklist. The agent does the rest.

Real results

Install

npx clawhub@latest install bertuccio-admachine/autoresearch

Or clone manually:

git clone https://github.com/bertuccio-admachine/openclaw-skill-autoresearch.git
# Drop the folder into your OpenClaw skills directory

Requirements

• Python 3.9+
• pip3 install anthropic
• ANTHROPIC_API_KEY set in environment (or OpenClaw auth profile configured)

Quick Start

1. Write a checklist (my-checklist.json):

[
  {"question": "Does the output include a specific number or result?"},
  {"question": "Is it free of buzzwords like 'revolutionary' or 'synergy'?"},
  {"question": "Does it include a clear, specific call to action?"},
  {"question": "Is the output under 150 words?"}
]

3–6 questions is the sweet spot. Each must be answerable yes or no.

2. Write test inputs (my-inputs.json):

[
  "Write landing page copy for an AI productivity tool",
  "Write landing page copy for a DTC skincare brand"
]

3. Run it:

python3 scripts/autoresearch.py \
  --skill /path/to/your/SKILL.md \
  --checklist my-checklist.json \
  --inputs my-inputs.json \
  --max-rounds 20 \
  --target 0.95 \
  --dashboard

Walk away. Check back when it's done.

Or just say it

With the skill installed in OpenClaw, you can just tell your agent:

"Run autoresearch on my landing-page skill"

The agent will ask for your checklist criteria, run the loop, and report back.

Live Dashboard

Pass --dashboard to get a live HTML dashboard that auto-refreshes every 8 seconds:

• Score chart over time
• Per-check pass/fail breakdown
• Full change log with keep/revert status

Outputs

The log file is the most valuable artifact. Every failure pattern becomes a permanent rule. Hand it to a future model and it picks up exactly where this one left off.

How to write a good checklist

Good questions:

• Binary — pass or fail, no "sort of"
• Specific — "mentions a number" not "is concrete"
• Independent — each checks exactly one thing

Bad questions:

• "Is the output high quality?" (too vague)
• "Is the tone right?" (too subjective)
• "Is it good?" (useless)

The tighter the checklist, the cleaner the output. Constraints do the work.

How it works

loop:
  run skill on all test inputs
  score each output against checklist  ← Claude as judge
  average scores across inputs
  propose ONE targeted change           ← Claude as improver
  run again with new skill
  if score improved → keep change
  else → revert
  repeat until target or max rounds

The scoring uses Claude itself as an evaluator — fast, consistent, and binary. The improvement step is also Claude, with full context of what's been tried and what's worked.

Example output

⚙️  Autoresearch: landing-page
   Checklist: 5 checks | Inputs: 3 | Target: 95%

📊 Baseline: 56.0%
   ✗ Does the headline include a specific number: 0%
   ✗ Is it free of buzzwords: 33%
   ✓ Does it include a CTA: 100%
   ✗ Does the first line address a pain point: 67%
   ✓ Is it under 150 words: 100%

🔄 Round 1: Added banned buzzwords list → 72.0% ✓ KEPT
🔄 Round 2: Added headline rule with number requirement → 88.0% ✓ KEPT  
🔄 Round 3: Added worked example → 92.0% ✓ KEPT
🔄 Round 4: Tighter word count → 84.0% ✗ reverted

✅ Done — 92.0% (was 56.0%) in 4 rounds

License

MIT — use it, fork it, improve it.

---

Built by Bertuccio for the Ad Machine team.
Inspired by Andrej Karpathy's autoresearch method.