sensei

# Sensei > "A true master teaches not by telling, but by refining." - The Skill Sensei Sensei automates the improvement of [Agent Skills](https://support.anthropic.com/en/articles/12512198-how-to-create-custom-skills) frontmatter compliance using the [Ralph loop pattern](https://github.com/soderlund/ralph) - iteratively improving skills until they reach Medium-High compliance with all tests passing. ## Table of Contents - [Overview](#overview) - [Quick Start](#quick-start) - [Prerequisites](#prerequisites) - [How It Works](#how-it-works) - [Configuration](#configuration) - [Scoring Criteria](#scoring-criteria) - [Examples](#examples) - [Troubleshooting](#troubleshooting) - [Contributing](#contributing) --- ## Overview ### The Problem Skills without proper frontmatter lead to **skill collision** - agents invoking the wrong skill for a given prompt. Common issues include: - **No triggers** - Agent doesn't know when to activate the skill - **No anti-triggers** - Agent doesn't know when NOT to use the skill - **Brief descriptions** - Not enough context for accurate matching - **Token bloat** - Oversized skills waste context window ### The Solution Sensei implements the "Ralph Wiggum" technique: 1. **Read** - Load the skill's current state and token count 2. **Score** - Evaluate frontmatter compliance 3. **Improve** - Add triggers, anti-triggers, compatibility 4. **Verify** - Run tests to ensure changes work 5. **Check Tokens** - Analyze token usage, gather suggestions 6. **Summary** - Display before/after with suggestions 7. **Prompt** - Ask user: Commit, Create Issue, or Skip? 8. **Repeat** - Until target score reached --- ## Quick Start ### Using with Copilot CLI #### Single Skill ``` Run sensei on my-skill-name ``` #### Single Skill (Fast Mode) ``` Run sensei on my-skill-name --fast ``` #### Multiple Skills ``` Run sensei on skill-a, skill-b, skill-c ``` #### All Low-Adherence Skills ``` Run sensei on all Low-adherence skills ``` #### All Skills ``` Run sensei on all skills ``` #### GEPA Mode (Deep Optimization) ``` Run sensei on my-skill-name --gepa Run sensei score my-skill-name ``` Score-only mode runs without LLM calls. ### Using Scripts Directly ```bash # Count tokens in all markdown files npm run tokens -- count # Count tokens in specific files npm run tokens -- count SKILL.md references/*.md # Check files against token limits npm run tokens -- check # Check with strict mode (exits 1 if limits exceeded) npm run tokens -- check --strict # Get optimization suggestions npm run tokens -- suggest # Score a skill directory (advisory checks) npm run tokens -- score . # Compare with previous commit npm run tokens -- compare HEAD~1 ``` ### GEPA Commands ```bash python scripts/src/gepa/auto_evaluator.py score --skill my-skill python scripts/src/gepa/auto_evaluator.py optimize --skill my-skill ``` ### Flags | Flag | Description | |------|-------------| | `--fast` | Skip tests for faster iteration | | `--gepa` | Use GEPA evolutionary optimization instead of template-based improvements | | `--skip-integration` | Skip integration tests (unit + trigger tests only) | > ⚠️ **Note:** Using `--fast` speeds up the loop significantly but may miss issues. Consider running full tests before final commit. --- ## Prerequisites ### Required 1. **Node.js 18+** - For running token management scripts ```bash node --version ``` 2. **Git** - For commits and comparisons ```bash git --version ``` ### Optional 3. **Test Framework** - Jest, pytest, or similar for trigger tests 4. **Python 3.10+** and **GEPA** - For evolutionary optimization (`pip install gepa`) ### Installation #### Option 1: Install as Copilot CLI Skill (Recommended) ```bash # Create the skills folder mkdir -p "$HOME/.copilot/skills" # Clone to your skills directory git clone https://github.com/spboyer/sensei.git "$HOME/.copilot/skills/sensei" # Install token CLI dependencies cd ~/.copilot/skills/sensei/scripts && npm install ``` The skill is now available in Copilot CLI. Invoke with: ``` Run sensei on my-skill-name ``` #### Option 2: Install in Project Skills Folder For project-specific installation: ```bash # From your project root mkdir -p .github/skills git clone https://github.com/spboyer/sensei.git .github/skills/sensei # Install dependencies cd .github/skills/sensei/scripts && npm install ``` #### Verify Installation ```bash # Test the token CLI cd ~/.copilot/skills/sensei # or your install path npm run tokens -- check # Should output token counts for all markdown files ``` --- ## How It Works ### The Ralph Loop ``` ┌─────────────────────────────────────────────────────────┐ │ START: User invokes "Run sensei on {skill-name}" │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 1. READ: Load skills/{skill-name}/SKILL.md │ │ Load tests/{skill-name}/ (if exists) │ │ Count tokens (baseline for comparison) │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 2. SCORE: Run rule-based compliance check │ │ • Check description length (> 150 chars?) │ │ • Check for trigger phrases ("USE FOR:") │ │ • Check for anti-triggers ("DO NOT USE FOR:") │ │ • Check for compatibility field │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌───────────────┐ │ Score >= M-H │──YES──▶ COMPLETE ✓ │ AND tests pass│ (next skill) └───────┬───────┘ │ NO ▼ ┌─────────────────────────────────────────────────────────┐ │ 3. SCAFFOLD: If tests/{skill-name}/ missing: │ │ Create tests from references/test-templates/ │ │ Creates prompts.md and framework-specific tests │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 4. IMPROVE FRONTMATTER: │ │ • Add "USE FOR:" with trigger phrases │ │ • Add "DO NOT USE FOR:" with anti-triggers │ │ • Add compatibility if applicable │ │ • Keep description under 1024 chars │ │ • OR with --gepa: GEPA evolutionary optimization │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 5. IMPROVE TESTS: │ │ • Update shouldTriggerPrompts (5+ prompts) │ │ • Update shouldNotTriggerPrompts (5+ prompts) │ │ • Match prompts to new frontmatter triggers │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 6. VERIFY: Run tests for the skill │ │ • If tests fail → fix and retry │ │ • If tests pass → continue │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 7. CHECK TOKENS: │ │ npm run tokens count {skill}/SKILL.md │ │ Verify under 500 token soft limit │ └─────────────────────┬───────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────┐ │ 8. SUMMARY: Display before/after comparison │ │ • Score change (Low → Medium-High) │ │ • Token delta (+/- tokens) │ │ • Unimplemented suggestions │ └─────────────────────┬───