⏳ This skill is pending AI review.

Scores will appear once the review pipeline completes.

v0.5.0

slop-grading

lukstei@lukstei⭐ 38 stars· last commit 17d ago· 0 open issues

>

Choose how to use this skill

You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.

1. Native installer

This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.

Do not guess an installer command or replace an existing version without reviewing the diff.

2. Complete package recommended

Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.

No complete ProSkills package is published for this listing yet.

3. Prompt-only

Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.

Need only the instruction file?

Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.

No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.

—/10

// RATINGS

⭐GitHub Stars
⭐⭐ 38 on GitHubGitHub ↗

Growing

🟢ProSkills Score
—
📍

Not yet listed on ClawHub or SkillsMP

// README

slop-grader

GitHub Actions Workflow Status NPM Version unpacked size NPM Downloads License: MIT

Rule-based slop grader for text files, powered by Jev. Runs every rule against every line in parallel. No skimming, no missed lines.

How it works

slop-grader runs as a two-step loop: grade text with the CLI, then paste the output to your AI agent to plan the improvements.

1. Grade the document

Run slop-grader on a document like examples/slop.md:

npx @lukstei/slop-grader@latest -r no-ai-slop examples/slop.md

2. Fix with an AI agent

Pass the output to your AI agent:

  • The agent distinguishes real violations from false positives and generates concrete replacements (Example with Gemini 3.8 Flash):

    Line 1 — banned_word

    • Original: # 🚀 The Ultimate Paradigm Shift in Modern Data Architecture
    • Fix: # Modern Data Architecture
    • Reason: Removes the banned phrase "paradigm shift" and decorative emoji.

    Line 7 — binary_contrast + faux_insight + colon_reveal

    • Original: What most people get wrong about databases is simple: it's not about speed, it's about trust.
    • Fix: Database design balances speed and trust.
    • Reason: Removes rhetorical framing and fake insight.
  • After your review the plan is applied to produce an improved document.

Features

  • Parallel exhaustive grading: Checks every rule against every line independently. No skimming.
  • Incremental line caching: Re-evaluates only edited lines; unchanged text resolves from cache with zero API calls. Toggle with --no-cache.
  • System One efficiency: Typed probabilities via Jev without text generation. Thousands of checks for cents.
  • Dynamic batching: Groups lines to token limits to minimize API calls. See evaluation details.
  • Context window padding: Evaluates line rules with $\pm 10$ lines of surrounding context. Boundary lines and sparse edits keep full visibility without paying for extra questions.
  • Line and document scope: Flags line patterns and rates whole documents on qualitative rubrics.
  • Plain Markdown rulesets: Write rules in Markdown; validate offline with --check.
  • Built-in rulesets: Ready-to-use rules for AI writing patterns, document scores, tech docs, and grammar.
  • Agent and CI ready: Terminal output for agent fix plans; structured JSON for pipelines.
  • Multi-provider: Works with TypeSafe AI and OpenRouter out of the box.

FAQ

Evaluation separates line-level checks (spotting specific patterns or phrases) from document-level checks (evaluating tone or overall structure).

Documents have hundreds of lines, but the number of rules is fixed. Sending one API request per line would mean hundreds of calls. Instead, slop-grader dynamically groups lines into batches sized to fit the model's context budget (up to 255 lines per batch) and evaluates each rule across its batch in a single call. Batches include up to 10 lines of surrounding document context padding so boundary lines and sparse edits retain neighboring visibility for cross-line checks. Every batch response is verified for complete answer-to-question parity; any dropped questions halt execution immediately without caching incomplete results.

Document rules run in a single request across the entire text.

Line rules often depend on nearby text: pronoun references, synonym repetition across sentences, or transitions between paragraphs. Evaluating a line in isolation makes these checks impossible.

slop-grader pads each evaluated line with up to 10 lines of surrounding text ($\pm 10$ lines):

  1. Region clustering: Lines slated for evaluation are grouped into continuous regions. Targets within 21 lines of each other merge into a single region so the model sees uninterrupted text without repeated lines.
  2. Structured omission: When distant lines or sparse edits share a batch, gaps between regions are marked with ...| [omitted lines], preserving uniform line structure for the model.
  3. Zero extra question cost: Padding lines exist strictly as non-evaluated context in the prompt state. Questions are generated only for active evaluation targets, so context lines add no question fees and don't count toward rule quotas.

This ensures single-line edits during incremental cache runs retain their full document surroundings rather than floating in isolation.

Line evaluations are cached by the hash of the line's content and the rule's criteria.

When you edit a document and run slop-grader again:

  1. Every line is matched against the local cache for each rule.
  2. Unchanged lines resolve immediately from the cache with zero API calls and zero cost.
  3. Only new or modified lines are batched and sent to the model.
  4. If a rule's instructions or criteria change, its cache invalidates automatically.

This makes repeated runs on large files or during editing loops nearly instantaneous. Pass --no-cache to bypass the cache, or --cache-dir <dir> to customize its location. See docs/CACHE.md for technical details on storage layout, hashing, and eviction.

Standard generative LLMs evaluate an entire document in a single prompt against a list of rules. On longer texts, they skip lines, miss rules, and report wrong line numbers.

Running a separate check for every line and rule with a generative LLM is impractical. A 300-line draft tested against 10 rules would require 3,000 text-generation requests, which is slow and expensive.

slop-grader uses a System One model (Jev). System One models answer discrete semantic questions with typed probabilities without generating text. Because these judgments return numbers instead of prose tokens, slop-grader can test every line against every rule separately and in parallel.

Cost scales with the number of rules and non-

// HOW IT'S BUILT

// FILES

2 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

  • SKILL.md7.9 KBupstream ↗
    ---
    name: slop-grading
    description: >
      Review the output of slop-grader, interpret flagged lines and document scores,
      distinguish genuine violations from false positives, and produce a concrete
      fix plan with edited text for each real issue.
    ---
    
    # Slop Grading — Review & Fix Plan
    
    You receive the output of `slop-grader` (human-readable text or JSON via `--json`) and the original source file. Available built-in rulesets can be listed with `slop-grader --list-rulesets` (or `-l`, `--json` to inspect scopes, rule counts, and rule IDs). Line evaluations are cached by default; pass `--no-cache` to force full re-evaluation, or `--concurrency <num>` to cap concurrent API requests (defaults to 5). Your job is to triage every flagged line, dismiss false positives, and produce a minimal fix plan with exact replacement text for every genuine violation.
    
    ## Output formats of slop-grader
    
    The tool emits either human-readable text (default) or structured JSON (via `--json` / `-j`).
    
    ### Format A — Human-readable text
    
    ```
    Rules:
      /absolute/path/to/no-ai-slop.md
      /absolute/path/to/article-scores.md
    
    A=banned_word, B=empty_adverb, C=importance_puffery   ← legend
    
    A,C           | L0003: The launch marks a pivotal moment for the company.
    B             | L0007: This is just a small update.
    
    ## Document Scores
    
    engagement       2.7/3   (confidence high)  "Holds attention — creates genuine curiosity..."
    narrative_arc    1.4/3   (confidence mid)   "Loosely organized" ↔ "Clear progression"
    closing_strength 0.3/3   (confidence high)  "Trails off or optimizes"
    
    ## Stats
    
    Rules applied:                6 (5 line, 1 document)
    Lines evaluated:              12
    Questions asked:              61
    API calls:                    6
    Questions evaluated via API:  61
    Cache hits:                   0
    ```
    
    - **Legend:** maps letters to rule IDs (`A`–`Z`, `AA`–`ZZ`).
    - **Flagged lines:** `<letters> | <line-marker>: <original text>`. Only lines that crossed the threshold (default 0.8) appear. If line rules are evaluated but no violations are found, `No line rule violations found.` is displayed.
    - **Document Scores:** probability-weighted mean position across 4 levels (0–3), confidence tier (`high` ≥ 0.8, `mid` 0.5–0.79, `low` < 0.5), and descriptions.
    - **Stats (optional):** execution metrics appended when `--stats` is passed. Ignore this block during triage.
    - **Debug (optional):** when `--debug` is passed, API calls are logged to stderr; ignore stderr output during triage.
    - Lines with no flags are clean — do not touch them.
    
    ### Format B — Structured JSON (`--json`)
    
    ```json
    {
      "file": "/abs/path/to/my-draft.txt",
      "rules": ["/abs/path/to/no-ai-slop.md"],
      "violations": {
        "lines": [
          { "lineNum": 1, "text": "Our platform empowers teams...", "rules": ["banned_word"] }
        ],
        "document": {
          "narrative_arc": { "score": 1.4, "max": 3, "confidence": 0.72, "label": "Loosely organized" }
        }
      },
      "stats": {
        "rules": 6,
        "lineRules": 5,
        "docRules": 1,
        "lines": 12,
        "questions": 61,
        "apiCalls": 6
      }
    }
    ```
    
    - **`violations.lines`:** list of flagged lines with 1-indexed `lineNum`, original `text`, and array of `rules`.
    - **`violations.document`:** document-level quality scores and boolean rule violations. Map numerical `confidence` to tiers: `high` (≥ 0.8), `mid` (0.5–0.79), `low` (< 0.5).
    - **`stats`:** execution metrics. Ignore during triage.
    - **Clean output:** `violations.lines` and `violations.document` are empty when no issues are detected.
    
    ## Step 1 — Parse
    
    1. **Check for clean output:** If no lines are flagged and no document scores need attention (or `violations.lines` and `violations.document` are empty in JSON), report that the document is clean and stop.
    2. **For human-readable text:** Read the legend. Map each letter back to its rule ID and the rule's plain-English meaning. List each flagged line with its rule(s) spelled out.
    3. **For JSON:** Read `violations.lines` directly; rule IDs and line numbers are already explicit. Map each `violations.document` confidence value to its tier (`high`, `mid`, `low`).
    
    ## Step 2 — Triage (dismiss false positives first)
    
    For each flag, ask: *does the rule genuinely apply here in context?*
    
    Common false positive patterns to dismiss without a fix:
    
    | Flag | Dismiss when… |
    |---|---|
    | `empty_adverb` | The adverb carries the writer's emphasis, contrast, or spoken rhythm |
    | `banned_word` | The word is used in a quote, technical name, or proper noun |
    | `binary_contrast` | The negation is genuine contrast, not a rhetorical reveal |
    | `dramatic_fragmentation` | The fragment is natural spoken prose, not a mic-drop device |
    | `passive_voice_overuse` | The actor is unknown or the object deserves emphasis |
    | `academic_semicolon` | The clauses are tightly parallel and the semicolon reads naturally |
    | `spelling_and_confused_words` | The English term is established in the target domain |
    | `colon_reveal` | The colon introduces a list or definition, not drama |
    | `synonym_cycling` | The different terms mark a real distinction, not variety for style |
    | `punctuation_and_typography` | The sentence is a list item or headline where a comma is grammatically optional |
    | `bold_lead_in_list` | The list is a genuine technical checklist, spec, API reference, or collection of distinct items where list structure aids scanning |
    
    When you dismiss a flag, state the reason in one sentence. Do not suggest a fix.
    
    ## Step 3 — Fix plan
    
    For every flag you did **not** dismiss, produce:
    
    ```
    Line <N> — <rule_id>
    Original: <exact original text>
    Fix:      <minimal rewrite — change as little as possible>
    Reason:   <one sentence>
    ```
    
    Rules for the fix:
    - **Minimum effective edit.** Change only what the rule requires. Leave the rest of the sentence untouched.
    - **Preserve the writer's voice.** Don't improve style beyond what the rule demands.
    - **No fix chains.** Fix one violation per line. If a line has multiple genuine flags, address each as a separate fix block.
    - **Legal flags are non-negotiable.** Rules from the legal rulebook (`scraped_photo_reference`, `missing_gross_price`, `unverified_ranking`, `disguised_commercial_intent`, etc.) are always genuine — never dismiss them as false positives. The fix may be a deletion, a label addition, or a disclosure insertion rather than a rewrite.
    
    ## Step 4 — Summary
    
    After all fix blocks, output a one-line count:
    
    ```
    X flags reviewed — Y dismissed as false positives, Z fixes applied.
    ```
    
    ## Step 5 — Document Scores (if present)
    
    If the output contains a Document Scores block, interpret it after the line-flag summary.
    
    For each score:
    - **Skip** axes where score ≥ 2.5 and confidence is `high` — the article is strong there, nothing to say.
    - **Flag for attention** any axis with score < 2.0 or confidence `low`.
    - For flagged axes, write one sentence naming the weakness and one concrete suggestion for how the author could move up one level.
    
    Format:
    ```
    Document Scores — 3 axes need attention:
    
    narrative_arc    1.4/3  (confidence mid)
      The sections relate to the topic but don't build — add a bridging
      sentence at the end of each section that sets up the next.
    
    closing_strength 0.3/3  (confidence high)
      The ending summarises rather than lands — replace the final paragraph
      with a single decision rule or provocation the reader takes away.
    ```
    
    Low-confidence scores (↔ display) are genuine uncertainty — name both levels and let the author judge which fits.
    
    ## Example
    
    ### Human-readable input
    
    **Input:**
    ```
    A=banned_word, B=importance_puffery, C=empty_adverb
    
    A,B           | L0001: Our platform empowers teams to unlock their full potential.
    C             | L0005: This is just a routing fix.
    ```
    
    **Output:**
    
    **L0001 — `banned_word`**
    Dismissed: "empowers" is on the banned list. Genuine violation.
    
    **L0001 — `importance_puffery`**
    Dismissed: "unlock their full potential" is vague puffery. Genuine violation.
    
    ```
    Line 1 — banned_word + importance_puffery
    Original: Our platform empowers 
  • README.mdupstream ↗

    Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/lukstei-slop-grader.md

// REPO STATS

38 stars
0 open issues
Last commit: 17d ago