⏳ This skill is pending AI review.

Scores will appear once the review pipeline completes.

version unknown

lintlang

hermes-labs-ai@hermes-labs-ai⭐ 147 stars· last commit 9d ago· 5 open issues

Use when writing or reviewing AI agent configs, system prompts, or tool definitions (JSON/YAML/Python) and you need to catch ambiguous tool descriptions, missing stop conditions, schema/description mismatches, or embedded prompts before they reach runtime. Deterministic static analysis, no LLM or network calls.

Choose how to use this skill

You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.

1. Native installer

This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.

Do not guess an installer command or replace an existing version without reviewing the diff.

2. Complete package recommended

Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.

No complete ProSkills package is published for this listing yet.

3. Prompt-only

Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.

Need only the instruction file?

Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.

No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.

—/10

// RATINGS

⭐GitHub Stars
⭐⭐⭐ 147 on GitHubGitHub ↗

Popular

🟢ProSkills Score
—
📍

Not yet listed on ClawHub or SkillsMP

// README

LintLang

Catch agent failures before your agent runs.

CI PyPI Downloads Python License OpenSSF Scorecard

Product page · Playground · PyPI · Docs · 简体中文

LintLang is a local, deterministic static linter for the instructions and tool interfaces an AI agent is given. It flags ambiguous tool choices, mixed output formats, schema gaps, missing bounds, and other setup defects before the agent runs.

Point it at a project directory. LintLang finds supported agent-facing content inside the files you already use: MCP and function-tool definitions nested in JSON/YAML, parameter schemas, system prompts, messages, output contracts, AGENTS.md, CLAUDE.md, GEMINI.md, SKILL.md, and supported Python prompt code.

uvx lintlang scan .

Below, by contrast, is unedited terminal output on a bundled fixture: two of the ten findings from lintlang scan samples/bad_tool_descriptions.yaml, long lines wrapped.

  ❌ FAIL — 1 CRITICAL, 1 HIGH, 5 MEDIUM, 3 LOW
  Inspected: 5 tools (4 described, 5 with a schema), system prompt

    !! [CRITICAL] H1.1 samples/bad_tool_descriptions.yaml:34  tool:process_ticket
      Tool 'process_ticket' has no description.
      → Add a specific, disambiguating description that explains WHEN to use this tool, not just WHAT it does.

    ~ [MEDIUM] H1.6 samples/bad_tool_descriptions.yaml:10  tool:get_user_info vs tool:fetch_user_data
      Tools 'get_user_info' and 'fetch_user_data' carry no differentia — every meaning-bearing term in one is
      present, or has a synonym, in the other. Both descriptions may be accurate and still give a model nothing
      to choose between them.
      Evidence: "'Get user info' vs 'Get user data from the system'"
      → Name a condition that selects one over the other. State what each tool is for that the other is NOT
      for — e.g. 'use X for orders already placed, use Y for carts not yet submitted'.

How LintLang differs

  • From LLM-as-judge reviewers: no model calls and no network access during a scan. pyyaml is the only runtime dependency, and the test suite runs the CLI with outbound sockets disabled (tests/test_first_run_recipe.py). Findings and verdicts are deterministic: the same tree produces byte-identical JSON and SARIF (tests/test_sarif.py); only the terminal summary's elapsed time varies. That is what lets it gate CI.
  • From generic linters: agent-aware. It reads tool descriptions, system prompts, SKILL.md, MCP definitions, and parameter schemas as instructions a model will act on, not as prose.
  • Every finding names the file and proposes a specific fix, and cites the line wherever the parser can justify one. LintLang never invents line numbers.

What LintLang catches

Agent configuration can be valid YAML, JSON, Markdown, or Python and still give a model bad instructions, bad choices, or an interface it cannot reliably use.

LintLang catches problems like:

  • Ambiguous tools — sibling tools that overlap without a clear reason for the model to choose one over another.
  • Missing bounds — retries, loops, or tool use without explicit stopping or progress conditions.
  • Schema mismatches — missing required fields, unclear parameters, and schemas that do not communicate enough intent.
  • Mixed output formats and missing priorities — a prompt that names more than one output format (H6 flags any two recognized formats, even when each is scoped to a case), and long instruction lists with no stated priority order (H5). LintLang does not detect semantic contradictions between two instructions, for example "always do X" next to "never do X".
  • SKILL.md defects — missing or invalid metadata, unclear usage criteria, and skill names that do not match their directory.
  • Context and message errors — stale project references, unbounded persistence, malformed roles, and broken tool-message sequences.
  • Embedded agent logic — supported Python prompts, literal tool definitions, and selected pipeline thresholds.

Each result says what LintLang inspected. Content with no recognized agent-facing structures is reported as SKIPPED, never PASS. Tool comparisons are within one parsed input; a directory scan does not combine tools from separate files into one selection namespace.

For exact extraction rules and detector behavior, see the technical reference.

Quickstart

Requires Python 3.10+.

Run once without installing:

uvx lintlang scan .

Or name a specific configuration source:

uvx lintlang scan AGENTS.md
uvx lintlang scan SKILL.md
uvx lintlang scan agent.yaml

Install with pip:

pip install lintlang
lintlang scan .

Or with Homebrew on macOS:

brew install hermes-labs-ai/tap/lintlang
lintlang scan .

Findings are advisory by default.

Block on HIGH or CRITICAL findings:

lintlang scan . --fail-on fail

Include MEDIUM findings in the gate:

lintlang scan . --fail-on review

LintLang also emits JSON, SARIF, and GitLab Code Quality reports for automation. See the GitLab CI guide for a copyable Code Quality job.

Put it in CI

Generate a pinned GitHub Actions workflow that scans the repository directory:

lintlang init --github --path .

The generated Action gates HIGH or CRITICAL findings by default. Use a narrower path when CI should check only one configuration source.

For an existing repository with known findings, record a reviewed baseline:

lintlang scan . --write-baseline .lintlang-baseline.json

Then gate new or changed findings:

lintlang scan . \
  --baseline .lintlang-baseline.json \
  --fail-on review

See GitHub CI and Code Scanning and baseline adoption.

Measured

What we can claim today, and what we can't:

  • More than 1,100 passing tests, run in CI on Python 3.10–3.13 on every pull request and push to main. Every tagged release from v0.3.1 through v0.8.2 points at a commit with a passing CI run; the release workflow checks tag/version parity and builds, it does not re-run the suite. Reproduce with pip install -e ".[dev]" && pytest -q.
  • Regression corpus (evals/corpus/cases.jsonl, 2 cases and 23 variants today): immutable case IDs with positive/negative controls per phrase class, each linked to a focused test. It guards detector boundaries against drift — it does not estimate accuracy or false-positive rates.
  • Sample detection check (evals/sample-detection-rate.sh): 4 deliberately-broken fixtures must fail, 1 clean fixture must pass. A release gate, not a benchmark.
  • Daily proof loop (`.github/workflo

// HOW IT'S BUILT

// FILES

2 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

  • .agents/skills/lintlang/SKILL.md2.8 KBupstream ↗
    ---
    name: lintlang
    description: Use when writing or reviewing AI agent configs, system prompts, or tool definitions (JSON/YAML/Python) and you need to catch ambiguous tool descriptions, missing stop conditions, schema/description mismatches, or embedded prompts before they reach runtime. Deterministic static analysis, no LLM or network calls.
    license: Apache-2.0
    compatibility: Requires Python 3.10+; installs via pip or runs standalone via `uvx lintlang`. Scans need no network access.
    ---
    
    # LintLang
    
    LintLang statically analyzes the natural-language instructions that control AI
    agents — system prompts, tool descriptions, and configs — catching ambiguous
    tools, missing limits, and mixed output formats before they reach an agent
    at runtime. It is zero-LLM: deterministic pattern and structural checks only,
    no model calls, no telemetry, no network access.
    
    ## Use it for
    
    - Linting tool descriptions before agents start choosing between them
      (detects pairs like `get_user_info` / `fetch_user_data` with no
      distinguishing term — check `H1.6`)
    - Checking prompts and configs for missing stop conditions, unbounded
      retries, and schema/description mismatches
    - Running a zero-LLM CI gate over YAML, JSON, prompt text, and Python source
    - Scanning `.py` files for embedded prompts and uncalibrated thresholds
      (detectors `P1`/`P2`)
    - Preflighting one present instruction plus explicit typed context before a
      host sends it to a model
    
    ## Do not use it for
    
    - Runtime evaluation of a live agent
    - Dynamic agent testing or behavioral benchmarking
    - Proving an agent is safe in production
    - Retrieving preferences from history, deciding truth, or rewriting/sending
      prompts on the agent's behalf
    
    ## Quickstart
    
    ```bash
    python -m pip install lintlang
    lintlang scan AGENTS.md
    ```
    
    Or without installing, via [uv](https://docs.astral.sh/uv/):
    
    ```bash
    uvx lintlang scan AGENTS.md
    ```
    
    Scan a fixture with a known finding:
    
    ```bash
    uvx lintlang scan samples/bad_tool_descriptions.yaml
    ```
    
    ## Output shape
    
    - Repository scan outcomes: `ERROR`, `PASS`, `REVIEW`, or `FAIL`
    - Structural findings by pattern `H1` through `H7`, plus Python pipeline
      findings `P1` and `P2`
    - JSON output for CI via `--format json`
    - Preflight states: `ALLOW`, `NOTICE`, `HOLD`, `UNAVAILABLE`, or `ERROR`
    - Preflight evidence uses exact code-point spans and stable `PF001`-`PF005`
      IDs
    
    ## Common gotchas
    
    - LintLang judges structure, not runtime model behavior — a config can pass
      every LintLang check and still fail at inference time.
    - Configs can be syntactically valid YAML/JSON while still under-specified
      for their intended use; LintLang flags this as `REVIEW`, not `FAIL`.
    - Preflight heuristic findings are notice-only; only exact contract/conflict
      rules may hold (`HOLD`).
    
    ## More
    
    Full docs, CLI reference, and CI integration:
    https://github.com/hermes-labs-ai/lintlang
    
  • README.mdupstream ↗

    Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/hermes-labs-ai-lintlang-agents-skills-lintlang.md

// REPO STATS

147 stars
5 open issues
Last commit: 9d ago