⏳ This skill is pending AI review.

Scores will appear once the review pipeline completes.

v4.5.1

agentic-readiness-assessment

exadel-inc@exadel-inc⭐ 25 stars· last commit 12d ago· 0 open issues

>

Choose how to use this skill

You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.

1. Native installer

This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.

Do not guess an installer command or replace an existing version without reviewing the diff.

2. Complete package recommended

Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.

No complete ProSkills package is published for this listing yet.

3. Prompt-only

Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.

Need only the instruction file?

Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.

No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.

—/10

// RATINGS

⭐GitHub Stars
⭐⭐ 25 on GitHubGitHub ↗

Growing

🟢ProSkills Score
—
📍

Not yet listed on ClawHub or SkillsMP

// README

Agentic Readiness Assessment

Exadel AI Agentic Readiness Assessment

Agentic Readiness Assessment is an agent plugin that evaluates a software repository for readiness to be developed and maintained by AI coding agents. It inspects the repository in place and produces an evidence-based readiness scorecard together with actionable recommendations.

Built by Exadel.

Description

The assessment establishes, with evidence, whether an AI coding agent can independently understand this repository, find the right change points, build a reproducible environment, validate its work, and prepare a change for delivery.

It scores 14 areas, weighted, for an applicable maximum of 105 points:

#AreaMax
1Agent guidance and navigation10
2Reproducible environment and dependency setup10
3Build, package, render, or deliverable validation10
4Lint, format check, typecheck, static analysis, or policy validation10
5Unit or component tests10
6Integration, contract, or functional tests5
7Runtime, API, CLI, or local-preview validation5
8Browser or UI validation5
9Coverage and feedback-loop quality5
10CI enforcement and parity with local validation10
11Safety, test isolation, artifacts, and cleanup5
12Delivery workflow from task through review or PR5
13Agent guardrails and permission scoping10
14Context economy5

Areas that do not apply to the repository's archetype are marked N/A and subtracted from the maximum, so the normalized score stays comparable across different kinds of repository.

Three gates sit above the score. Gate 1 anchors on setup, Gate 2 on the deliverable, Gate 3 on whichever area holds the archetype's primary verification surface. A gate passes only when its anchor area is fully verified, so a high average can never hide a failed gate.

Evidence, not configuration. A command that was not executed is a claim. The assessment runs safe commands and records each one with its working directory, result and artifacts. A capability is verified only when a check proved it, never because a config file mentions it.

A probe proves the loop. One small reversible edit confirms the repository can actually take a change from edit to green, then it is reverted. Nothing else is modified.

Guardrails have a hard cap. A permission bypass, a blanket allow-all policy, an unguarded secret file or a committed plaintext credential caps area 13 at 2 regardless of what else is in place, and each one becomes its own P0 or P1 fix record. Paths are reported, values never are.

The run produces a status of Ready, Partially ready or Not ready, paired with a verification confidence of High, Medium or Low, so a weak score and a weakly-evidenced score never read the same.

The plugin ships two Agent Skills, both of which run entirely locally, need no backend or MCP server, and send nothing anywhere:

Supported clients

This is a skill written in the open Agent Skills format. It includes native manifests for Cursor-compatible Agent Plugins, Google Antigravity, Codex, and Claude Code:

The three packages point to the same skills and require no backend or MCP server.

Clients that read the Agent Skills format directly, including Google Antigravity, Zed, Warp, Windsurf and Amp, load each skill from its folder under skills/ with no conversion.

Installation

Cursor

Once published, install it from the Cursor marketplace.

Claude Code

Add this repository as a marketplace and install the plugin:

claude plugin marketplace add exadel-inc/agentic-readiness-assessment
claude plugin install agentic-readiness-assessment@exadel-agent-plugins

To try it without adding a marketplace, clone the repository and load it directly with claude --plugin-dir ..

ChatGPT and Codex

Public installation will be available after OpenAI review and publication. The native Codex package is already present in the repository for submission and local packaging.

Google Antigravity

Install the plugin from GitHub with the Antigravity CLI:

agy plugin install https://github.com/exadel-inc/agentic-readiness-assessment

Or from a local clone:

agy plugin install /path/to/agentic-readiness-assessment

GitHub Copilot CLI

copilot plugin marketplace add exadel-inc/agentic-readiness-assessment

JetBrains Junie

Open /extensions, go to the Marketplaces tab, choose Add marketplace and paste the repository URL. Junie reads the .claude-plugin/marketplace.json in this repository.

Any client that reads Agent Skills

npx skills add exadel-inc/agentic-readiness-assessment

For Zed, Warp, Windsurf, Amp and anything else that discovers skills on the filesystem, copy skills/agentic-readiness-assessment/ into the client's skills directory, usually .agents/skills/ in the repository you want to assess. skills/agent-tool-call-latency/ audits the machine rather than one repository, so it belongs in the user-level skills directory, such as ~/.agents/skills/.

From source

Clone the repository and point a compatible client at the repository root:

git clone https://github.com/exadel-inc/agentic-readiness-assessment.git

Getting Started

Ask your agent for an assessment from inside the repository you want to evaluate:

  • "Run an agentic readiness assessment on this repository."
  • "Generate an AI-readiness scorecard for this codebase."
  • "How ready is this repo for AI coding agents, and what should we fix first?"

There are no arguments or configuration. The skill reads the repository in your current working directory.

It writes exactly one file: reports/agentic-readiness.md, creating reports/ if it does not exist. That single document holds both the decision content and the supporting evidence, in thirteen fixed sections:

  1. Verdict
  2. Run and Scope
  3. Glossary
  4. Readiness Scorecard
  5. Mandatory Gates
  6. What to Fix
  7. What Can Be Delegated Today
  8. What Is Already Good
  9. Surface Resolution
  10. Golden Path
  11. Probe
  12. Commands Executed
  13. Confidence and Limits

Sections 1 to 8 are the decision document, ordered so a reader has the whole readiness picture before reaching the action list. Sections 9 to 13 are the evidence behind it.

Nothing else in your repository is changed. The one exception is the probe, a single reversible edit that is reverted before the report is written.

Example report

examples/fs-demo-project/agentic-readiness.md is a full run against a Next.js 14 application with Prisma and PostgreSQL. It scored 71 of 100, Partially ready, at Medium confidence. The golden path and the test suite verified cleanly; a port conflict on the audit host blocked the database and, with it, the probe. That is the useful case to read, because the report separates what the repository is missing from what this particular machine could not prove.

More runs, and what was redacted before publishing, in examples/.

Agent tool-call latency

Modern coding agen

// HOW IT'S BUILT

// FILES

7 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

  • skills/agentic-readiness-assessment/SKILL.md32 KBff221da43dacupstream ↗
    ---
    name: agentic-readiness-assessment
    description: >
      Evaluates a software repository for readiness to be developed and maintained
      by AI coding agents, and produces an evidence-based readiness scorecard. Use
      when the user requests an agentic-readiness assessment, readiness scorecard,
      AI-readiness audit of a codebase, or recommendations for improving
      AI-assisted development workflows.
    license: Apache-2.0
    ---
    
    # Agentic Readiness Assessment
    
    Read the `version` field of the plugin's `plugin.json` and record it as `prompt_version` in the report run block, so reports produced by different revisions stay comparable. Write `unknown` if the manifest cannot be read.
    
    ## Objective
    
    Assess whether an AI coding agent can independently understand this repository, find the correct change points, establish a reproducible environment, validate work, and prepare a change for delivery. Produce an evidence-backed improvement backlog for maintainers.
    
    This is an audit. Do not implement features or refactor production code. Create only one file: `reports/agentic-readiness.md`, a single document holding both the decision content and the supporting evidence.
    
    The only permitted change to any other tracked file is the single reversible probe edit described in *Probe*, which you revert before writing the report. Preserve all pre-existing user changes.
    
    The deliverable of this task is that report. Any standing instruction to output only code, or to treat an analysis document as a failed task, applies to feature work and not to this audit.
    
    ## Host Execution Contract
    
    Read `references/agent-execution-contract.md` before collecting the baseline. Under the explicit constraints in Run and Scope, record the host's model identity, effective permission profile, command boundary, run stop condition, and context telemetry availability. Write `unavailable` for any field the host cannot evidence, classify it as an `agent-environment` limitation, and never invent a tool restriction, model pin, or telemetry control. After roughly five attempts to obtain unavailable evidence, stop and record the limitation.
    
    ## Evidence and Safety Rules
    
    1. Cite repository-relative paths and line numbers for repository claims.
    2. Cover every applicable workspace or component; do not silently sample a monorepo.
    3. Prefer commands from agent instructions, documentation, task runners, package scripts, and CI. Label each command `documented` or `inferred`.
    4. Run relevant safe commands. Record exact commands, working directories, results, durations when measured, and artifacts.
    5. A command you did not execute is a claim, not evidence. Do not mark a capability verified because configuration exists when a safe executable check was available. A reachable health endpoint does not prove the system behind it performs meaningful work.
    6. Do not deploy, publish, push, run production migrations, use real credentials or customer data, call paid services, or execute destructive or privileged commands.
    7. Never bulk-restore. Do not run `git restore`, `git checkout -- .`, `git clean`, or `git reset` across the tree. Revert the probe by rewriting the exact path you edited.
    8. Stop only services you started, and delete only artifacts you created. Never create, update, or commit a lockfile or equivalent resolution data.
    9. Run linters and formatters in check or diff mode. Never auto-fix.
    10. Use reasonable timeouts. Leave no background processes running.
    11. Write `Not found` when evidence is absent. Never infer a capability, and never reinterpret a failure as a pass.
    
    **Never let your own tooling become the finding.** If a command fails because of a flag you chose, a wrapper you lack, a missing timing utility, or a typo, retry it correctly and log only the meaningful attempt. Offline, cached-only, or sandbox-restricted flags are your constraints, not the repository's — a failure they cause is never evidence against the repository.
    
    Classify commands before execution:
    
    - **local-read:** Reads repository or machine state. Run.
    - **local-build:** Produces local build, cache, test, or ignored artifacts. Run when documented and safe.
    - **external-read:** Reads from a registry or source already required by the repository. Run only when permitted; record the endpoint.
    - **external-write:** Pushes, publishes, deploys, or mutates remote state. Do not run.
    - **privileged:** Requires root, `sudo`, or machine-wide changes. Do not run.
    - **paid:** Calls a billed external service. Do not run.
    
    Run `python3 <skill-directory>/scripts/validate_report.py reports/agentic-readiness.md` from the audited repository after the provisional report write; `<skill-directory>` is the directory containing this `SKILL.md`, not a path inside the audited repository. This is a `local-read` command that validates the report contract without changing the repository.
    
    Being documented does not make a command safe. A refused command is not a skipped category: record it, name its class, and score on the safe evidence that remains.
    
    ### Dependency Installation
    
    Attempt the repository's frozen or locked installation first, in the ecosystem's own idiom. If it fails because resolution data is missing, incomplete, or incorrect, that is a repository finding — record it, then retry **once** with the ecosystem's non-frozen install using flags that cannot create or modify resolution files, for example `npm install --package-lock=false`.
    
    Label that result **unfrozen fallback**. Evidence obtained after a successful fallback is valid for every downstream area: proving that typecheck, tests, and the build actually pass is more useful than reporting them unproven. The setup area itself scores at most half when the fallback was required. If even the fallback fails, or if the ecosystem offers no non-mutating equivalent, record the blocker and move on.
    
    ## Blockers and Fix Ownership
    
    Keep repository readiness separate from limitations of the audit environment:
    
    - **Repository blocker:** Repository-owned instructions, dependencies, commands, fixtures, or controls are missing, incorrect, unsafe, contradictory, or nondeterministic.
    - **Environment blocker:** The repository documents a sufficient requirement, but the audit machine lacks the required tool, service, network access, or permission.
    - **External prerequisite:** A credential or external service is required; assess whether a documented local or test substitute exists.
    - **N/A:** The capability is genuinely irrelevant to the repository archetype.
    
    Route each fix to one owner:
    
    - **repo:** A file or control in the audited repository.
    - **agent-environment:** The image, hook, runtime, or toolchain that hosts the coding agent.
    - **platform:** Organization or infrastructure configuration outside both the repository and agent environment.
    - **external-service:** A third-party service, credential, or policy outside local control.
    
    A missing build tool is never a documentation fix. If `make`, a container runtime, a package manager, or a browser is absent, the owner is `agent-environment`; documenting the prerequisite instead is a deferred repair. Missing prerequisite documentation is itself a separate repository defect.
    
    Do not invent an external file path or configuration key. If the exact target cannot be established, name the required artifact generically and mark the target unverified.
    
    ## Audit Method
    
    ### Delegation
    
    Read `references/assessment-decision-record.md` before baseline collection. Use the audit agent only to interpret repository evidence; use deterministic commands for score arithmetic and `scripts/validate_report.py` for report validation. The requesting human owns scope constraints and must approve an irreversible operation before it runs; otherwise stop and record the command as refused. Keep the delegation map's Cost / Complexity / Risk discipline in the evidence and report the named detection mechanism for any limitation.
    
    ### 1. Baseline and Scope
    
    Capture the baseline before anything else: tracked and untracked worktree state, stash list, commit, and branch. A dirty tree or pre-existing untracked directory is the baseline, not an error. Record the platform, explicit constraints, and relevant installed tools.
    
    Classify the repository as an application, service, frontend, library, CLI, infrastructure, documentation, data pipeline, monorepo, or an applicable combination. List all components and workspaces in scope.
    
    ### 2. Resolve the Archetype's Surfaces
    
    Before scoring, name the concrete command filling each role, or `N/A` with justification.
    
    | Role | Application / service | Library / SDK | Infrastructure | Documentation | Data / model |
    |---|---|---|---|---|---|
    | Deliverable | Build the app or image | Package or compile | Validate or lint the plan | Render the site | Build the pipeline or compile the schema |
    | Primary verification | Unit or component tests | Unit tests, API compatibility | Policy or conformance tests | Link, schema, front-matter checks | Schema validation, golden-record tests |
    
    If a role is `N/A`, name the nearest executable substitute and say what it does not prove. A repository with no executable verification surface at all cannot be Ready.
    
    ### 3. Reconstruct the Golden Path
    
    Determine the intended route from a clean checkout to a delivered change:
    
    `setup -> targeted feedback -> full validation -> review or PR`
    
    Inspect:
    
    - `AGENTS.md`, `CLAUDE.md`, repository-local rules, commands, prompts, and skills;
    - README, architecture, contribution, ownership, and delivery documentation;
    - manifests, dependency-resolution files, runtime versions, environment examples, containers, and fixtures;
    - task runners, build scripts, test configuration, hooks, permission and policy files, and CI.
    
    Trace one representative feature from its entry point through business logic, persistence or an external boundary, UI when applicable, and tests. Assess whether a newcomer can discover where a similar change and its tests belong. Do not edit the feature.
    
    ### 4. Exercise Applicable Validation
    
    For every component, find and safely exercise the applicable surfaces: dependency bootstrap; build, packaging, rendering, or schema validation; static analysis; unit tests; integration or contract tests; runtime, API, CLI, or local-preview validation; browser validation; coverage evidence.
    
    Start services only through a documented local or disposable workflow. If execution is unsafe or blocked, record why and classify the blocker; do not invent a substitute that changes the intended workflow.
    
    ### 5. Check Operability and Enforcement
    
    Compare documentation, executable scripts, observed behavior, and CI. Look for:
    
    - commands whose failures or exit codes are masked;
    - local checks that required CI does not enforce;
    - CI jobs that configure a tool without exercising meaningful behavior;
    - mutable, unpinned, or non-reproducible setup;
    - stale paths, contradictions, or undocumented generated files;
    - unclear ownership, component boundaries, change locations, or test placement;
    - tests requiring undocumented services, fixtures, credentials, or cleanup;
    - slow, noisy, flaky, or ambiguous feedback;
    - missing artifacts or failure diagnostics;
    - unsafe required operations or missing test-safe substitutes;
    - missing branch, commit, review, ownership, or task-to-PR guidance.
    
    Report strengths as well as gaps. Explicitly distinguish an **enforced control** from a documented intention.
    
    ### Cost Discipline
    
    There is no fixed command budget, but an unanswered question is cheaper than an endless search. After roughly five attempts to resolve the same question, stop and record it as `not determined`, then score the affected area on the evidence you have. Never restart a completed phase.
    
    ## Probe
    
    The command inventory proves only that commands exit 0. The probe proves the repository can guide a change to a *verified* fix. It is conditional.
    
    **Run the probe when** the Gate 3 verification command executed in this environment — pass or fail — and the repository has code covered by it.
    
    1. Prefer a genuine pre-existing failing check. Otherwise introduce one small, reversible defect inside the Gate 3 scope: invert a boolean, shift a boundary, drop a required field, change a public return value, violate a policy rule or fixture schema, or break a required document constraint.
    2. Run the verification and confirm it catches the defect. **If nothing fails, that is a P0 or P1 fix record** — the path ships without a guard.
    3. Fix and re-run until green.
    4. Revert by rewriting the exact path you edited, then compare the full tree, tracked and untracked, against the baseline. The report is the only expected difference.
    
    Record **locatability** (did the failure name file and line — quote it), **loop cost** (commands and wall-clock from edit to trustworthy green), **signal quality** (legible alone, or needed a debugger or outside knowledge), and **blockers** (what made a one-line change expensive).
    
    **Do not run the probe when** the Gate 3 surface is repository-blocked or environment-blocked. Record `Probe: not attempted` with the blocking Fix Record IDs. Do not substitute an unrelated surface: proving that a documentation-mirror or metadata check detects drift says nothing about whether application changes are verifiable, and a substitute never satisfies Gate 4. If you run one anyway, label it `non-representative` and state plainly what it does not prove.
    
    ## Score Readiness
    
    Score these areas across the full applicable scope. `Max` is the area's weight.
    
    | # | Area | Max | Gate |
    |---|---|---:|---|
    | 1 | Agent guidance and navigation | 10 | |
    | 2 | Reproducible environment and dependency setup | 10 | **Gate 1** |
    | 3 | Build, package, render, or deliverable validation | 10 | **Gate 2** |
    | 4 | Lint, format check, typecheck, static analysis, or policy validation | 10 | |
    | 5 | Unit or component tests | 10 | |
    | 6 | Integration, contract, or functional tests | 5 | |
    | 7 | Runtime, API, CLI, or local-preview validation | 5 | |
    | 8 | Browser or UI validation | 5 | |
    | 9 | Coverage and feedback-loop quality | 5 | |
    | 10 | CI enforcement and parity with local validation | 10 | |
    | 11 | Safety, test isolation, artifacts, and cleanup | 5 | |
    | 12 | Delivery workflow from task through review or PR | 5 | |
    | 13 | Agent guardrails and permission scoping | 10 | |
    | 14 | Context economy | 5 | |
    
    Applicable maximum is 105 when every area applies.
    
    Readiness status and points:
    
    - **Verified:** full area points — executed successfully across the applicable scope, or proven by a currently required CI job.
    - **Partial:** half the area points, rounded down — useful but incomplete, verified for only part of the scope, obtained through the unfrozen fallback, or documented but not fully proven.
    - **Repository-blocked:** 0 — a repository-owned gap prevents reliable agent work.
    - **N/A:** excluded from the applicable maximum; explain why.
    
    Verification state: **Executed**, **Executed (unfrozen fallback)**, **CI-proven**, **Environment-blocked**, **Not run: unsafe**, or **Not applicable**.
    
    An environment blocker is a confidence limitation, not an automatic zero, and never full credit: award at most half the area points, and only when repository evidence is coherent. Score the *property*, not a named command — there is no universal setup command. Award area 2 for pinned resolution in the ecosystem's own idiom, declared toolchain versions, documented external prerequisites, and an executed clean-checkout path; state whether setup was tested clean or only in a warmed workspace.
    
    ### Area 13 — Agent Guardrails
    
    Score by capability rather than status. These are not interchangeable: instrumentation that reports an unsafe action afterward is not a control that prevents it. Named mechanisms are examples; credit any runtime's equivalent, including mechanisms outside the agent runtime.
    
    | Capability | Pts | Earns credit |
    |---|---:|---|
    | Prevention | 4 | Blocks an unsafe action before it happens: a pre-tool gate denying reads or writes to secret paths, a scoped allow-list, protected paths, approval before an outward-facing write. |
    | Observation | 2 | Records what the agent did: post-edit hooks, audit logs, artifact capture, secret redaction. |
    | Completion | 2 | A tracked run-finished signal, so lifecycle state cannot go stale. |
    | Enforcement | 2 | Controls outside the agent's cooperation: CI that actually runs the verification suite, pre-commit with a deterministic install path, a non-privileged runtime user, resource limits, restricted egress, branch protection. |
    
    **Negative controls cap area 13 at 2 regardless of what else exists**, and each is its own P0 or P1 fix record: a permission bypass in any committed script, task runner, container definition, or CI job; a blanket allow-all policy; a readable secret file with no guard; a plaintext credential in the repository or shipped configuration. Report the path, never the value.
    
    A rule that lives only in prose is described, not enforced; it scores under area 1.
    
    ### Area 14 — Context Economy
    
    Score the cost of reading this repository well enough to change it: oversized files on the paths an agent must actually open, duplicated or contradictory instruction sources, stale instructions that contradict the code, and undocumented generated files an agent may waste effort editing.
    
    Measure, but tie every measurement to observed friction. Counting files over roughly 800 lines is evidence only when one of them sits on the change path you traced, or on the path a Fix Record requires editing. Exclude vendored, generated, and resolution files. A raw size statistic with no friction attached is not a finding.
    
    ### Gates
    
    Each gate is anchored to a scored area. A gate's result **is** its anchor area's score; never report a separate gate number.
    
    | Gate | Anchor |
    |---|---|
    | Gate 1 — setup | Area 2 |
    | Gate 2 — deliverable | Area 3 |
    | Gate 3 — primary verification | The area holding the archetype's primary verification surface, named explicitly: usually area 5, area 6 where functional tests are the real coverage, or area 4 for documentation and infrastructure archetypes |
    | Gate 4 — probe | The probe result, or `not attempted` with its blocking Fix Record IDs |
    
    Gates 1 to 3 pass only when the anchor area is **Verified** at full points. State which area anchors Gate 3 and why.
    
    Gate 4 passes when the probe's verification detected the defect and the loop returned to green. It fails when the probe ran and nothing detected the defect. When Gate 3 was blocked, Gate 4 is `not attempted`: it neither passes nor fails, and it caps confidence rather than forcing a status.
    
    ### Score and Status
    
    `normalized score = round(points earned / applicable maximum * 100)`
    
    Report the raw total, the applicable maximum, every N/A subtraction, and the addends on one line so a reviewer can re-add them without re-reading the repository.
    
    Verification confidence:
    
    - **High:** Full scope audited; at least 80% of applicable executable areas, including every gate, are executed or CI-proven; the probe completed when required.
    - **Medium:** Full scope audited; 50–79% are executed or CI-proven, or one gate is environment-blocked with coherent repository evidence.
    - **Low:** Less than 50% are proven, multiple gates are unproven, a required probe did not complete, or important scope is unaudited.
    
    Overall status:
    
    - **Ready:** Every gate passes, the probe passed when required, score is at least 80, confidence is High or Medium, no P0 repository blocker remains, and no negative control was found.
    - **Partially ready:** Score 50–79; or score at least 80 with a gate that fails only for environment reasons, Low confidence, or another material validation gap.
    - **Not ready:** Score below 50, any gate's anchor area is Repository-blocked, the probe ran and nothing detected the defect, or an agent cannot reliably understand, set up, change, or validate the repository.
    
    Never let a high average hide a failed gate. Treat the score as a comparison and trend signal; use evidence, gates, and fixes for decisions.
    
    ## Create Fix Records
    
    Create one stable record per problem: `F-01`, `F-02`, and so on. Findings and actions are one object, so they cannot drift apart. Include:
    
    - **Problem:** One concrete sentence.
    - **Blocks:** The specific agent capability lost — "blocks producing the frontend deliverable", not "hurts maintainability".
    - **Evidence:** A `path:line`, executed command with its result, or quoted failure. For a non-`repo` owner, cite the command you ran in this environment and what it returned.
    - **Priority:** P0, P1, or P2.
    - **Owner:** `repo`, `agent-environment`, `platform`, or `external-service`. Choose the narrowest that actually fixes it.
    - **Target:** The file, control, or generic external artifact that changes. Mark unverifiable external targets as unverified.
    - **Fix:** An imperative action. If unknown, name the investigation required and mark it `needs-investigation`.
    - **Verify:** A command or objectively observable result that closes the record. Never prose that a reader cannot run or check.
    - **Level:** `code`, `control`, `docs`, or `instructions`.
    
    Priorities:
    
    - **P0:** Unattended work stops and needs a human.
    - **P1:** The agent can proceed incorrectly or without meaningful validation.
    - **P2:** The agent succeeds with avoidable cost, ambiguity, or weak feedback.
    
    Every failed gate and every negative control has a record. Prefer `code`, then an executable `control`, before `docs` or `instructions`. Documentation is not a sufficient fix for a broken command or missing enforcement.
    
    Order records by priority, then by expected effort when reasonably known, so the cheapest high-priority wins come first. Keep environment and platform fixes separate from the repository improvement backlog.
    
    ## Output Quality Rules
    
    - Give every line a consumer and a decision or action it enables. There are two consumers: the maintainer deciding what to repair and delegate, and the agent consuming structured state to plan or re-run.
    - State each problem authoritatively in its Fix Record. Elsewhere cite its ID — but a table an FDE acts on must still carry complete wording. **Never write a bare `See F-01` in a cell whose purpose is to tell someone what to do.**
    - Every scorecard cell cites a command row or a `path:line`.
    - Do not add consistently empty columns or write `not measured` repeatedly. Include duration only when measured.
    - Do not narrate compliance with this prompt. Avoid decorative metadata and hedged filler.
    - Do not create a finding without a fix, or a fix without a finding.
    - State each fact where the report assigns it and do not restate it elsewhere. The score, status, and confidence belong in the run block; the verdict paragraph interprets them in prose without repeating the commit or prompt version.
    - No table exceeds seven columns; one line per cell; escape literal pipes as `\|`.
    - Write the provisional report in one file-editing operation. Never assemble it with shell heredocs. The finalization exception in Final Validation permits the bounded rewrites needed to validate and truthfully record that command.
    
    ## Report
    
    Write `reports/agentic-readiness.md` with an `# Agentic Readiness — <repository>` title, then these thirteen `##` sections in this order. Sections 1 to 8 are the decision document, ordered so a reader gets the whole readiness picture before the action list: verdict, then scope, then the scorecard, then what it means and what to do about it. Sections 9 to 13 are the supporting evidence.
    
    ### 1. Verdict
    
    One short paragraph: the score, overall status, confidence, a one-sentence readiness conclusion, and the most expensive repository blocker named by ID. No commit, no prompt version, no table.
    
    ### 2. Run and Scope
    
    A single fenced yaml block with exactly these top-level keys, in this order: `audited_by`, `prompt_version`, `started`, `completed`, `commit`, `branch`, `platform`, `archetype`, `scope`, `baseline_worktree`, `final_worktree`, `clean_state_setup` (verified, unfrozen fallback, warmed-workspace-only, or blocked with reason), `normalized_score`, `raw_total`, `applicable_maximum`, `status`, `confidence`, and a `gates` map. Follow `references/report-contract.md` for the machine-checked values and gate shape.
    
    Then list components, workspaces, and explicit constraints, including the five host-execution fields from `references/agent-execution-contract.md` or `unavailable` with their `agent-environment` limitation.
    
    ### 3. Glossary
    
    Reproduce the table below verbatim. It is fixed boilerplate: do not add terms, reword definitions, or expand it with repository-specific detail. Immediately below the table, add exactly one sentence in this format: `Gate 3 anchor: Area <4, 5, or 6> — <why this is the primary verification surface>.` Add no other content to this section.
    
    | Term | Meaning |
    |---|---|
    | Area | One of 14 scored readiness dimensions. Each has a weight shown as `Max`. |
    | Gate | One of 4 mandatory checks. A gate's result **is** its anchor area's score, not a separate number. Gates 1–3 pass only at full points. |
    | Probe | Gate 4. A deliberate small defect introduced and reverted, to test whether the repository's own verification catches a real break. |
    | `F-nn` | A Fix Record: one stable ID per problem, holding the finding and its fix together. |
    | Priority | P0 stops unattended work and needs a human. P1 lets the agent proceed incorrectly or unvalidated. P2 costs avoidable time or clarity. |
    | Owner | Who fixes it: `repo`, `agent-environment` (the agent's image or toolchain), `platform` (org or infra config), or `external-service`. |
    | Readiness status | Per area: Verified (full points), Partial (half, rounded down), Repository-blocked (0), or N/A (excluded from the maximum). |
    | Verification state | How the evidence was obtained: Executed, Executed (unfrozen fallback), CI-proven, Environment-blocked, Not run: unsafe, or Not applicable. |
    | Level | What a fix changes: `code`, `control`, `docs`, or `instructions`. |
    | Normalized score | `points earned / applicable maximum * 100`. The applicable maximum is 105 minus the weight of every N/A area, so it is rarely 100. |
    | Repository blocker | The repository itself is missing, wrong, unsafe, or contradictory. Counts against readiness. |
    | Environment blocker | The repository documents enough, but this audit machine lacked a tool, service, or permission. Caps confidence; never a full zero. |
    | Enforced control | Something that actually fails when violated. A rule that lives only in prose is documented guidance, not an enforced control. |
    | Negative control | A safety defect such as a permission bypass, allow-all policy, unguarded secret file, or committed plaintext credential. Caps area 13 at 2. |
    | Unfrozen fallback | The locked install failed, so a non-mutating unlocked install was used instead. Valid evidence, but caps the setup area at half. |
    
    ### 4. Readiness Scorecard
    
    Use:
    
    `Area | Applicability | Status | Score | Verification | Evidence / Result | Fix IDs`
    
    Include all fourteen areas, then the area 13 capability breakdown and any negative controls found. Follow `references/report-contract.md` for the machine-checked score and negative-control notation. Close with the raw total, applicable maximum, every N/A subtraction, the addends on one line, and the normalized score. Gate results belong in the next section; confidence belongs in sections 1 and 13.
    
    The `Fix IDs` column lists IDs only. Do not restate problems here.
    
    ### 5. Mandatory Gates
    
    Show each gate, its anchor area, its score, verification state, and blocking Fix Record IDs. Include the probe result or the reason it was not attempted.
    
    ### 6. What to Fix
    
    Open with a compact index table, repository-owned fixes first, then environment or platform fixes:
    
    `ID | Priority / Owner | Problem and blocked capability | Fix and target | Verify`
    
    Every cell must be actionable on its own. Follow `references/report-contract.md` for the compact index. Each `F-nn` ID appears exactly once in the index and has exactly one matching record. Then give the authoritative full record for every `F-nn` ID, in ID order, with all nine fields. Start each record with `### F-nn`, then put each field on its own line as `**Problem:**`, `**Blocks:**`, `**Evidence:**`, `**Priority:**`, `**Owner:**`, `**Target:**`, `**Fix:**`, `**Verify:**`, and `**Level:**`. The index exists for triage and the records exist for execution; keep both, and keep their wording consistent.
    
    ### 7. What Can Be Delegated Today
    
    Rate the work an AI coding agent can independently undertake today — research, implementation, testing, deliverable validation, runtime or API validation, browser validation, and autonomous task-to-PR delivery — as Yes, Partially, No, or N/A, each with the gate or Fix Record IDs behind it. Rate autonomous delivery **Yes** only when every gate passes, the probe succeeded, and area 13 scored prevention points above zero.
    
    ### 8. What Is Already Good
    
    Give three to five evidence-backed strengths. Mark each as an enforced control or documented guidance, so nobody removes something load-bearing. If there are none, write `No strengths identified.` and say why.
    
    ### 9. Surface Resolution
    
    Resolve each role and gate to an exact command with `Role | Resolved command | Source | Status | Justification`, including substitutes attempted and what they do not prove. The scorecard and gates cite these commands, so every command named in a scorecard cell must appear here.
    
    ### 10. Golden Path
    
    Describe the clean-checkout workflow from setup through review or PR, and include the representative feature trace.
    
    ### 11. Probe
    
    What was broken and where, how it was detected, locatability, loop cost, signal quality, blockers, and the baseline comparison result. If the probe was not attempted, state the blocking IDs and stop. If a non-representative substitute was used, say what it does not prove.
    
    ### 12. Commands Executed
    
    List only meaningful commands actually executed, plus commands refused as unsafe with their class:
    
    `Command | Directory | Source / Safety Class | Purpose | Result | Notes / Artifacts`
    
    Add a duration column only when durations were measured consistently. Exclude attempts that failed because of your own tooling.
    
    ### 13. Confidence and Limits
    
    Explain the confidence rating, important environment blockers, and any unaudited scope in no more than three short paragraphs. Then list every command or scope that could not be verified, its safety or blocker classification, and which conclusion it leaves uncertain. Nothing else.
    
    ## Final Validation
    
    Re-read the report and verify:
    
    - all thirteen sections exist in order, and every table is well formed and within seven columns;
    - the glossary matches the prompt's table verbatim and names the Gate 3 anchor;
    - each gate result equals its anchor area's score;
    - the scorecard addends sum to the stated raw total, and the normalized score follows from it;
    - every Fix Record has all nine fields, and every `Verify` is runnable or objectively observable;
    - every ID in the index table has a full record, and their shared fields agree;
    - no cell that requires action contains only a cross-reference;
    - the probe section reports either a completed probe or an explicit reason it was not attempted;
    - the report contains only commands actually executed or explicitly refused as unsafe;
    - the final worktree differs from the baseline only by the report and known audit-created ignored artifacts, with the probe edit reverted.
    - `python3 <skill-directory>/scripts/validate_report.py reports/agentic-readiness.md` exits 0;
    - any unavailable host-execution control is recorded in Run and Scope constraints and Confidence and Limits as an `agent-environment` limitation.
    
    Finalization exception: write the provisional report, run the validator, and make one corrective rewrite only if it fails. Run the validator once more after that correction. Then make one finalization rewrite that changes only Commands Executed and Confidence and Limits to record the last validator command and result. The finalization rewrite is permitted even when the provisional report passed; no other rewrite is allowed. Record unresolved validator errors in Confidence and Limits; say that you rewrote it, then stop.
    
    Then respond with the report path, score, overall status, confidence, failed gates, the probe result, and the first repository-owned fix.
    
  • skills/agentic-readiness-assessment/scripts/validate_report.py16 KB18d591aff675upstream ↗
    from __future__ import annotations
    
    import argparse
    import re
    import sys
    from pathlib import Path
    
    SECTIONS = (
        "Verdict",
        "Run and Scope",
        "Glossary",
        "Readiness Scorecard",
        "Mandatory Gates",
        "What to Fix",
        "What Can Be Delegated Today",
        "What Is Already Good",
        "Surface Resolution",
        "Golden Path",
        "Probe",
        "Commands Executed",
        "Confidence and Limits",
    )
    RUN_KEYS = (
        "audited_by",
        "prompt_version",
        "started",
        "completed",
        "commit",
        "branch",
        "platform",
        "archetype",
        "scope",
        "baseline_worktree",
        "final_worktree",
        "clean_state_setup",
        "normalized_score",
        "raw_total",
        "applicable_maximum",
        "status",
        "confidence",
        "gates",
    )
    AREAS = (
        ("Agent guidance and navigation", 10),
        ("Reproducible environment and dependency setup", 10),
        ("Build, package, render, or deliverable validation", 10),
        ("Lint, format check, typecheck, static analysis, or policy validation", 10),
        ("Unit or component tests", 10),
        ("Integration, contract, or functional tests", 5),
        ("Runtime, API, CLI, or local-preview validation", 5),
        ("Browser or UI validation", 5),
        ("Coverage and feedback-loop quality", 5),
        ("CI enforcement and parity with local validation", 10),
        ("Safety, test isolation, artifacts, and cleanup", 5),
        ("Delivery workflow from task through review or PR", 5),
        ("Agent guardrails and permission scoping", 10),
        ("Context economy", 5),
    )
    FIX_RECORD_FIELDS = ("Problem", "Blocks", "Evidence", "Priority", "Owner", "Target", "Fix", "Verify", "Level")
    AREA_STATUSES = ("Verified", "Partial", "Repository-blocked", "N/A")
    INDEX_HEADER = "| ID | Priority / Owner | Problem and blocked capability | Fix and target | Verify |"
    INDEX_SEPARATOR = "| --- | --- | --- | --- | --- |"
    GLOSSARY_TABLE = """| Term | Meaning |
    |---|---|
    | Area | One of 14 scored readiness dimensions. Each has a weight shown as `Max`. |
    | Gate | One of 4 mandatory checks. A gate's result **is** its anchor area's score, not a separate number. Gates 1–3 pass only at full points. |
    | Probe | Gate 4. A deliberate small defect introduced and reverted, to test whether the repository's own verification catches a real break. |
    | `F-nn` | A Fix Record: one stable ID per problem, holding the finding and its fix together. |
    | Priority | P0 stops unattended work and needs a human. P1 lets the agent proceed incorrectly or unvalidated. P2 costs avoidable time or clarity. |
    | Owner | Who fixes it: `repo`, `agent-environment` (the agent's image or toolchain), `platform` (org or infra config), or `external-service`. |
    | Readiness status | Per area: Verified (full points), Partial (half, rounded down), Repository-blocked (0), or N/A (excluded from the maximum). |
    | Verification state | How the evidence was obtained: Executed, Executed (unfrozen fallback), CI-proven, Environment-blocked, Not run: unsafe, or Not applicable. |
    | Level | What a fix changes: `code`, `control`, `docs`, or `instructions`. |
    | Normalized score | `points earned / applicable maximum * 100`. The applicable maximum is 105 minus the weight of every N/A area, so it is rarely 100. |
    | Repository blocker | The repository itself is missing, wrong, unsafe, or contradictory. Counts against readiness. |
    | Environment blocker | The repository documents enough, but this audit machine lacked a tool, service, or permission. Caps confidence; never a full zero. |
    | Enforced control | Something that actually fails when violated. A rule that lives only in prose is documented guidance, not an enforced control. |
    | Negative control | A safety defect such as a permission bypass, allow-all policy, unguarded secret file, or committed plaintext credential. Caps area 13 at 2. |
    | Unfrozen fallback | The locked install failed, so a non-mutating unlocked install was used instead. Valid evidence, but caps the setup area at half. |"""
    
    
    def section(content: str, heading: str) -> str | None:
        match = re.search(rf"^## {re.escape(heading)}\n(.*?)(?=^## |\Z)", content, re.MULTILINE | re.DOTALL)
        return match.group(1) if match else None
    
    
    def yaml_block(content: str) -> str | None:
        run_scope = section(content, "Run and Scope")
        if run_scope is None:
            return None
        match = re.search(r"^```yaml\n(.*?)^```$", run_scope, re.MULTILINE | re.DOTALL)
        return match.group(1) if match else None
    
    
    def integer(yaml: str, key: str, errors: list[str]) -> int | None:
        match = re.search(rf"^{re.escape(key)}:\s*(\d+)\s*$", yaml, re.MULTILINE)
        if not match:
            errors.append(f"invalid Run and Scope integer: {key}")
            return None
        return int(match.group(1))
    
    
    def allowed_value(yaml: str, key: str, values: tuple[str, ...], errors: list[str]) -> None:
        match = re.search(rf"^{re.escape(key)}:\s*(.+?)\s*$", yaml, re.MULTILINE)
        if match and match.group(1) not in values:
            errors.append(f"invalid Run and Scope {key}: {match.group(1)}")
    
    
    def validate_scorecard(content: str, errors: list[str]) -> tuple[dict[int, int], dict[int, str], dict[int, int], int, int, bool]:
        scorecard = section(content, "Readiness Scorecard")
        scores: dict[int, int] = {}
        statuses: dict[int, str] = {}
        maximums: dict[int, int] = {}
        total = 0
        applicable_maximum = 0
        if scorecard is None:
            return scores, statuses, maximums, total, applicable_maximum, False
    
        for number, (name, maximum) in enumerate(AREAS, start=1):
            match = re.search(
                rf"^\| {re.escape(name)} \| [^|]* \| ([^|]*) \| ([^|]*) \|",
                scorecard,
                re.MULTILINE,
            )
            if not match:
                errors.append(f"missing scorecard row: {name}")
                continue
            status = match.group(1).strip()
            score = match.group(2).strip()
            statuses[number] = status
            maximums[number] = maximum
            if status not in AREA_STATUSES:
                errors.append(f"invalid status for {name}: {status}")
            expected = {
                "Verified": maximum,
                "Partial": maximum // 2,
                "Repository-blocked": 0,
            }.get(status)
            if status == "N/A":
                if score != "N/A":
                    errors.append(f"invalid score for {name}: {score}")
                continue
            applicable_maximum += maximum
            if expected is None or score != f"{expected}/{maximum}":
                errors.append(f"invalid score for {name}: {score}")
                continue
            scores[number] = expected
            total += expected
        negative_controls = re.search(r"^Negative controls:\s*(.+?)\s*$", scorecard, re.MULTILINE)
        if negative_controls is None:
            errors.append("missing negative controls declaration")
            has_negative_control = False
        elif negative_controls.group(1) == "none":
            has_negative_control = False
        elif re.fullmatch(r"F-\d\d(?:, F-\d\d)*", negative_controls.group(1)):
            has_negative_control = True
        else:
            errors.append("invalid negative controls declaration")
            has_negative_control = False
        return scores, statuses, maximums, total, applicable_maximum, has_negative_control
    
    
    def validate_gates(yaml: str, scores: dict[int, int], errors: list[str]) -> tuple[dict[str, int], str | None]:
        expected = {
            "setup": (2,),
            "deliverable": (3,),
            "verification": (4, 5, 6),
        }
        gate_anchors: dict[str, int] = {}
        for gate, anchors in expected.items():
            match = re.search(
                rf"^  {gate}:\n    anchor:\s*(\d+)\n    score:\s*(\d+)\s*$",
                yaml,
                re.MULTILINE,
            )
            if not match:
                errors.append(f"missing gate entry: {gate}")
                continue
            anchor, score = (int(value) for value in match.groups())
            if anchor not in anchors:
                errors.append(f"invalid gate anchor: {gate}")
                continue
            gate_anchors[gate] = anchor
            if scores.get(anchor) != score:
                errors.append(f"gate {gate} score does not match anchor area {anchor}")
    
        probe = re.search(r"^  probe:\n    result:\s*(pass|fail|not attempted)\s*$", yaml, re.MULTILINE)
        if not probe:
            errors.append("missing gate entry: probe")
        return gate_anchors, probe.group(1) if probe else None
    
    
    def validate_overall_status(yaml: str, normalized_score: int | None, scores: dict[int, int], statuses: dict[int, str], maximums: dict[int, int], gate_anchors: dict[str, int], probe: str | None, has_p0: bool, has_negative_control: bool, errors: list[str]) -> None:
        status = re.search(r"^status:\s*(.+?)\s*$", yaml, re.MULTILINE)
        confidence = re.search(r"^confidence:\s*(.+?)\s*$", yaml, re.MULTILINE)
        if not status:
            return
        gates_pass = all(
            statuses.get(anchor) == "Verified" and scores.get(anchor) == maximums.get(anchor)
            for anchor in gate_anchors.values()
        ) and len(gate_anchors) == 3
        gate_blocked = any(statuses.get(anchor) == "Repository-blocked" for anchor in gate_anchors.values())
        ready = normalized_score is not None and normalized_score >= 80 and confidence is not None and confidence.group(1) in ("High", "Medium") and probe == "pass" and gates_pass and not has_p0 and not has_negative_control
        not_ready = normalized_score is not None and (normalized_score < 50 or gate_blocked or probe == "fail" or has_p0)
    
        if status.group(1) == "Ready" and not ready:
            errors.append("Ready status is not supported by parsed report fields")
        elif status.group(1) == "Partially ready" and (ready or not_ready):
            errors.append("Partially ready status is not supported by parsed report fields")
        elif status.group(1) == "Not ready" and not not_ready:
            errors.append("Not ready status is not supported by parsed report fields")
    
    
    def split_table_row(row: str) -> list[str]:
        return re.split(r"(?<!\\)\|", row)
    
    
    def validate_fix_records(content: str, errors: list[str]) -> bool:
        fixes = section(content, "What to Fix")
        if fixes is None:
            return False
    
        normalized_fixes = fixes.strip()
        if not normalized_fixes.startswith(INDEX_HEADER):
            errors.append("missing What to Fix compact index")
            index_rows: list[str] = []
        else:
            after_header = normalized_fixes[len(INDEX_HEADER):]
            if not after_header.startswith(f"\n{INDEX_SEPARATOR}"):
                errors.append("malformed What to Fix compact index separator")
                index_rows = []
            else:
                index_rows = []
                for line in after_header[len(INDEX_SEPARATOR) + 2:].splitlines():
                    if not line:
                        break
                    if not line.startswith("|"):
                        break
                    index_rows.append(line)
    
        index_ids: list[str] = []
        for row in index_rows:
            cells = split_table_row(row)
            if len(cells) != 7 or not row.endswith("|"):
                errors.append("malformed What to Fix compact index row")
                continue
            value = cells[1].strip()
            if not re.fullmatch(r"F-\d\d", value):
                errors.append(f"malformed Fix Record index ID: {value}")
            else:
                index_ids.append(value)
        for identifier in set(index_ids):
            if index_ids.count(identifier) != 1:
                errors.append(f"duplicate Fix Record index ID: {identifier}")
    
        records: list[tuple[str, str]] = []
        declarations = list(re.finditer(r"^### (F-.*)$", fixes, re.MULTILINE))
        for position, declaration in enumerate(declarations):
            identifier = declaration.group(1)
            if not re.fullmatch(r"F-\d\d", identifier):
                errors.append(f"malformed Fix Record declaration: {identifier}")
                continue
            end = declarations[position + 1].start() if position + 1 < len(declarations) else len(fixes)
            records.append((identifier, fixes[declaration.end():end]))
    
        record_ids = [identifier for identifier, _ in records]
        for identifier in set(index_ids):
            if record_ids.count(identifier) != 1:
                errors.append(f"index Fix Record {identifier} does not have exactly one matching record")
        for identifier in set(record_ids):
            if identifier not in index_ids:
                errors.append(f"Fix Record {identifier} is not declared in the index")
    
        for identifier, record in records:
            for field in FIX_RECORD_FIELDS:
                if not re.search(rf"^\*\*{re.escape(field)}:\*\*\s+\S", record, re.MULTILINE):
                    errors.append(f"{identifier} is missing Fix Record field: {field}")
        return any(re.search(r"^\| F-\d\d \| P0 /", row) for row in index_rows)
    
    
    def validate(path: Path) -> list[str]:
        try:
            content = path.read_text(encoding="utf-8")
        except OSError:
            return [f"cannot read report: {path}"]
    
        errors: list[str] = []
        if not re.search(r"^# Agentic Readiness — .+", content, re.MULTILINE):
            errors.append("missing report title")
    
        headings = re.findall(r"^## (.+)$", content, re.MULTILINE)
        missing = [heading for heading in SECTIONS if heading not in headings]
        errors.extend(f"missing section: {heading}" for heading in missing)
        if not missing and tuple(headings) != SECTIONS:
            errors.append("sections are not in the required order")
    
        glossary = section(content, "Glossary")
        normalized_glossary = glossary.strip() if glossary is not None else ""
        anchor_sentence = r"Gate 3 anchor: Area ([456]) — [^\n]+\."
        glossary_anchor: int | None = None
        if glossary is None or not re.search(r"^\| Term \| Meaning \|$", normalized_glossary, re.MULTILINE):
            errors.append("missing glossary table header")
        elif not normalized_glossary.startswith(GLOSSARY_TABLE):
            errors.append("glossary does not match the fixed table")
        elif not normalized_glossary[len(GLOSSARY_TABLE):].strip():
            errors.append("missing Gate 3 anchor sentence")
        else:
            anchor = re.fullmatch(rf"{re.escape(GLOSSARY_TABLE)}\n\n{anchor_sentence}", normalized_glossary)
            if not anchor:
                errors.append("glossary does not match the fixed contract")
            else:
                glossary_anchor = int(anchor.group(1))
    
        yaml = yaml_block(content)
        if yaml is None:
            errors.append("missing Run and Scope YAML block")
        else:
            top_level_keys = tuple(re.findall(r"^([a-z_]+):", yaml, re.MULTILINE))
            for key in RUN_KEYS:
                if not re.search(rf"^{re.escape(key)}:\s", yaml, re.MULTILINE):
                    errors.append(f"missing Run and Scope key: {key}")
            if top_level_keys != RUN_KEYS:
                errors.append("Run and Scope keys are not in the required order")
            allowed_value(yaml, "status", ("Ready", "Partially ready", "Not ready"), errors)
            allowed_value(yaml, "confidence", ("High", "Medium", "Low"), errors)
            scores, area_statuses, area_maximums, scorecard_total, scorecard_maximum, has_negative_control = validate_scorecard(content, errors)
            raw_total = integer(yaml, "raw_total", errors)
            applicable_maximum = integer(yaml, "applicable_maximum", errors)
            normalized_score = integer(yaml, "normalized_score", errors)
            if raw_total is not None and raw_total != scorecard_total:
                errors.append("raw_total does not equal scorecard total")
            if applicable_maximum is not None and applicable_maximum != scorecard_maximum:
                errors.append("applicable_maximum does not equal scorecard maximum")
            if raw_total is not None and applicable_maximum not in (None, 0) and normalized_score is not None:
                if normalized_score != round(raw_total / applicable_maximum * 100):
                    errors.append("normalized_score does not match raw_total and applicable_maximum")
            gate_anchors, probe = validate_gates(yaml, scores, errors)
            if glossary_anchor is not None and gate_anchors.get("verification") != glossary_anchor:
                errors.append("Glossary Gate 3 anchor does not match gates.verification.anchor")
            has_p0 = validate_fix_records(content, errors)
            validate_overall_status(yaml, normalized_score, scores, area_statuses, area_maximums, gate_anchors, probe, has_p0, has_negative_control, errors)
            return errors
    
        validate_fix_records(content, errors)
        return errors
    
    
    def main(argv: list[str] | None = None) -> int:
        parser = argparse.ArgumentParser(description="Validate an agentic-readiness report contract.")
        parser.add_argument("report", type=Path)
        args = parser.parse_args(argv)
    
        errors = validate(args.report)
        if errors:
            print("\n".join(errors), file=sys.stderr)
            return 1
        return 0
    
    
    if __name__ == "__main__":
        raise SystemExit(main())
    
  • skills/agentic-readiness-assessment/references/agent-execution-contract.md1.4 KB28d725f51b0bupstream ↗
    # Agent execution contract
    
    `agentic-readiness-assessment` is a **coding-agent workflow**: agentic repository investigation determines the relevant change and validation surfaces; deterministic tooling classifies commands, executes safe checks, and validates the finished report; a human controls scope and any irreversible action.
    
    Before an assessment starts, the host runtime must provide or explicitly withhold the following evidence:
    
    | Evidence | Record | If unavailable |
    |---|---|---|
    | Model identity | Provider, model identifier, and version or alias | `unavailable`; do not claim a pinned model. |
    | Effective permission profile | The host's tool and path restrictions | `unavailable`; do not claim a tool allowlist. |
    | Command boundary | Which command classes the host can run or refuse | `unavailable`; classify the resulting limitation. |
    | Run stop condition | The host's turn, time, or cancellation limit | `unavailable`; retain the assessment's five-attempt discipline. |
    | Context telemetry | Whether token usage, truncation, or context-limit signals are exposed | `unavailable`; do not infer context safety. |
    
    Record the five fields or `unavailable` under the explicit constraints in **Run and Scope**. Missing host evidence is an `agent-environment` limitation on the assessment's confidence, not a repository finding. A portable prompt can request and record this evidence; it cannot enforce host permissions, pin a model, or add context telemetry.
    
  • skills/agentic-readiness-assessment/references/assessment-decision-record.md3.4 KBcae0883d9f11upstream ↗
    # Assessment Delegation and Decision Record
    
    Use this record before running an assessment. It names who owns each step so the audit agent does not silently become the owner of arithmetic, command safety, or an irreversible decision.
    
    ## Delegation map
    
    | Step | Owner | Reversibility / stakes | Accountability | Detection mechanism |
    |---|---|---|---|---|
    | Interpret repository structure and evidence | Audit agent | Reversible analysis; an unsupported conclusion can misdirect a repair backlog. | The audit agent must cite evidence and label uncertainty. | Every repository claim carries a path and line or command result; any uncited claim is rejected in report review. Owner: report reviewer. |
    | Execute approved commands, calculate totals, and validate report shape | Deterministic tooling | Commands may create local artifacts; a wrong score or malformed report makes a decision unreliable. | The audit agent selects only safe commands and records their result. | The command inventory contains every executed command; the validator must exit `0`; score addends must differ from the stated raw total by `0`. Owner: audit agent. |
    | Set scope constraints and approve irreversible actions | Human | Irreversible or external actions can change customer, production, or shared state. | The requesting human owns scope and authorization. | Stop before any irreversible, external-write, privileged, paid, or otherwise unclassified action; no such command may appear as executed without approval. Owner: requesting human. |
    
    ## Decision records
    
    ### Command execution
    
    **Decision:** Classify each command and execute only `local-read` or safe `local-build` work without additional approval.
    
    **Cost:** Command classification and recording add a small amount of audit time.
    
    **Complexity:** The agent must keep a command inventory and clean up only artifacts it created.
    
    **Risk:** A command can mutate state or contact an external system despite looking routine.
    
    **Detection mechanism:** The Commands Executed inventory is reviewed for every command with a non-local safety class; threshold: `0` unapproved external-write, privileged, paid, or unclassified commands. Owner: report reviewer.
    
    ### Score calculation
    
    **Decision:** Calculate raw totals, applicable maximum, and normalized score with deterministic tooling, not model arithmetic.
    
    **Cost:** One local calculation command per report.
    
    **Complexity:** The agent must preserve the scorecard addends and record the calculation inputs.
    
    **Risk:** A transcription or arithmetic error can produce a plausible but incorrect readiness status.
    
    **Detection mechanism:** Re-add the scorecard values with a local calculation; threshold: stated raw total and normalized score differ by `0` from the calculated values. Owner: audit agent.
    
    ### Report validation
    
    **Decision:** Run `python3 <skill-directory>/scripts/validate_report.py reports/agentic-readiness.md` after the provisional report write, where `<skill-directory>` contains this skill's `SKILL.md`.
    
    **Cost:** A local Python process and, at most, one corrective rewrite.
    
    **Complexity:** The report must retain the fixed headings, Run and Scope keys, and glossary shape the validator checks, then use the bounded finalization rewrite to record the validator result.
    
    **Risk:** A malformed report can look complete while breaking automated consumption or audit comparison.
    
    **Detection mechanism:** Validator process exit status; threshold: exit code must be `0` after at most one corrective rewrite. Owner: audit agent.
    
  • skills/agentic-readiness-assessment/references/report-contract.md1.0 KB62e5b053fbecupstream ↗
    # Report Contract
    
    Use this reference only while assembling the report.
    
    ## Run and Scope
    
    The YAML block has the top-level keys listed in `SKILL.md`, in that order. `raw_total` and `applicable_maximum` are integers; `normalized_score` is `round(raw_total / applicable_maximum * 100)`.
    
    ```yaml
    gates:
      setup:
        anchor: 2
        score: <earned area score>
      deliverable:
        anchor: 3
        score: <earned area score>
      verification:
        anchor: <4, 5, or 6>
        score: <earned area score>
      probe:
        result: <pass|fail|not attempted>
    ```
    
    Each gate score equals its anchor area’s earned score.
    
    ## Scorecard and findings
    
    Each area score is `earned/max`; `N/A` uses `N/A`. Add exactly one line after the scorecard: `Negative controls: none` or `Negative controls: F-nn, F-nn`.
    
    Start What to Fix with:
    
    ```markdown
    | ID | Priority / Owner | Problem and blocked capability | Fix and target | Verify |
    | --- | --- | --- | --- | --- |
    ```
    
    Each indexed `F-nn` has one matching `### F-nn` record with the nine fields specified in `SKILL.md`.
    
  • skills/agentic-readiness-assessment/tests/test_validate_report.py13 KB177a6b70de77upstream ↗
    from __future__ import annotations
    
    import sys
    import tempfile
    import unittest
    from pathlib import Path
    
    sys.path.insert(0, str(Path(__file__).parents[1] / "scripts"))
    import validate_report
    
    
    SECTIONS = (
        "Verdict",
        "Run and Scope",
        "Glossary",
        "Readiness Scorecard",
        "Mandatory Gates",
        "What to Fix",
        "What Can Be Delegated Today",
        "What Is Already Good",
        "Surface Resolution",
        "Golden Path",
        "Probe",
        "Commands Executed",
        "Confidence and Limits",
    )
    AREAS = (
        ("Agent guidance and navigation", 10),
        ("Reproducible environment and dependency setup", 10),
        ("Build, package, render, or deliverable validation", 10),
        ("Lint, format check, typecheck, static analysis, or policy validation", 10),
        ("Unit or component tests", 10),
        ("Integration, contract, or functional tests", 5),
        ("Runtime, API, CLI, or local-preview validation", 5),
        ("Browser or UI validation", 5),
        ("Coverage and feedback-loop quality", 5),
        ("CI enforcement and parity with local validation", 10),
        ("Safety, test isolation, artifacts, and cleanup", 5),
        ("Delivery workflow from task through review or PR", 5),
        ("Agent guardrails and permission scoping", 10),
        ("Context economy", 5),
    )
    FIX_FIELDS = ("Problem", "Blocks", "Evidence", "Priority", "Owner", "Target", "Fix", "Verify", "Level")
    GLOSSARY = """| Term | Meaning |
    |---|---|
    | Area | One of 14 scored readiness dimensions. Each has a weight shown as `Max`. |
    | Gate | One of 4 mandatory checks. A gate's result **is** its anchor area's score, not a separate number. Gates 1–3 pass only at full points. |
    | Probe | Gate 4. A deliberate small defect introduced and reverted, to test whether the repository's own verification catches a real break. |
    | `F-nn` | A Fix Record: one stable ID per problem, holding the finding and its fix together. |
    | Priority | P0 stops unattended work and needs a human. P1 lets the agent proceed incorrectly or unvalidated. P2 costs avoidable time or clarity. |
    | Owner | Who fixes it: `repo`, `agent-environment` (the agent's image or toolchain), `platform` (org or infra config), or `external-service`. |
    | Readiness status | Per area: Verified (full points), Partial (half, rounded down), Repository-blocked (0), or N/A (excluded from the maximum). |
    | Verification state | How the evidence was obtained: Executed, Executed (unfrozen fallback), CI-proven, Environment-blocked, Not run: unsafe, or Not applicable. |
    | Level | What a fix changes: `code`, `control`, `docs`, or `instructions`. |
    | Normalized score | `points earned / applicable maximum * 100`. The applicable maximum is 105 minus the weight of every N/A area, so it is rarely 100. |
    | Repository blocker | The repository itself is missing, wrong, unsafe, or contradictory. Counts against readiness. |
    | Environment blocker | The repository documents enough, but this audit machine lacked a tool, service, or permission. Caps confidence; never a full zero. |
    | Enforced control | Something that actually fails when violated. A rule that lives only in prose is documented guidance, not an enforced control. |
    | Negative control | A safety defect such as a permission bypass, allow-all policy, unguarded secret file, or committed plaintext credential. Caps area 13 at 2. |
    | Unfrozen fallback | The locked install failed, so a non-mutating unlocked install was used instead. Valid evidence, but caps the setup area at half. |
    
    Gate 3 anchor: Area 5 — unit tests are the primary verification surface for this repository."""
    
    
    def run_scope(*, raw_total: int = 105, normalized_score: int = 100, setup_score: int = 10, status: str = "Ready", confidence: str = "High", probe: str = "pass") -> str:
        return f"""```yaml
    audited_by: agent
    prompt_version: 4.4.0
    started: now
    completed: now
    commit: abc123
    branch: main
    platform: test
    archetype: documentation
    scope: repository
    baseline_worktree: clean
    final_worktree: clean
    clean_state_setup: verified
    normalized_score: {normalized_score}
    raw_total: {raw_total}
    applicable_maximum: 105
    status: {status}
    confidence: {confidence}
    gates:
      setup:
        anchor: 2
        score: {setup_score}
      deliverable:
        anchor: 3
        score: 10
      verification:
        anchor: 5
        score: 10
      probe:
        result: {probe}
    ```"""
    
    
    def scorecard(*, setup_status: str = "Verified") -> str:
        rows = ["| Area | Applicability | Status | Score | Verification | Evidence / Result | Fix IDs |"]
        rows.append("| --- | --- | --- | --- | --- | --- | --- |")
        for name, maximum in AREAS:
            status = setup_status if name == "Reproducible environment and dependency setup" else "Verified"
            score = maximum if status == "Verified" else maximum // 2
            rows.append(f"| {name} | applicable | {status} | {score}/{maximum} | Executed | evidence | |")
        rows.extend(("", "Negative controls: none"))
        return "\n".join(rows)
    
    
    def fix_records(*, complete: bool = True) -> str:
        fields = FIX_FIELDS if complete else FIX_FIELDS[:-1]
        body = ["| ID | Priority / Owner | Problem and blocked capability | Fix and target | Verify |", "| --- | --- | --- | --- | --- |", "| F-01 | P2 / repo | problem | fix | command |", "", "### F-01"]
        body.extend(f"**{field}:** value" for field in fields)
        return "\n".join(body)
    
    
    def report(*, sections: tuple[str, ...] = SECTIONS, run_scope_body: str | None = None, glossary: str = GLOSSARY, scorecard_body: str | None = None, fixes: str | None = None) -> str:
        bodies = {
            "Run and Scope": run_scope() if run_scope_body is None else run_scope_body,
            "Glossary": glossary,
            "Readiness Scorecard": scorecard() if scorecard_body is None else scorecard_body,
            "What to Fix": fix_records() if fixes is None else fixes,
        }
        parts = ["# Agentic Readiness — Example"]
        for section in sections:
            parts.append(f"## {section}\n\n{bodies.get(section, 'Content')}")
        return "\n\n".join(parts)
    
    
    class ValidateReportTests(unittest.TestCase):
        def validate(self, content: str) -> list[str]:
            with tempfile.TemporaryDirectory() as directory:
                path = Path(directory) / "agentic-readiness.md"
                path.write_text(content, encoding="utf-8")
                return validate_report.validate(path)
    
        def test_accepts_a_report_with_the_required_contract(self) -> None:
            self.assertEqual(self.validate(report()), [])
    
        def test_accepts_a_closing_line_after_the_last_section(self) -> None:
            closing = "---\n\nThis report covers one repository. See [Exadel](https://exadel.com/)."
            self.assertEqual(self.validate(report() + "\n\n" + closing), [])
    
        def test_rejects_a_missing_required_section(self) -> None:
            sections = tuple(section for section in SECTIONS if section != "Probe")
            self.assertIn("missing section: Probe", self.validate(report(sections=sections)))
    
        def test_rejects_sections_out_of_order(self) -> None:
            sections = list(SECTIONS)
            sections[0], sections[1] = sections[1], sections[0]
            self.assertIn("sections are not in the required order", self.validate(report(sections=tuple(sections))))
    
        def test_rejects_an_unreadable_report(self) -> None:
            self.assertEqual(validate_report.validate(Path("does-not-exist.md")), ["cannot read report: does-not-exist.md"])
    
        def test_rejects_glossary_text_that_is_not_a_header_row(self) -> None:
            errors = self.validate(report(glossary="The required header is | Term | Meaning |."))
            self.assertIn("missing glossary table header", errors)
    
        def test_rejects_an_incomplete_fixed_glossary(self) -> None:
            errors = self.validate(report(glossary=GLOSSARY.replace("| Probe |", "| Test probe |")))
            self.assertIn("glossary does not match the fixed table", errors)
    
        def test_rejects_a_missing_gate_three_anchor_sentence(self) -> None:
            errors = self.validate(report(glossary=GLOSSARY.rsplit("\n\n", 1)[0]))
            self.assertIn("missing Gate 3 anchor sentence", errors)
    
        def test_rejects_glossary_content_after_the_gate_three_anchor_sentence(self) -> None:
            errors = self.validate(report(glossary=f"{GLOSSARY}\n\nExtra prose."))
            self.assertIn("glossary does not match the fixed contract", errors)
    
        def test_rejects_a_yaml_block_outside_run_and_scope(self) -> None:
            malformed = report(run_scope_body="Content").replace("## Confidence and Limits\n\nContent", f"## Confidence and Limits\n\n{run_scope()}")
            self.assertIn("missing Run and Scope YAML block", self.validate(malformed))
    
        def test_rejects_run_and_scope_keys_out_of_order(self) -> None:
            malformed = run_scope().replace("audited_by: agent\nprompt_version: 4.4.0", "prompt_version: 4.4.0\naudited_by: agent")
            self.assertIn("Run and Scope keys are not in the required order", self.validate(report(run_scope_body=malformed)))
    
        def test_rejects_a_missing_gate_map_entry(self) -> None:
            malformed = run_scope().replace("  probe:\n    result: pass\n", "")
            self.assertIn("missing gate entry: probe", self.validate(report(run_scope_body=malformed)))
    
        def test_rejects_score_arithmetic_that_does_not_match_scorecard(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope(raw_total=104, normalized_score=100)))
            self.assertIn("raw_total does not equal scorecard total", errors)
            self.assertIn("normalized_score does not match raw_total and applicable_maximum", errors)
    
        def test_rejects_an_invalid_area_status_and_score(self) -> None:
            errors = self.validate(report(scorecard_body=scorecard(setup_status="Ready")))
            self.assertIn("invalid status for Reproducible environment and dependency setup: Ready", errors)
            self.assertIn("invalid score for Reproducible environment and dependency setup: 5/10", errors)
    
        def test_rejects_a_gate_score_that_does_not_match_its_anchor(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope(setup_score=5)))
            self.assertIn("gate setup score does not match anchor area 2", errors)
    
        def test_rejects_a_glossary_gate_three_anchor_that_differs_from_the_gate_map(self) -> None:
            errors = self.validate(report(glossary=GLOSSARY.replace("Area 5 —", "Area 4 —")))
            self.assertIn("Glossary Gate 3 anchor does not match gates.verification.anchor", errors)
    
        def test_accepts_an_escaped_pipe_in_a_compact_index_row(self) -> None:
            fixes = fix_records().replace("| F-01 | P2 / repo | problem | fix | command |", "| F-01 | P2 / repo | command \\| pipeline | fix | command |")
            self.assertEqual(self.validate(report(fixes=fixes)), [])
    
        def test_rejects_ready_with_a_p0_fix_record(self) -> None:
            fixes = fix_records().replace("P2 / repo", "P0 / repo")
            errors = self.validate(report(fixes=fixes))
            self.assertIn("Ready status is not supported by parsed report fields", errors)
    
        def test_rejects_ready_with_a_negative_control(self) -> None:
            scorecard_with_negative_control = scorecard().replace("Negative controls: none", "Negative controls: F-01")
            errors = self.validate(report(scorecard_body=scorecard_with_negative_control))
            self.assertIn("Ready status is not supported by parsed report fields", errors)
    
        def test_rejects_not_ready_without_a_mechanical_blocker(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope(status="Not ready")))
            self.assertIn("Not ready status is not supported by parsed report fields", errors)
    
        def test_rejects_an_invalid_overall_status(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope().replace("status: Ready", "status: Unknown")))
            self.assertIn("invalid Run and Scope status: Unknown", errors)
    
        def test_rejects_ready_when_confidence_is_low(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope(confidence="Low")))
            self.assertIn("Ready status is not supported by parsed report fields", errors)
    
        def test_rejects_partially_ready_when_the_probe_failed(self) -> None:
            errors = self.validate(report(run_scope_body=run_scope(status="Partially ready", probe="fail")))
            self.assertIn("Partially ready status is not supported by parsed report fields", errors)
    
        def test_rejects_a_missing_compact_index_with_no_records(self) -> None:
            self.assertIn("missing What to Fix compact index", self.validate(report(fixes="No findings.")))
    
        def test_rejects_a_malformed_compact_index_separator(self) -> None:
            malformed = fix_records().replace("| --- | --- | --- | --- | --- |", "| --- | --- |")
            self.assertIn("malformed What to Fix compact index separator", self.validate(report(fixes=malformed)))
    
        def test_rejects_an_incomplete_fix_record(self) -> None:
            self.assertIn("F-01 is missing Fix Record field: Level", self.validate(report(fixes=fix_records(complete=False))))
    
        def test_rejects_an_index_id_without_one_matching_record(self) -> None:
            malformed = fix_records().replace("### F-01", "### F-02")
            errors = self.validate(report(fixes=malformed))
            self.assertIn("index Fix Record F-01 does not have exactly one matching record", errors)
            self.assertIn("Fix Record F-02 is not declared in the index", errors)
    
        def test_rejects_a_malformed_fix_record_declaration(self) -> None:
            malformed = fix_records().replace("### F-01", "### F-1")
            self.assertIn("malformed Fix Record declaration: F-1", self.validate(report(fixes=malformed)))
    
    
    if __name__ == "__main__":
        unittest.main()
    
  • README.mdupstream ↗

    Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/exadel-inc-agentic-readiness-assessment-skills-agentic-readiness-assessment.md

// REPO STATS

25 stars
0 open issues
Last commit: 12d ago