triage

Decision Kit

Decision Kit is a public GitHub Copilot Canvas demonstration for making fast, inspectable decisions from real context. TypeSafe AI's Jev model chooses a typed branch and exposes its probabilities. A separate chat model explains that bounded result and proposes next steps without changing it.

flowchart LR
    A[Scenario + edited context] --> B[Loopback Canvas server]
    B --> C[OpenRouter Decisions API]
    C -->|choice + score + noul| D[Confidence gate]
    D -->|above threshold| E[Recommended action]
    D -->|below threshold or review signal| F[Human review]
    D --> G[OpenRouter chat completion]
    G --> H[Bounded rationale + next steps]
    E --> I[Context-selected Canvas UI]
    F --> I
    H --> I

What the workbench demonstrates

The Canvas offers incident response, feature prioritization, and customer escalation presets. You can select a scenario, edit its context, and watch two explicit stages progress through Server-Sent Events.

Jev is not a chatbot. One request asks three independent questions using TypeSafe's choice, score, and noul primitives. The result includes the selected branch, option probabilities, confidence, urgency score, and latency. Decision Kit applies a 0.72 confidence threshold and an explicit human-review signal. Below the threshold, it withholds automated certainty and routes the case to a human.

The LLM receives the source context and Jev's bounded result only after that gate is computed. It writes a concise rationale and next-step plan, but it cannot invisibly replace the branch, probabilities, score, or review requirement. The interface labels Jev output as authoritative and the LLM narrative as explanatory.

The UI selects from a small vocabulary instead of rendering every possible panel: urgency signal, option distribution, confidence gate, recommended action, human review, and audit trace.

Setup

Requirements:

• GitHub Copilot CLI with project extension Canvas support
• Node.js 20 or newer for local tests
• An OpenRouter account for live requests

Environment variables:

Do not put credentials in the repository. The extension requests OPENROUTER_API_KEY through the Copilot SDK's requestedEnvironmentVariables flow. If the key is absent or access is denied, the extension retries without the secret and remains usable in clearly labeled deterministic demo mode. A failed live request remains an explicit error; it does not fall back to a fixture.

Open and use the Canvas

The project extension is discovered from .github/extensions/decision-kit/extension.mjs.

1. Open this repository in a Copilot project session.
2. Reload extensions after changing extension files.
3. Ask Copilot to open the decision-kit Canvas, or use the Canvas catalog.
4. Pick a preset, edit the context, and select Run decision.

Host integrations can invoke run_preset and reset_state. Both actions validate inputs with JSON Schema. The renderer communicates only with its per-instance HTTP server on 127.0.0.1 using an ephemeral port.

Passive delivery nudge and evaluations

The first delivery slice is a possible polish-loop nudge, independent of the Canvas and OpenRouter. It requires no self-report, feedback form, model call, or API key. It does not change Jev decisions or grant execution permissions.

The repository hook in .github/hooks/decision-kit.json uses Copilot's documented postToolUse event and additionalContext output. On a host that supports that output, the reminder goes to the agent after a successful tool call, not to a user approval dialog. Start a new host session after installing the hook. Node.js 20+ and Git must be available to the hook process.

What triggers it

Three narrow spacing or corner-radius replacements in the same file, within the last ten successful tool observations and fifteen minutes, produce at most one reminder per session. The agent is asked to check the existing agreed outcome, preserve necessary usability and accessibility work, and verify the current slice before the next authorized commit or PR. The hook never declares work complete, changes the contract, commits, opens a PR, or deploys.

The classifier supports edit and str_replace_editor replacement arguments (path or file_path, old_str/new_str or oldString/newString, including JSON-encoded arguments). It recognizes only whole snippets of spacing or radius declarations in CSS, SCSS, HTML, JavaScript, and JSX/TSX files. Contrast, focus, mixed behavior changes, failed edits, and unsupported patches do not qualify. Larger replacements and apply_patch are deliberately not inferred to be cosmetic. Spacing can still be important: the nudge is advisory, not a finding of wasted work. These thresholds are initial heuristics, not calibrated facts.

Set DECISION_KIT_DELIVERY_MODE in the host environment:

You can also disable this hook file with its top-level disableAllHooks setting. Missing or incompatible host events mean reduced coverage, not proof that no drift occurred. The implementation is tested with documented payloads; live host delivery is not verified by the unit tests. There is no automatic cross-host installation or device synchronization.

Local observations and privacy

State lives in decision-kit/ inside the current worktree's Git metadata directory, outside tracked files. It contains hashed session IDs and relative file identifiers, event timestamps, bounded recent event categories, counts, and hook processing time. Hashes are identifiers, not anonymization. Prompts, source text, tool results, credentials, and raw file paths are not persisted or sent over the network.

State is isolated by worktree and session. Atomic writes and a per-session lock prevent duplicate nudges from overlapping calls; a contended lock skips the observation rather than waiting. Unsupported, stale, or same-timestamp duplicate events are conservatively skipped. A killed process can leave a lock; after stopping the host, remove the matching .lock directory to resume observation. Corrupt state is not silently reset and re-nudged.

There is no automatic retention cleanup in this slice. After stopping the host, deleting decision-kit/ from the directory reported by git rev-parse --absolute-git-dir clears these observations and reminder suppression. Do not delete the Git metadata directory itself. Cloud-agent sandbox data is ephemeral and is not uploaded.

Run the targeted evaluations

npm run eval:delivery
npm run delivery:report

The eval suite replays synthetic positive and negative cases and tests the outcome evaluator's guardrails. It is a regression baseline, not evidence of faster or better real delivery. The report summarizes local observations and nudge outputs. It leaves cost, verified delivery time, quality, and human attention as null: these hooks do not provide those facts. Nudge output is not proof of host receipt, agent compliance, or human attention saved. Reported hook time excludes process startup and final persistence.

delivery-evaluation.mjs exports evaluateDeliveryOutcomes({ policy, slices }) for future passive outcome adapters. No user data-entry step or live experiment is installed. Its input contract is: