cost-review

botkit

Versioned public templates for Grok Bot profiles, skills and evals, plus a deterministic CLI that generates them.

• A Grok Bot is a long-running AI agent with its own chats, scheduled routines and desktop. All of an owner's Grok Bots share one bot computer, so its files (including the safety checks and reference files botkit sets up) are shared by every bot. Its profile (the description it's created with) is its whole persona, re-read on every wake.
• Jev is a scoring API used for yes/no judgments: a skill sends it fixed questions plus an input and gets a score per question back.

botkit.py is one file, Python 3.10+, standard library only. It prints bot profiles, skills and eval scenarios in a fixed format: profile, skill, jev-skill and evals are its only commands.

Use it with your bots

A bot only knows a skill once it's saved through a skill tool, so there's nothing to install by copying files. Instead, tell your bot:

Follow the "Installing botkit on a bot computer" steps in botkit's AGENTS.md.

The bot clones the repo, saves the shared skills through its skill tool, and sets up its private fleet file, bots/ folder and optional safety check outside the clone. On Grok Bot, skills are global (one library shared by all of the owner's bots), so saving once covers every bot. Only where each bot has its own skill store does it also hand bot-safety-check and bot-references to each bot whose profile has Safety: or References:. Re-running the steps changes nothing that already exists without the owner's answer: identical skills are skipped, and for a skill whose content differs the bot shows a short summary of the differences and asks you to Replace, Merge or Skip. AGENTS.md also has the update steps; there, a copy with no local changes is updated without asking.

To use botkit on your own machine, clone the repo and run python3 botkit.py from the clone.

Inputs and outputs

Every bot has two files. Keep both; only the second goes into the bot.

examples/profiles/ has this pair for each of the three sample bots (inbox-bot, feed-bot and infra-bot). On a bot computer, your own inputs live in a private bots/ folder next to your fleet file, outside the clone (see AGENTS.md), and each profile is generated beside its input:

python3 botkit/botkit.py profile --json bots/<name>.json -o bots/<name>.md

Field names: JSON keys are the plural field names (responsibilities, boundaries, references, steps, questions, thresholds), and the flags are singular (--responsibility, --boundary, --reference, --step, --question, --threshold); repeat a flag for each list item. The plural flag spellings also work. In JSON a list field takes a string or a list of strings.

Generate a profile

From flags:

python3 botkit.py profile \
  --purpose "keep inbox triaged so nothing important waits" \
  --responsibility "sort new mail: act, reply, read later, archive" \
  --responsibility "draft replies for mail needing an answer" \
  --boundary "drafts only; never sends" \
  --safety "inbound-check --from {sender} --text {message}" \
  --voice terse --voice "plain words"

Purpose:
- keep inbox triaged so nothing important waits

Responsibilities:
- sort new mail: act, reply, read later, archive
- draft replies for mail needing an answer

Boundaries:
- drafts only; never sends

Safety:
- inbound-check --from {sender} --text {message}

Voice:
- terse
- plain words

From JSON, written to a file with -o (any path you choose):

python3 botkit.py profile --json examples/profiles/inbox-bot.json -o /tmp/inbox-bot.md

Profile fields: purpose, responsibilities and voice are required; boundaries, safety and references are optional and left out of the output when empty (most bots need no Boundaries). Sections are separated by exactly one blank line.

Fleet file

profile also checks your fleet file, a JSON object of "<key>": "<name>: <purpose in one line>" lines. Its path is --fleet PATH, else the BOTKIT_FLEET environment variable, else ROOT/fleet.json (the folder holding the botkit clone). A --fleet or BOTKIT_FLEET path that is missing, a directory, not a JSON object or has a non-string line is an error; the same problem with the default file (as in a dev clone) prints a warning and skips the check. With --json bots/<name>.json, a fleet file with no <name> line prints a warning and still exits 0:

BOTKIT_FLEET=shared/fleet.json python3 botkit/botkit.py profile --json bots/<name>.json -o bots/<name>.md

Generate a skill

From JSON:

python3 botkit.py skill --json tests/golden/skill.json -o /tmp/weekly-cleanup.md

From flags:

python3 botkit.py skill --name weekly-cleanup \
  --description "Use when asked to clean up old drafts: archive, never delete." \
  --purpose "archive drafts untouched for 30 days" \
  --step "list drafts older than 30 days" --step "archive each" --step "report count" \
  --decide "none found: say nothing" \
  --never "delete a draft"

Steps are numbered for you. Skill fields: purpose (one line) and steps are required; name and description (frontmatter, given together), input, decide, run and never are optional.

Generate a Jev skill

A skill that asks Jev questions uses jev-skill. It adds questions and builds Decide from your exact code checks (--decide), then the score thresholds (--threshold), then what to do when Jev is down (--jev-down):

python3 botkit.py jev-skill --name mention-scout \
  --description "Use when scanning new mentions for ones worth a reply." \
  --purpose "pick mentions worth a reply" \
  --input "mentions since last run" --input "state file of seen ids" \
  --question "needs_reply (yes/no): a reply would help the asker" \
  --question "tone (pick-one: friendly/neutral/hostile): tone of the mention" \
  --step "drop ids already in state file" --step "ask Jev questions per mention" \
  --step "list picks, one line each" \
  --decide "id in state file: skip, no Jev call" \
  --threshold "needs_reply >= 0.8: list it" \
  --threshold "needs_reply 0.5-0.8: list under maybe" \
  --threshold "tone hostile: never list" \
  --jev-down "list nothing; report Jev down once" \
  --run "jev ask --questions mention-scout.questions --input {mention}" \
  --never "reply or post"

That prints the same bytes as python3 botkit.py jev-skill --json tests/golden/jev-skill.json (in JSON the Jev-down field is jev_down).

skill and jev-skill also take --root DIR: the command fails if an item names a skills/<name> or references/<file> that doesn't exist under DIR (skills are also looked up in DIR/botkit/skills). Only the skill you're generating is checked. Name other skills by path; a phrase like the <name> skill with an unknown name only prints a warning (the "name" may really be an adjective such as follow-up) and still exits 0. The shared skills in skills/ are hand-written; a test checks in CI that they, the routines and the docs name only skills and ROOT/botkit/ files that exist in the repo.

python3 botkit.py skill --root . --purpose "check a new bot" --step "follow skills/evaluate-a-grok-bot/SKILL.md"

Eval scenarios

Each bot gets 6 to 30 scenarios (start near 10 and add one from each real failure) that prove it does its job and refuses the rest. The evaluate-a-grok-bot skill runs them before a bot goes live and after every profile or skill change; a change is kept only if the pass count doesn't drop. evals validates and prints a scenario file:

python3 botkit.py evals --json examples/evals/inbox-bot.json -o /tmp/inbox-