snifftest

Sniff Test

Does the draft pass the sniff test?

Sniff Test reads Markdown and plain text and checks it against your house rules. It prints one line per flag with the file, the line, the rule that tripped and how sure it is.

There are two kinds of rule. Countable rules are regular expressions. Long dashes, a paragraph with three colons, a banned word, every sentence the same length. They run on your machine, cost nothing and send nothing anywhere. Judgment rules need something read rather than counted. A closing sentence that only restates the paragraph, a claim hedged three times, a line that talks the reader out of the offer. For those it sends one paragraph at a time to a small hosted judgment model and gets back one probability per rule. It only does that after you have said yes.

Two things it is not. There is no general model in the loop; the judgment model returns one probability per question and nothing else. And it will not rewrite your text. A flag is a sentence for you to fix.

npm install --global snifftest
snifftest check --dry-run draft.md

That runs the countable rules and nothing leaves your machine. To run the judgment rules too, put a TypeSafe key in TYPESAFE_API_KEY and drop --dry-run. The first time, it prints exactly what it is about to send and where, and waits for your answer.

It exits 0 when nothing tripped, 1 when at least one rule did, 2 when the tool could not do its job, and 3 when the judgment rules needed a yes and did not get one.

The nose is the mascot. It is one hand-drawn SVG with six expressions, in assets/nose/, and snifftest serve opens a local page where you type and the nose reacts.

Reading a flag

draft.md:14 dash_present 1.00 A long dash. Give the sentence a full stop instead.
draft.md:31 restating_closer 0.84 The last sentence says the paragraph again. End one sentence earlier.

The number is how sure the checker is, between 0 and 1, and it says nothing about how bad the problem is. A countable rule always scores 1.00. A judgment rule counts as a flag at or above the threshold, which is 0.7 unless the ruleset or --threshold says otherwise. A reading between 0.4 and 0.6 is reported as no judgment and never becomes a flag or a pass. That band exists because the judgment model answers about 0.5 on text it cannot read, and a flat middle number is the one failure that looks like a clean draft.

--format json prints the same flags for a machine, with every reading including the ones under the threshold. snifftest rules prints the ruleset in force and which file each rule came from, which is the quickest answer to "why did that flag".

The rules

The default ruleset ships in rules/default.yaml. Fifteen rules, and the file itself is the documentation. Ten apply to most prose and five are conventions of copy meant to sell.

The five rules tagged marketing sit out an ordinary run. Ask for them on a landing page with --only marketing, which runs those five and nothing else. To run everything, write a project file that extends the default and sets off_by_default: [].

Fenced code is dropped before either kind of rule sees it. Headings, tables, front matter, link definitions and HTML comments are never sent to the judgment rules, and the countable rules only read the ones a rule names.

Writing your own

A .snifftest.yaml in your project root is picked up on every run. It can extend the default ruleset and change the rules it disagrees with, matched by id, so a house that wants five colons instead of three writes one rule and inherits the other fourteen.

version: 1
extends: default
threshold: 0.7

rules:
  # A countable rule of your own is a pattern and a message.
  - id: no_utilize
    kind: regex
    pattern: "\\butili[sz]e"
    flags: i
    message: "Use. The word is use."

  # Override a shipped rule by reusing its id.
  - id: colon_heavy
    kind: regex
    builtin: colon_count
    min: 5
    message: "Five colons in one paragraph."

  # A judgment rule is a description, its near misses, and two criteria.
  - id: passive_apology
    kind: judgment
    what: |
      A sentence that apologises for the document itself: for its length,
      its lateness, or the writer's lack of expertise.
    not_for: |
      An apology to a person for a thing that happened.
      A stated limit of scope.
    examples:
      - "Sorry this is so long."
      - "I am no expert, but here goes."
    criteria:
      true: "At least one sentence apologises for the document or the writer."
      false: "No sentence apologises for the document or the writer."
    message: "The draft is apologising for itself. Cut the line."

A countable rule names either a builtin or a pattern, never both. The built-ins are dash_present, colon_count, sentence_rhythm, slop_vocab and banned_words, and each takes the tuning keys the default file shows. A rule may carry tags, and a countable rule may name the kinds of block it applies to with chunks. A judgment rule's not_for is what keeps it honest. A rule with a what and no not_for will flag things you did not mean.

A pattern with a nested quantifier is refused when the ruleset loads, and every pattern runs against a capped slice of text. Neither is a proof that a pattern cannot run for a long time. See SECURITY.md.

To propose a rule for the shipped set, open a rule proposal issue. CONTRIBUTING.md says what one needs.

Consent, and what leaves the machine

Nothing is sent before the answer to one question is yes. The countable rules run first, on your machine, whatever happens next. Then, if the ruleset has judgment rules and you did not pass --dry-run, the checker looks for a key in TYPESAFE_API_KEY. Without one it prints the countable verdict, says the judgment rules could not run, and exits 2. With one, it prints this and waits:

Sniff Test is about to use the judgment rules, which run on a model.

  What leaves this machine: one paragraph of your text at a time, from 3 files,
    together with the wording of these rules: not_x_but_y, tricolon, stacked_hedging, rhetorical_opener, restating_closer.
  Where it goes: https://api.typesafe.ai/v1/systemone (TypeSafe), over HTTPS, with your TYPESAFE_API_KEY.
  What comes back: one probability per rule per paragraph.
  What is never sent: file names, file paths, anything outside the text you pointed at.
  There is no t