# claude-history-ingest

Adopt this skill: https://proskills.md/skills/other/ar9av-obsidian-wiki-skills-claude-history-ingest

Page: https://proskills.md/skills/other/ar9av-obsidian-wiki-skills-claude-history-ingest
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: .skills/claude-history-ingest/SKILL.md
size: 8198
upstream: https://github.com/ar9av/obsidian-wiki/blob/main/.skills/claude-history-ingest/SKILL.md

````text
---
name: claude-history-ingest
description: >
  Ingest Claude Code conversation history into the Obsidian wiki. Use this skill when the user wants to mine
  their past Claude conversations for knowledge, import their ~/.claude folder, extract insights from
  previous coding sessions, or says things like "process my Claude history", "add my conversations to the wiki",
  "what have I discussed with Claude before". Also triggers when the user mentions their .claude folder,
  Claude projects, session data, past conversation logs, local-agent-mode sessions, or audit logs.
---

# Claude History Ingest — Conversation Mining

You are extracting knowledge from the user's past Claude Code conversations and distilling it into the Obsidian wiki. Conversations are rich but messy — your job is to find the signal and compile it.

This skill can be invoked directly or via the `wiki-history-ingest` router (`/wiki-history-ingest claude`).

## Before You Start

**Writing profile:** Before drafting or rewriting natural-language Markdown, read and apply the `Writing Profile Resolution` section in `llm-wiki/SKILL.md`. Framework schema, provenance, safety, and operation-specific requirements take precedence.
`WRITING.md` preferences apply only to newly drafted or rewritten natural-language Markdown; preserve source content and structured records.

1. **Resolve config** — follow the Config Resolution Protocol in `llm-wiki/SKILL.md` (inline `@name` override → walk up CWD for `.env` → global config → prompt setup). This gives `OBSIDIAN_VAULT_PATH` and `CLAUDE_HISTORY_PATH` (defaults to `~/.claude`)
2. Read `.manifest.json` at the vault root to check what's already been ingested
3. Read `index.md` at the vault root to know what the wiki already contains
4. **Project Scoping** — read `WIKI_SKIP_PROJECTS` from config (comma-separated substrings). Exclude any project directory whose name contains one of them from **every** step below (scan, delta, sampling, manifest writes). If the user names extra projects to skip this run, add them. Apply the exclusion **once, uniformly** — don't hand-write `grep -v` filters into individual commands, which drifts between the scan and manifest steps.

## Ingest Modes

### Append Mode (default)

Check `.manifest.json` for each source file (conversation JSONL, memory file). Only process:

- Files not in the manifest (new conversations, new memory files, new projects)
- Files whose modification time is newer than their `ingested_at` in the manifest

This is usually what you want — the user ran a few new sessions and wants to capture the delta.

> **Portable keys when comparing.** Manifest keys follow the source key contract (v2) in `llm-wiki/SKILL.md` → `.manifest.json`. A session under `$HOME` is keyed `~`-relative (`~/.claude/projects/.../abc.jsonl`), never by the expanded machine path; vault-relative and pseudo-keys are the other two forms. Before deciding a file is "new", resolve the stored key the same way the tool does (expand `~`/env vars, resolve vault-relative against the vault root) — otherwise an already-tracked file looks new and gets re-ingested. The `scripts/manifest.py` helper does this for you:
>
> ```bash
> # New/modified sources, honoring WIKI_SKIP_PROJECTS + --skip:
> python3 "$OBSIDIAN_WIKI_REPO/scripts/manifest.py" delta "$OBSIDIAN_VAULT_PATH" \
>   --scan "$CLAUDE_HISTORY_PATH/projects/*/memory/*.md"
> # One-time repair if the manifest still holds legacy absolute keys:
> python3 "$OBSIDIAN_WIKI_REPO/scripts/manifest.py" migrate "$OBSIDIAN_VAULT_PATH" --dry-run
> ```
>
> The helper is optional — if it's unavailable, apply the same resolution inline before every manifest lookup and write.

### Pre-extraction (recommended — run before ingest)

Raw JSONL files are 80-90% noise: `tool_use` blocks, `thinking` blocks, `progress` events, and
`file-history-snapshot` entries dominate by byte count.  The `scripts/extract-jsonl.py` helper
strips all of that and writes compact signal-only JSON to `~/.claude/extracted/`, achieving
**50–200× file-size reduction** (e.g. 12 MB JSONL → 64 KB extracted).  This lets the skill read
5–10× more conversations per run within the same token budget.

Run it as a pre-step before invoking this skill:

```bash
# First run — extract everything (skip excluded projects)
python3 "$OBSIDIAN_WIKI_REPO/scripts/extract-jsonl.py" --skip tsg,autom8

# Incremental — only sessions modified in the last day
python3 "$OBSIDIAN_WIKI_REPO/scripts/extract-jsonl.py" \
    --since "$(date -v-1d +%Y-%m-%d)" --skip tsg,autom8
```

Extracted files live at `~/.claude/extracted/<project-dir>/<session-id>.json` and contain:

```json
{
  "session_id": "uuid",
  "project": "-Users-name-myapp",
  "cwd": "/Users/name/myapp",
  "start_ts": "...",
  "end_ts": "...",
  "n_turns": 18,
  "n_user_words": 620,
  "turns": [
    {"role": "user",      "text": "..."},
    {"role": "assistant", "text": "..."}
  ]
}
```

**When Step 3 reads conversations, always prefer the extracted file over the raw JSONL.** (See Step 3.)

If `extract-jsonl.py` was not run first, fall back to raw JSONL — but note the coverage will be
shallower because each raw file costs far more tokens to read.

### Conversation Sampling Heuristic

A history path can hold hundreds of conversation JSONLs — do not try to read them all. Per project:

- **If the project already has memory files** (`memory/*.md`), ingest those first (they are
  pre-distilled signal), then **also process conversations not yet in the manifest** — new
  conversations should still be captured even for memory-rich projects.
- **If the project has no memory files**, read only the **3 most recent** conversations (by mtime)
  to characterize it. Prefer pre-extracted files (see above) — they are cheap enough that you can
  read 5–10 in the same token budget as 1 raw JSONL.
- Always report what you sampled vs skipped (e.g. "agenttower: 7 memory files + 4 new conversations
  ingested, 14 unchanged conversations skipped"), so the coverage gap is visible rather than silent.

### Full Mode

Process everything regardless of manifest. Use after a `wiki-rebuild` or if the user explicitly asks.

## Claude Code Data Layout

Claude Code stores data in two locations. Scan **both**.

### Source 1: `~/.claude/` (CLI sessions)

```
~/.claude/
├── projects/                          # Per-project directories
│   ├── -Users-name-project-a/         # Path-derived name (slashes → dashes)
│   │   ├── <session-uuid>.jsonl       # Conversation data (JSONL)
│   │   └── memory/                    # Structured memories
│   │       ├── MEMORY.md              # Memory index
│   │       ├── user_*.md              # User profile memories
│   │       ├── feedback_*.md          # Workflow feedback memories
│   │       └── project_*.md           # Project context memories
│   ├── -Users-name-project-b/
│   │   └── ...
├── sessions/                          # Session metadata (JSON)
│   └── <pid>.json                     # {pid, sessionId, cwd, startedAt, kind, entrypoint}
├── history.jsonl                      # Global session history
├── tasks/                             # Subagent task data
├── plans/                             # Saved plans
└── settings.json
```

### Source 2: `~/Library/Application Support/Claude/local-agent-mode-sessions/` (Desktop app agent sessions)

> **Pre-check first.** Many users are CLI-only and have no desktop sessions. Before walking the structure below, confirm it's non-empty:
> ```bash
> DESKTOP_SESSIONS="$HOME/Library/Application Support/Claude/local-agent-mode-sessions"
> [ -d "$DESKTOP_SESSIONS" ] && find "$DESKTOP_SESSIONS" -name "audit.jsonl" | head -1
> ```
> If that prints nothing, skip this entire section (Source 2 + Step 3b) and don't narrate it.

The Claude desktop app stores local agent mode sessions here. The structure is deeply nested:

```
~/Library/Application Support/Claude/local-agent-mode-sessions/
└── <outer-uuid>/
    └── <inner-uuid>/
        ├── local_<session-uuid>.json          # S
````

## File: README.md
upstream: https://github.com/ar9av/obsidian-wiki/blob/main/README.md

````text
<h1 align="center">obsidian-wiki</h1>

<p align="center"><b>A unified digital brain you grow with your AI agents.</b></p>

<p align="center">
  It remembers what you figure out, connects it to what you already know,<br>
  and answers when you ask.
</p>

<p align="center">
  <a href="https://pypi.org/project/obsidian-wiki/"><img src="https://img.shields.io/pypi/v/obsidian-wiki?color=blue" alt="PyPI" /></a>
  <a href="https://github.com/ar9av/obsidian-wiki/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a>
  <a href="https://x.com/_ar9av"><img src="https://img.shields.io/badge/@__ar9av-black?logo=x&logoColor=white" alt="X" /></a>
  <a href="https://discord.gg/FH2PRX754c"><img src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white" alt="Discord" /></a>
</p>

<p align="center">
  <img width="768" alt="obsidian-wiki" src="assets/hero.png" />
</p>

<p align="center">
  <img width="768" alt="obsidian-wiki CLI: query, lint and context-pack on a demo vault" src="assets/demo.gif" />
</p>

<p align="center">
  English | <a href="https://github.com/Ar9av/obsidian-wiki/blob/main/README_TW.md">繁體中文</a>
</p>

---

You solve a hard problem on a Tuesday. Three months later, in a different repo, you solve it again from scratch — because the answer lived in a chat log you'll never find.

This fixes that. Point it at a folder, tell your agent what to remember, and it compiles what you learn into interconnected markdown you own. The pattern comes from Andrej Karpathy's [LLM Wiki gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f): compile knowledge once and keep it current, instead of asking an LLM the same questions forever or re-running RAG every time.

**Your second brain. Your AI agent is how you grow it.**

Every skill here is a markdown file that any agent — Claude Code, Cursor, Codex, Windsurf, Gemini CLI, and [a dozen more](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/agents.md) — reads and runs. No runtime, no API keys, no vendor.

## 60 seconds

```bash
pip install obsidian-wiki
obsidian-wiki setup --vault ~/brain
```

That also registers the two session hooks, so every new session starts with who you are and what's open, and ends by capturing anything reusable. Pass `--no-hooks` to skip them.

Using `uv` or `pipx`? `uv tool install obsidian-wiki` and `pipx install obsidian-wiki` work the same way. (Not `uvx` — see [Installation](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/installation.md#install-via-pip-uv-or-pipx-recommended).)

Then open any project in your agent and say **"set up my wiki"**.

Prefer not to touch a terminal? Give your agent this and it'll do the whole thing:

```text
https://github.com/Ar9av/obsidian-wiki — set up my wiki
```

Using Claude Code? Install it as a plugin instead — no Python needed:

```text
/plugin marketplace add Ar9av/obsidian-wiki
/plugin install obsidian-wiki@obsidian-wiki
```

Plugin skills are namespaced (`/obsidian-wiki:wiki-ingest`), and the plugin ships the skills only — not the session hooks or the `obsidian-wiki` CLI.

Other paths — `git clone`, Claude Code plugin, Skills CLI, multiple vaults → **[Installation](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/installation.md)**

## What you actually do

<p align="center">
  <img width="760" alt="The loop: feed it, it compiles into linked pages, ask it, keep it honest; every session starts from what the last one left behind" src="https://raw.githubusercontent.com/Ar9av/obsidian-wiki/main/assets/loop.svg" />
</p>

**Start with three.** `/wiki-setup` once, `/wiki-capture` to save what you just worked out, `/wiki-query` to get it back. Everything below is there when you need it.

**Feed it.** Anything text-shaped: docs, PDFs, chat exports, meeting transcripts, screenshots, URLs.

```text
/wiki-ingest ~/research
/wiki-update                        # distill the repo you're standing in (code-graph aware)
/wiki-capture                       # save this conversation
/wiki-history-ingest claude         # mine everything you've ever asked Claude
```

**Ask it.** Answers come back with `[[wikilink]]` citations, not vibes.

```text
/wiki-query what do I know about rate limiting?
/wiki-narrate MCP security          # a cited briefing on a topic
/wiki-digest week                   # what did I learn this week?
```

**Find that session you can't name.**

```bash
obsidian-wiki sessions-build
obsidian-wiki sessions-query "the auth bug with the weird retry loop"
```

**Keep it honest.** The vault gets messy on its own; these clean it.

```text
/wiki-lint            # broken links, orphans, contradictions
/wiki-dedup           # "RSC" and "React Server Components" are one page now
/cross-linker         # weave new pages into the graph
/wiki-status          # what's ingested, what's pending, where the hubs are
```

All 39 skills → **[Skills Reference](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/skills.md)**

## See it

Open the vault in Obsidian and hit the graph view (Cmd/Ctrl+P → "Open graph view"). Say **"color my graph"** and it tints nodes by tag, category, or visibility.

Want to look before you build? [`examples/demo-vault/`](https://github.com/Ar9av/obsidian-wiki/tree/main/examples/demo-vault) is an 18-page sample vault on building reliable LLM agents, already colored by folder. Clone the repo, choose *Open folder as vault* in Obsidian, and pick that folder.

<p align="center">
  <img width="900" alt="obsidian-wiki graph view" src="https://github.com/user-attachments/assets/f2980840-4b5b-438a-8264-5ad1de42f483" />
</p>

Or export the whole graph to `graph.json`, GraphML (Gephi/yEd), Neo4j Cypher, Postgres SQL, or a self-contained interactive `graph.html`.

## Use it from Python

```python
from obsidian_wiki import Memory

memory = Memory("~/brain")
memory.remember("stack", "Python, FastAPI", confidence=0.9)
memory.add("Postgres was chosen over MySQL for partial indexes.")
memory.search("postgres")
memory.recap()                       # inject at session start
```

No API key, no embedding model, no vector store, no network — the vault is markdown on disk, so everything your agent remembers is readable, greppable and diffable. `user_id` scopes per-person memory while knowledge stays shared.

More → **[Python API](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/python-api.md)**

## It remembers between sessions

A session starts with the vault's memory already in context — who you are, what threads are open, what changed recently — and ends by capturing anything reusable before the context closes.

<p align="center">
  <img width="720" alt="session lifecycle: recap injected at start, capture nudged at stop" src="https://github.com/Ar9av/obsidian-wiki/blob/main/docs/images/memory-session-lifecycle.png?raw=true" />
</p>

`index.md`, `log.md`, `hot.md`, and the owner profile and todo index under `_meta/` all go through one writer that takes a lock and writes atomically, so parallel agents can't drop each other's updates.

More → **[Memory Surface](https://github.com/Ar9av/obsidian-wiki/blob/main/docs/memory.md)**

## Why this and not a notes folder

- **It compiles, it doesn't accumulate.** New knowledge merges into existing pages. Contradictions get flagged. Nothing gets duplicated.
- **It only reads what changed.** A manifest tracks every source ingested, so the second run processes the delta — not your whole library again.
- **You can tell knowledge from guessing.** Every claim is tagged `extracted`, `^[inferred]`, or `^[ambiguous]`, and lint flags pages drifting into speculation.
- **Queries stay cheap as it grows.** Titles, tags, and summaries get read before page bodies. 20 pages or 2000, roughly the same cost.
- **It's yours.** Plain markdown in a folder. Push it to a private repo, open it in Obsidian, grep it, delete it. No service, no lock-in, nothing leaves your machine.
- **It carries context across sessions.** An owner profile, open t
````
