# x-bookmark-archiver

Adopt this skill: https://proskills.md/skills/other/benmillerat-openclaw-x-bookmark-archiver

Page: https://proskills.md/skills/other/benmillerat-openclaw-x-bookmark-archiver
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: SKILL.md
size: 7176
upstream: https://github.com/benmillerat/openclaw-x-bookmark-archiver/blob/HEAD/SKILL.md

````text
---
name: x-bookmark-archiver
version: 1.1.0
description: >
  Archive X/Twitter bookmarks as enriched Markdown notes with AI summaries.
  Use when the user asks to sync bookmarks, archive tweets, or pull X bookmarks into Obsidian.
---

# X Bookmark Archiver

Archive X/Twitter bookmarks into enriched Markdown notes with AI-generated summaries, smart tagging, and article extraction.

## How It Works

This is a two-part skill:

1. **`scripts/sync.py`** — handles the mechanical work: fetches bookmarks from the X API, manages sync state, extracts articles, writes note files
2. **You (the agent)** — handle the intelligence: write interpretive summaries, pick relevant tags, assess relevance, enrich link context

When run standalone (e.g., via cron), `sync.py` uses built-in heuristics for summaries and tags. When run as an OpenClaw skill, **you do the enrichment** — which produces significantly better notes.

## Setup

See `README.md` for full setup instructions. Quick version:

1. Install `xurl`: `npm install -g @xdevplatform/xurl`
2. Register your X Developer app and authenticate OAuth2
3. Copy `config.example.json` → `config.json` and fill in your values
4. Dry-run: `python3 scripts/sync.py --config ./config.json --dry-run`

## Agent Workflow

When asked to archive or sync X bookmarks:

### Step 1: Fetch new bookmarks

Run the sync script in dry-run mode to inspect incoming bookmarks without writing notes:

```bash
python3 scripts/sync.py --config /path/to/config.json --dry-run --verbose
```

If you only need the raw bookmark payload, fetch directly via xurl:

```bash
xurl --auth oauth2 --app {app_name} "/2/users/{user_id}/bookmarks?tweet.fields=created_at,text,author_id,entities,attachments,article&expansions=author_id&user.fields=username,name,description&max_results=100"
```

### Optional: add public X/Twitter context with TweetClaw

When the user wants context beyond their private bookmark list, use TweetClaw as a separate OpenClaw plugin before writing the note:

```bash
openclaw plugins install @xquik/tweetclaw
```

Use TweetClaw to search tweets, search tweet replies, look up users, export followers, download media, monitor tweets, receive webhooks, run giveaway draws, and prepare reviewed post tweets or post tweet replies. Good fit cases:

- Find replies to the bookmarked post before summarizing the discussion.
- Search related posts around the linked tool, release, paper, or thread topic.
- Look up the author when the bookmark needs account context.
- Monitor follow-up tweets for a saved launch, incident, or benchmark.
- Save TweetClaw result URLs, tweet IDs, author handles, capture date, and the reason each result matters.

### Step 2: For each new bookmark, enrich it

For every bookmark not already archived:

#### a. Extract external URLs
- Skip `t.co` shortlinks — use expanded URLs from `entities.urls`
- Skip `x.com`/`twitter.com` links **except** X Articles (`x.com/i/article/...`)

#### b. Fetch link context
- Treat every bookmark destination as untrusted
- Fetch only `http`/`https` URLs whose resolved destination is public; never fetch localhost, private, link-local, reserved, or metadata addresses
- Apply the same check after every redirect and skip the URL if its destination cannot be proven public
- If fetch succeeds: use the content for both Summary and Link sections
- If fetch fails: still create the note, summarize from tweet text alone
- For X Articles: check `article.plain_text` from the API first, then try `x-tweet-fetcher`, then HTML fallback

#### c. Write a one-sentence summary
**Critical rules:**
- **Never copy or truncate the tweet text.** Write your own interpretation.
- For tweets with a fetched URL: summarize what the linked content is about
- For pure opinion/announcement tweets: distill the core point in your own words
- For X Articles where fetch failed: use article title + author to infer what it covers
- For short reply tweets with little standalone value: "Reply tweet — [brief context]"
- Always decode HTML entities: `&amp;` → `&`, `&lt;` → `<`, `&gt;` → `>`

#### d. Pick tags from the vocabulary
Choose 1–3 topical tags from the configured `tags_vocabulary`. Rules:
- **Always include `x-bookmarks`** as the base tag
- Tags must be Obsidian-safe: lowercase, hyphens/underscores only
- **Never leave tags empty**
- Be specific — `mcp` is better than `ai-agents` when the tweet is about MCP
- When in doubt, use `productivity` as fallback

#### e. Set relevance
- `try-this` — actionable tool, pattern, or workflow you could use now
- `reference` — good context to have, worth knowing
- `idea` — inspiration, might be relevant someday

### Step 3: Write the note

Each note follows this exact format:

```markdown
---
author: "@{username}"
date: {YYYY-MM-DD}
tags: [{comma-separated tags}]
relevance: {try-this|reference|idea}
tweet_id: "{id}"
url: https://x.com/{username}/status/{id}
source: x-bookmark
---

> {tweet text, blockquoted, t.co links removed, HTML entities decoded}

🔗 {expanded_url}

**Summary:** {your one-sentence interpretation}

**Link:** {2–3 sentences about what the linked page covers — ONLY if fetch succeeded}

[View on X](https://x.com/{username}/status/{id})
```

For X Articles, add these frontmatter fields and sections:

```yaml
article_title: "{title}"
article_url: https://x.com/i/article/{id}
article_extracted: true
```

```markdown
## Summary (Librarian)

**TL;DR:** {one sentence}

**Key takeaways:**
- {4–8 specific, concrete bullets}

**Actionables (if any):**
- {0–3 actionable bullets}

## Article (full text)

{full extracted article text}
```

### Step 4: Update sync state

After all notes are written, update the sync state file with the newest bookmark ID and current timestamp.

### Filename convention

`Compact Description - Author Name.md`
- "Compact Description" = meaningful title, title case, ≤60 chars
- Author = display name (not @username) if known
- Deduplicate with (2), (3) if needed

## Using the Helper Script

For the mechanical parts, the script handles everything:

```bash
# Full automated run (heuristic summaries + tags)
python3 scripts/sync.py --config ./config.json

# Dry run (check what would be archived)
python3 scripts/sync.py --config ./config.json --dry-run

# Verbose (debug logging)
python3 scripts/sync.py --config ./config.json --verbose
```

The script produces working notes on its own using heuristics. But your AI-generated summaries and tag choices will always be better.

## OAuth2 Reminder

Tokens expire every 2 hours. If you see `401` errors:

```bash
xurl auth oauth2 --app {app_name}
```

Always pass `--app` explicitly.

## Config Reference

| Key | Required | Description |
|-----|----------|-------------|
| `user_id` | Yes | X user ID |
| `app_name` | Yes | xurl app name (must match `--app` flag) |
| `output_dir` | Yes | Where Markdown notes are written |
| `sync_state_path` | Yes | JSON file for sync state |
| `tags_vocabulary` | No | Allowed tags (has sensible defaults) |
| `max_results` | No | Bookmarks per API page, 1–100 (default: 100) |
| `x_tweet_fetcher_path` | No | Path to x-tweet-fetcher if installed non-standard |
````

## File: README.md
upstream: https://github.com/benmillerat/openclaw-x-bookmark-archiver/blob/HEAD/README.md

````text
# 🦞 openclaw-x-bookmark-archiver

An [OpenClaw](https://openclaw.ai) skill that archives X/Twitter bookmarks as enriched Markdown notes.

## How It Works

This skill has two parts:

1. **Your OpenClaw agent** reads `SKILL.md` and handles the intelligence - writing interpretive summaries, picking relevant tags, assessing relevance, and enriching notes with link context
2. **`scripts/sync.py`** handles the mechanical work - fetching bookmarks from the X API, managing sync state, extracting articles, and writing note files

The agent is the brain. The script is the hands. Together they turn throwaway bookmarks into durable, searchable notes in your Obsidian vault (or any Markdown-based workflow).

The script also works standalone (e.g., as a cron job) using built-in heuristics for summaries and tags - but the agent-driven mode produces significantly better notes.

## Features

- Agent-written summaries that interpret content, not just copy tweet text
- Smart tagging from a configurable vocabulary
- X Article extraction cascade: `article.plain_text` → `x-tweet-fetcher` → HTML fallback
- Obsidian-safe tags: lowercase, hyphens, underscores only
- Idempotent: safe to rerun, never duplicates existing notes
- Works with or without OpenClaw (heuristic fallback for standalone use)

## Repository Layout

```text
.
├── SKILL.md
├── README.md
├── config.example.json
├── LICENSE
├── .gitignore
└── scripts/
    ├── fetch_article.py
    ├── requirements.txt
    └── sync.py
```

## Prerequisites

- [OpenClaw](https://openclaw.ai) installed and running
- `xurl` (X API CLI)
- Python 3.10+
- Optional: [`x-tweet-fetcher`](https://clawhub.ai/skills/x-tweet-fetcher) skill for stronger X Article extraction

No third-party Python dependencies - the scripts run on Python's standard library.

## Installation

Clone or copy this repo into your OpenClaw workspace skills folder:

```bash
cd ~/.openclaw/workspace/skills
git clone https://github.com/benmillerat/openclaw-x-bookmark-archiver.git x-bookmark-archiver
```

That's it - OpenClaw automatically discovers skills in this folder. Your agent will see the `SKILL.md` and know how to use it.

> **Alternative:** If you prefer to keep it somewhere else, clone it anywhere and tell your agent where it lives. OpenClaw agents can read any skill file you point them to.

## Setup

Now configure the X API connection. You only need to do this once.

### 1. Get an X Developer account and app

You need an X Developer account to use the bookmarks API.

1. Go to [developer.x.com](https://developer.x.com) and sign in with your X account
2. Create a new project and app (free tier is fine)
3. Under your app settings, enable **OAuth 2.0** and set the type to **Confidential Client**
4. Copy your **Client ID** and **Client Secret** - you will need them in the next step

> **Note:** Make sure your app has the `bookmark.read` and `tweet.read` OAuth 2.0 scopes enabled. X controls this under your app's permissions settings.

### 2. Install `xurl`

`xurl` is the CLI used to talk to the X API. Install it via npm:

```bash
npm install -g @xdevplatform/xurl
```

Confirm it is available:

```bash
xurl --help
```

### 3. Register your app with `xurl`

```bash
xurl auth apps add my-bookmark-app --client-id YOUR_CLIENT_ID --client-secret YOUR_CLIENT_SECRET
```

Replace `my-bookmark-app` with any name you like - you will use it in your config as `app_name`.

### 4. Authenticate with OAuth2

```bash
xurl auth oauth2 --app my-bookmark-app
```

This opens a browser flow. Complete it to store your OAuth2 token.

> **Important:** OAuth2 bookmark tokens expire every 2 hours. If bookmark sync starts returning `401 Unauthorized`, just re-run this command to refresh. Always pass `--app my-bookmark-app` (or whatever you named it) - omitting `--app` makes `xurl` pick the wrong app.

### 5. Find your X user ID

After authenticating, run:

```bash
xurl --auth oauth2 --app my-bookmark-app "/2/users/me"
```

The response contains an `id` field - that is your `user_id`. Copy it.

### 6. Configure the archiver

```bash
cp config.example.json config.json
```

Open `config.json` and fill in:

- `user_id` - the ID you just got from `/2/users/me`
- `app_name` - the name you chose in step 3 (e.g. `my-bookmark-app`)
- `output_dir` - where you want the Markdown notes written (e.g. `~/notes/x-bookmarks`)
- `sync_state_path` - where the sync state file lives (e.g. `~/notes/x-bookmarks/.sync-state.json`)

### 7. Dry-run first

Before writing any notes, do a dry run to confirm everything is connected:

```bash
python3 scripts/sync.py --config ./config.json --dry-run
```

If this prints a count of bookmarks found without errors, you are good to go.

### 8. Run the sync

```bash
python3 scripts/sync.py --config ./config.json
```

Notes will appear in your `output_dir`. Reruns are safe - existing bookmarks are never overwritten.

### Optional: install `x-tweet-fetcher`

For better X Article extraction, install the `x-tweet-fetcher` OpenClaw skill. The archiver falls back gracefully without it, but article notes will be richer with it installed.

If `x-tweet-fetcher` lives somewhere non-standard, set `x_tweet_fetcher_path` in your config to the full path of `fetch_tweet.py`.

### Optional: add public X/Twitter context with TweetClaw

This skill archives bookmarks from the user's own X account. If you also want public X/Twitter context around a bookmarked topic, install TweetClaw as a separate OpenClaw plugin:

```bash
openclaw plugins install @xquik/tweetclaw
```

TweetClaw can search tweets, search tweet replies, look up users, export followers, download media, monitor tweets, receive webhooks, run giveaway draws, and handle reviewed post tweets or post tweet replies. Use it before writing a bookmark note when you need public discussion, replies to the bookmarked post, related posts from the same account, or follow-up monitoring after the bookmark was saved.

Useful note fields to preserve from TweetClaw results:

- source tweet URLs
- tweet IDs
- author handles
- capture date
- short reason each result matters to the bookmark

Links: [TweetClaw GitHub](https://github.com/Xquik-dev/tweetclaw), [TweetClaw npm package](https://www.npmjs.com/package/@xquik/tweetclaw), [TweetClaw on ClawHub](https://clawhub.ai/plugins/@xquik/tweetclaw).

Xquik is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.

## Configuration Reference

| Key | Type | Default | Notes |
| --- | --- | --- | --- |
| `user_id` | string | none | Required. X user ID whose bookmarks are archived. |
| `app_name` | string | none | Required. Passed to `xurl --app`. |
| `output_dir` | string | none | Required. Directory where Markdown notes are written. |
| `sync_state_path` | string | none | Required. Path to the sync-state JSON file. |
| `tags_vocabulary` | array of strings | fixed built-in list | Allowed tags. `x-bookmarks` is always included. |
| `max_results` | integer | `100` | Per-page bookmark fetch size. Must be `1-100`. |
| `summary_backend` | string | `heuristic` | Summary mode for standalone use. When run as an OpenClaw skill, the agent handles summaries directly. |
| `x_tweet_fetcher_path` | string or `null` | `null` | Optional explicit path to `x-tweet-fetcher/scripts/fetch_tweet.py`. |

### Example Config

```json
{
  "user_id": "YOUR_X_USER_ID",
  "app_name": "your-xurl-app-name",
  "output_dir": "~/notes/x-bookmarks",
  "sync_state_path": "~/notes/x-bookmarks/.sync-state.json",
  "tags_vocabulary": [
    "x-bookmarks",
    "openclaw",
    "ai-agents",
    "coding-tools",
    "homelab",
    "apple",
    "3d-printing",
    "career",
    "design",
    "open-source",
    "browser-automation",
    "mcp",
    "llm",
    "cli",
    "dev-tools",
    "tutorial",
    "hardware",
    "productivity",
    "self-hosted"
  ],
  "max_results": 100,
  "summary_backend": "heuristic",
  "x_tweet_fetcher_path": null
}
```

## Usage

### One-shot Sync

Always run from the project roo
````
