# weibo-scraper

Adopt this skill: https://proskills.md/skills/other/rrrrrredy-weibo-scraper

Page: https://proskills.md/skills/other/rrrrrredy-weibo-scraper
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: SKILL.md
size: 5510
upstream: https://github.com/rrrrrredy/weibo-scraper/blob/HEAD/SKILL.md

````text
---
name: weibo-scraper
description: 'Weibo public content reader for visible posts that do not require login. Uses web search and Playwright visitor-mode rendering to fetch public user timelines or post text. Triggers: check Weibo, scrape Weibo, Weibo user, read Weibo post, Weibo timeline, fetch Weibo content. Not for: login-required or fans-only content; extracting account cookies; private messages; large-scale batch scraping; non-Weibo platforms.'
---

# weibo-scraper 1.0.0

No login required. Fetches visible public Weibo posts via web search + Playwright visitor-mode rendering.

---

## Routing Contract

### When to use

- User provides a Weibo username, UID, profile URL, or public post URL and asks to read, summarize, or list visible public posts.
- User asks for a small recent timeline sample or one public post's text/media indicators.

### When not to use

- User asks for fans-only, login-only, private, deleted, hidden, DM, or account-specific content.
- User asks to extract cookies, use the user's logged-in account, bypass access controls, or run large-scale scraping.
- User asks about another platform; route to the matching platform skill.

### Required inputs

- Username, UID, profile URL, post URL, or enough search terms to locate the public Weibo account.
- Desired scope: one post, recent N visible posts, or profile overview. Default to recent visible posts only.

### Output

- Fixed output: source URL/UID, scrape time, fetched count, visible post text, publish time when available, media indicator, and visibility caveats.
- Do not claim the timeline is complete unless the public page exposes enough chronological evidence.

### Failure handling

- If search cannot locate the account after three attempts, stop and ask for the profile URL or UID.
- If Playwright renders an empty page or selectors changed, report the failed URL and selector evidence.
- If content appears restricted, state the visibility limitation and do not attempt login.

---

## ⚠️ Gotchas

⚠️ **Weibo ajax API returns -100 without login** → `https://m.weibo.cn/api/container/getIndex` etc. always return errno=-100 when not logged in. Do not attempt direct API calls.

⚠️ **Camoufox may fail to initialize** → Camoufox downloads browser binaries on first run, which can timeout in some network environments. Use standard Playwright instead.

⚠️ **CSS selector `.detail_wbtext_4CRf9` may change** → Weibo frontend updates class names periodically. When the selector fails, use a fallback: find the timestamp element in page HTML and read surrounding text blocks.

⚠️ **Search result ordering is not strictly chronological** → Web search results are ranked by relevance, not strict reverse-chronological order. For "absolute latest N posts", open the user's profile page and sort by timestamp manually.

⚠️ **Unlisted content is inaccessible** → Posts set to "followers only" cannot be fetched. If content appears empty or post count seems low, inform the user about potential visibility restrictions.

---

## 🛑 Hard Stop

- Same web search query fails more than **3 times** (empty results or errors) → Stop immediately, tell user: "Cannot locate this user via search. Please verify the Weibo username or provide the profile URL directly."
- Playwright fails to render content on **3 consecutive** URLs → Stop, inform user the CSS selector may have changed, awaiting manual update.

---

## First Use

Run the dependency check before first use:
```bash
bash scripts/setup.sh
```
> This skill depends on Playwright for browser automation and a web search tool.

---

## Tool Dependencies

| Tool | Purpose |
|------|---------|
| agent-browser (recommended) | Open Weibo pages with visitor cookie to read post content (auto-detects and uses when available) |
| Playwright | Fallback browser automation when agent-browser is not available |
| Web search (Bing/Brave/etc.) | Locate Weibo user UID and post URLs via `site:weibo.com` search |

---

## Workflow

### A. Fetch a User's Recent N Posts

**Step 1: Locate user UID**
```
Web search: @username site:weibo.com
→ Extract UID from weibo.com/u/<UID> format URL
→ If post URLs are found directly, skip to Step 3
```

**Step 2: Search for user's post list**
```
Web search: site:weibo.com/u/<UID>
→ Collect returned post URLs (typically 5-10)
```

**Step 3: Read post content**
```
Playwright: open post URL
→ Read CSS selector: .detail_wbtext_4CRf9
→ If selector fails: find timestamp element in page, read surrounding text blocks
→ Extract: post text + publish time (from page title or timestamp element)
```

**Step 4: Format output**
```
Sort by time (descending, based on publish time field)
Output format:
---
[time] Post content
(Images/video: [contains media, text shown above])
---
```

### B. Get User Profile Overview
```
Playwright: open https://weibo.com/u/<UID> or https://weibo.com/<username>
→ Read post list from page (visitor cookie mode, no login required)
→ Note: without login, visible posts are limited (typically 10-15)
```

---

## Output Format

```
User: @username (UID: XXXXXXXXXX)
Scraped at: YYYY-MM-DD HH:MM
---
1. [2026-04-06 10:30] Post content...
2. [2026-04-05 15:20] Post content...
...
Total: N posts fetched, M contain media (text only shown)
```

---

> 微博 API 技术笔记见 `references/weibo-api-notes.md`。

## Changelog

- 1.0.0: Initial release. Web search + Playwright visitor cookie approach for public Weibo content scraping without login.
````

## File: README.md
upstream: https://github.com/rrrrrredy/weibo-scraper/blob/HEAD/README.md

````text
# weibo-scraper

Weibo public content scraper — no login required.

> OpenClaw Skill — works with [OpenClaw](https://github.com/openclaw/openclaw) AI agents

## What It Does

Fetches public Weibo posts without needing a Weibo account. Uses web search to locate user profiles and post URLs, then opens pages via Playwright with a visitor cookie to extract post text and timestamps. Supports user timeline browsing and individual post content extraction.

## Quick Start

```bash
openclaw skill install weibo-scraper
# Or:
git clone https://github.com/rrrrrredy/weibo-scraper.git ~/.openclaw/skills/weibo-scraper
```

Run dependency setup on first use:
```bash
bash scripts/setup.sh
```

## Features

- **No login required**: visitor cookie mode for public content access
- **User timeline**: fetch a user's recent public posts (typically 10-15 visible)
- **Post extraction**: read full text content from individual post URLs
- **Web search integration**: locate user UIDs and post URLs via `site:weibo.com` queries
- **Fallback selectors**: handles CSS class name changes with alternative extraction methods
- **Hard stop protection**: auto-stops after 3 consecutive failures to prevent infinite retries

## Usage

Trigger with natural language:

- "查一下 @某用户 的微博"
- "看看这个微博用户最近发了什么"
- "抓取微博内容"
- "搜微博" / "微博动态"

## Project Structure

```
weibo-scraper/
├── SKILL.md                # Skill definition and workflow
├── scripts/
│   └── setup.sh            # Dependency setup script
├── references/
│   └── weibo-api-notes.md  # API notes and technical details
├── README.md
├── LICENSE
└── .gitignore
```

## Requirements

- OpenClaw agent runtime
- Playwright (installed via setup.sh)
- Web search tool (Bing/Brave/etc.)
- Upstream proxy for network access (configured in environment)

## License

[MIT](LICENSE)
````
