⏳ This skill is pending AI review.
Scores will appear once the review pipeline completes.
scrapling-official
Scrape or crawl websites, extract data from web pages, handle anti-bot protections (Cloudflare, fingerprinting), bypass JavaScript-heavy sites, write Python scrapers or spiders, or when web_fetch fails or returns empty/incomplete content.
Choose how to use this skill
You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.
1. Native installer
This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.
Do not guess an installer command or replace an existing version without reviewing the diff.
2. Complete package recommended
Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.
No complete ProSkills package is published for this listing yet.3. Prompt-only
Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.
Need only the instruction file?
Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.
No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.
// RATINGS
Not yet listed on ClawHub or SkillsMP
// README
🕷️ Openclaw Scrape Skill — Rhaone
This skill brings full-featured web scraping capabilities directly into your OpenClaw agents — anti-bot bypass, JavaScript rendering, adaptive element tracking, proxy rotation, and a complete spider crawling framework. No guesswork, no manual docs hunting.
What This Skill Does
- Static scraping — fast HTTP requests with real browser TLS fingerprints (no browser overhead)
- JS-rendered pages — full Playwright/Patchright browser automation
- Anti-bot bypass — auto-solves Cloudflare Turnstile, spoofs canvas/WebRTC/fingerprints
- Adaptive parsing — learns from website changes and relocates elements automatically
- Spider framework — concurrent multi-page crawls with pause/resume and proxy rotation
- XHR/API interception — capture background API calls and get raw JSON directly
- Structured data — JSON-LD, OpenGraph, Twitter Card extraction out of the box
Includes a JS-heavy fallback chain, structured failure reporting, and model-adaptive token optimization so agents running on any LLM (small local models to large cloud models) use it efficiently.
Installation in OpenClaw
clawhub install scrapling-official
Or download the ZIP directly and load it manually in OpenClaw.
Skill Files
SKILL.md ← Main skill definition (OpenClaw entry point)
LICENSE.txt
examples/
01_fetcher_session.py ← HTTP session with TLS fingerprinting
02_dynamic_session.py ← Playwright browser automation
03_stealthy_session.py ← Stealth browser + Cloudflare bypass
04_spider.py ← Full concurrent spider with JSON export
05_structured_extraction.py ← JSON-LD, OpenGraph, XHR capture
06_error_recovery.py ← Escalation chain + structured failure report
references/
fetching/ ← Static, Dynamic, Stealthy fetcher docs
parsing/ ← Selector, XPath, adaptive scraping docs
spiders/ ← Spider framework, sessions, proxy, advanced
patterns/
common-sites.md ← E-commerce, news, directories, reviews, jobs
troubleshooting.md ← Diagnosis guide, visual debug, all error types
mcp-server.md ← MCP server integration
migrating_from_beautifulsoup.md ← API comparison for BS4 users
Setup (once)
pip install "scrapling[all]>=0.4.3"
scrapling install --force
Or via Docker (CLI only, no Python code):
docker pull pyd4vinci/scrapling
Credits
This skill is built on top of Scrapling by @D4Vinci — an adaptive Python web scraping framework.
- GitHub: D4Vinci/Scrapling
- Docs: scrapling.readthedocs.io
- PyPI: pypi.org/project/scrapling
- License: BSD 3-Clause
License
BSD 3-Clause — see LICENSE.txt
// HOW IT'S BUILT
KEY FILES