⏳ This skill is pending AI review.

Scores will appear once the review pipeline completes.

version unknown

data-update

sixian-li@sixian-li⭐ 529 stars· last commit 1mo ago· 0 open issues

Operate and assess the Quant workspace market-data layer through the tested data-update CLI. Use when checking QQQ/SPY or constituent freshness, reviewing supported versus missing data capabilities, diagnosing quality, running a SPY-current-member shadow update, inspecting update reports, or previewing/importing purchased CSV/ZIP/JSON/XLSX/XLS data without silently changing approved datasets.

Choose how to use this skill

You do not need every option. Choose the path your AI client supports. The stable page stays the same; versioned files are immutable.

1. Native installer

This listing has no registered native installer command. Use the complete package or source fallback below, depending on what your client supports.

Do not guess an installer command or replace an existing version without reviewing the diff.

2. Complete package recommended

Download the ZIP when available. It includes SKILL.md plus the references, security notes and version metadata.

No complete ProSkills package is published for this listing yet.

3. Prompt-only

Copy the prompt above when the agent can read the stable page or when you want to adopt the workflow without installing a skill.

Need only the instruction file?

Download SKILL.md only if your client requires a single file. The complete ZIP is safer for a full installation because it preserves the references and release context.

No path installs or executes anything by itself. Your agent still needs access to the project files. Before updating, compare the installed version and review the diff.

—/10

// RATINGS

⭐GitHub Stars
⭐⭐⭐⭐ 529 on GitHubGitHub ↗

Popular

🟢ProSkills Score
—
📍

Not yet listed on ClawHub or SkillsMP

// README

PlainBacktest

English | 中文

Say It Simply, Test It Properly.

Describe your trading strategy in everyday language. PlainBacktest works with your coding agent to turn it into explicit rules, run the backtest, and reconcile orders and daily account balances against an independent ledger. Explicit execution timing, recorded data and code hashes, and reproducible reports make the calculation open to inspection.

Source code and bundled data: Sixian-Li/plainbacktest.

Strategy in natural language → Agent + Skill clarify the rules → Freeze the experiment
→ Data checks → PyBroker execution + independent ledger → Report and research record

Your agent interprets the strategy and implements its rules; the Python framework handles reproducible calculation and validation. The repository includes three Skills, curated local data, research code, and 83 experiment definitions. Start with an offline example, then use your agent to continue the research.

Run your first example

Clone the repository and run the following commands. The verified environment is macOS with Python 3.13. Installing dependencies requires an internet connection; running the example needs no market-data API, credentials, or live data.

git clone https://github.com/Sixian-Li/plainbacktest.git
cd plainbacktest
python3.13 -m venv backtest/.venv
backtest/.venv/bin/python -m pip install -r backtest/requirements.lock
backtest/.venv/bin/python backtest/scripts/release_data.py check
backtest/.venv/bin/python backtest/scripts/quickstart.py

Open the outputs/example_…/report.html path printed at the end. Each execution creates a new directory containing the input data, frozen rules, daily account state, orders, metrics, code and data hashes, and reconciliation results.

The default example uses QQQ from 2016-01-01 through 2026-08-04. It holds QQQ when SMA20 is above SMA100 and stays in cash otherwise. Signals are confirmed at the close and executed at the next session's open, with a cost of 5 bps per side. Initial capital is $100,000; fractional shares are allowed, with no leverage. The example uses the framework's PyBroker execution engine, independent ledger, and v5 report template.

Change the parameters or choose another approved symbol:

backtest/.venv/bin/python backtest/scripts/quickstart.py --symbol SPY --fast 50 --slow 200

This exploratory example checks installation, execution, and reconciliation. It does not mark existing experiments as validated or replace parameter robustness and out-of-sample analysis. Moving averages use earlier history for warmup. Adjusted OHLC prices approximate total returns without modeling individual cash dividend payments.

Research with natural language

Open this directory with an agent that supports project Skills. Have it read AGENTS.md, then describe a strategy. For example:

Use quant-backtest to study QQQ with an SMA20/SMA100 rule. If the fast SMA is above the slow SMA at the close, hold a fully invested position from the next session's open; otherwise, exit to cash. Define the data range, costs, warmup, and benchmark before implementing the strategy, reconciling the ledgers, and explaining the results.

The project overview and quickstart are available in English and Chinese. Detailed research documents and report text are currently mainly in Chinese.

All three Skills are versioned with the repository, each with a SKILL.md entry point:

SkillPurpose
quant-backtestTurn a strategy description into an experiment, execution, independent reconciliation, report, and research record
data-updateInspect data snapshots and manage isolated updates and candidate review when needed
quant-tidyMaintain directories, experiment lineage, and workspace structure

These Skills are part of the project. Agents that support .agents/skills can discover them directly; other agents can read the relevant SKILL.md and its references. They do not depend on the author's global Skill installation. See the research architecture for Git provenance, run lifecycles, validation gates, and worktree conventions used in formal research. The example also works from a downloaded source ZIP.

What's included

  • backtest/quantkit/, scripts/, and tests/: execution, ledgers, strategies, metrics, data handling, and validation.
  • backtest/report_templates/: self-contained Plotly reports. The default v5 template explains the strategy before presenting results.
  • backtest/experiments/: 83 definitions and their lineage across three research programs: DER, ROT, and TIM.
  • data/: 1,303 canonical daily-price CSVs, trading calendars, membership tables, quality evidence, and two original Nasdaq-100 ZIP archives. See the data documentation.
  • research/: indicator and market-observation tools without strategy account ledgers.

This is an independent distribution snapshot. Historical runs and large collections of old reports are not bundled, and the active/latest run pointers have been cleared. Historical findings from the source workspace have not been revalidated here. Some experiments require earlier training artifacts or original archives that are not included; see the distribution scope. The source revision and data hashes at the time of copying are recorded in source_snapshot.json.

This version uses the bundled data by default. yfinance is a transitive dependency of PyBroker, but it has not been integrated as this project's default data provider. The quickstart does not call Yahoo. The existing shadow-update tools are optional advanced features.

Validation and optional labs

backtest/.venv/bin/python -m pip check
backtest/.venv/bin/python backtest/scripts/release_data.py check
cd backtest
.venv/bin/python -m unittest discover -s tests -t . -v
.venv/bin/python -m scripts.build_research_catalog --check
.venv/bin/python -m scripts.audit_workspace --workspace ..

Tests that require original archives absent from this distribution report an explicit skip, not a pass. See the verification record for results. The distributed test suite does not implicitly rebuild the bundled canonical data.

Two optional Streamlit labs are included. Launch either from the project root:

backtest/.venv/bin/python -m streamlit run backtest/dual_sma_lab/app.py
backtest/.venv/bin/python -m streamlit run research/indicator_lab/app.py

Browser interaction checks and PDF export also require Node.js and Chrome/Chromium. Set QUANT_CHROME_PATH to use a specific browser executable. The standard example and Python tests do not require them. Other operating systems have not yet been fully verified.

Licensing

Original project code, Skills, and documentation are licensed under MIT. The bundled data under data/ is licensed under CC BY 4.0; attribution details are in the data documentation. The data rights holder has confirmed ownership and authorized this license.

Third-party dependencies retain their own licenses. In particular, PyBroker 1.2.12 uses Apache 2.0 with Commons Clause, which includes a restriction on selling the software as defined in that license. The dependency stack is therefore not entirely MIT-licensed. See third-party notices for the original terms.

// HOW IT'S BUILT

// FILES

4 files shipped with this skill. Inline text is the catalog copy. The upstream link is the source repository.

  • .agents/skills/data-update/SKILL.md6.2 KBabe24ba21fe2upstream ↗
    ---
    name: data-update
    description: Operate and assess the Quant workspace market-data layer through the tested data-update CLI. Use when checking QQQ/SPY or constituent freshness, reviewing supported versus missing data capabilities, diagnosing quality, running a SPY-current-member shadow update, inspecting update reports, or previewing/importing purchased CSV/ZIP/JSON/XLSX/XLS data without silently changing approved datasets.
    ---
    
    # Data Update
    
    Use the workspace CLI as the single execution path. Keep update logic in the tested project code; do not reimplement provider calls in the skill.
    
    ## Distributed project
    
    This skill is versioned at `.agents/skills/data-update/` in this repository. Locate the root from the current checkout; do not depend on a user-level skill or the author’s original workspace. Read `README.md` and `backtest/docs/release/scope.md` first. The distribution contains canonical data and experiment definitions, not the complete source archives or historical runs.
    
    For the bundled snapshot, begin with `backtest/.venv/bin/python backtest/scripts/release_data.py status` and `check`. These are offline and read-only. The live CLI below is for optional data operations; its source/archive audit may report omitted historical files. Do not rebuild canonical data implicitly. Data reuse is authorized under `data/LICENSE`; quality-failed and pending-review products keep their original quality gates.
    
    ## Locate and inspect
    
    1. Locate the Quant root containing `catalog.md`, `log.md`, `data/`, and `backtest/`.
    2. Read `data/data_update_registry.json` before any mutating operation.
    3. Run commands from the Quant root with:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py <command>
    ```
    
    4. Never display API keys. The CLI resolves them from environment variables or macOS Keychain.
    
    Read [contracts.md](references/contracts.md) when interpreting statuses, running a full update, or importing purchased files.
    
    ## Route the request
    
    ### Check the last update or health
    
    Run:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py status
    backtest/.venv/bin/python backtest/scripts/data_update.py check
    ```
    
    Use `check --deep` only for an explicit full audit, after material data work, or before enabling promotion. It verifies large raw files and runs the full test suite, so expect it to take longer.
    
    Report separately:
    
    - approved database date;
    - latest completed XNYS session;
    - current-member coverage;
    - tracked QQQ/SPY/VOO dates and whether their shadow update exists;
    - capability boundaries, especially production promotion, Nasdaq-100 point-in-time membership, and hedge-asset coverage;
    - latest source-validation and shadow-update run;
    - warnings versus blocking failures;
    - whether production writes and scheduling are enabled.
    
    ### Manually update
    
    Run an isolated shadow update:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py shadow-update
    ```
    
    For a fast diagnostic, pass `--symbols AAPL,FERG,BRK.B,BF.B --tiingo all`. For a requested full current-universe run, omit `--symbols` and `--limit`; warn that Twelve Data's free 8-credit/minute limit makes 503 symbols take roughly 63 minutes for the single adjusted-price request.
    
    Treat exit code 2 as “review required,” not automatically as a crash. Open the `report_json` referenced by `data/processed/updates/sp500_shadow/latest.json` and explain the exact symbol statuses.
    
    For a campaign rerun after code-only fixes, use immutable provider archives instead of spending primary-source credits again. `--replay-twelve-run RUN_ID` reuses a complete Twelve Data archive; `--replay-tiingo-runs RUN_ID[,RUN_ID...]` reuses available Tiingo payloads and fetches only missing required symbols. Never use replay to stand in for a new completed XNYS session.
    
    If a live run stops after writing only some Twelve Data batches, preserve that raw run and resume into a new run ID:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py shadow-update \
      --resume-twelve-run PARTIAL_RUN_ID \
      --run-id NEW_RUN_ID
    ```
    
    Resume reuses valid archived symbols, fetches only missing or stale symbols, and never modifies the source archive. Inspect `resume_source_issues` in the new report. Do not call a partial transport failure a completed campaign day.
    
    If one archived symbol contains a transient latest-session value that later fails the independent cross-check, preserve the archive and refetch only that selected symbol while resuming:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py shadow-update \
      --resume-twelve-run SOURCE_RUN_ID \
      --refresh-symbols CBOE \
      --replay-tiingo-runs SOURCE_RUN_ID \
      --run-id NEW_RUN_ID
    ```
    
    Use `--refresh-symbols` only with `--resume-twelve-run`. Require the replacement payload to pass the unchanged candidate and cross-source gates; do not use this option to overwrite or hide the original failed evidence.
    
    Do not edit `data/processed/daily/equities/`. While `production_writes_enabled=false`, no command may promote candidates into the approved database.
    
    ### Import newly purchased data
    
    Require the exact source path. Always preview first:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py import-purchased \
      --source /exact/external/path --deep
    ```
    
    Summarize file count, bytes, root SHA256, formats, and inspection failures. Apply only after the user explicitly authorizes copying that exact source and supplies or confirms provider and acquisition date:
    
    ```bash
    backtest/.venv/bin/python backtest/scripts/data_update.py import-purchased \
      --source /exact/external/path \
      --provider provider-name \
      --acquired-date YYYY-MM-DD \
      --label optional-label \
      --apply
    ```
    
    Applied files remain immutable under `data/raw/purchased/` with status `pending_review`. Never infer permission to merge them into approved prices, rebuild historical membership, or delete the external source.
    
    ## Finish safely
    
    1. Inspect the generated machine report and human-readable report when present.
    2. Run `check`; use `check --deep` after an applied purchase import or material implementation change.
    3. If material workspace files changed, also follow `quant-tidy` for `catalog.md`, `log.md`, audit, and tests.
    4. State exactly what was updated, what stayed shadow/pending, the last covered session, and any user action required.
    
  • .agents/skills/data-update/references/contracts.md4.7 KB310f4229e172upstream ↗
    # Data-update contracts
    
    ## Authoritative paths
    
    - Operations contract: `data/data_update_registry.json`
    - Historical baseline contract: `data/sp500_history_registry.json`
    - Current dated universe: `data/processed/universes/sp500/current_constituents.csv`
    - Approved current/history prices: `data/processed/daily/equities/`
    - Shadow updates: `data/processed/updates/sp500_shadow/`
    - Archived public responses: `data/raw/public/market_data_updates/`
    - Purchased intake registry: `data/purchased_import_registry.json`
    - Immutable purchased intake: `data/raw/purchased/`
    - Nasdaq-100 candidate contract: `data/nasdaq100_history_registry.json`
    - Nasdaq-100 pending-review products: `data/processed/universes/nasdaq100/pending_review/`
    
    ## Capability boundary
    
    Treat `data/data_update_registry.json.capabilities` as authoritative. The S&P 500 point-in-time baseline, current-member shadow update, provider replay, and purchased intake are ready. Nasdaq-100 point-in-time membership is implemented as `candidate_pending_review`, not approved: use its frozen manifest and hashes only for explicitly authorized exploratory research, preserve the one-XNYS-session membership lag, and block strategy promotion while provenance, authorization, identity, adjustment, and early-history reviews remain open. Approved-price promotion and the scheduler remain disabled; QQQ/SPY shadow updates and a hedge-asset universe are not implemented. Report these boundaries explicitly instead of implying that the current SPY-constituent updater or Nasdaq candidate covers every strategy input.
    
    ## Status meanings
    
    - `ready_candidate`: overlap, date coverage, and adjusted-return continuity passed; still shadow-only.
    - `ready_bootstrap_candidate`: dated official identity plus independent security evidence allow a post-baseline shadow candidate; it is not production approval and pre-baseline history is not spliced.
    - `bootstrap_needs_identity_review`: public prices exist for a missing current constituent, but security identity/history must be approved before any merge.
    - `review`: one or more date, overlap, freshness, or cross-source gates need inspection.
    - `fetch_or_analysis_failed`: the provider or parser failed for that symbol; other symbols may still be valid.
    - `pending_review`: a purchased batch was copied and hash-verified but is not an approved input.
    - `candidate_pending_review`: a reproducible derived candidate passed machine parsing and hash gates but still has unresolved human-review fields. It may support explicitly authorized exploratory research with frozen provenance; it is not an approved production input and cannot support promotion claims.
    
    ## Non-negotiable boundaries
    
    - A shadow candidate is not an approved database row.
    - A successful Tiingo/Twelve comparison does not prove historical security identity.
    - Price level differences caused by different adjustment bases require an anchor; compare overlapping daily returns before scaling new rows.
    - A historical adjustment-basis jump requires registered public corporate-action evidence, a stable recent return tail, and same-run Tiingo coverage; an undocumented jump blocks the run.
    - Volume is warning-only because provider semantics differ materially.
    - Never fill missing bars, splice ticker histories, or merge purchased files silently.
    - Never expose credentials in argv, reports, source files, or chat.
    
    ## Free-tier behavior
    
    - Twelve Data: 8 credits/minute and 800/day; one time-series symbol consumes one credit. The CLI waits across minute windows and uses a single adjusted-price request per symbol.
    - Tiingo: 500 unique symbols/month, 50 requests/hour, and 1,000/day. The configured 480-symbol monthly budget leaves headroom; the monthly offset rotates the otherwise omitted current constituents over time.
    - Eastmoney: opportunistic validator only until repeated transport-stability runs pass.
    
    ## Five-session campaign
    
    - A qualifying day covers all 503 dated current equities, has no primary-provider failure, produces only allowed shadow statuses, passes at least 30 Tiingo cross-checks, and writes zero production rows.
    - Every bootstrap candidate and every symbol with a documented historical adjustment-basis warning must be included in that day's Tiingo sample.
    - Progress is the latest consecutive XNYS-session streak. Weekends and exchange holidays do not break it; a missing or failed trading session restarts the streak.
    
    ## Purchased intake phases
    
    1. Preview external source, inventory hashes, and inspect supported containers.
    2. Explicitly apply a reviewed source into immutable raw storage.
    3. Review provider schema, ticker identity, adjustment semantics, coverage, and licensing.
    4. Build a separate candidate transformation and compare against approved overlap.
    5. Promote only through a separately implemented, tested, explicitly authorized gate.
    
  • .agents/skills/data-update/agents/openai.yaml236 B1a09d11a83aeupstream ↗
    interface:
      display_name: "Data Update"
      short_description: "Check and safely operate Quant market-data workflows"
      default_prompt: "Use $data-update to check Quant data freshness, capability gaps, and run the safe update workflow."
    
  • README.mdupstream ↗

    Contents are not inlined in the catalog. Open the upstream file.

Plain-text copy, no login: /skill-bundles/sixian-li-plain-backtest-agents-skills-data-update.md

// REPO STATS

529 stars
0 open issues
Last commit: 1mo ago