# deeppapernote

Adopt this skill: https://proskills.md/skills/other/917dhj-deeppapernote-skills-deeppapernote

Page: https://proskills.md/skills/other/917dhj-deeppapernote-skills-deeppapernote
This document is the public skill bundle for that page. It contains SKILL.md and every helper file the catalog carries. No login is required.

## File: skills/deeppapernote/SKILL.md
size: 22148
sha: 2216b61e66fd9070975037a58c563279abd1d30a
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/SKILL.md

```text
---
name: deeppapernote
description: Generate a high-quality deep-reading note for a single paper and write it into an Obsidian-style vault. Use when the user gives a paper title, DOI, URL, arXiv ID, Zotero item, or local PDF and wants a polished Markdown note with strong structure, evidence-based analysis, and figure placeholders.
---

# DeepPaperNote

Turn one paper into a replication-grade deep-reading note and save it to the workspace or Obsidian target of the resolved configuration. Each run handles one paper. A rewrite of the abstract is never a finished note.

## Standard

The note reconstructs the paper's argument as one story that runs through its middle sections: the problem it takes on, how the work answers it, what the evidence shows, what the paper does not prove, and why it is worth keeping. The paper type sets the middle sections and the questions they must answer (`references/paper-types.md`).

Write as a senior colleague explaining the paper to a smart researcher from an adjacent field: someone who reads papers but does not know this paper's subfield, its jargon, or its baselines. Lead each paragraph with its point, explain a term the first time it appears, and keep the technical depth the paper type calls for: equations and implementation detail for method, system, and theory papers; design, measurement, and argument for the others.

These hold at every step:

- **Evidence-first:** `*_raw_sections.jsonl` and `*_source_manifest.json` are the canonical reading material. Finish only from inspected full-text evidence, never from the title, abstract, or headings alone.
- **Model-first:** scripts gather and structure evidence. You choose the paper type, emphasis, contributions, mechanism, limitations, and prose; script suggestions are hints.
- **Fail closed:** when the evidence cannot support a deep note, stop as Stop Conditions describes.
- **Placeholder-first figures:** every major figure and table starts as a placeholder. It becomes an image only after Visual Review accepts it and Formal Save materializes it. Image extraction is optional and never reduces textual coverage.

## Language Integrity Contract

After Configuration Readiness, resolve one `output_language` (`zh-CN` or `en`) and pass it as `--language` to every script. `source_manifest.language_hint` describes the source text only and never selects the note profile.

The value travels through Save Target Admission → Figure Plan → Figure/Table Decisions → Synthesis Bundle → Note Plan → Grounding Lint → Final Note Lint → Final Quality Review → Final Readability Review → Formal Save. Every artifact records it as `output_language`, and each script stops on a missing or mismatched value. Final Note Lint records `note_sha256`, so any edit after lint means rerunning it. Formal Save checks every binding before any save side effect. `references/output-language.md` describes the two profiles.

## Workflow

Run the steps in order. A step is complete at its **done when**, never because a partial artifact exists. At any point, `scripts/run_state.py status --workdir <run_directory> --prefix <run_prefix>` reports verified stages, stale artifacts, and the `next_action`; `references/run-state.md` explains how to bind the artifacts you write.

1. **Configuration Readiness.** Resolve Run Overrides from the explicit request, CLI, and process environment. Read User Configuration only for fields they leave unset; `references/user-configuration.md` covers admission, migration, repair, and Preference Changes. **Done when** the save mode, its target, and the output language are resolved and valid.
2. **Acquire.** `scripts/run_pipeline.py acquire --input <paper> --prefix <run_prefix>` resolves the identity, collects metadata, fetches the PDF, and extracts the raw source text, following Sources below. Without `--workdir` it picks a directory under `~/.deeppapernote/runs/` and prints `workdir=`; reuse that value in every later step, or pass a writable `--workdir` when a sandbox blocks the default. **Done when** the source manifest exists and is not `blocked`.
3. **Save Target Admission.** In Obsidian mode, run `scripts/write_obsidian_note.py --preflight` with the resolved title, `--language`, Vault, `--source-manifest`, and `--output <run_directory>/<run_prefix>_admission.json`. Its result is authoritative:

   | Result | Action |
   |---|---|
   | reuse of an existing directory | Use that directory and its domain. A verified PDF-only directory needs no confirmation. |
   | `same_language_note_exists` | Ask whether to overwrite the reported note, and say what its `user_notes` result means: `present` keeps the user's My Notes block (about `chars` characters) and regenerates everything else; `none` found no notes of the user's; `heading_missing` found no My Notes heading, so nothing can be kept. When `legacy_format` is true, add that the kept block predates the current note format and may be old model-written text. Edits elsewhere in the note are replaced. Carry its `existing_read_date`, when non-empty, into the new note's `read_date`. On yes, rerun with `--overwrite-existing-note --expected-existing-note-sha256 <reported_sha256>` and carry it to Formal Save. On no, stop without writing. |
   | several matching directories or sources | Ask for the selection as `references/paper-archive.md` describes, and carry it through Formal Save. |
   | `domain_required`, `new_domain_reason_required`, or `domain_name_invalid` | Choose the domain from the receipt's `domain_overview` as `references/domain-routing.md` describes, then rerun with `--domain`, adding `--new-domain-reason` for a new domain. Carry both to Formal Save. |
   | any other conflict | Report it and stop. |

   Workspace mode runs the same preflight with `--save-mode workspace` only to choose the domain. **Done when** an admitted result with its `domain` is saved.
4. **Prepare.** `scripts/run_pipeline.py prepare --workdir <run_directory> --prefix <run_prefix> --admission <admission.json>` (workspace mode omits `--admission`) extracts structure and PDF assets, plans figures, builds the complete figure/table decision table, and writes the synthesis bundle, all offline. It refuses to replace decisions that hold Visual Review work unless the user accepts repeating those reviews and you pass `--discard-reviews`. Rerun a failed phase whole. **Done when** the bundle exists.
5. **Visual Review and Figure/Table Decision Freeze.** Record a verdict for every `review_pending` item with `scripts/plan_figure_table_decisions.py review`, send any Figure/Table the note needs into review with `promote`, and drop a usable candidate the note has no use for with a recorded reason, as `references/figure-placement.md` describes. Every decision must satisfy `writing_contract.figure_table_contract`. **Done when** `scripts/plan_figure_table_decisions.py summary --decisions <decisions.json>` lists no `review_pending_labels`; its table gives each insert's embed.
6. **Note plan.** Read the bundle and the raw sections, choose the paper type as `references/paper-types.md` describes, and fill every value of `writing_contract.note_plan_contract.template` to create a short JSON `note_plan` with the exact `output_language`. Set `reader_research_direction` to the research direction the request states, verbatim; when the request states none and `research_memory.status` is `ok`, copy `research_memory.suggested_reader_research_direction`; otherwise leave it empty. Write `storyline` as three to five sentences that tell this paper's own story the way the authors build it, from the problem through the idea and the evidence to what it means; it is the outline the middle sections follow, not text for the note. Fill `key_prior_work` with the three to five works this paper builds on, and put in `connections` only those `connection_candidates` from the bundle that truly relate to this paper, each with a relation key and a one-sentence reason. `references/evidence-first.md` shows a complete plan. **Done when** Grounding passes.
7. **Grounding.** `scripts/lint_grounding.py --note-plan <note_plan> --source-manifest <manifest> --bundle-json <bundle> --figure-decisions <decisions.json> --language <language> --output <grounding.json>`. **Done when** it reports no issue; keep its JSON for Formal Save.
8. **Draft.** Write the note from the plan, following Note Shape, as one Markdown file in UTF-8 without BOM and with LF line endings. **Done when** the complete draft is on disk.
9. **Final Note Lint.** `scripts/lint_note.py --input <note> --plan-file <note_plan> --language <language> --output <lint.json>`. **Done when** every reported `passes_*` gate is `true`. A failed style gate is normal for several rounds: fix the reported text and rerun. Each `zh-CN` mixed-language issue names the English it counted in `english_run` or `english_words`; translate that wording, or wrap a retained proper noun in backticks, keeping formulas inline and proper nouns intact. Escalate to the user only when the same failures repeat unchanged across several rounds.
10. **Final Quality Review and Final Readability Review.** Follow `references/final-quality-review.md` for the reviewer prompt, the recorded `final_quality_review`, the two-revision limit, and the `final_readability_review` receipt. A readability edit repeats Final Note Lint and a fresh quality review. **Done when** both reviews pass for the current note bytes.
11. **Formal Save.** Run `scripts/write_obsidian_note.py` with `--content-file`, `--plan-file`, `--lint-json`, `--grounding-lint-json`, `--source-manifest`, `--figure-decisions`, `--final-quality-review`, `--final-readability-review`, the admitted `--domain` and `--new-domain-reason`, and any approved overwrite hash, each naming the exact file the linters and reviewers checked. It repeats admission, refuses missing, failed, or stale gates, refuses unresolvable `[[links]]`, and materializes each `insert` into the paper-local `images/` directory. **Done when** the save succeeds.

Both lint artifacts bind the plan through `note_plan_sha256`, and Grounding Lint also binds the source manifest and decisions. These are exact-byte hashes, so reformatting JSON invalidates them: rerun both linters after a plan change, Grounding Lint after a source or decision change, and Final Note Lint after a note edit.

| Save Target | Formal Save |
|---|---|
| `save_mode=obsidian` with a usable Vault | Save to that Vault. |
| `save_mode=obsidian` and the save fails | Keep the Vault target and recover within scope; otherwise report `blocked`, and do not switch to workspace. |
| `save_mode=workspace` | Save inside the workspace output root. |

Running `scripts/run_pipeline.py` without a phase performs both phases in workspace mode only; Obsidian mode needs Save Target Admission between them.

A normal request finishes in one run: text, figure decisions, image materialization, and save. Produce a text-only note only when the user asks for one.

## Sources

Prefer the strongest source in this order:

1. local PDF path given by the user
2. local Zotero item and local Zotero attachment if available
3. verified local Obsidian PDF for the accepted work and requested version
4. DOI and publisher metadata
5. arXiv or open-access PDF sources
6. Semantic Scholar or OpenAlex for metadata backfill

`acquire` checks the desktop library through the Zotero Local API first. Its `--zotero-mode` is `auto` by default and falls back to the web providers when Zotero has no match; `off` makes no Local API request, and `required` demands a Zotero match. An explicit Zotero key must resolve locally. A trusted JSON artifact or explicit local PDF stays authoritative and skips this lookup. A confident local-library hit outranks any title-only web match. When a library match exposes only an attachment key and filename, `scripts/locate_zotero_attachment.py` finds the PDF under Zotero storage. In Obsidian mode, a matching local archive is verified and reused before any download; `references/paper-archive.md` covers version and source selection and Connector handoff.

## Stop Conditions

Stop, report the blocked stage and the remaining stages, and ask for what would unblock it when:

- the paper identity is ambiguous, including several equally plausible local items: ask for a DOI, arXiv ID, or exact Zotero key
- no PDF is available after every supported acquisition path: local PDF, Zotero attachment, metadata `pdf_url`, direct PDF URL, arXiv or open-access sources, an accessible publisher PDF, DOI enrichment, and any other implemented fetch path
- the source manifest is `blocked` with `blocked_code: unreadable_text`: ask for an OCR version or another copy, since rerunning cannot fix it
- the PDF or evidence is too weak for a real deep note: ask for a better PDF, OCR, or other source material

When any other step fails, retry it or take a fallback this skill names. Otherwise stop as above. An abstract-only, provisional, or partial note is never the finished output.

## Note Shape

- Start with an Obsidian YAML properties block above the `#` title heading. Include `tags` with exactly one `papers/<domain>` value, the admitted `domain`, and useful `aliases`; add `date`, `doi`, or `arxiv_id` when known, and omit unknown fields. Always include the fields of `writing_contract.frontmatter_contract`: `paper_type` matching the plan, `methods`, `datasets`, and `tasks` as lists of at most 5 central items, and `read_date` as `YYYY-MM-DD`. A list value that names a term in the bundle's `glossary_terms` is written as that term's `link`, copied verbatim; every other value is plain text. Formal Save refuses a frontmatter link that is not a glossary term note.
- Use real `#`, `##`, and `###` headings. The top-level sections are exactly `writing_contract.section_order`: the fixed head, the middle sections of `contracts_by_paper_type[paper_type]`, then the fixed tail, in that order. `references/paper-types.md` explains how to choose the type, answer its core questions, and write the weak-evidence callout. A non-trivial paper gets several meaningful `###` subsections in its middle sections.
- Core Information is a fixed metadata block holding only `writing_contract.core_info_fields`, in order. Analysis belongs in later sections.
- Abstract renders the original abstract faithfully in the configured language, translated in `zh-CN` mode, with no later evidence or judgment added.
- Contributions lists the paper's actual innovations and why each matters.
- A paper type with a `workflow` gets that subsection in its host section: an overview chain, then a step table that mirrors `note_plan.workflow` (`references/paper-types.md`).
- Write the middle sections in order, following `storyline`: each section picks up where the previous one left off, and a paragraph that serves no part of the story is cut. Within a section, write ordinary paragraphs, one point each, conclusion first and details after. Explain each term or acronym in one plain sentence the first time it appears, and keep an established English name rather than translating it word for word.
- Keep the numbers that carry a conclusion in the prose and move the other results into tables, so every main experiment stays in the note. A comparison of three or more systems, settings, datasets, metrics, or ablation rows becomes a compact Markdown table with a sentence saying what it shows; tables are welcome wherever they make the comparison clearer. Formulas follow the type's `formula_rules`, as `$...$` or `$$...$$`.
- Takeaways holds exactly the two subsections of `writing_contract.takeaways_contract`, one or two items each, and each item is one or two sentences: reusable ideas that name the paper's mechanism, where it transfers, and its precondition, then follow-up questions or experiments specific to this paper, at least one of which tests the paper's own claims or evidence. With a non-empty `reader_research_direction`, connect at least one transferable idea to that direction whenever a credible transfer exists, start every item that connects to it with `takeaways_contract.judgment_marker`, and say plainly when no credible transfer exists.
- Connections follows `writing_contract.connections_contract`: the in-vault subsection lists exactly the planned `connections` as `[[target|Title]]` entries with their relation labels, or its fixed empty statement; the key-prior-work subsection lists the planned `key_prior_work` as plain text. Never write the bibliography, and never write a `[[link]]` whose target is not a bundle candidate. Prose outside Connections carries no `[[link]]`; paper-glossary links terms in the body after saving. Formal Save refuses links it cannot find in the vault, and any link in workspace mode. A recommended next paper must already appear here. `references/obsidian-format.md` shows the format.
- My Notes is the last section and belongs to the user. The draft holds exactly `writing_contract.my_notes_contract.guidance` there. Formal Save keeps the user's own block from the note it replaces.
- A short paper still gets a complete deep note: spend the space on protocol details, ablations, limitations, and replication implications.
- The readability pass checks the story holds together, every term is explained on first use, and each paragraph leads with its point; it rewrites leftover language into natural prose in the configured language and keeps stable proper nouns. It adds no new facts or filler.

`references/final-writing.md` and `references/obsidian-format.md` hold the full writing and Markdown rules.

## Reporting

Say each phrase only when its condition holds:

| Phrase | Condition |
|---|---|
| `笔记已完成` | The workflow is complete through Formal Save. |
| `已生成草稿` | The draft exists, but lint, a review, or the save is pending. |
| `已通过校验` | Final Note Lint ran and passed. |
| `已保存到 Obsidian` | Formal Save succeeded. |

Lint is a floor, not polish: while the readability review is pending, say the draft passed script lint but has not finished its final language review. When a run stops early, name the current stage and the remaining stages.

Final user report:

- After a successful Formal Save, report in the user's conversation language, and derive every claim from current-run artifacts; the `summary` of `scripts/run_state.py status` carries these fields.
- Lead with the final note link or path, the save mode, and the actual saved domain, read from the note path under the configured papers root in Obsidian mode or the output root in workspace mode. When admission reused an existing directory, report that directory's existing domain. When the save receipt's `new_domain` is true, say that this domain folder was created, give its `new_domain_reason`, and suggest the organize-wiki skill to add the domain to the user's convention.
- Then report, in order:
  1. the paper title and strongest verified identifier
  2. the Grounding Lint, Final Note Lint, Final Quality Review, and Final Readability Review results, plus the warning count
  3. the materialized and retained-placeholder figure/table counts, plus each dropped item with its reason from `figure_drops`
  4. whether the generated part of the saved note matches the Final Note Lint `note_sha256`
- When `reader_research_direction` came from `research_memory`, say in one line that the transferable ideas followed the user's research memory.
- When the save receipt's `glossary_followup.status` is `suggested`, suggest running paper-glossary on this paper and give its `source_manifest` and `note_path`; when `regenerated` is true, add that term links the glossary had added to the replaced note return only after that rerun. For any other status, say nothing about the glossary.
- When the save replaced an existing note, report the save receipt's `user_notes.status`: `preserved` kept the user's My Notes; `untouched` found none to keep; `heading_missing` found no My Notes heading. Only when `user_notes.legacy_format` is true, add that the kept block predates the current note format and may be old model-written text the user can delete.
- When a save receipt `replaced_images` entry lists other notes in `also_embedded_by`, say in one line that the new crop of that figure also shows in those notes.
- Add overwrite actions, preference changes, or user-relevant warnings only when they occurred.

## References

`SKILL.md` and the synthesis bundle carry a normal run. Read a reference only when its row applies:

| Reference | When |
|---|---|
| `references/user-configuration.md` | Configuration Readiness |
| `references/paper-archive.md` | existing Obsidian directories, local PDF versions, Connector handoff |
| `references/domain-routing.md` | choosing the domain folder, the `papers/` tag, the domains file |
| `references/figure-placement.md` | Visual Review, promotion, drops, and placement |
| `references/run-state.md` | reading `run_state.py status` and binding artifacts |
| `references/final-quality-review.md` | Final Quality Review and Final Readability Review |
| `references/output-language.md` | language profiles, while drafting or debugging either language |
| `references/evidence-first.md` | a complete note plan example |
| `references/deep-analysis.md`, `references/final-writing.md` | a complex paper or a draft that is too shallow |
| `references/obsidian-format.md` | Markdown, vault, frontmatter, and Connections formatting |
| `references/paper-types.md` | choosing the paper type, core answers, workflow, weak-evidence callout |
| `references/note-quality.md` | final review |
| `references/metadata-sources.md` | incomplete metadata |
| `references/architecture.md` | repository maintenance |

## Environment

- The scripts need Python 3.10 or newer. When `python3` is older, use another interpreter of 3.10 or newer, such as `python3.12`, for the whole run; when none exists, stop and report the version found.
- Run `scripts/check_environment.py` only when a concrete dependency or integration problem blocks a step, and explain the required dependencies, optional enhancements, and downgrade behavior directly.
```

## File: skills/deeppapernote/scripts/_domain_routing.py
size: 8349
sha: 073e168aabe67202b3c543147b48f1ceccb4a0fc
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_domain_routing.py

```text
"""Domain routing: the overview the model reads and the checks on the domain it chooses.

The model picks the domain folder for a new paper directory. This module only
describes the existing domains and validates the choice; it never picks one.
"""

from __future__ import annotations

import json
import re
from pathlib import Path
from typing import Any

from _frontmatter import FrontmatterError, frontmatter_block, parse_frontmatter
from _note_paths import existing_domain_dirs, resolve_note_output_mode, safe_relative_path_parts

DOMAINS_FILE = Path("DeepPaperNote") / "domains.md"
SAMPLE_TITLES = 8
TAG_PREFIX = "papers/"
FIELD_KEYS = {
    "定义": "definition",
    "definition": "definition",
    "收录": "includes",
    "includes": "includes",
    "不收录": "excludes",
    "excludes": "excludes",
    "代表论文": "representative_papers",
    "representative papers": "representative_papers",
}
DOMAIN_HEADING_RE = re.compile(r"^##[ \t]+(?!#)(.+?)[ \t]*#*[ \t]*$")
FIELD_RE = re.compile(r"^[ \t]*[-*][ \t]+([^:：]+?)[ \t]*[:：][ \t]*(.*?)[ \t]*$")
NEW_DOMAIN_NAME_RE = re.compile(r"[\w-]+")
DOMAIN_CHOICE_CODES = frozenset(
    {"domain_required", "domain_name_invalid", "new_domain_reason_required"}
)

class DomainChoiceError(Exception):
    def __init__(self, code: str, message: str):
        super().__init__(message)
        self.code = code
        self.message = message

def domain_locations(config: dict[str, Any]) -> tuple[Path, Path | None]:
    """The folder that holds the domains, and the domains file (Obsidian mode only)."""
    output_mode, root = resolve_note_output_mode(config)
    if output_mode != "obsidian":
        return root, None
    papers_dir = str(config.get("papers_dir", "Research/Papers")).strip() or "Research/Papers"
    return root / Path(*safe_relative_path_parts(papers_dir, "papers_dir")), root / DOMAINS_FILE

def parse_domains_file(text: str) -> dict[str, dict[str, str]]:
    """Each `##` heading names a domain; `- Label: text` lines under it fill its fields."""
    text = text.removeprefix("﻿").replace("\r\n", "\n")
    try:
        block = frontmatter_block(text)
    except FrontmatterError:
        block = None
    if block is not None:
        text = text[4 + len(block):].partition("\n")[2]
    domains: dict[str, dict[str, str]] = {}
    current: dict[str, str] | None = None
    for line in text.split("\n"):
        heading = DOMAIN_HEADING_RE.match(line)
        if heading:
            current = domains.setdefault(heading.group(1).strip(), {})
            continue
        if line.startswith("#"):
            current = None
            continue
        field = FIELD_RE.match(line) if current is not None else None
        if field:
            key = FIELD_KEYS.get(field.group(1).strip().lower())
            if key and field.group(2) and key not in current:
                current[key] = field.group(2)
    return domains

def read_domains_file(path: Path | None) -> dict[str, dict[str, str]] | None:
    if path is None or not path.is_file():
        return None
    try:
        return parse_domains_file(path.read_text(encoding="utf-8", errors="replace"))
    except OSError:
        return None

def paper_folders(domain_dir: Path) -> list[Path]:
    if not domain_dir.is_dir():
        return []
    return [
        child for child in sorted(domain_dir.iterdir())
        if child.is_dir() and not child.name.startswith(".")
    ]

def paper_title(folder: Path) -> str:
    """The sidecar title, else a note's `title` property or H1, else the folder name."""
    try:
        title = json.loads((folder / ".deeppapernote.json").read_text(encoding="utf-8"))["title"]
        if isinstance(title, str) and title.strip():
            return title.strip()
    except (OSError, ValueError, KeyError, TypeError):
        pass
    for note in sorted(folder.glob("*.md")):
        try:
            text = note.read_text(encoding="utf-8", errors="replace")[:4000]
        except OSError:
            continue
        try:
            fields = parse_frontmatter(text) or {}
        except FrontmatterError:
            fields = {}
        title = fields.get("title")
        if isinstance(title, str) and title.strip():
            return title.strip()
        heading = re.search(r"^# (.+)$", text, flags=re.MULTILINE)
        if heading:
            return heading.group(1).strip()
    return folder.name.replace("_", " ")

def domain_overview(config: dict[str, Any]) -> dict[str, Any]:
    """What the model reads before choosing: every domain with its definition or sample titles."""
    base, domains_file = domain_locations(config)
    convention = read_domains_file(domains_file)
    on_disk = existing_domain_dirs(config)
    domains = []
    for name in sorted(set(on_disk) | set(convention or {})):
        folders = paper_folders(base / name) if name in on_disk else []
        entry: dict[str, Any] = {
            "name": name,
            "paper_count": len(folders),
            "on_disk": name in on_disk,
            "in_convention": name in (convention or {}),
        }
        if convention and name in convention:
            entry.update(convention[name])
        else:
            entry["sample_titles"] = [paper_title(folder) for folder in folders[:SAMPLE_TITLES]]
        domains.append(entry)
    return {
        "source": "folders" if convention is None else "domains_file",
        "domains_file": None if convention is None else str(domains_file),
        "domains": domains,
        "guide": "references/domain-routing.md",
    }

def resolve_domain_choice(
    config: dict[str, Any], domain: str, new_domain_reason: str = ""
) -> tuple[str, bool]:
    """Return (domain, is_new) for the model's choice, or raise DomainChoiceError."""
    name = (domain or "").strip()
    if not name:
        raise DomainChoiceError(
            "domain_required",
            "Choose a domain from domain_overview and rerun with --domain.",
        )
    if (
        "/" in name or "\\" in name or name.startswith(".")
        or any(ord(char) < 32 for char in name)
    ):
        raise DomainChoiceError(
            "domain_name_invalid", f"A domain is one folder name inside the papers root: {name}"
        )
    base, domains_file = domain_locations(config)
    known = set(existing_domain_dirs(config)) | set(read_domains_file(domains_file) or {})
    for existing in sorted(known):
        if existing.casefold() == name.casefold():
            return existing, False
    if (base / name).exists():
        raise DomainChoiceError(
            "domain_name_invalid", f"{name} already exists and is not a domain folder."
        )
    if not NEW_DOMAIN_NAME_RE.fullmatch(name) or name.isdigit():
        raise DomainChoiceError(
            "domain_name_invalid",
            f"A new domain name must work as a tag: letters, digits, _ and - only: {name}",
        )
    if not (new_domain_reason or "").strip():
        raise DomainChoiceError(
            "new_domain_reason_required",
            f"{name} is a new domain; pass --new-domain-reason saying why no existing domain fits.",
        )
    return name, True

def saved_domain(config: dict[str, Any], note_path: Path) -> str:
    """The domain a note is saved under: its paper folder's parent below the papers root."""
    base, _ = domain_locations(config)
    parent = Path(note_path).parent.parent
    try:
        parts = parent.resolve().relative_to(base.resolve()).parts
    except ValueError:
        parts = ()
    return "/".join(parts) or parent.name

def domain_tag(domain: str) -> str:
    return TAG_PREFIX + re.sub(r"\s+", "_", domain.strip())

def domain_tag_issue(text: str, domain: str) -> str:
    """Empty when the note carries exactly one papers/ tag and it names the saved domain."""
    try:
        fields = parse_frontmatter(text) or {}
    except FrontmatterError:
        fields = {}
    tags = fields.get("tags")
    values = tags if isinstance(tags, list) else [tags] if isinstance(tags, str) else []
    found = [tag.strip() for tag in values if tag.strip().startswith(TAG_PREFIX)]
    expected = domain_tag(domain)
    if found == [expected]:
        return ""
    return (
        f"The note must carry exactly one {TAG_PREFIX} tag, {expected}, matching the saved "
        f"domain {domain}; found {', '.join(found) or 'none'}."
    )
```

## File: skills/deeppapernote/scripts/_evidence_heuristics.py
size: 8271
sha: a8fbfd8e40562b51c839cbf1f0685d53613416bf
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_evidence_heuristics.py

```text
"""Keyword heuristics that pick candidate evidence sentences from paper text."""

from __future__ import annotations

import re

from _pdf_text import split_sentences
from common import normalize_title, normalize_whitespace

def infer_paper_type(title: str, abstract: str) -> tuple[str, str]:
    lower = f"{title} {abstract}".lower()
    survey_terms = [
        "survey",
        "tutorial",
        "overview",
        "systematic review",
        "literature review",
        "scoping review",
        "meta-analysis",
        "meta analysis",
    ]
    if any(re.search(rf"(?<![a-z0-9]){re.escape(token)}(?![a-z0-9])", lower) for token in survey_terms):
        return "survey_or_review", "The paper is an explicit survey, review, tutorial, overview, or meta-analysis."
    if title.strip().lower().startswith("position:"):
        return "position", "The title declares a position paper."
    if any(token in lower for token in ["depression", "anxiety", "mental health", "clinical", "patient", "psychiatric", "psychological", "hospital"]):
        return "human_subjects_empirical", "The paper is closer to an empirical study of human participants."
    if any(token in lower for token in ["semi-structured interview", "focus group", "thematic analysis", "ethnograph", "grounded theory", "qualitative study"]):
        return "qualitative_empirical", "The paper reports qualitative fieldwork or thematic analysis."
    if any(token in lower for token in ["humanities", "theoretical framing", "interpretive", "conceptual analysis", "argument structure", "philosoph"]):
        return "humanities_argument", "The paper emphasizes interpretation or conceptual argument."
    if any(token in lower for token in ["benchmark", "leaderboard", "evaluation suite", "dataset", "corpus"]):
        return "benchmark_or_dataset", "The paper emphasizes benchmark, dataset, or evaluation design."
    if any(token in lower for token in ["we prove", "theorem", "lower bound", "upper bound", "sample complexity"]):
        return "theory", "The paper centers on proved results."
    return "method", "The paper is best treated as a method-focused technical paper."

def extract_dataset_candidates(text: str) -> list[str]:
    found: list[str] = []
    seen = set()
    for sentence in split_sentences(text):
        if not any(token in sentence.lower() for token in ["dataset", "benchmark", "corpus", "participants", "patients"]):
            continue
        candidates = re.findall(r"\b[A-Z][A-Za-z0-9+\-]{2,}(?:[ -][A-Z][A-Za-z0-9+\-]{2,})?\b", sentence)
        for candidate in candidates:
            norm = candidate.lower()
            if norm in seen:
                continue
            seen.add(norm)
            found.append(candidate)
            if len(found) >= 8:
                return found
    return found

def extract_metric_claims(text: str) -> list[str]:
    claims: list[str] = []
    seen = set()
    for sentence in split_sentences(text):
        lower = sentence.lower()
        if not re.search(r"\d", sentence):
            continue
        if not any(token in lower for token in ["accuracy", "f1", "auc", "auprc", "mae", "rmse", "score", "%", "outperform", "improv", "bac"]):
            continue
        normalized = normalize_whitespace(sentence)
        key = normalize_title(normalized)
        if key in seen:
            continue
        seen.add(key)
        claims.append(normalized)
        if len(claims) >= 8:
            break
    return claims

def extract_negative_claims(text: str, *, limit: int = 6) -> list[str]:
    claims: list[str] = []
    seen = set()
    explicit_negative_tokens = [
        "worse",
        "degrade",
        "degraded",
        "drop",
        "dropped",
        "decrease",
        "decreased",
        "unstable",
        "instability",
        "fail",
        "failed",
        "fails",
        "collapse",
        "collapsed",
        "underperform",
        "underperformed",
        "sensitive",
        "sensitivity",
        "trade-off",
        "tradeoff",
        "hurt performance",
        "hurts performance",
    ]
    ablation_tokens = [
        "without",
        "w/o",
        "remove",
        "removed",
        "removing",
        "omit",
        "omits",
        "omitted",
        "omitting",
        "ablation",
    ]
    performance_tokens = [
        "accuracy",
        "f1",
        "auc",
        "auprc",
        "mae",
        "rmse",
        "score",
        "performance",
        "%",
        "result",
        "results",
        "training",
        "converge",
        "convergence",
        "stable",
        "stability",
        "baseline",
    ]
    positive_only_tokens = [
        "outperform",
        "improv",
        "better than",
        "achieve",
        "state-of-the-art",
        "sota",
    ]

    for sentence in split_sentences(text):
        normalized = normalize_whitespace(sentence)
        if not normalized:
            continue
        lower = normalized.lower()
        has_explicit_negative = any(token in lower for token in explicit_negative_tokens)
        has_ablation_marker = any(token in lower for token in ablation_tokens)
        has_performance_context = any(token in lower for token in performance_tokens) or bool(re.search(r"\d", normalized))
        has_positive_only = any(token in lower for token in positive_only_tokens)

        if not has_explicit_negative:
            if not (has_ablation_marker and has_performance_context):
                continue
            if has_positive_only and not any(token in lower for token in ["trade-off", "tradeoff", "sensitive", "stability"]):
                continue

        key = normalize_title(normalized)
        if key in seen:
            continue
        seen.add(key)
        claims.append(normalized)
        if len(claims) >= limit:
            break
    return claims

def extract_mechanism_flow_sentences(text: str, *, limit: int = 8) -> list[str]:
    claims: list[str] = []
    seen = set()
    action_tokens = [
        "encode",
        "encoded",
        "encoding",
        "extract",
        "extracted",
        "project",
        "projected",
        "pool",
        "pooled",
        "fuse",
        "fused",
        "fusion",
        "concat",
        "concatenate",
        "query",
        "queried",
        "align",
        "aligned",
        "compress",
        "compressed",
        "send",
        "sent",
        "feed",
        "fed",
        "generate",
        "generated",
        "predict",
        "predicted",
        "decode",
        "decoded",
        "update",
        "updated",
        "freeze",
        "frozen",
        "fine-tune",
        "finetune",
    ]
    flow_tokens = [
        "input",
        "inputs",
        "output",
        "outputs",
        "token",
        "tokens",
        "feature",
        "features",
        "representation",
        "representations",
        "encoder",
        "decoder",
        "attention",
        "module",
        "modules",
        "llm",
        "language model",
        "query token",
        "cross-attention",
        "projection",
        "state",
        "states",
    ]

    for sentence in split_sentences(text):
        normalized = normalize_whitespace(sentence)
        if not normalized:
            continue
        lower = normalized.lower()
        if not any(token in lower for token in action_tokens):
            continue
        if not any(token in lower for token in flow_tokens):
            continue
        key = normalize_title(normalized)
        if key in seen:
            continue
        seen.add(key)
        claims.append(normalized)
        if len(claims) >= limit:
            break
    return claims

def pick_sentences_by_keywords(text: str, keywords: list[str], *, limit: int = 5) -> list[str]:
    picked: list[str] = []
    seen = set()
    for sentence in split_sentences(text):
        lower = sentence.lower()
        if not any(keyword in lower for keyword in keywords):
            continue
        normalized = normalize_title(sentence)
        if normalized in seen:
            continue
        seen.add(normalized)
        picked.append(normalize_whitespace(sentence))
        if len(picked) >= limit:
            break
    return picked
```

## File: skills/deeppapernote/scripts/_final_quality.py
size: 12658
sha: a69bba2ab754a4b91d6701bbca31f1dd7b7deb75
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_final_quality.py

```text
from __future__ import annotations

import hashlib
import json
import math
import re

# Version 2 added the takeaways check; version 3 added the connections check; version 4
# added the core_questions and narrative checks. The readability receipt and the revision
# journal kept their shape and stay at version 1.
QUALITY_REVIEW_SCHEMA_VERSION = 4
RECEIPT_SCHEMA_VERSION = 1
QUALITY_CHECKS = (
    "central_claims", "mechanism_result_map", "claim_boundaries", "takeaways", "connections",
    "core_questions", "narrative",
)
# Plan lists each check reviews; a blocking finding must point at an item of one of them.
QUALITY_CHECK_PLAN_FIELDS = {
    "central_claims": ("central_claims",),
    "mechanism_result_map": ("mechanism_result_map",),
    "claim_boundaries": ("claim_boundaries",),
    "takeaways": ("reuse_takeaways", "followup_questions"),
    "connections": ("connections", "key_prior_work"),
}
# The two v4 checks review plan fields that are not lists: core_answers is keyed by
# question id, and storyline is one string.
QUESTION_REF_RE = re.compile(r"/core_answers/([a-z_]+\.[a-z_]+)")
STORYLINE_REF = "/storyline"
# A vault may hold no related notes; the connections check then reviews key_prior_work alone.
MAY_BE_EMPTY_PLAN_FIELDS = frozenset({"connections"})
SHA256_RE = re.compile(r"[0-9a-f]{64}")
PLAN_REF_RE = re.compile(
    r"/(central_claims|mechanism_result_map|claim_boundaries|reuse_takeaways|followup_questions"
    r"|connections|key_prior_work)"
    r"/(0|[1-9][0-9]*)(/what_it_does_not_prove)?"
)

def strict_json_loads(text: str) -> dict:
    def object_pairs(pairs: list[tuple[str, object]]) -> dict:
        result = {}
        for key, value in pairs:
            if key in result:
                raise ValueError(f"Duplicate JSON field: {key}")
            result[key] = value
        return result

    def reject_constant(value: str) -> None:
        raise ValueError(f"Nonfinite JSON value: {value}")

    def finite_float(value: str) -> float:
        result = float(value)
        if not math.isfinite(result):
            reject_constant(value)
        return result

    value = json.loads(
        text,
        object_pairs_hook=object_pairs,
        parse_constant=reject_constant,
        parse_float=finite_float,
    )
    if not isinstance(value, dict):
        raise ValueError("Final review artifact must be a JSON object.")
    return value

def canonical_note_text(raw: bytes) -> str:
    if raw.startswith(b"\xef\xbb\xbf") or b"\r" in raw:
        raise ValueError(
            "Final draft must be canonical UTF-8 without BOM and with LF line endings."
        )
    try:
        return raw.decode("utf-8")
    except UnicodeDecodeError as exc:
        raise ValueError("Final draft must be canonical UTF-8.") from exc

def _keys(value: object, keys: set[str], name: str) -> None:
    if not isinstance(value, dict) or set(value) != keys:
        raise ValueError(f"{name} requires exactly these fields: {', '.join(sorted(keys))}.")

def _version_language(
    value: dict, language: str, name: str, *, version: int = RECEIPT_SCHEMA_VERSION, hint: str = ""
) -> None:
    if type(value["schema_version"]) is not int or value["schema_version"] != version:
        raise ValueError(f"{name} requires schema_version {version}.{hint}")
    if language not in {"en", "zh-CN"} or value["output_language"] != language:
        raise ValueError(f"{name} output_language does not match {language}.")

def _sha(value: object, name: str, *, empty: bool = False) -> None:
    if empty and value == "":
        return
    if not isinstance(value, str) or SHA256_RE.fullmatch(value) is None:
        raise ValueError(f"{name} requires a lowercase SHA-256.")

def _valid_plan_ref(reference: str, check: str, plan: dict) -> bool:
    if check == "core_questions":
        match = QUESTION_REF_RE.fullmatch(reference)
        return match is not None and match.group(1) in plan.get("core_answers", {})
    if check == "narrative":
        return reference == STORYLINE_REF
    match = PLAN_REF_RE.fullmatch(reference)
    if match is None:
        return False
    group, index, suffix = match.groups()
    if suffix:
        if check != "claim_boundaries" or group != "central_claims":
            return False
    elif group not in QUALITY_CHECK_PLAN_FIELDS[check]:
        return False
    items = plan.get(group)
    if not isinstance(items, list) or int(index) >= len(items):
        return False
    return not suffix or (
        isinstance(items[int(index)], dict) and "what_it_does_not_prove" in items[int(index)]
    )

def validate_quality(
    review: dict,
    *,
    note_sha256: str,
    plan_sha256: str,
    plan: dict,
    language: str,
) -> str:
    _keys(
        review,
        {
            "schema_version",
            "output_language",
            "verdict",
            "reviewed_note_sha256",
            "reviewed_plan_sha256",
            "checks",
            "blocking_findings",
        },
        "Final Quality Review",
    )
    _version_language(
        review, language, "Final Quality Review",
        version=QUALITY_REVIEW_SCHEMA_VERSION,
        hint=" A review written before the core_questions and narrative checks is stale; rerun the full review.",
    )
    for key, expected in (
        ("reviewed_note_sha256", note_sha256),
        ("reviewed_plan_sha256", plan_sha256),
    ):
        _sha(review[key], key)
        if review[key] != expected:
            raise ValueError(f"Final Quality Review {key} is stale; rerun the full review.")
    if (
        not isinstance(plan, dict)
        or plan.get("output_language") != language
        or any(
            not isinstance(plan.get(field), list)
            or (not plan[field] and field not in MAY_BE_EMPTY_PLAN_FIELDS)
            for fields in QUALITY_CHECK_PLAN_FIELDS.values()
            for field in fields
        )
        or not isinstance(plan.get("core_answers"), dict)
        or not plan["core_answers"]
        or not isinstance(plan.get("storyline"), str)
        or not plan["storyline"].strip()
    ):
        raise ValueError(
            "Note Plan requires the current language, nonempty quality-check lists, "
            "core_answers, and a storyline."
        )
    _keys(review["checks"], set(QUALITY_CHECKS), "Final Quality Review checks")
    if any(type(value) is not bool for value in review["checks"].values()):
        raise ValueError("Final Quality Review checks must be literal booleans.")
    if review["verdict"] not in ("pass", "fail"):
        raise ValueError("Final Quality Review verdict must be pass or fail.")
    findings = review["blocking_findings"]
    if not isinstance(findings, list):
        raise ValueError("Final Quality Review blocking_findings must be an array.")
    false_checks = {key for key, value in review["checks"].items() if not value}
    covered = set()
    for finding in findings:
        _keys(finding, {"check", "plan_ref", "reason", "required_revision"}, "Blocking finding")
        if any(not isinstance(value, str) or not value.strip() for value in finding.values()):
            raise ValueError("Blocking finding fields must be nonempty strings.")
        check = finding["check"]
        if check not in false_checks or not _valid_plan_ref(finding["plan_ref"], check, plan):
            raise ValueError("Blocking finding must reference an existing item of a false check.")
        covered.add(check)
    if (review["verdict"] == "pass" and (false_checks or findings)) or (
        review["verdict"] == "fail" and (not false_checks or covered != false_checks)
    ):
        raise ValueError("Final Quality Review verdict, checks, and findings are inconsistent.")
    return hashlib.sha256(
        json.dumps(review, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode(
            "utf-8"
        )
    ).hexdigest()

def validate_readability(receipt: dict, *, note_sha256: str, language: str) -> None:
    _keys(
        receipt,
        {
            "schema_version",
            "output_language",
            "verdict",
            "reviewed_note_sha256",
        },
        "Final Readability Review",
    )
    _version_language(receipt, language, "Final Readability Review")
    _sha(receipt["reviewed_note_sha256"], "reviewed_note_sha256")
    if receipt["verdict"] != "pass" or receipt["reviewed_note_sha256"] != note_sha256:
        raise ValueError("Final Readability Review must pass for the current final note bytes.")

def validate_state(state: dict, *, language: str) -> None:
    _keys(
        state,
        {
            "schema_version",
            "output_language",
            "revisions_used",
            "last_review",
            "pending_note_sha256",
            "exhausted_note_sha256",
            "seen_review_ids",
        },
        "Final quality state",
    )
    _version_language(state, language, "Final quality state")
    if type(state["revisions_used"]) is not int or not 0 <= state["revisions_used"] <= 2:
        raise ValueError("Final quality state revision count must be between zero and two.")
    last = state["last_review"]
    _keys(last, {"review_id", "note_sha256", "plan_sha256", "verdict"}, "Recorded review")
    for key in ("review_id", "note_sha256", "plan_sha256"):
        _sha(last[key], key)
    if last["verdict"] not in ("pass", "fail"):
        raise ValueError("Recorded review verdict must be pass or fail.")
    seen = state["seen_review_ids"]
    if not isinstance(seen, list) or not seen:
        raise ValueError("Final quality state requires seen review identities.")
    for identity in seen:
        _sha(identity, "seen review identity")
    if len(set(seen)) != len(seen) or last["review_id"] not in seen:
        raise ValueError("Final quality state review identities are inconsistent.")
    for key in ("pending_note_sha256", "exhausted_note_sha256"):
        _sha(state[key], key, empty=True)
    pending, exhausted = state["pending_note_sha256"], state["exhausted_note_sha256"]
    if (pending and pending != last["note_sha256"]) or (last["verdict"] == "fail" and not pending):
        raise ValueError("Final quality state pending revision is inconsistent.")
    if exhausted and (state["revisions_used"] != 2 or exhausted != pending):
        raise ValueError("Final quality state exhaustion is inconsistent.")
    if last["verdict"] == "fail" and state["revisions_used"] == 2 and not exhausted:
        raise ValueError("Final quality state must retain its exhausted revision anchor.")

def advance_state(state: dict | None, review: dict, review_id: str, *, language: str) -> dict:
    if state is None:
        state = {
            "schema_version": 1,
            "output_language": language,
            "revisions_used": 0,
            "pending_note_sha256": "",
            "exhausted_note_sha256": "",
            "seen_review_ids": [],
        }
    else:
        validate_state(state, language=language)
        if review_id in state["seen_review_ids"]:
            if review_id != state["last_review"]["review_id"]:
                raise ValueError("Old review replay cannot restore a previous passing observation.")
            return state
    result = {**state, "seen_review_ids": [*state["seen_review_ids"], review_id]}
    note = review["reviewed_note_sha256"]
    if result["exhausted_note_sha256"] and note != result["exhausted_note_sha256"]:
        raise ValueError("Final quality revision limit exhausted; a third revision is not allowed.")
    if result["pending_note_sha256"] and note != result["pending_note_sha256"]:
        if result["revisions_used"] == 2:
            raise ValueError("Final quality revision limit exhausted.")
        result["revisions_used"] += 1
        result["pending_note_sha256"] = ""
    if review["verdict"] == "fail":
        result["pending_note_sha256"] = note
        if result["revisions_used"] == 2:
            result["exhausted_note_sha256"] = note
    result["last_review"] = {
        "review_id": review_id,
        "note_sha256": note,
        "plan_sha256": review["reviewed_plan_sha256"],
        "verdict": review["verdict"],
    }
    return result

def require_recorded_pass(state: dict, review: dict, review_id: str, *, language: str) -> None:
    validate_state(state, language=language)
    expected = {
        "review_id": review_id,
        "note_sha256": review["reviewed_note_sha256"],
        "plan_sha256": review["reviewed_plan_sha256"],
        "verdict": "pass",
    }
    if review["verdict"] != "pass" or state["last_review"] != expected:
        raise ValueError("Formal Save requires a current recorded passing Final Quality Review.")
```

## File: skills/deeppapernote/scripts/_frontmatter.py
size: 3415
sha: 7e82bcb4a88d405583d5186fc1466e3556533745
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_frontmatter.py

```text
"""Read the small YAML subset DeepPaperNote notes use for Obsidian properties.

Supported shapes, one key per line at the top level:

    key: scalar
    key: "quoted scalar"
    key: [a, "b, c", "[[Link]]"]
    key:
      - item
      - "[[Link]]"

Anything else (nested maps, orphan list items, duplicate keys) raises
FrontmatterError, so lint fails closed instead of guessing.
"""

from __future__ import annotations

import re

KEY_RE = re.compile(r"^([A-Za-z_][\w-]*)\s*:(?:\s+(.*?))?\s*$")
ITEM_RE = re.compile(r"^\s+-\s+(.*?)\s*$|^\s+-\s*$")

class FrontmatterError(ValueError):
    pass

def frontmatter_block(text: str) -> str | None:
    """Return the text between the opening and closing `---` lines, or None."""
    normalized = text.removeprefix("﻿").replace("\r\n", "\n")
    if not normalized.startswith("---\n"):
        return None
    end = re.search(r"^---[ \t]*$", normalized[4:], flags=re.MULTILINE)
    if end is None:
        raise FrontmatterError("frontmatter is not closed")
    return normalized[4 : 4 + end.start()]

def _unquote(value: str) -> str:
    value = value.strip()
    if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
        return value[1:-1]
    return value

def _flow_list(value: str) -> list[str]:
    inner = value[1:-1]
    items: list[str] = []
    current: list[str] = []
    quote = ""
    for ch in inner:
        if quote:
            current.append(ch)
            if ch == quote:
                quote = ""
        elif ch in "\"'":
            quote = ch
            current.append(ch)
        elif ch == ",":
            items.append("".join(current))
            current = []
        else:
            current.append(ch)
    if quote:
        raise FrontmatterError("unterminated quote in list")
    items.append("".join(current))
    parsed = [_unquote(item) for item in items]
    return [] if parsed == [""] else parsed

def _scalar_or_list(value: str) -> str | list[str]:
    if value.startswith("["):
        if not value.endswith("]") or value.startswith("[["):
            if value.startswith("[[") and value.endswith("]]"):
                return value  # an unquoted wikilink scalar
            raise FrontmatterError(f"unterminated list: {value}")
        return _flow_list(value)
    return _unquote(value)

def parse_frontmatter(text: str) -> dict[str, str | list[str]] | None:
    block = frontmatter_block(text)
    if block is None:
        return None
    fields: dict[str, str | list[str]] = {}
    current_list: list[str] | None = None
    for line in block.split("\n"):
        if not line.strip() or line.lstrip().startswith("#"):
            continue
        if line[0].isspace():
            item = ITEM_RE.match(line)
            if item is None or current_list is None:
                raise FrontmatterError(f"unsupported line: {line.strip()}")
            current_list.append(_unquote(item.group(1) or ""))
            continue
        match = KEY_RE.match(line)
        if match is None:
            raise FrontmatterError(f"unsupported line: {line.strip()}")
        key, value = match.group(1), match.group(2) or ""
        if key in fields:
            raise FrontmatterError(f"duplicate key: {key}")
        if value:
            fields[key] = _scalar_or_list(value)
            current_list = None
        else:
            current_list = []
            fields[key] = current_list
    return fields
```

## File: skills/deeppapernote/scripts/_identity.py
size: 66991
sha: bb0f47360716b505f8b87a636444fa3710d3a9f8
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_identity.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_identity.py

## File: skills/deeppapernote/scripts/_local_pdf_title.py
size: 9960
sha: 82ea224a7a7a94005fe69f4857cb5cc4992a95e1
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_local_pdf_title.py

```text
"""Read title and identifier hints from a local PDF."""

from __future__ import annotations

import re
from pathlib import Path
from typing import Any

from _identity import (
    candidate_priority_score,
    choose_best_title_match,
    clean_local_pdf_stem,
    extract_arxiv_id,
    extract_doi,
    is_probable_local_pdf_artifact_title,
    normalize_identity_title,
    publication_quality_score,
    title_similarity,
)
from _pdf_text import clean_pdf_line
from common import (
    fitz,
    normalize_whitespace,
)

TITLE_MIN_LETTER_SHARE = 0.6
TITLE_MAX_GARBLE_SHARE = 0.05
TITLE_GARBLE_CHARS = set("!\"#$%*<=>@[\\]^_`{|}~")
TITLE_LINE_SIZE_RATIO = 0.9
TITLE_LINE_SPACING = 1.6
TITLE_FOOTNOTE_MARKS_RE = re.compile(r"[\s*∗†‡§¶]+$")
CJK_CHAR_RE = re.compile(r"[㐀-鿿]")

def is_mostly_letters(text: str) -> bool:
    """Tell readable words from a garbled text layer, which decodes to stray symbols."""
    visible = [char for char in TITLE_FOOTNOTE_MARKS_RE.sub("", text) if not char.isspace()]
    if not visible:
        return False
    letters = sum(1 for char in visible if char.isalpha())
    garble = sum(1 for char in visible if char in TITLE_GARBLE_CHARS)
    return (
        letters / len(visible) >= TITLE_MIN_LETTER_SHARE
        and garble / len(visible) <= TITLE_MAX_GARBLE_SHARE
    )

def is_plausible_pdf_title_line(line: str) -> bool:
    normalized = clean_pdf_line(line)
    lower = normalized.lower()
    # A Chinese title has no spaces between words, so only its length counts.
    single_word = len(normalized.split()) < 2 and not CJK_CHAR_RE.search(normalized)
    if len(normalized) < 8 or (len(normalized) < 20 and single_word):
        return False
    if not is_mostly_letters(normalized):
        return False
    if normalized.count(",") >= 3:
        return False
    if any(token in lower for token in ["doi.org/", "http://", "https://", "www.", "check for updates"]):
        return False
    if lower in {"abstract", "article", "preprint"}:
        return False
    if lower.startswith("npj |") or lower.startswith("arxiv:") or lower.startswith("submitted to"):
        return False
    if any(re.search(pattern, lower) for pattern in LOW_CONFIDENCE_LOCAL_TITLE_PATTERNS):
        return False
    if " doi:" in lower or lower.startswith("doi:"):
        return False
    return True

def join_title_lines(lines: list[str]) -> str:
    title = ""
    for line in lines:
        if not title:
            title = line
        elif title.endswith("-") and line[:1].islower():
            title = title[:-1] + line
        elif CJK_CHAR_RE.match(title[-1]) and CJK_CHAR_RE.match(line[0]):
            title += line
        else:
            title = f"{title} {line}"
    return TITLE_FOOTNOTE_MARKS_RE.sub("", title)

def first_page_title_from_layout(page: Any) -> str:
    """Return the first page's title: its largest plausible line plus the adjacent lines set
    in the same type, so a journal name or volume line printed above it is passed over."""
    try:
        blocks = page.get_text("dict").get("blocks", [])
    except Exception:
        return ""
    lines = []
    for block in blocks:
        for line in block.get("lines", []):
            if tuple(line.get("dir", (1.0, 0.0))) != (1.0, 0.0):
                continue
            spans = line.get("spans", [])
            text = clean_pdf_line("".join(str(span.get("text", "")) for span in spans))
            if text:
                sizes = [
                    float(span.get("size", 0))
                    for span in spans
                    if str(span.get("text", "")).strip()
                ]
                size = max(sizes)
                lines.append({"text": text, "size": size, "top": float(line["bbox"][1])})
    candidates = [line for line in lines if is_plausible_pdf_title_line(line["text"])]
    if not candidates:
        return ""
    anchor = max(candidates, key=lambda line: (line["size"], -line["top"]))
    ordered = sorted(lines, key=lambda line: line["top"])
    start = end = ordered.index(anchor)

    def continues_title(edge: dict, neighbour: dict) -> bool:
        return (
            neighbour["size"] >= anchor["size"] * TITLE_LINE_SIZE_RATIO
            and abs(neighbour["top"] - edge["top"]) <= anchor["size"] * TITLE_LINE_SPACING
            and is_mostly_letters(neighbour["text"])
        )

    while start > 0 and continues_title(ordered[start], ordered[start - 1]):
        start -= 1
    while end + 1 < len(ordered) and continues_title(ordered[end], ordered[end + 1]):
        end += 1
    return join_title_lines([line["text"] for line in ordered[start : end + 1]])

def first_page_title_candidate(first_page_text: str) -> str:
    for raw_line in (first_page_text or "").splitlines():
        if is_plausible_pdf_title_line(raw_line):
            return clean_pdf_line(raw_line)
    return ""

def extract_local_pdf_hints(pdf_path: Path) -> dict[str, Any]:
    raw_title = normalize_whitespace(pdf_path.stem.replace("_", " "))
    cleaned_title = clean_local_pdf_stem(pdf_path.stem)
    hints: dict[str, Any] = {
        "title": cleaned_title or raw_title,
        "local_pdf_title_source": "local_pdf_stem_used",
    }
    if is_probable_local_pdf_artifact_title(raw_title):
        hints["local_pdf_artifact_title"] = True
    if fitz is None:
        return hints

    metadata_title = ""
    metadata_subject = ""
    first_page_text = ""
    layout_title = ""
    try:
        doc = fitz.open(pdf_path)
    except Exception:
        return hints
    try:
        metadata = doc.metadata or {}
        metadata_title = normalize_whitespace(str(metadata.get("title", "")))
        metadata_subject = normalize_whitespace(str(metadata.get("subject", "")))
        if len(doc):
            first_page_text = doc[0].get_text("text")
            layout_title = first_page_title_from_layout(doc[0])
    except Exception:
        return hints
    finally:
        doc.close()

    if metadata_title:
        hints["title"] = metadata_title
        hints["local_pdf_title_source"] = "pdf_metadata_title_used"
        # Tools that write the metadata can drop part of the printed title, such as a
        # venue's "Position:" lead-in, so identity resolution also sees the printed one.
        if layout_title and normalize_identity_title(layout_title) != normalize_identity_title(
            metadata_title
        ):
            hints["local_pdf_first_page_title"] = layout_title
    else:
        page_title = layout_title or first_page_title_candidate(first_page_text)
        if page_title:
            hints["title"] = page_title
            hints["local_pdf_title_source"] = "first_page_title_used"

    searchable = "\n".join(part for part in [metadata_subject, metadata_title, first_page_text] if part)
    doi = extract_doi(searchable)
    if doi:
        hints["doi"] = doi
    arxiv_id = extract_arxiv_id(searchable)
    if arxiv_id:
        hints["arxiv_id"] = arxiv_id

    return hints

def choose_local_pdf_corrected_title(base: dict[str, Any], candidates: list[dict[str, Any]]) -> str:
    current_title = normalize_whitespace(str(base.get("title", "")))
    if not current_title:
        return ""
    titled_candidates = [candidate for candidate in candidates if normalize_whitespace(str(candidate.get("title", "")))]
    if local_pdf_title_needs_identity_correction(base):
        identity_matches = [
            candidate for candidate in titled_candidates if external_identity_matches(base, candidate)
        ]
        if identity_matches:
            best_identity = sorted(
                identity_matches,
                key=lambda item: (
                    candidate_priority_score(item),
                    publication_quality_score(item),
                    len(normalize_whitespace(str(item.get("title", "")))),
                ),
                reverse=True,
            )[0]
            return normalize_whitespace(str(best_identity.get("title", "")))

    if not is_probable_local_pdf_artifact_title(current_title):
        return ""
    best = choose_best_title_match(current_title, titled_candidates)
    if not best:
        return ""
    candidate_title = normalize_whitespace(str(best.get("title", "")))
    if not candidate_title:
        return ""
    if title_similarity(current_title, candidate_title) < 0.55:
        return ""
    if not (best.get("doi") or best.get("arxiv_id") or publication_quality_score(best) >= 2):
        return ""
    return candidate_title

LOW_CONFIDENCE_LOCAL_TITLE_PATTERNS = (
    r"^(?:research|original|review)\s+article$",
    r"^provided proper attribution\b",
    r"^published as\b",
    r"^submitted to\b",
    r"^accepted (?:as|to|for)\b",
    r"^conference paper\b",
    r"^proceedings of\b",
    r"^arxiv\s*:",
    r"^doi\s*:",
)

def local_pdf_title_needs_identity_correction(base: dict[str, Any]) -> bool:
    title = normalize_whitespace(str(base.get("title", "")))
    if not title:
        return False
    if base.get("local_pdf_artifact_title") or is_probable_local_pdf_artifact_title(title):
        return True
    source = normalize_whitespace(str(base.get("local_pdf_title_source", "")))
    if source not in {"first_page_title_used", "local_pdf_stem_used", "pdf_metadata_title_used"}:
        return False
    lowered = title.lower()
    return any(re.search(pattern, lowered) for pattern in LOW_CONFIDENCE_LOCAL_TITLE_PATTERNS)

def external_identity_matches(base: dict[str, Any], candidate: dict[str, Any]) -> bool:
    base_doi = extract_doi(str(base.get("doi", ""))) or ""
    candidate_doi = extract_doi(str(candidate.get("doi", ""))) or ""
    if base_doi and candidate_doi and base_doi.lower() == candidate_doi.lower():
        return True
    base_arxiv = normalize_whitespace(str(base.get("arxiv_id", "")))
    candidate_arxiv = normalize_whitespace(str(candidate.get("arxiv_id", "")))
    return bool(base_arxiv and candidate_arxiv and base_arxiv == candidate_arxiv)
```

## File: skills/deeppapernote/scripts/_metadata_sources.py
size: 24055
sha: 571ffce223ccbbdd04e40f715ba6403c16f661cc
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_metadata_sources.py

```text
"""Fetch and normalize paper metadata from arXiv, Crossref, Semantic Scholar, and OpenAlex."""

from __future__ import annotations

import json
import re
import ssl
import urllib.parse
import urllib.request
import xml.etree.ElementTree as ET
from copy import deepcopy
from pathlib import Path
from typing import Any

from _identity import (
    _record_arxiv_id,
    _string_field,
    adjudicate_identity_observations,
    apply_identity_confidence,
    choose_best_title_match,
    clean_local_pdf_stem,
    extract_arxiv_id,
    extract_doi,
    infer_source_type,
    normalize_identity_title,
    paper_id_for_record,
)
from _local_pdf_title import extract_local_pdf_hints
from common import (
    env_config_value,
    normalize_title,
    normalize_whitespace,
    strip_tags,
)

try:
    import certifi  # type: ignore
except ImportError:  # pragma: no cover
    certifi = None

ARXIV_NS = {
    "atom": "http://www.w3.org/2005/Atom",
    "arxiv": "http://arxiv.org/schemas/atom",
}

SEMANTIC_SCHOLAR_SEARCH_URL = "https://api.semanticscholar.org/graph/v1/paper/search"
OPENALEX_WORKS_URL = "https://api.openalex.org/works"
CROSSREF_WORKS_URL = "https://api.crossref.org/works"
DEFAULT_USER_AGENT = "DeepPaperNote/0.1"

def fallback_arxiv_record(arxiv_id: str, source_type: str, source_url: str = "") -> dict[str, Any]:
    paper = {
        "status": "ok",
        "source_type": source_type,
        "source_url": source_url or f"https://arxiv.org/abs/{arxiv_id}",
        "arxiv_id": arxiv_id,
        "pdf_url": f"https://arxiv.org/pdf/{arxiv_id}.pdf",
        "metadata_sources": [source_type],
    }
    paper["paper_id"] = paper_id_for_record(paper)
    return apply_identity_confidence(paper)

def http_get_text(url: str, *, timeout: int = 30, headers: dict[str, str] | None = None) -> str:
    request = urllib.request.Request(url, headers=headers or {"User-Agent": DEFAULT_USER_AGENT})
    with urllib.request.urlopen(request, timeout=timeout, context=https_ssl_context(url)) as response:
        return response.read().decode("utf-8")

def http_get_json(url: str, *, timeout: int = 30, headers: dict[str, str] | None = None) -> dict[str, Any]:
    return json.loads(http_get_text(url, timeout=timeout, headers=headers))

def http_get_bytes(url: str, *, timeout: int = 60, headers: dict[str, str] | None = None) -> bytes:
    request = urllib.request.Request(url, headers=headers or {"User-Agent": DEFAULT_USER_AGENT})
    with urllib.request.urlopen(request, timeout=timeout, context=https_ssl_context(url)) as response:
        return response.read()

def https_ssl_context(url: str) -> ssl.SSLContext | None:
    if not str(url).lower().startswith("https://") or certifi is None:
        return None
    try:
        return ssl.create_default_context(cafile=certifi.where())
    except Exception:
        return None

def semantic_scholar_headers() -> dict[str, str]:
    headers = {"User-Agent": DEFAULT_USER_AGENT}
    api_key = env_config_value("DEEPPAPERNOTE_SEMANTIC_SCHOLAR_API_KEY", "SEMANTIC_SCHOLAR_API_KEY")
    if api_key:
        headers["x-api-key"] = api_key
    return headers

def parse_arxiv_xml(xml_content: str) -> list[dict[str, Any]]:
    papers: list[dict[str, Any]] = []
    root = ET.fromstring(xml_content)
    for entry in root.findall("atom:entry", ARXIV_NS):
        paper: dict[str, Any] = {
            "source": "arxiv",
            "source_type": "arxiv",
            "metadata_sources": ["arxiv"],
        }
        id_elem = entry.find("atom:id", ARXIV_NS)
        if id_elem is not None and id_elem.text:
            paper["source_url"] = normalize_whitespace(id_elem.text)
            paper["url"] = paper["source_url"]
            arxiv_id = extract_arxiv_id(paper["source_url"])
            if arxiv_id:
                paper["arxiv_id"] = arxiv_id

        title_elem = entry.find("atom:title", ARXIV_NS)
        paper["title"] = normalize_whitespace(title_elem.text if title_elem is not None else "")

        summary_elem = entry.find("atom:summary", ARXIV_NS)
        paper["abstract"] = normalize_whitespace(summary_elem.text if summary_elem is not None else "")

        journal_ref_elem = entry.find("arxiv:journal_ref", ARXIV_NS)
        journal_ref = normalize_whitespace(journal_ref_elem.text if journal_ref_elem is not None else "")
        if journal_ref:
            paper["venue"] = journal_ref

        doi_elem = entry.find("arxiv:doi", ARXIV_NS)
        if doi_elem is not None and doi_elem.text:
            paper["doi"] = normalize_whitespace(doi_elem.text)

        authors = []
        for author in entry.findall("atom:author", ARXIV_NS):
            name_elem = author.find("atom:name", ARXIV_NS)
            if name_elem is not None and name_elem.text:
                authors.append(normalize_whitespace(name_elem.text))
        paper["authors"] = authors

        published_elem = entry.find("atom:published", ARXIV_NS)
        if published_elem is not None and published_elem.text:
            paper["published"] = normalize_whitespace(published_elem.text)
            if re.match(r"^\d{4}", paper["published"]):
                paper["year"] = paper["published"][:4]

        for link in entry.findall("atom:link", ARXIV_NS):
            if link.get("title") == "pdf" and link.get("href"):
                paper["pdf_url"] = str(link.get("href"))
                break

        papers.append(paper)
    return papers

def fetch_arxiv_entries(*, search_query: str = "", id_list: str = "", max_results: int = 10) -> list[dict[str, Any]]:
    params = urllib.parse.urlencode(
        {
            "search_query": search_query,
            "id_list": id_list,
            "start": 0,
            "max_results": max_results,
        }
    )
    try:
        xml_content = http_get_text(f"https://export.arxiv.org/api/query?{params}")
    except Exception:
        return []
    if not normalize_whitespace(xml_content):
        return []
    try:
        return parse_arxiv_xml(xml_content)
    except Exception:
        return []

def safe_fetch_arxiv_entries(*, search_query: str = "", id_list: str = "", max_results: int = 10) -> list[dict[str, Any]]:
    try:
        return fetch_arxiv_entries(search_query=search_query, id_list=id_list, max_results=max_results)
    except Exception:
        return []

def normalize_crossref_work(item: dict[str, Any]) -> dict[str, Any]:
    title = normalize_whitespace(" ".join(item.get("title") or []))
    # Crossref splits "LightGCN: Simplifying ..." into title and subtitle.
    subtitle = normalize_whitespace(" ".join(item.get("subtitle") or []))
    if subtitle and subtitle.casefold() not in title.casefold():
        title = f"{title}: {subtitle}" if title else subtitle
    authors = []
    affiliations = []
    for author in item.get("author", []) or []:
        given = normalize_whitespace(str(author.get("given", "")))
        family = normalize_whitespace(str(author.get("family", "")))
        name = normalize_whitespace(" ".join(part for part in [given, family] if part))
        if name:
            authors.append(name)
        for aff in author.get("affiliation", []) or []:
            aff_name = normalize_whitespace(str(aff.get("name", "")))
            if aff_name and aff_name not in affiliations:
                affiliations.append(aff_name)
    venue = normalize_whitespace(" ".join(item.get("container-title") or []))
    published = (
        item.get("published-print", {}).get("date-parts")
        or item.get("published-online", {}).get("date-parts")
        or item.get("issued", {}).get("date-parts")
        or []
    )
    year = ""
    if published and isinstance(published, list) and isinstance(published[0], list) and published[0]:
        year = str(published[0][0])
    doi = normalize_whitespace(str(item.get("DOI", "")))
    source_url = normalize_whitespace(str(item.get("URL", "")))
    return {
        "title": title,
        "authors": authors,
        "affiliations": affiliations,
        "venue": venue,
        "doi": doi,
        "source": "crossref",
        "source_type": "crossref",
        "source_url": source_url,
        "year": year,
        "published": year,
        "abstract": strip_tags(str(item.get("abstract", ""))),
        "metadata_sources": ["crossref"],
    }

def fetch_crossref_by_doi(doi: str) -> dict[str, Any] | None:
    url = f"{CROSSREF_WORKS_URL}/{urllib.parse.quote(doi)}"
    try:
        data = http_get_json(url, headers={"User-Agent": DEFAULT_USER_AGENT})
    except Exception:
        return None
    message = data.get("message") or {}
    if not isinstance(message, dict):
        return None
    return normalize_crossref_work(message)

def search_crossref_by_title(title: str, *, limit: int = 5) -> list[dict[str, Any]]:
    params = urllib.parse.urlencode({"query.title": title, "rows": limit})
    try:
        data = http_get_json(f"{CROSSREF_WORKS_URL}?{params}", headers={"User-Agent": DEFAULT_USER_AGENT})
    except Exception:
        return []
    items = data.get("message", {}).get("items", []) or []
    return [normalize_crossref_work(item) for item in items if isinstance(item, dict)]

def normalize_semantic_scholar_paper(paper: dict[str, Any]) -> dict[str, Any]:
    ext_ids = paper.get("externalIds") or {}
    doi = normalize_whitespace(str(ext_ids.get("DOI", "")))
    arxiv_id = normalize_whitespace(str(ext_ids.get("ArXiv", "")))
    affiliations: list[str] = []
    authors: list[str] = []
    for author in paper.get("authors", []) or []:
        if not isinstance(author, dict):
            continue
        name = normalize_whitespace(str(author.get("name", "")))
        if name:
            authors.append(name)
        raw_affs = author.get("affiliations", []) or []
        if isinstance(raw_affs, str):
            raw_affs = [raw_affs]
        for aff in raw_affs:
            aff_name = normalize_whitespace(str(aff))
            if aff_name and aff_name not in affiliations:
                affiliations.append(aff_name)
    result = {
        "title": normalize_whitespace(str(paper.get("title", ""))),
        "abstract": normalize_whitespace(str(paper.get("abstract", ""))),
        "authors": authors,
        "affiliations": affiliations,
        "venue": normalize_whitespace(str(paper.get("venue", ""))),
        "year": normalize_whitespace(str(paper.get("year", ""))),
        "doi": doi,
        "arxiv_id": arxiv_id,
        "source": "semantic_scholar",
        "source_type": "semantic_scholar",
        "source_url": normalize_whitespace(str(paper.get("url", ""))),
        "metadata_sources": ["semantic_scholar"],
    }
    if arxiv_id and not result.get("pdf_url"):
        result["pdf_url"] = f"https://arxiv.org/pdf/{arxiv_id}.pdf"
    return result

def search_semantic_scholar(query: str, *, limit: int = 5) -> list[dict[str, Any]]:
    params = urllib.parse.urlencode(
        {
            "query": query,
            "limit": limit,
            "fields": "title,abstract,year,venue,url,externalIds,authors.name,authors.affiliations",
        }
    )
    try:
        data = http_get_json(
            f"{SEMANTIC_SCHOLAR_SEARCH_URL}?{params}",
            headers=semantic_scholar_headers(),
        )
    except Exception:
        return []
    items = data.get("data", []) or []
    return [normalize_semantic_scholar_paper(item) for item in items if isinstance(item, dict)]

def normalize_openalex_work(item: dict[str, Any]) -> dict[str, Any]:
    title = normalize_whitespace(str(item.get("display_name", "")))
    authors = []
    affiliations = []
    for authorship in item.get("authorships", []) or []:
        if not isinstance(authorship, dict):
            continue
        author = authorship.get("author", {}) or {}
        name = normalize_whitespace(str(author.get("display_name", "")))
        if name:
            authors.append(name)
        for institution in authorship.get("institutions", []) or []:
            if not isinstance(institution, dict):
                continue
            inst_name = normalize_whitespace(str(institution.get("display_name", "")))
            if inst_name and inst_name not in affiliations:
                affiliations.append(inst_name)
    ids = item.get("ids", {}) or {}
    doi_url = normalize_whitespace(str(ids.get("doi", "")))
    doi = extract_doi(doi_url or normalize_whitespace(str(item.get("doi", "")))) or ""
    primary_location = item.get("primary_location", {}) or {}
    pdf_url = normalize_whitespace(str((primary_location.get("pdf_url") or "")))
    landing_page_url = normalize_whitespace(str(primary_location.get("landing_page_url") or ""))
    best_oa = item.get("best_oa_location", {}) or {}
    if not pdf_url:
        pdf_url = normalize_whitespace(str(best_oa.get("pdf_url") or ""))
    if not landing_page_url:
        landing_page_url = normalize_whitespace(str(best_oa.get("landing_page_url") or ""))
    venue = normalize_whitespace(str((primary_location.get("source", {}) or {}).get("display_name", "")))
    year = normalize_whitespace(str(item.get("publication_year", "")))
    return {
        "title": title,
        "authors": authors,
        "affiliations": affiliations,
        "venue": venue,
        "year": year,
        "doi": doi,
        "source": "openalex",
        "source_type": "openalex",
        "source_url": landing_page_url or normalize_whitespace(str(item.get("id", ""))),
        "pdf_url": pdf_url,
        "abstract": "",
        "metadata_sources": ["openalex"],
    }

def fetch_openalex_by_doi(doi: str) -> dict[str, Any] | None:
    url = f"{OPENALEX_WORKS_URL}/https://doi.org/{urllib.parse.quote(doi, safe='')}"
    try:
        data = http_get_json(url, headers={"User-Agent": DEFAULT_USER_AGENT})
    except Exception:
        return None
    if not isinstance(data, dict):
        return None
    return normalize_openalex_work(data)

def search_openalex_by_title(title: str, *, limit: int = 5) -> list[dict[str, Any]]:
    params = urllib.parse.urlencode({"search": title, "per-page": limit})
    try:
        data = http_get_json(f"{OPENALEX_WORKS_URL}?{params}", headers={"User-Agent": DEFAULT_USER_AGENT})
    except Exception:
        return []
    items = data.get("results", []) or []
    return [normalize_openalex_work(item) for item in items if isinstance(item, dict)]

def merge_metadata_records(*records: dict[str, Any]) -> dict[str, Any]:
    merged: dict[str, Any] = {}
    metadata_sources: list[str] = []
    additive_list_fields = {"affiliations", "metadata_sources", "identity_confidence_reasons"}
    for record in records:
        if not isinstance(record, dict):
            continue
        for key, value in record.items():
            if value in ("", None, [], {}):
                continue
            if key == "authors":
                if not merged.get("authors"):
                    values = value if isinstance(value, list) else [value]
                    seen = set()
                    deduped = []
                    for item in values:
                        cleaned = normalize_whitespace(str(item))
                        marker = normalize_title(cleaned)
                        if cleaned and marker and marker not in seen:
                            deduped.append(cleaned)
                            seen.add(marker)
                    if deduped:
                        merged["authors"] = deduped
                continue
            if key in additive_list_fields:
                current = merged.setdefault(key, [])
                if not isinstance(current, list):
                    current = []
                    merged[key] = current
                values = value if isinstance(value, list) else [value]
                for item in values:
                    cleaned = normalize_whitespace(str(item))
                    if cleaned and cleaned not in current:
                        current.append(cleaned)
                continue
            if key not in merged or merged[key] in ("", None):
                merged[key] = value
    for record in records:
        if isinstance(record, dict):
            for source in record.get("metadata_sources", []) or []:
                source_name = normalize_whitespace(str(source))
                if source_name and source_name not in metadata_sources:
                    metadata_sources.append(source_name)
    if metadata_sources:
        merged["metadata_sources"] = metadata_sources
    merged["paper_id"] = paper_id_for_record(merged)
    return merged

def _has_strong_work_identifier(record: dict[str, Any]) -> bool:
    paper_id = _string_field(record, "paper_id").lower()
    return bool(
        extract_doi(_string_field(record, "doi"))
        or extract_doi(_string_field(record, "source_url"))
        or _record_arxiv_id(record)
        or paper_id.startswith(("doi:", "arxiv:"))
    )

def resolve_reference(value: str) -> dict[str, Any]:
    source_type = infer_source_type(value)
    stripped = (value or "").strip()
    if source_type == "local_pdf":
        path = Path(stripped).expanduser().resolve()
        hints = extract_local_pdf_hints(path)
        paper = {
            "status": "ok",
            "source_type": "local_pdf",
            "source_url": str(path),
            "local_pdf_path": str(path),
            "title": normalize_whitespace(str(hints.get("title", ""))) or clean_local_pdf_stem(path.stem) or path.stem.replace("_", " "),
            "metadata_sources": ["local_pdf"],
        }
        if hints.get("local_pdf_title_source"):
            paper["local_pdf_title_source"] = hints["local_pdf_title_source"]
        if hints.get("local_pdf_artifact_title"):
            paper["local_pdf_artifact_title"] = True
        if hints.get("local_pdf_first_page_title"):
            paper["local_pdf_first_page_title"] = hints["local_pdf_first_page_title"]
        doi = normalize_whitespace(str(hints.get("doi", "")))
        arxiv_id = normalize_whitespace(str(hints.get("arxiv_id", "")))
        if doi:
            paper["doi"] = doi
        if arxiv_id:
            paper["arxiv_id"] = arxiv_id
        paper["paper_id"] = paper_id_for_record(paper)
        return apply_identity_confidence(paper)
    if source_type == "arxiv_id":
        arxiv_id = extract_arxiv_id(stripped) or ""
        if arxiv_id:
            return fallback_arxiv_record(arxiv_id, "arxiv_id")
    if source_type == "arxiv_url":
        arxiv_id = extract_arxiv_id(stripped) or ""
        if arxiv_id:
            return fallback_arxiv_record(arxiv_id, "arxiv_url", source_url=stripped)
    if source_type in {"doi", "doi_url"}:
        doi = extract_doi(stripped) or ""
        paper = {
            "status": "ok",
            "source_type": "doi",
            "source_url": f"https://doi.org/{doi}",
            "doi": doi,
            "metadata_sources": ["doi"],
        }
        paper["paper_id"] = paper_id_for_record(paper)
        return apply_identity_confidence(paper)
    if source_type == "pdf_url":
        filename = Path(urllib.parse.urlparse(stripped).path).stem or "paper"
        paper = {
            "status": "ok",
            "source_type": "pdf_url",
            "source_url": stripped,
            "pdf_url": stripped,
            "title": filename.replace("_", " "),
            "metadata_sources": ["pdf_url"],
        }
        paper["paper_id"] = paper_id_for_record(paper)
        return apply_identity_confidence(paper)
    if source_type == "url":
        doi = extract_doi(stripped)
        if doi:
            return resolve_reference(doi)
        paper = {
            "status": "ok",
            "source_type": "url",
            "source_url": stripped,
            "metadata_sources": ["url"],
        }
        paper["paper_id"] = paper_id_for_record(paper)
        return apply_identity_confidence(paper)
    if source_type == "zotero_key":
        paper = {
            "status": "ok",
            "source_type": "zotero_key",
            "zotero_key": stripped,
            "source_url": "",
            "metadata_sources": ["zotero_key"],
        }
        paper["paper_id"] = paper_id_for_record(paper)
        return apply_identity_confidence(paper)

    title = stripped
    paper = {
        "status": "ok",
        "source_type": "title_query",
        "title": title,
        "source_url": "",
        "metadata_sources": ["title_query"],
    }
    paper["paper_id"] = paper_id_for_record(paper)
    return apply_identity_confidence(paper)

def collect_metadata_observations(record: dict[str, Any]) -> list[dict[str, Any]]:
    base = dict(record)
    observations = [
        deepcopy(item)
        for item in base.get("identity_observations", []) or []
        if isinstance(item, dict)
    ]
    doi = normalize_whitespace(str(base.get("doi", "")))
    title = normalize_whitespace(str(base.get("title", "")))
    arxiv_id = normalize_whitespace(str(base.get("arxiv_id", "")))

    def append(
        provider: str,
        kind: str,
        value: str,
        candidate: dict[str, Any] | None,
        relation: dict[str, str] | None = None,
    ) -> None:
        if not candidate:
            return
        observation = {
            "provider": provider,
            "retrieved_by": {"kind": kind, "value": value},
            "record": deepcopy(candidate),
        }
        if relation:
            observation["relation"] = relation
        if observation not in observations:
            observations.append(observation)

    if doi:
        crossref = fetch_crossref_by_doi(doi)
        append("crossref", "doi", doi, crossref)
        openalex = fetch_openalex_by_doi(doi)
        append("openalex", "doi", doi, openalex)
        sem = choose_best_title_match(title or doi, search_semantic_scholar(doi, limit=3))
        append("semantic_scholar", "doi", doi, sem)

    if arxiv_id:
        arxiv = safe_fetch_arxiv_entries(id_list=arxiv_id, max_results=1)
        if arxiv:
            append("arxiv", "arxiv_id", arxiv_id, arxiv[0])

    if title:
        sem = choose_best_title_match(title, search_semantic_scholar(title, limit=5))
        append("semantic_scholar", "title", title, sem)
        oa = choose_best_title_match(title, search_openalex_by_title(title, limit=5))
        append("openalex", "title", title, oa)
        cross = choose_best_title_match(title, search_crossref_by_title(title, limit=5))
        append("crossref", "title", title, cross)
        arxiv_candidates = safe_fetch_arxiv_entries(
            search_query=f'ti:"{title}"', max_results=5
        )
        arxiv = choose_best_title_match(title, arxiv_candidates)
        exact_matches = [
            candidate
            for candidate in arxiv_candidates
            if normalize_identity_title(_string_field(candidate, "title"))
            == normalize_identity_title(title)
        ]
        relation = (
            {
                "kind": "arxiv_lookup",
                "match_kind": "title",
                "match_resolution": "unique_exact",
            }
            if arxiv is not None and len(exact_matches) == 1 and arxiv == exact_matches[0]
            else None
        )
        append("arxiv", "title", title, arxiv, relation)

    return observations

def enrich_metadata(record: dict[str, Any]) -> dict[str, Any]:
    base = dict(record)
    observations = collect_metadata_observations(base)
    merged = adjudicate_identity_observations(base, observations)["accepted_metadata"]
    if not merged.get("year") and merged.get("published") and re.match(r"^\d{4}", str(merged["published"])):
        merged["year"] = str(merged["published"])[:4]
    if merged.get("doi") and not merged.get("source_url"):
        merged["source_url"] = f"https://doi.org/{merged['doi']}"
    if merged.get("arxiv_id") and not merged.get("pdf_url"):
        merged["pdf_url"] = f"https://arxiv.org/pdf/{merged['arxiv_id']}.pdf"
    if merged.get("arxiv_id") and not merged.get("doi"):
        merged["doi"] = f"10.48550/arXiv.{merged['arxiv_id']}"
    merged["paper_id"] = paper_id_for_record(merged)
    return apply_identity_confidence(merged)
```

## File: skills/deeppapernote/scripts/_note_paths.py
size: 5556
sha: d2d73343b7e0c560d295c83f3e39429597df2779
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_note_paths.py

```text
"""Resolve where a note is saved: the save target, the domain folders, and the note path."""

from __future__ import annotations

import re
from pathlib import Path, PureWindowsPath
from typing import Any

from common import slugify_filename

def configured_obsidian_vault(config: dict[str, Any]) -> Path | None:
    if str(config.get("save_mode", "")).strip() == "workspace":
        return None
    vault = str(config.get("obsidian_vault", "")).strip()
    if not vault:
        return None
    vault_path = Path(vault).expanduser().resolve()
    if not vault_path.exists() or not vault_path.is_dir():
        raise RuntimeError(f"Configured Obsidian vault does not exist: {vault_path}")
    return vault_path

def require_obsidian_vault(config: dict[str, Any]) -> Path:
    vault_path = configured_obsidian_vault(config)
    if vault_path is None:
        raise RuntimeError("Missing Obsidian Vault in User Configuration or the current Run Override.")
    return vault_path

def resolve_note_output_mode(config: dict[str, Any]) -> tuple[str, Path]:
    save_mode = str(config.get("save_mode", "")).strip()
    if save_mode == "obsidian":
        return ("obsidian", require_obsidian_vault(config))
    if not save_mode:
        vault_path = configured_obsidian_vault(config)
        if vault_path is not None:
            return ("obsidian", vault_path)
    workspace_root = Path.cwd().resolve()
    output_dir = str(config.get("workspace_output_dir", "DeepPaperNote_output")).strip() or "DeepPaperNote_output"
    output_path = workspace_root / Path(
        *safe_relative_path_parts(output_dir, "workspace_output_dir")
    )
    return ("workspace", require_path_within(workspace_root, output_path, "workspace_output_dir"))

def safe_relative_path_parts(value: str | Path, field: str) -> tuple[str, ...]:
    raw = str(value).strip()
    if not raw:
        return ()
    windows_path = PureWindowsPath(raw)
    if (
        Path(raw).is_absolute()
        or windows_path.is_absolute()
        or windows_path.drive
        or windows_path.root
    ):
        raise RuntimeError(
            f"Unsafe {field}: expected a relative path inside the Save Target: {raw}"
        )
    parts = tuple(part for part in re.split(r"[\\/]+", raw) if part and part != ".")
    if ".." in parts:
        raise RuntimeError(
            f"Unsafe {field}: expected a relative path inside the Save Target: {raw}"
        )
    return parts

def require_path_within(root: str | Path, path: str | Path, field: str) -> Path:
    root_path = Path(root).expanduser().resolve()
    candidate = Path(path).expanduser().resolve()
    try:
        candidate.relative_to(root_path)
    except ValueError as exc:
        raise RuntimeError(f"Unsafe {field}: path escapes the Save Target: {path}") from exc
    return Path(path)

def resolve_note_asset_dir(note_path: str | Path, asset_subdir: str) -> Path:
    asset_subdir_parts = safe_relative_path_parts(asset_subdir, "asset_subdir")
    if not asset_subdir_parts:
        raise RuntimeError(
            "Unsafe asset_subdir: expected a relative directory inside the Save Target"
        )
    note_dir = Path(note_path).parent
    return require_path_within(
        note_dir,
        note_dir / Path(*asset_subdir_parts),
        "asset_subdir",
    )

def is_probable_paper_folder(path: Path) -> bool:
    if not path.is_dir():
        return False
    return any(
        (path / marker).exists()
        for marker in (
            f"{path.name}.md",
            f"{path.name}.zh-CN.md",
            f"{path.name}.en.md",
            ".deeppapernote.json",
        )
    )

def existing_domain_dirs(config: dict[str, Any]) -> list[str]:
    output_mode, root_path = resolve_note_output_mode(config)
    papers_dir = str(config.get("papers_dir", "Research/Papers")).strip() or "Research/Papers"
    base_dir = root_path / Path(papers_dir) if output_mode == "obsidian" else root_path
    if not base_dir.exists() or not base_dir.is_dir():
        return []
    names: list[str] = []
    for child in sorted(base_dir.iterdir()):
        if not child.is_dir():
            continue
        if is_probable_paper_folder(child):
            continue
        names.append(child.name)
    return names

def resolve_obsidian_note_path(
    config: dict[str, Any],
    *,
    title: str,
    subdir: str = "",
    filename: str = "",
) -> Path:
    def path_from_parts(parts: tuple[str, ...]) -> Path:
        return Path(*parts) if parts else Path()

    output_mode, root_path = resolve_note_output_mode(config)
    papers_dir = str(config.get("papers_dir", "Research/Papers")).strip() or "Research/Papers"
    papers_dir_parts = safe_relative_path_parts(papers_dir, "papers_dir")
    relative_dir = path_from_parts(papers_dir_parts) if output_mode == "obsidian" else Path()
    if subdir:
        relative_dir = relative_dir / path_from_parts(safe_relative_path_parts(subdir, "subdir"))
    note_slug = slugify_filename(title)
    target_name = filename.strip() or f"{note_slug}.md"
    target_name_parts = safe_relative_path_parts(target_name, "filename")
    target_name_path = path_from_parts(target_name_parts)
    folder_name = target_name_path.stem or note_slug
    target_path = root_path / relative_dir / folder_name / target_name_path
    save_target = (
        root_path / path_from_parts(papers_dir_parts)
        if output_mode == "obsidian"
        else root_path
    )
    require_path_within(root_path, save_target, "papers_dir")
    return require_path_within(save_target, target_path, "note path")
```

## File: skills/deeppapernote/scripts/_pdf_text.py
size: 16807
sha: eb30da34f7c391f40848e5de19f9fecaef036c40
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_pdf_text.py

```text
"""Extract sections, appendix pages, and caption lines from PDF text."""

from __future__ import annotations

import re
import unicodedata
from pathlib import Path
from typing import Any

from common import fitz, normalize_title, normalize_whitespace

PDF_LIGATURE_MAP = {
    "ß": "ss",
    "ﬀ": "ff",
    "ﬁ": "fi",
    "ﬂ": "fl",
    "ﬃ": "ffi",
    "ﬄ": "ffl",
}

def normalize_pdf_text_artifacts(text: str) -> str:
    normalized = text or ""
    for original, replacement in PDF_LIGATURE_MAP.items():
        normalized = normalized.replace(original, replacement)
    return normalized

def split_sentences(text: str) -> list[str]:
    text = re.sub(r"\s+", " ", text or "").strip()
    if not text:
        return []
    parts = re.split(r"(?<=[.!?。！？])\s+", text)
    return [part.strip() for part in parts if part.strip()]

def clean_pdf_line(line: str) -> str:
    line = re.sub(r"\s+", " ", normalize_pdf_text_artifacts(line or "")).strip()
    if not line:
        return ""
    if re.fullmatch(r"\d+", line):
        return ""
    if re.fullmatch(r"page \d+", line.lower()):
        return ""
    if len(line) <= 2 and not re.search(r"[㐀-鿿]", line):
        return ""
    return line

def normalize_heading(line: str) -> str:
    line = normalize_pdf_text_artifacts(line or "").strip().lower()
    line = re.sub(r"^\s*(?:section\s*)?\d+(\.\d+)*[\s.、．:：-]*", "", line)
    line = re.sub(r"^\s*[一二三四五六七八九十百千]+[、．.:\s-]*", "", line)
    line = re.sub(r"^\s*[ivxlcdm]+[.)\s]+", "", line)
    line = re.sub(r"[^a-z0-9㐀-鿿\s]", " ", line)
    line = re.sub(r"(?<=[㐀-鿿])\s+(?=[㐀-鿿])", "", line)
    return re.sub(r"\s+", " ", line).strip()

SECTION_ALIASES = {
    "abstract": {"abstract", "摘要"},
    "introduction": {
        "introduction",
        "background",
        "preliminaries",
        "preliminary",
        "related work",
        "literature review",
        "引言",
        "绪论",
        "背景",
        "相关工作",
        "文献综述",
    },
    "method": {
        "method",
        "methods",
        "approach",
        "approaches",
        "methodology",
        "framework",
        "model",
        "models",
        "materials",
        "materials and methods",
        "study design",
        "方法",
        "方法学",
        "研究方法",
        "材料与方法",
        "实验方法",
        "研究设计",
        "模型",
        "框架",
        "系统设计",
    },
    "data": {
        "data",
        "dataset",
        "datasets",
        "corpus",
        "data and materials",
        "数据",
        "数据集",
        "语料库",
    },
    "experiment": {
        "experiment",
        "experiments",
        "evaluation",
        "evaluations",
        "results",
        "experimental results",
        "evaluation results",
        "analysis",
        "findings",
        "ablations",
        "ablation",
        "实验",
        "实验结果",
        "结果",
        "研究结果",
        "评价",
        "评估",
        "分析",
        "发现",
        "消融",
    },
    "conclusion": {
        "conclusion",
        "conclusions",
        "discussion",
        "discussions",
        "future work",
        "limitations",
        "limitation",
        "结论",
        "总结",
        "讨论",
        "局限",
        "不足",
        "未来工作",
    },
}

STOP_SECTION_ALIASES = {
    "references",
    "bibliography",
    "appendix",
    "appendices",
    "supplementary material",
    "acknowledgments",
    "acknowledgements",
    "参考文献",
    "附录",
    "补充材料",
    "致谢",
}

STOP_SECTION_REASONS = {
    "references": "references",
    "bibliography": "references",
    "参考文献": "references",
    "appendix": "appendix",
    "appendices": "appendix",
    "supplementary material": "appendix",
    "附录": "appendix",
    "补充材料": "appendix",
    "acknowledgments": "acknowledgments",
    "acknowledgements": "acknowledgments",
    "致谢": "acknowledgments",
}

def match_section_heading(line: str) -> str | None:
    normalized = normalize_heading(line)
    if not normalized:
        return None
    if normalized in STOP_SECTION_ALIASES:
        return "stop"
    for section, aliases in SECTION_ALIASES.items():
        if normalized in aliases:
            return section
    return None

def stop_section_reason(line: str, *, allow_prefix: bool = False) -> str:
    normalized = normalize_heading(line)
    if not normalized:
        return ""
    reason = STOP_SECTION_REASONS.get(normalized, "")
    if reason or not allow_prefix:
        return reason
    if normalized.startswith(("references ", "bibliography ")):
        return "references"
    if normalized.startswith(("appendix ", "appendices ", "supplementary material ")):
        return "appendix"
    if normalized.startswith(("acknowledgments ", "acknowledgements ")):
        return "acknowledgments"
    return ""

def pdf_coverage_summary(pdf_path: Path, max_pages: int | None = None) -> dict[str, Any]:
    summary: dict[str, Any] = {
        "total_pages": None,
        "text_max_pages": max_pages,
        "text_pages_scanned": 0,
        "truncated_due_to_page_limit": False,
        "appendix_detected": False,
        "appendix_start_page": None,
        "references_start_page": None,
        "section_stop_reason": "",
        "section_stop_page": None,
    }
    if fitz is None or not pdf_path.is_file():
        return summary

    try:
        doc = fitz.open(pdf_path)
    except Exception:
        return summary

    try:
        total_pages = len(doc)
        page_limit = total_pages if max_pages is None else min(total_pages, max_pages)
        summary["total_pages"] = total_pages
        summary["text_pages_scanned"] = page_limit
        summary["truncated_due_to_page_limit"] = max_pages is not None and total_pages > max_pages

        for page_index in range(total_pages):
            page_number = page_index + 1
            text = doc[page_index].get_text("text")
            for raw_line in text.splitlines():
                line = clean_pdf_line(raw_line)
                if not line:
                    continue
                exact_reason = stop_section_reason(line)
                detected_reason = exact_reason or stop_section_reason(line, allow_prefix=True)
                if not detected_reason:
                    continue
                if detected_reason == "references" and summary["references_start_page"] is None:
                    summary["references_start_page"] = page_number
                if detected_reason == "appendix" and summary["appendix_start_page"] is None:
                    summary["appendix_start_page"] = page_number
                    summary["appendix_detected"] = True
                if exact_reason and not summary["section_stop_reason"]:
                    summary["section_stop_reason"] = exact_reason
                    summary["section_stop_page"] = page_number
                break
    finally:
        doc.close()
    return summary

def extract_appendix_page_texts(
    pdf_path: Path,
    appendix_start_page: int | None,
) -> list[dict[str, Any]]:
    if fitz is None or not pdf_path.is_file() or not appendix_start_page:
        return []

    try:
        doc = fitz.open(pdf_path)
    except Exception:
        return []

    pages: list[dict[str, Any]] = []
    try:
        start_index = max(int(appendix_start_page) - 1, 0)
        for page_index in range(start_index, len(doc)):
            text = doc[page_index].get_text("text")
            cleaned = normalize_whitespace(text)
            if cleaned:
                pages.append({"page": page_index + 1, "text": text})
    finally:
        doc.close()
    return pages

def appendix_section_title(line: str) -> str:
    cleaned = clean_pdf_line(line)
    if not cleaned:
        return ""
    lower = cleaned.lower()
    if lower in {"appendix", "appendices", "supplementary material"}:
        return ""
    if re.match(r"^fig(?:ure)?\.?\s*\d+", cleaned, re.IGNORECASE):
        return ""
    if re.match(r"^table\.?\s*\d+", cleaned, re.IGNORECASE):
        return ""
    if len(cleaned) > 120:
        return ""
    if re.match(r"^(?:appendix\s+)?[A-Z]\.?\s+.{3,}$", cleaned, re.IGNORECASE):
        return cleaned
    return ""

def extract_appendix_index(
    pdf_path: Path,
    pdf_coverage: dict[str, Any] | None = None,
    *,
    max_sections: int = 20,
    max_captions: int = 24,
) -> dict[str, Any]:
    pdf_coverage = pdf_coverage or {}
    start_page = pdf_coverage.get("appendix_start_page")
    index: dict[str, Any] = {
        "appendix_detected": bool(pdf_coverage.get("appendix_detected")),
        "start_page": start_page,
        "sections": [],
        "figure_captions": [],
        "table_captions": [],
    }
    if not index["appendix_detected"] or not start_page:
        return index

    seen_sections = set()
    seen_figure_captions = set()
    seen_table_captions = set()
    for page in extract_appendix_page_texts(pdf_path, int(start_page)):
        page_number = int(page.get("page", 0) or 0)
        text = str(page.get("text", ""))
        for raw_line in text.splitlines():
            title = appendix_section_title(raw_line)
            marker = normalize_title(title)
            if title and marker not in seen_sections and len(index["sections"]) < max_sections:
                seen_sections.add(marker)
                index["sections"].append({"title": title, "page": page_number})

        for caption in extract_caption_lines(text, "figure"):
            marker = f"{caption.get('id', '').lower()}::{caption.get('caption', '').lower()}"
            if marker in seen_figure_captions or len(index["figure_captions"]) >= max_captions:
                continue
            seen_figure_captions.add(marker)
            index["figure_captions"].append({**caption, "page_hint": f"p.{page_number}"})

        for caption in extract_caption_lines(text, "table"):
            marker = f"{caption.get('id', '').lower()}::{caption.get('caption', '').lower()}"
            if marker in seen_table_captions or len(index["table_captions"]) >= max_captions:
                continue
            seen_table_captions.add(marker)
            index["table_captions"].append({**caption, "page_hint": f"p.{page_number}"})
    return index

def extract_pdf_sections(pdf_path: Path, max_pages: int | None = None) -> dict[str, str]:
    if fitz is None:
        return {}
    sections: dict[str, list[str]] = {"preamble": []}
    current = "preamble"
    doc = fitz.open(pdf_path)
    try:
        page_limit = len(doc) if max_pages is None else min(len(doc), max_pages)
        for page_index in range(page_limit):
            text = doc[page_index].get_text("text")
            reached_stop = False
            for raw_line in text.splitlines():
                line = clean_pdf_line(raw_line)
                if not line:
                    continue
                heading = match_section_heading(line)
                if heading == "stop":
                    reached_stop = True
                    break
                if heading:
                    current = heading
                    sections.setdefault(current, [])
                    continue
                sections.setdefault(current, []).append(line)
            if reached_stop:
                break
    finally:
        doc.close()

    collapsed = {}
    for key, value in sections.items():
        if not value:
            continue
        text = re.sub(r"\s+", " ", " ".join(value)).strip()
        if text:
            collapsed[key] = text
    return collapsed

def extract_pdf_text(pdf_path: Path, max_pages: int | None = None) -> str:
    if fitz is None:
        return ""
    doc = fitz.open(pdf_path)
    try:
        page_limit = len(doc) if max_pages is None else min(len(doc), max_pages)
        texts = [doc[i].get_text("text") for i in range(page_limit)]
    finally:
        doc.close()
    return "\n".join(texts)

def normalize_caption_label(label: str) -> str:
    label = normalize_whitespace(unicodedata.normalize("NFKC", label))
    chinese_match = re.match(r"^(图|表)\s*([A-Z]?\d+[a-z]?)$", label, re.IGNORECASE)
    if chinese_match:
        return f"{chinese_match.group(1)} {chinese_match.group(2)}"
    extended_figure_match = re.match(
        r"^extended\s+data\s+fig(?:ure)?\.?\s*(\d+[a-z]?)$",
        label,
        re.IGNORECASE,
    )
    if extended_figure_match:
        return f"Extended Data Fig {extended_figure_match.group(1)}"
    extended_table_match = re.match(
        r"^extended\s+data\s+table\.?\s*(\d+[a-z]?)$",
        label,
        re.IGNORECASE,
    )
    if extended_table_match:
        return f"Extended Data Table {extended_table_match.group(1)}"
    scheme_match = re.match(r"^(scheme|algorithm)\.?\s*(\d+[a-z]?)$", label, re.IGNORECASE)
    if scheme_match:
        return f"{scheme_match.group(1).capitalize()} {scheme_match.group(2)}"
    supplementary_match = re.match(
        r"^(supplementary)\s+(fig(?:ure)?|table)\.?\s*(\d+[a-z]?)$",
        label,
        re.IGNORECASE,
    )
    if supplementary_match:
        return f"{supplementary_match.group(1)} {supplementary_match.group(2)} {supplementary_match.group(3)}"
    english_match = re.match(
        r"^(fig(?:ure)?|table)\.?\s*([AS]?\d+[a-z]?)$",
        label,
        re.IGNORECASE,
    )
    if english_match:
        return f"{english_match.group(1)} {english_match.group(2)}"
    return label

CAPTION_REFERENCE_VERBS = {
    "show",
    "shows",
    "illustrate",
    "illustrates",
    "plot",
    "plots",
    "present",
    "presents",
    "report",
    "reports",
    "depict",
    "depicts",
    "compare",
    "compares",
    "summarize",
    "summarizes",
    "demonstrate",
    "demonstrates",
}

def caption_label_key(label: str) -> str:
    normalized = normalize_caption_label(label).lower()
    normalized = re.sub(r"\bfigure\b", "fig", normalized)
    normalized = re.sub(r"\bfig\.\s*", "fig ", normalized)
    normalized = re.sub(r"\btable\.\s*", "table ", normalized)
    return normalize_whitespace(normalized)

def caption_preference_score(label: str, caption: str) -> int:
    cleaned_caption = normalize_whitespace(caption)
    lowered_label = normalize_whitespace(label).lower()
    first_word_match = re.match(r"^([A-Za-z][A-Za-z-]*)\b", cleaned_caption)
    first_word = first_word_match.group(1).lower() if first_word_match else ""
    score = len(cleaned_caption)
    if lowered_label.startswith(("figure", "table")):
        score += 25
    if first_word in CAPTION_REFERENCE_VERBS or cleaned_caption[:1].islower():
        score -= 80
    if len(cleaned_caption) < 12:
        score -= 20
    return score

def extract_caption_lines(pdf_text: str, kind: str) -> list[dict[str, str]]:
    grouped: dict[str, dict[str, str]] = {}
    scores: dict[str, int] = {}
    order: list[str] = []
    lines = [clean_pdf_line(line) for line in pdf_text.splitlines()]
    if kind == "figure":
        pattern = re.compile(
            r"^((?:"
            r"supplementary\s+fig(?:ure)?\.?\s*\d+[a-z]?"
            r"|extended\s+data\s+fig(?:ure)?\.?\s*\d+[a-z]?"
            r"|scheme\.?\s*\d+[a-z]?"
            r"|algorithm\.?\s*\d+[a-z]?"
            r"|fig(?:ure)?\.?\s*[AS]?\d+[a-z]?"
            r"|图\.?\s*[A-Z]?\d+[a-z]?"
            r"))(?!\.\d)(?:[:：.。,\s、|—–-]+|$)(.*)$",
            re.IGNORECASE,
        )
    else:
        pattern = re.compile(
            r"^((?:"
            r"supplementary\s+table\.?\s*\d+[a-z]?"
            r"|extended\s+data\s+table\.?\s*\d+[a-z]?"
            r"|table\.?\s*[AS]?\d+[a-z]?"
            r"|表\.?\s*[A-Z]?\d+[a-z]?"
            r"))(?!\.\d)(?:[:：.。,\s、|—–-]+|$)(.*)$",
            re.IGNORECASE,
        )
    for idx, line in enumerate(lines):
        if not line:
            continue
        match = pattern.match(line)
        if not match:
            continue
        label = normalize_caption_label(match.group(1))
        caption = normalize_whitespace(match.group(2))
        if not caption and idx + 1 < len(lines):
            caption = normalize_whitespace(lines[idx + 1])
        key = caption_label_key(label)
        if not key:
            continue
        candidate = {"id": label, "caption": caption}
        score = caption_preference_score(label, caption)
        if key not in grouped:
            grouped[key] = candidate
            scores[key] = score
            order.append(key)
            continue
        if score > scores[key]:
            grouped[key] = candidate
            scores[key] = score
    return [grouped[key] for key in order]
```

## File: skills/deeppapernote/scripts/_user_notes.py
size: 4668
sha: 83e476d25b3b7523c337e7dc2997ef24e3c6ae2c
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_user_notes.py

```text
"""Keep the user-owned My Notes section across note regeneration.

The My Notes section is the last top-level section of a note. Everything from
its heading line to the end of the file belongs to the user. A generated draft
holds only the standard guidance there; Formal Save swaps in the user's block
from the note it replaces.
"""

from __future__ import annotations

import hashlib
import re

from localization import note_schema

def _heading_re(language: str) -> re.Pattern[str]:
    heading = note_schema(language)["sections"]["my_notes"]
    return re.compile(rf"^##[ \t]+{re.escape(heading)}[ \t]*$", flags=re.MULTILINE)

def split_user_notes(text: str, language: str) -> tuple[str, str] | None:
    """Return (generated part, user block), or None when the heading is absent."""
    match = _heading_re(language).search(text)
    if match is None:
        return None
    return text[: match.start()], text[match.start():]

def _is_legacy_format(generated: str, language: str) -> bool:
    """A note saved before the Takeaways section existed; its My Notes may be model text."""
    heading = note_schema(language)["sections"]["takeaways"]
    return re.search(rf"^##[ \t]+{re.escape(heading)}[ \t]*$", generated, flags=re.MULTILINE) is None

def _block_body(block: str) -> str:
    _, _, body = block.partition("\n")
    return body

def is_default_block(block: str, language: str) -> bool:
    return _block_body(block).strip() == note_schema(language)["my_notes_guidance"]

def _content_chars(block: str) -> int:
    return sum(1 for ch in _block_body(block) if not ch.isspace())

def _normalize(text: str) -> str:
    return text.removeprefix("﻿").replace("\r\n", "\n").replace("\r", "\n")

def inspect_existing_user_notes(text: str, language: str) -> dict:
    """Describe the user content of a note about to be replaced, for the overwrite prompt."""
    parts = split_user_notes(_normalize(text), language)
    if parts is None:
        return {"status": "heading_missing", "chars": 0, "legacy_format": False}
    generated, block = parts
    if is_default_block(block, language):
        return {"status": "none", "chars": 0, "legacy_format": False}
    return {
        "status": "present",
        "chars": _content_chars(block),
        "legacy_format": _is_legacy_format(generated, language),
    }

def merge_user_notes(
    draft_text: str,
    existing_text: str | None,
    language: str,
    *,
    recorded_saved_sha256: str = "",
) -> tuple[str, dict]:
    """Carry the user's My Notes block from the existing note into the reviewed draft.

    The block is treated as untouched, and the draft's guidance kept, when the
    existing note is byte-identical to what Formal Save last recorded or when the
    block is exactly the standard guidance. Any other block is kept as it is.
    """
    if existing_text is None:
        return draft_text, {"status": "no_existing_note", "chars": 0, "user_notes_sha256": "", "legacy_format": False}
    if recorded_saved_sha256 and (
        hashlib.sha256(existing_text.encode("utf-8")).hexdigest() == recorded_saved_sha256
    ):
        return draft_text, {"status": "untouched", "chars": 0, "user_notes_sha256": "", "legacy_format": False}
    existing_parts = split_user_notes(_normalize(existing_text), language)
    if existing_parts is None:
        return draft_text, {"status": "heading_missing", "chars": 0, "user_notes_sha256": "", "legacy_format": False}
    existing_generated, block = existing_parts
    if is_default_block(block, language):
        return draft_text, {"status": "untouched", "chars": 0, "user_notes_sha256": "", "legacy_format": False}
    draft_parts = split_user_notes(draft_text, language)
    if draft_parts is None:
        raise ValueError(
            "The draft has no My Notes section, so the user's notes in the existing note "
            "cannot be kept; restore the section and rerun Final Note Lint."
        )
    if not block.endswith("\n"):
        block += "\n"
    return draft_parts[0] + block, {
        "status": "preserved",
        "chars": _content_chars(block),
        "user_notes_sha256": hashlib.sha256(block.encode("utf-8")).hexdigest(),
        "legacy_format": _is_legacy_format(existing_generated, language),
    }

def restore_generated_text(saved_text: str, draft_text: str, language: str) -> str:
    """Rebuild the reviewed draft from a saved note by swapping the user block back out."""
    saved_parts = split_user_notes(saved_text, language)
    draft_parts = split_user_notes(draft_text, language)
    if saved_parts is None or draft_parts is None:
        return saved_text
    return saved_parts[0] + draft_parts[1]
```

## File: skills/deeppapernote/scripts/_zotero_local.py
size: 31352
sha: 0fce5c43188953d0845c922087a9b1fd4e44b838
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_zotero_local.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/_zotero_local.py

## File: skills/deeppapernote/scripts/build_identity_contract.py
size: 2575
sha: 4c929e04d378da222a6538f472d2e7c7411057eb
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/build_identity_contract.py

```text
#!/usr/bin/env python3
"""Emit the accepted acquisition Paper Identity contract for trusted inputs."""

from __future__ import annotations

import argparse

from _identity import (
    ACCEPTED_IDENTITY_VERDICTS,
    build_canonical_identity_artifact,
    build_identity_repair_trace,
)
from common import (
    emit,
    maybe_load_json_record,
    require_ok_input_artifact,
)

def parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(description=__doc__ or "build identity contract")
    p.add_argument("--input", required=True, help="Metadata JSON path or JSON string.")
    p.add_argument("--resolve", required=True, help="Resolve artifact path for provenance.")
    p.add_argument("--trace-output", required=True, help="Identity repair trace JSON output path.")
    p.add_argument("--output", required=True, help="Canonical Identity Artifact JSON output path.")
    return p

def load_required_record(value: str, consumer: str, *, producer: str) -> dict:
    record = maybe_load_json_record(value)
    if record is None:
        raise SystemExit(f"{consumer} requires a JSON acquisition artifact input.")
    accepted = dict(require_ok_input_artifact(record, consumer))
    if str(accepted.get("script", "")) != producer:
        raise SystemExit(f"{consumer} requires a {producer} producer artifact.")
    return accepted

def main() -> None:
    args = parser().parse_args()
    metadata = load_required_record(
        args.input,
        "build_identity_contract.py",
        producer="collect_metadata.py",
    )
    if not isinstance(metadata.get("identity_observations"), list):
        raise SystemExit(
            "build_identity_contract.py requires metadata identity_observations."
        )
    source_record = load_required_record(
        args.resolve,
        "build_identity_contract.py",
        producer="resolve_paper.py",
    )

    trace = build_identity_repair_trace(
        metadata,
        source_record=source_record,
        resolve_artifact_path=args.resolve,
        metadata_artifact_path=args.input,
    )
    emit(trace, args.trace_output)

    identity = build_canonical_identity_artifact(
        metadata,
        source_record=source_record,
        repair_trace_path=args.trace_output,
        resolve_artifact_path=args.resolve,
        metadata_artifact_path=args.input,
    )
    emit(identity, args.output)
    verdict = str(identity.get("identity_verdict", "")).lower().replace("-", "_")
    if verdict not in ACCEPTED_IDENTITY_VERDICTS:
        raise SystemExit(1)

if __name__ == "__main__":
    main()
```

## File: skills/deeppapernote/scripts/build_synthesis_bundle.py
size: 25122
sha: 320f0160714530df16a30e159314a437e8ddf3ef
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/build_synthesis_bundle.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/build_synthesis_bundle.py

## File: skills/deeppapernote/scripts/check_environment.py
size: 5131
sha: 68f980e872dc0afd311b17ed259f2230de76cd5a
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/check_environment.py

```text
#!/usr/bin/env python3
"""Inspect the local DeepPaperNote environment for maintenance and troubleshooting."""

from __future__ import annotations

import argparse
import importlib.util
import shutil
import sys
from pathlib import Path

from _zotero_local import probe_zotero_local_api
from common import emit, env_config_value, runtime_config
from user_configuration import inspect_configuration

def parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(description=__doc__ or "check environment")
    p.add_argument("--output", default="", help="Optional JSON output path.")
    return p

def import_available(module_name: str) -> bool:
    return importlib.util.find_spec(module_name) is not None

def find_obsidian_candidates() -> list[str]:
    roots = [
        Path.home() / "Documents",
        Path.home() / "Desktop",
    ]
    results: list[str] = []
    seen: set[str] = set()
    for root in roots:
        if not root.exists():
            continue
        try:
            for path in root.rglob("*"):
                if not path.is_dir():
                    continue
                if path == root:
                    continue
                name = path.name.lower()
                if "obsidian" not in name and "vault" not in name:
                    continue
                resolved = str(path.resolve())
                if resolved in seen:
                    continue
                seen.add(resolved)
                results.append(resolved)
                if len(results) >= 8:
                    return results
        except Exception:
            continue
    return results

def find_local_zotero_hints() -> list[str]:
    candidates = [
        Path.home() / "Zotero",
        Path.home() / "Library" / "Application Support" / "Zotero",
    ]
    hits: list[str] = []
    for path in candidates:
        if path.exists():
            hits.append(str(path.resolve()))
    return hits

def main() -> None:
    args = parser().parse_args()
    configuration = inspect_configuration()
    config = runtime_config() if configuration["state"] == "ready" else {}
    zotero_local_api = probe_zotero_local_api()

    obsidian_vault = str(config.get("obsidian_vault", "")).strip()
    obsidian_vault_exists = bool(obsidian_vault) and Path(obsidian_vault).expanduser().exists()

    tesseract_path = shutil.which("tesseract") or ""
    pdftoppm_path = shutil.which("pdftoppm") or ""

    payload = {
        "status": "ok",
        "script": "check_environment.py",
        "tool_role": "maintenance",
        "user_configuration": configuration,
        "python": {
            "executable": sys.executable,
            "version": sys.version.split()[0],
            "fitz_installed": import_available("fitz"),
            "pytesseract_installed": import_available("pytesseract"),
            "pillow_installed": import_available("PIL"),
        },
        "obsidian": {
            "configured": bool(obsidian_vault),
            "vault_path": obsidian_vault,
            "vault_exists": obsidian_vault_exists,
            "papers_dir": str(config.get("papers_dir", "")),
            "output_dir": str(config.get("output_dir", "")),
            "candidate_vaults": find_obsidian_candidates(),
        },
        "workspace_fallback": {
            "available": True,
            "current_working_directory": str(Path.cwd().resolve()),
            "workspace_output_dir": str(config.get("workspace_output_dir", "DeepPaperNote_output")),
            "note": (
                "With save_mode=workspace, DeepPaperNote saves under the current working directory."
            ),
        },
        "zotero": {
            "local_hints": find_local_zotero_hints(),
            "local_api": zotero_local_api,
            "local_api_available": bool(zotero_local_api.get("ready")),
            "local_api_status": str(zotero_local_api.get("status", "error")),
            "local_api_version": str(zotero_local_api.get("api_version", "")),
            "local_api_schema_version": str(zotero_local_api.get("schema_version", "")),
            "mcp_available_from_script": False,
            "session_integration_checked_by_script": False,
            "note": (
                "The built-in read-only Local API check is reported here. Session-scoped "
                "library integrations must still be checked by the active agent at runtime."
            ),
        },
        "ocr": {
            "tesseract_installed": bool(tesseract_path),
            "tesseract_path": tesseract_path,
            "pytesseract_installed": import_available("pytesseract"),
            "pillow_installed": import_available("PIL"),
            "pdftoppm_installed": bool(pdftoppm_path),
            "pdftoppm_path": pdftoppm_path,
        },
        "metadata": {
            "maintenance_utility": True,
            "semantic_scholar_api_key_configured": bool(
                env_config_value(
                    "DEEPPAPERNOTE_SEMANTIC_SCHOLAR_API_KEY", "SEMANTIC_SCHOLAR_API_KEY"
                )
            ),
        },
    }
    emit(payload, args.output)

if __name__ == "__main__":
    main()
```

## File: skills/deeppapernote/scripts/citation_links.py
size: 11009
sha: 3c43afa2984d907b705a878082b14137e79f2601
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/citation_links.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/citation_links.py

## File: skills/deeppapernote/scripts/collect_metadata.py
size: 1198
sha: de837d1954b3eb1f11db0ab47ec60a3e8b37c090
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/collect_metadata.py

```text
#!/usr/bin/env python3
"""Collect and merge metadata from Zotero, DOI, Semantic Scholar, OpenAlex, arXiv, and publisher pages."""

from __future__ import annotations

from _identity import paper_id_for_record
from _metadata_sources import (
    collect_metadata_observations,
    resolve_reference,
)
from common import (
    base_parser,
    emit,
    maybe_load_json_record,
    require_ok_input_artifact,
)

def main() -> None:
    parser = base_parser(__doc__ or "collect metadata")
    args = parser.parse_args()

    if not args.input:
        raise SystemExit("collect_metadata.py requires --input.")

    input_record = maybe_load_json_record(args.input)
    if input_record is not None:
        record = dict(require_ok_input_artifact(input_record, "collect_metadata.py"))
    else:
        record = resolve_reference(args.input)

    metadata = dict(record)
    metadata["identity_observations"] = collect_metadata_observations(record)
    metadata["paper_id"] = args.paper_id or metadata.get("paper_id") or paper_id_for_record(metadata)
    metadata["status"] = "ok"
    metadata["script"] = "collect_metadata.py"
    emit(metadata, args.output)

if __name__ == "__main__":
    main()
```

## File: skills/deeppapernote/scripts/common.py
size: 7313
sha: 70d6933c65ca9947773fe012e39221c934c9ede0
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/common.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/common.py

## File: skills/deeppapernote/scripts/contracts.py
size: 29454
sha: cb82accd5e4fb7f2cad431422aec25fdb31dd09b
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/contracts.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/contracts.py

## File: skills/deeppapernote/scripts/create_input_record.py
size: 3213
sha: ce5ba413ecde581545d8071102a0644ab2356fb0
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/create_input_record.py

```text
#!/usr/bin/env python3
"""Create a deterministic paper input record from trusted metadata such as Zotero results."""

from __future__ import annotations

import argparse
import json

from _identity import paper_id_for_record
from common import emit, normalize_whitespace

def parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(description=__doc__ or "create input record")
    p.add_argument("--title", required=True, help="Canonical paper title.")
    p.add_argument("--translated-title", default="", help="Optional translated title.")
    p.add_argument("--authors-json", default="[]", help="JSON array of author names.")
    p.add_argument("--affiliations-json", default="[]", help="JSON array of affiliations.")
    p.add_argument("--year", default="", help="Publication year.")
    p.add_argument("--venue", default="", help="Venue or journal.")
    p.add_argument("--doi", default="", help="DOI if known.")
    p.add_argument("--arxiv-id", default="", help="arXiv id if known.")
    p.add_argument("--source-url", default="", help="Canonical source URL.")
    p.add_argument("--pdf-url", default="", help="Preferred PDF URL if known.")
    p.add_argument("--local-pdf-path", default="", help="Local PDF path if already available.")
    p.add_argument("--zotero-key", default="", help="Local Zotero item key.")
    p.add_argument("--abstract", default="", help="Abstract text when available.")
    p.add_argument("--source-type", default="", help="Optional explicit source type.")
    p.add_argument("--output", default="", help="Output JSON path.")
    return p

def parse_json_list(raw: str) -> list[str]:
    try:
        data = json.loads(raw)
    except Exception:
        return []
    if not isinstance(data, list):
        return []
    values: list[str] = []
    for item in data:
        cleaned = normalize_whitespace(str(item))
        if cleaned:
            values.append(cleaned)
    return values

def main() -> None:
    args = parser().parse_args()
    record = {
        "status": "ok",
        "script": "create_input_record.py",
        "title": normalize_whitespace(args.title),
        "translated_title": normalize_whitespace(args.translated_title),
        "authors": parse_json_list(args.authors_json),
        "affiliations": parse_json_list(args.affiliations_json),
        "year": normalize_whitespace(args.year),
        "venue": normalize_whitespace(args.venue),
        "doi": normalize_whitespace(args.doi),
        "arxiv_id": normalize_whitespace(args.arxiv_id),
        "source_url": normalize_whitespace(args.source_url),
        "pdf_url": normalize_whitespace(args.pdf_url),
        "local_pdf_path": normalize_whitespace(args.local_pdf_path),
        "zotero_key": normalize_whitespace(args.zotero_key),
        "abstract": normalize_whitespace(args.abstract),
        "source_type": normalize_whitespace(args.source_type)
        or ("zotero_seed" if normalize_whitespace(args.zotero_key) else "seed_record"),
        "metadata_sources": ["zotero_seed"] if normalize_whitespace(args.zotero_key) else ["seed_record"],
    }
    record["paper_id"] = paper_id_for_record(record)
    emit(record, args.output)

if __name__ == "__main__":
    main()
```

## File: skills/deeppapernote/scripts/extract_evidence.py
size: 31092
sha: ade4465833a63c832b7870288c514732598f8c18
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_evidence.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_evidence.py

## File: skills/deeppapernote/scripts/extract_pdf_assets.py
size: 64880
sha: 8f65f2d397e03c19f77953424ced48ed3e50d14b
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_pdf_assets.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_pdf_assets.py

## File: skills/deeppapernote/scripts/extract_source_text.py
size: 16272
sha: 01a31fb7285eb152bd73f216f46833a2246d28f8
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_source_text.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/extract_source_text.py

## File: skills/deeppapernote/scripts/fetch_pdf.py
size: 12000
sha: a303edb6defb8b6ae8b7b5e4b3ab2c8d3c7da700
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/fetch_pdf.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/fetch_pdf.py

## File: skills/deeppapernote/scripts/find_archived_source.py
size: 1723
sha: 8b37bf9ef558d56fa7c95d46352e8c5de2105e1d
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/find_archived_source.py

```text
#!/usr/bin/env python3
"""Find a verified local source without changing the Vault."""

from __future__ import annotations

import argparse
import re
from pathlib import Path

from common import emit, maybe_load_json_record, runtime_config
from paper_archive import ArchiveError, find_source

def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--input", required=True, help="Metadata JSON or original paper reference.")
    parser.add_argument("--vault", default="")
    parser.add_argument(
        "--reference", default="", help="Original user reference; preserves explicit versions."
    )
    parser.add_argument("--target-directory", default="")
    parser.add_argument("--source-sha256", default="")
    parser.add_argument("--output", default="")
    args = parser.parse_args()
    record = maybe_load_json_record(args.input) or {"source_url": args.input}
    if re.fullmatch(r"\d{4}\.\d{4,5}(?:v\d+)?", args.input):
        record["arxiv_id"] = args.input
    vault = args.vault or runtime_config().get("obsidian_vault", "")
    try:
        if not vault or not Path(vault).expanduser().is_dir():
            raise ArchiveError("invalid_vault", [vault])
        source = find_source(
            Path(vault).expanduser().resolve(),
            record,
            args.reference or args.input,
            args.target_directory,
            args.source_sha256,
        )
        emit({"status": "ok" if source else "not_found", **(source or {})}, args.output)
    except ArchiveError as exc:
        emit({"status": "blocked", "conflict_code": exc.code, "candidates": exc.paths}, args.output)
        raise SystemExit(2) from exc

if __name__ == "__main__":
    main()
```

## File: skills/deeppapernote/scripts/glossary_terms.py
size: 9773
sha: 53e154898cb4745e15fda4a147b03e35824d231f
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/glossary_terms.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/glossary_terms.py

## File: skills/deeppapernote/scripts/lint_grounding.py
size: 29858
sha: 60a3b367f3f1658dc42b642ff1ff48a6897f70ef
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/lint_grounding.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/lint_grounding.py

## File: skills/deeppapernote/scripts/lint_note.py
size: 77749
sha: e1547cf21cdaef0df181ad1ebb1c3c24ee23f6ff
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/lint_note.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/lint_note.py

## File: skills/deeppapernote/scripts/localization.py
size: 5393
sha: 07d4d0b73ee95036a2a04025ed52e502df6c20c8
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/localization.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/localization.py

## File: skills/deeppapernote/scripts/locate_zotero_attachment.py
size: 3721
sha: 5dd240340ac5bb840c089648d25148f9921806b0
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/locate_zotero_attachment.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/locate_zotero_attachment.py

## File: skills/deeppapernote/scripts/paper_archive.py
size: 21675
sha: 5e6745c266443fc278b400e314d593dadc24478a
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/paper_archive.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/paper_archive.py

## File: skills/deeppapernote/scripts/pdf_captions.py
size: 15048
sha: 048f9b866034528944994a56c9a0b79cf37a2593
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/pdf_captions.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/pdf_captions.py

## File: skills/deeppapernote/scripts/plan_figure_table_decisions.py
size: 34683
sha: 42525b473ed642953cee28110549e1c8ae7b2b6b
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/plan_figure_table_decisions.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/plan_figure_table_decisions.py

## File: skills/deeppapernote/scripts/plan_figures.py
size: 21551
sha: 2f901df149165d1f4b166b22f04b567e73ba8eb3
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/plan_figures.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/plan_figures.py

## File: skills/deeppapernote/scripts/research_memory.py
size: 3348
sha: f2965e6035e2307e48f8aab882d537650de2afb1
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/research_memory.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/research_memory.py

## File: skills/deeppapernote/scripts/resolve_paper.py
size: 7220
sha: 9b0fb12b9263528b92fc3d276f4ff4c7fc1a65c2
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/resolve_paper.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/resolve_paper.py

## File: skills/deeppapernote/scripts/run_pipeline.py
size: 14424
sha: 285d5abc3dc49a29489a3747e58b690886645ceb
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/run_pipeline.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/run_pipeline.py

## File: skills/deeppapernote/scripts/run_state.py
size: 53718
sha: fc08b85ad7e6c07a4245e29be9149b851e4d90db
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/run_state.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/run_state.py

## File: skills/deeppapernote/scripts/source_corpus.py
size: 17314
sha: 63c6df8a543af20012842d9bd81155c597af07f4
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/source_corpus.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/source_corpus.py

## File: skills/deeppapernote/scripts/type_profiles.py
size: 44639
sha: 3f44489ee4c63a6388ae2621a4a0d3e69a2c7edc
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/type_profiles.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/type_profiles.py

## File: skills/deeppapernote/scripts/user_configuration.py
size: 16238
sha: 8281cfafb10c3e77a79317d4c35f3b235bf8414f
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/user_configuration.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/user_configuration.py

## File: skills/deeppapernote/scripts/vault_connections.py
size: 11390
sha: f48c1a817bc389dee45f9fe63f5d3fa333b657e6
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/vault_connections.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/vault_connections.py

## File: skills/deeppapernote/scripts/write_obsidian_note.py
size: 51121
sha: f7ce30b157933c810e97dc7a2d6f12b49a2e1fdf
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/write_obsidian_note.py

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/scripts/write_obsidian_note.py

## File: skills/deeppapernote/references/architecture.md
size: 2869
sha: 330fb58300b89d24e56c112dd420470621fc55c2
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/architecture.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/architecture.md

## File: skills/deeppapernote/references/deep-analysis.md
size: 9540
sha: 80041690dd7af1d7952cddc2c7d3d63f96a0bc96
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/deep-analysis.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/deep-analysis.md

## File: skills/deeppapernote/references/domain-routing.md
size: 3229
sha: b661e5016619ab71e72ac66a5663331a243b36fe
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/domain-routing.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/domain-routing.md

## File: skills/deeppapernote/references/evidence-first.md
size: 12736
sha: 98c9192ea82e7338adfa61b8167981bfa6a3341f
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/evidence-first.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/evidence-first.md

## File: skills/deeppapernote/references/figure-placement.md
size: 11556
sha: 66f15a29709e594693d3014893c7b337019f205e
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/figure-placement.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/figure-placement.md

## File: skills/deeppapernote/references/final-quality-review.md
size: 9791
sha: 28404fd1bf0990a0336d0ef9a5c9e209be46aef9
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/final-quality-review.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/final-quality-review.md

## File: skills/deeppapernote/references/final-writing.md
size: 19308
sha: 813c8e20841144699aa2e15046bf82637e7beb3d
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/final-writing.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/final-writing.md

## File: skills/deeppapernote/references/metadata-sources.md
size: 1428
sha: b6ba84aa9b4c33fb2694f7b2c298f5ede31f4339
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/metadata-sources.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/metadata-sources.md

## File: skills/deeppapernote/references/note-quality.md
size: 5128
sha: bfafe50c57e09a8e1256c3393589c965e5ba79c7
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/note-quality.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/note-quality.md

## File: skills/deeppapernote/references/obsidian-format.md
size: 12900
sha: ca5fa88dd0b6cdfea988dfb93710b598d9b3d3ee
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/obsidian-format.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/obsidian-format.md

## File: skills/deeppapernote/references/output-language.md
size: 5011
sha: 7a6457d5153492f0d626f7df7fb0e7645325b089
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/output-language.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/output-language.md

## File: skills/deeppapernote/references/paper-archive.md
size: 4935
sha: 5da103a9b30c48aca841032792e0f8912d75ecf4
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/paper-archive.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/paper-archive.md

## File: skills/deeppapernote/references/paper-types.md
size: 6136
sha: 8c8d611eb6f19529f613a27519a4e79c171e0411
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/paper-types.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/paper-types.md

## File: skills/deeppapernote/references/run-state.md
size: 7652
sha: b3740c659f218fa73a4b4063b9d5149efc38c95e
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/run-state.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/run-state.md

## File: skills/deeppapernote/references/user-configuration.md
size: 5429
sha: 3ada4636da2f89cb61967a22ab57efd0f05986be
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/user-configuration.md

Content is not inlined in the catalog. Read the upstream file: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/references/user-configuration.md

## File: skills/deeppapernote/agents/openai.yaml
size: 283
sha: 519b6b0f88c5ef87ec2d3fd5ea3d77f15758e9b6
upstream: https://github.com/917dhj/deeppapernote/blob/main/skills/deeppapernote/agents/openai.yaml

```text
interface:
  display_name: "DeepPaperNote"
  short_description: "Generate a high-quality deep-reading note for one paper with a raw-source manifest workflow and Obsidian-oriented save semantics."
  default_prompt: "Use $deeppapernote to create the deep-reading note for this paper."
```

## File: README.md
upstream: https://github.com/917dhj/deeppapernote/blob/main/README.md

````text
<div align="center">

# DeepPaperNote

**Turn one complex paper into an Obsidian note you will actually want to keep.**

[English](./README.md) | [简体中文](./README.zh-CN.md)

[![Homepage](https://img.shields.io/badge/homepage-online-2563eb)](https://917dhj.github.io/DeepPaperNote/)
[![Release](https://img.shields.io/github/v/release/917Dhj/DeepPaperNote?display_name=tag&color=0f766e)](https://github.com/917Dhj/DeepPaperNote/releases)
[![License](https://img.shields.io/badge/license-MIT-c9a227)](./LICENSE)
[![Agents](https://img.shields.io/badge/agents-Claude%20Code%20%2B%20Codex-7c3aed)](./skills/deeppapernote/SKILL.md)
[![Output](https://img.shields.io/badge/output-Obsidian-16a34a)](./skills/deeppapernote/references/obsidian-format.md)

</div>

[![DeepPaperNote Hero](./assets/hero-deep-reading-workshop.png)](https://917dhj.github.io/DeepPaperNote/)

<p align="center">
  <em>Read one paper deeply. Add one durable page to your academic wiki.</em>
</p>

**When you study an important paper, the hard part is rarely the reading. It is turning what you understood into a note you can still use later.** Finding the source, matching figures to the text, copying numbers over, and tracing the method often take the better part of a day. And the papers you have read end up scattered, with nothing connecting one to another.

**DeepPaperNote is an agent skill that runs in Claude Code, Codex, and any other agent that supports skills.** Hand it a paper, and it reads the full text, writes an illustrated deep-reading note, saves it into your Obsidian vault, and links it to the other papers you have read. **Every paper you read grows your academic wiki a little.**

- 🚀 **One sentence, one note.** No prompts to learn, no templates to set up. Say "Generate a deep-reading note for this paper" with a title, DOI, arXiv ID, or PDF, and it finds the paper, reads it in full, extracts the figures, and saves the note into your vault.
- 🎓 **Reads like a domain expert.** From CS and AI to medicine, psychology, and the humanities, from new methods and benchmarks to surveys and clinical trials, your papers can all go to DeepPaperNote. Each of 10 paper types gets its own way of reading, drawn from more than 50 reviewer guidelines and reporting standards such as the NeurIPS review criteria and CONSORT, and tested on 34 real papers.
- 🎯 **Accurate and complete.** Every conclusion traces back to the paper, numbers come straight from its text and tables, and every main experiment is covered. When the paper does not report something, the note says so instead of guessing. Before saving, every note goes through an independent review and a strict quality check, and when the full text cannot be read, it never makes one up.
- 🖼️ **Figures, tables, and formulas in the text.** Figures and tables are cropped from the PDF and placed right where the note discusses them, and key formulas render in Obsidian. No more flipping back to the PDF.
- 🧠 **A vault that knows you better the more you use it.** Each note links to related papers in your vault and goes into the domain folders you already use. Record your research directions, and each note's Takeaways tell you what the paper means for your own work.

## 📰 News

- **[v3.0.0]** 🎓 Notes read each paper by its type: 10 paper types each get their own way of reading and note structure, every figure is visually reviewed before it enters the note, and an independent quality review runs before saving. [Release notes](https://github.com/917Dhj/DeepPaperNote/releases/tag/v3.0.0)
- **[v3.0.0]** 🧠 Every note joins your vault: notes gain Takeaways and Connections sections and carry structured properties, and two new skills, [`research-memory`](./skills/research-memory/README.md) and [`organize-wiki`](./skills/organize-wiki/README.md), tailor the takeaways to your research directions and keep your domain folders in order.
- **[v2.3.1]** 🔌 Added support for [DeepPaperNote Connector](https://github.com/917Dhj/DeepPaperNote-Connector): papers you save from your browser can get their notes in the same paper directory. [Release notes](https://github.com/917Dhj/DeepPaperNote/releases/tag/v2.3.1)

News lists only the three most recent user-facing milestones. See the [changelog](./CHANGELOG.md) and [GitHub Releases](https://github.com/917Dhj/DeepPaperNote/releases) for the full history.

## 🚀 Quick Start

### 1. Install

**Claude Code**: install from the plugin marketplace

```bash
claude plugin marketplace add 917Dhj/DeepPaperNote
claude plugin install deeppapernote@deeppapernote
```

**Codex and other agents that support skills**: install with the skills CLI

```bash
npx skills add 917Dhj/DeepPaperNote
```

The plugin installs all four skills at once. The skills CLI lets you choose which skills to install and which agents receive them. To try it out, `deeppapernote` alone is enough; see [below](#-from-paper-to-wiki) for what the other three do.

### 2. Install the PDF dependency

```bash
python3 -m pip install PyMuPDF
```

Python 3.10 or newer is required. If you use `uv`, run `uv pip install PyMuPDF` in the Python environment DeepPaperNote will use.

### 3. Read a paper

Tell your agent in one sentence, with the paper's title, DOI, URL, arXiv ID, or local PDF:

```text
Generate a deep-reading note for this paper: <paper>
Turn this paper into an Obsidian note: <paper>
```

You can also call it with the slash command `/deeppapernote <paper>`.

The first time, it asks you two things: whether notes should be in English or Chinese, and whether to save them in the current workspace or your Obsidian vault. It remembers your answers and does not ask again. To use a different language or location for one paper, just say so in the request. For environment variables and other settings, see [User Configuration](./skills/deeppapernote/references/user-configuration.md).

### 4. Update to a new version

**Claude Code**:

```bash
claude plugin marketplace update deeppapernote
claude plugin update deeppapernote@deeppapernote
```

Restart Claude Code to apply the update.

**skills CLI**:

```bash
npx skills update
```

## 👀 What a Note Looks Like

![A DeepPaperNote deep-reading note for Attention Is All You Need](./assets/demo-note.en.webp)

Every note has three parts: the opening and closing are the same for every paper, and the body in between follows the paper's type.

| Part | Sections |
| --- | --- |
| **Opening** | Core Information · Abstract · Contributions · One-Sentence Summary |
| **Body** | Method paper: Background and Problem → Limitations of Prior Work → Key Idea and Challenges → Method → Experimental Evidence → Deep Analysis<br>Benchmark: Evaluation Gap → Task and Design Goals → Construction and Quality Control → Evaluation Framework → Findings → What It Really Measures<br>Survey: Scope and Search → Taxonomy → Directions → Consensus, Disagreement, and Gaps<br>…10 paper types in all |
| **Closing** | Limitations · Takeaways · Connections · My Notes |

The closing sections are where the note meets your vault:

- **Takeaways**: Transferable Ideas and Next Steps, covering which of the paper's ideas carry over to other problems and what to do after reading it. Once you set up research memory, this part is written against your own research directions.
- **Connections**: links to related paper notes in your vault, and the key prior work you need to understand this paper.
- **My Notes**: yours to write in, and kept exactly as it is when the note is regenerated.

<details>
<summary><b>What appears in your vault after saving</b></summary>

Each paper gets its own folder, and figures and tables are named after their labels in the paper:

```text
<your papers folder>/
└── NLP/                       ← domain folder, reusing the ones you already have
    └── Attention_Is_All_You_Need/
        ├── Attention_Is_All_You_Need.en.md
        └── images/
            ├── figure_1.png
            └── table_2.png
```

Each note opens with structured properties you can query and filter with Obsidi
````
