feat: initial project scaffold and Stage 2 ingestion pipeline

- Project foundation: config, models, tools, llm, logging, prompts
- Tool schemas: extract_entities.json, search_landscape.json
- Stage 2 ingestion:
- ingestion/pubmed.py: PubMed Entrez client (batch fetch, rate-limit aware)
- ingestion/pmc.py: PMC XML full-text fetcher (JATS section parser)
- ingestion/semantic_scholar.py: citation count enrichment via S2 batch API
- ingestion/clinicaltrials.py: ClinicalTrials.gov v2 ALS trials client
- scripts/ingest_papers.py: orchestrates PubMed + PMC + citations
- scripts/ingest_trials.py: fetches recruiting ALS trials
- Stub modules for Stage 3-5 (rag/, graph/, agents/, app.py, main.py)
- MIT License, .gitignore (excludes data/, .env, logs/)
- CLAUDE.md architecture guide

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.env.example

@@ -0,0 +1,15 @@
+# Required
+ANTHROPIC_API_KEY=your_anthropic_api_key_here
+
+# Required for PubMed Entrez API (NCBI requires email for all Entrez requests)
+ENTREZ_EMAIL=your_email@example.com
+
+# Optional: increases PubMed Entrez rate limit from 3/s to 10/s
+NCBI_API_KEY=your_ncbi_api_key_here
+
+# Optional: LLM provider override (default: anthropic)
+LLM_PROVIDER=anthropic
+
+# Optional: logging
+CANDLE_LOG_LEVEL=WARNING
+CANDLE_LOG_DIR=logs

.gitignore

@@ -0,0 +1,31 @@
+# Environment
+.env
+
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+.venv/
+*.egg-info/
+dist/
+build/
+
+# Data (large files — generated by pipeline scripts)
+data/papers/
+data/trials/
+data/extracted/
+data/graph/
+data/chroma/
+
+# Logs
+logs/
+
+# uv
+.python-version
+
+# macOS
+.DS_Store
+
+# IDE
+.vscode/
+.idea/

CLAUDE.md

@@ -0,0 +1,94 @@
+# Candle-Fire — Architecture Guide
+
+## What This Is
+
+Candle-fire is a physician-facing ALS research intelligence tool. A physician asks a free-text question ("What's the evidence for tofersen targeting SOD1?") and gets a synthesized, cited answer grounded in ~500 curated ALS papers, enriched by a knowledge graph.
+
+**Sibling project**: beacon (patient-facing clinical trial finder at `../beacon`). Follow the same conventions.
+
+## Two-Layer Intelligence
+
+1. **Knowledge Graph (KG)**: NetworkX DiGraph linking Gene → Protein → Compound → Pathway → Phenotype → ClinicalTrial. Used to expand query entities before retrieval (e.g., "tofersen" → SOD1 → oxidative stress → related compounds).
+
+2. **RAG (Vector Search)**: ChromaDB collection of ~500 ALS paper abstracts/full-text. Citation-count-weighted re-ranking. Used to retrieve evidence passages for synthesis.
+
+**Query pipeline**: KG expansion first, then RAG retrieval with expanded entity context, then Claude synthesis.
+
+## Module Responsibilities
+
+| File/Dir | Responsibility |
+|---|---|
+| `config.py` | All constants: model names, file paths, ALS seed entities, API endpoints |
+| `models.py` | Dataclasses: `ALSPaper`, `ExtractedEntity`, `EntityRelationship`, `ResearchLandscape` |
+| `prompts.py` | System prompts for extraction agent and synthesis agent |
+| `tools.py` | Tool schema loader — reads JSON from `data/tools/`, exports typed tool params |
+| `llm.py` | LLM provider abstraction (Anthropic/OpenAI switchable via `LLM_PROVIDER` env var) |
+| `logging_config.py` | Structured JSON rotating log to `logs/candle_fire.log` |
+| `ingestion/pubmed.py` | PubMed Entrez client: fetch abstracts + metadata by MeSH query or PMID list |
+| `ingestion/pmc.py` | PMC XML full-text fetcher: structured section text for Open Access papers |
+| `ingestion/clinicaltrials.py` | ClinicalTrials.gov v2 client for ALS trials (no geo/distance, unlike beacon) |
+| `ingestion/semantic_scholar.py` | Citation count enrichment per PMID via Semantic Scholar API |
+| `extraction/extractor.py` | Claude Sonnet NER: batch 10 papers/call, exponential backoff, resumable via `.progress.json` |
+| `extraction/normalizer.py` | Entity name → canonical ID: HGNC alias table → PubChem → REST fallback |
+| `graph/builder.py` | Build NetworkX DiGraph from `entities.jsonl`; upsert nodes+edges; citation-weighted confidence |
+| `graph/query.py` | Graph traversal: `expand_query_entities()`, `find_trials_for_target()`, `get_entity_evidence()` |
+| `graph/serializer.py` | Save/load graph: pickle (fast load at startup) + JSON (human-readable export) |
+| `rag/indexer.py` | Build ChromaDB collection; section-aware chunking; `citation_count` in metadata |
+| `rag/retriever.py` | `search()`, `search_by_entities()`, citation-weighted re-ranking |
+| `agents/research_agent.py` | Multi-step synthesis agent (streaming): entity extraction → KG expansion → RAG → synthesis |
+| `app.py` | Gradio UI: loads graph + ChromaDB once at startup, streams responses |
+| `main.py` | CLI interface (Rich console) |
+| `scripts/` | Offline pipeline scripts: run once in order (ingest → extract → build_graph → build_index) |
+
+## Offline Pipeline Run Order
+
+Run these once to build the knowledge assets. Each is resumable.
+
+```bash
+# 1. Ingest papers from PubMed + PMC full text + Semantic Scholar citation counts
+uv run python scripts/ingest_papers.py
+
+# 2. Ingest ALS clinical trials (can run in parallel with step 1)
+uv run python scripts/ingest_trials.py
+
+# 3. Extract entities from papers using Claude (resumable — safe to interrupt)
+uv run python scripts/extract_entities.py
+
+# 4. Build knowledge graph
+uv run python scripts/build_graph.py
+
+# 5. Build ChromaDB vector index
+uv run python scripts/build_index.py
+```
+
+## Key Invariants
+
+- **Node key = `canonical_id`**, never raw entity name. Two papers mentioning "TDP-43" and "TARDBP" must produce one node.
+- **ChromaDB metadata values must be scalars** (str/int/float). Lists → comma-separated strings, deserialized on retrieval.
+- **KG expansion precedes RAG retrieval** in the agent loop. Never query ChromaDB with the raw user question alone.
+- **All heavy compute is offline**. No PubMed/extraction calls at query time.
+
+## Data File Locations
+
+```
+data/papers/papers.jsonl          — 500 ALS paper records (ALSPaper)
+data/trials/trials.jsonl          — ALS clinical trial records
+data/extracted/entities.jsonl     — per-paper NER output
+data/extracted/canonical_ids.json — entity name → canonical ID registry
+data/extracted/.progress.json     — extraction resumability tracker
+data/graph/als_graph.pkl          — NetworkX DiGraph (fast load)
+data/graph/als_graph.json         — human-readable graph export
+data/chroma/                      — ChromaDB SQLite store
+data/tools/                       — Claude tool input schemas (JSON)
+```
+
+## Environment Variables
+
+| Variable | Required | Default | Purpose |
+|---|---|---|---|
+| `ANTHROPIC_API_KEY` | Yes | — | Claude API |
+| `ENTREZ_EMAIL` | Yes | — | NCBI Entrez (required by NCBI) |
+| `NCBI_API_KEY` | No | — | Raises Entrez rate limit 3→10 req/s |
+| `LLM_PROVIDER` | No | `anthropic` | Switch to `openai` |
+| `CANDLE_LOG_LEVEL` | No | `WARNING` | Console log verbosity |
+| `CANDLE_LOG_DIR` | No | `logs` | Log file directory |

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 linkan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agents/research_agent.py

@@ -0,0 +1,2 @@
+"""Multi-step ALS research synthesis agent (streaming)."""
+# Stage 3 implementation (RAG-only), upgraded in Stage 4 (KG+RAG)

app.py

@@ -0,0 +1,2 @@
+"""Gradio web UI for candle-fire."""
+# Stage 5 implementation

config.py

@@ -0,0 +1,79 @@
+from pathlib import Path
+
+# Models
+SYNTHESIS_MODEL = "claude-sonnet-4-6"
+EXTRACTION_MODEL = "claude-sonnet-4-6"
+
+# External API endpoints
+CTGOV_BASE = "https://clinicaltrials.gov/api/v2/studies"
+SEMANTIC_SCHOLAR_BASE = "https://api.semanticscholar.org/graph/v1"
+HGNC_REST_BASE = "https://rest.genenames.org"
+PUBCHEM_REST_BASE = "https://pubchem.ncbi.nlm.nih.gov/rest/pug"
+
+# Data paths
+DATA_DIR = Path(__file__).parent / "data"
+PAPERS_PATH = DATA_DIR / "papers" / "papers.jsonl"
+TRIALS_PATH = DATA_DIR / "trials" / "trials.jsonl"
+ENTITIES_PATH = DATA_DIR / "extracted" / "entities.jsonl"
+CANONICAL_IDS_PATH = DATA_DIR / "extracted" / "canonical_ids.json"
+EXTRACTION_PROGRESS_PATH = DATA_DIR / "extracted" / ".progress.json"
+GRAPH_PICKLE_PATH = DATA_DIR / "graph" / "als_graph.pkl"
+GRAPH_JSON_PATH = DATA_DIR / "graph" / "als_graph.json"
+CHROMA_DIR = DATA_DIR / "chroma"
+CHROMA_COLLECTION = "als_papers"
+
+# PubMed ingestion defaults
+PUBMED_DEFAULT_QUERY = (
+    '"amyotrophic lateral sclerosis"[MeSH Major Topic] '
+    'AND ("2018"[PDAT]:"2024"[PDAT]) '
+    "AND hasabstract[text]"
+)
+PUBMED_DEFAULT_MAX = 500
+PUBMED_BATCH_SIZE = 200  # PMIDs per Entrez efetch call
+
+# Entity extraction
+EXTRACTION_BATCH_SIZE = 10  # papers per Claude call
+
+# RAG
+CHROMA_N_RESULTS = 10
+CHROMA_ENTITY_N_RESULTS = 15
+
+# Knowledge graph
+KG_EXPANSION_HOPS = 1  # hops for query entity expansion
+KG_MIN_EDGE_CONFIDENCE = 0.3  # edges below this are excluded from traversal
+
+# ALS seed entities — pre-populate the graph before paper-derived extraction
+ALS_SEED_ENTITIES = {
+    "genes": [
+        "SOD1", "TARDBP", "FUS", "C9orf72", "ATXN2",
+        "TBK1", "OPTN", "UBQLN2", "VCP", "NEK1",
+        "ANG", "SETX", "SIGMAR1", "CHCHD10", "MATR3",
+    ],
+    "proteins": [
+        "TDP-43", "FUS protein", "SOD1 protein", "Alsin",
+        "Optineurin", "p62", "Ubiquilin-2",
+    ],
+    "compounds": [
+        "riluzole", "edaravone", "tofersen", "AMX0035",
+        "mexiletine", "masitinib", "bosutinib",
+    ],
+    "mechanisms": [
+        "glutamate excitotoxicity", "oxidative stress",
+        "neuroinflammation", "protein aggregation",
+        "RNA metabolism dysfunction", "mitochondrial dysfunction",
+        "axonal transport defect", "autophagy impairment",
+    ],
+    "phenotypes": [
+        "upper motor neuron degeneration", "lower motor neuron degeneration",
+        "bulbar onset ALS", "spinal onset ALS",
+        "frontotemporal dementia", "respiratory failure",
+    ],
+}
+
+# ALS condition synonyms for ClinicalTrials.gov queries
+ALS_CONDITION_TERMS = [
+    "Amyotrophic Lateral Sclerosis",
+    "ALS",
+    "Motor Neuron Disease",
+    "Lou Gehrig's Disease",
+]

data/tools/extract_entities.json

@@ -0,0 +1,67 @@
+{
+  "type": "object",
+  "properties": {
+    "pmid": {
+      "type": "string",
+      "description": "The PubMed ID of the paper being analyzed."
+    },
+    "entities": {
+      "type": "array",
+      "description": "Biomedical entities identified in the abstract.",
+      "items": {
+        "type": "object",
+        "properties": {
+          "type": {
+            "type": "string",
+            "enum": ["Gene", "Protein", "Compound", "Pathway", "Phenotype", "Mechanism"],
+            "description": "Category of the biomedical entity."
+          },
+          "name": {
+            "type": "string",
+            "description": "Exact name as it appears in the text."
+          },
+          "confidence": {
+            "type": "number",
+            "minimum": 0.0,
+            "maximum": 1.0,
+            "description": "Confidence that this is correctly identified (1.0 = unambiguous)."
+          },
+          "mentions": {
+            "type": "integer",
+            "minimum": 1,
+            "description": "Number of times this entity is mentioned in the abstract."
+          }
+        },
+        "required": ["type", "name", "confidence", "mentions"]
+      }
+    },
+    "relationships": {
+      "type": "array",
+      "description": "Relationships between entities identified in the abstract.",
+      "items": {
+        "type": "object",
+        "properties": {
+          "source": {
+            "type": "string",
+            "description": "Name of the source entity (must match an entity in the entities list)."
+          },
+          "target": {
+            "type": "string",
+            "description": "Name of the target entity (must match an entity in the entities list)."
+          },
+          "type": {
+            "type": "string",
+            "enum": ["BINDS", "INHIBITS", "ASSOCIATED_WITH", "TESTED_IN", "EXPRESSED_IN", "CO_OCCURS"],
+            "description": "Type of relationship between source and target."
+          },
+          "evidence_text": {
+            "type": "string",
+            "description": "Brief excerpt from the abstract that supports this relationship."
+          }
+        },
+        "required": ["source", "target", "type", "evidence_text"]
+      }
+    }
+  },
+  "required": ["pmid", "entities", "relationships"]
+}

data/tools/search_landscape.json

@@ -0,0 +1,18 @@
+{
+  "type": "object",
+  "properties": {
+    "query_entities": {
+      "type": "array",
+      "description": "Genes, proteins, compounds, pathways, mechanisms, or phenotypes identified in the physician's question. Include all specific biological terms mentioned.",
+      "items": {
+        "type": "string"
+      },
+      "minItems": 1
+    },
+    "query_text": {
+      "type": "string",
+      "description": "The original physician question, used verbatim for semantic vector search."
+    }
+  },
+  "required": ["query_entities", "query_text"]
+}

extraction/extractor.py

@@ -0,0 +1,2 @@
+"""Claude Sonnet entity extractor — batch 10 papers/call, resumable."""
+# Stage 4 implementation

extraction/normalizer.py

@@ -0,0 +1,2 @@
+"""Entity name → canonical ID mapping (HGNC aliases + PubChem + REST fallback)."""
+# Stage 4 implementation

graph/builder.py

@@ -0,0 +1,2 @@
+"""NetworkX DiGraph construction from extracted entities and trials."""
+# Stage 4 implementation

graph/query.py

@@ -0,0 +1,2 @@
+"""Graph traversal utilities: entity expansion, trial lookup, evidence retrieval."""
+# Stage 4 implementation

graph/serializer.py

@@ -0,0 +1,2 @@
+"""Save and load the ALS knowledge graph (pickle + JSON)."""
+# Stage 4 implementation

ingestion/clinicaltrials.py

@@ -0,0 +1,118 @@
+"""ClinicalTrials.gov v2 client for ALS trials (adapted from beacon/trials_api.py)."""
+from __future__ import annotations
+
+import re
+import time
+
+import httpx
+
+from config import CTGOV_BASE
+from logging_config import get_logger
+
+_logger = get_logger("ingestion.clinicaltrials")
+
+# Known ALS-relevant targets for heuristic entity linking
+_KNOWN_TARGETS: list[tuple[re.Pattern, str]] = [
+    (re.compile(r"\bSOD1\b", re.IGNORECASE), "SOD1"),
+    (re.compile(r"\bTARDBP\b", re.IGNORECASE), "TARDBP"),
+    (re.compile(r"\bTDP-?43\b", re.IGNORECASE), "TARDBP"),
+    (re.compile(r"\bFUS\b", re.IGNORECASE), "FUS"),
+    (re.compile(r"\bC9orf72\b", re.IGNORECASE), "C9orf72"),
+    (re.compile(r"\bATXN2\b", re.IGNORECASE), "ATXN2"),
+    (re.compile(r"\bTBK1\b", re.IGNORECASE), "TBK1"),
+    (re.compile(r"\bNEK1\b", re.IGNORECASE), "NEK1"),
+    (re.compile(r"\bVCP\b", re.IGNORECASE), "VCP"),
+    (re.compile(r"\briluzole\b", re.IGNORECASE), "riluzole"),
+    (re.compile(r"\bedaravone\b", re.IGNORECASE), "edaravone"),
+    (re.compile(r"\btofersen\b", re.IGNORECASE), "tofersen"),
+    (re.compile(r"\bAMX0035\b", re.IGNORECASE), "AMX0035"),
+    (re.compile(r"\bmasitinib\b", re.IGNORECASE), "masitinib"),
+    (re.compile(r"\bbosutinib\b", re.IGNORECASE), "bosutinib"),
+    (re.compile(r"\bmexiletine\b", re.IGNORECASE), "mexiletine"),
+    (re.compile(r"\bantisense oligonucleotide\b", re.IGNORECASE), "antisense oligonucleotide"),
+    (re.compile(r"\bASO\b"), "antisense oligonucleotide"),
+    (re.compile(r"\bsiRNA\b", re.IGNORECASE), "siRNA"),
+    (re.compile(r"\bstem cell\b", re.IGNORECASE), "stem cell"),
+    (re.compile(r"\bgene therapy\b", re.IGNORECASE), "gene therapy"),
+]
+
+
+def fetch_als_trials(status: str = "RECRUITING") -> list[dict]:
+    """Fetch ALS interventional trials. Returns flat dicts ready for JSONL serialization."""
+    params: dict[str, str | int] = {
+        "query.cond": "Amyotrophic Lateral Sclerosis",
+        "filter.overallStatus": status,
+        "aggFilters": "studyType:int",
+        "pageSize": 1000,
+        "format": "json",
+    }
+
+    all_studies: list[dict] = []
+    while True:
+        for attempt in range(3):
+            try:
+                resp = httpx.get(CTGOV_BASE, params=params, timeout=30)
+                resp.raise_for_status()
+                body = resp.json()
+                break
+            except httpx.HTTPError as exc:
+                if attempt == 2:
+                    raise
+                wait = 2 ** attempt
+                _logger.warning(f"ClinicalTrials.gov error (attempt {attempt + 1}): {exc}")
+                time.sleep(wait)
+
+        page_studies = body.get("studies", [])
+        all_studies.extend(page_studies)
+        next_token = body.get("nextPageToken")
+        _logger.debug(
+            "ClinicalTrials.gov page",
+            extra={"data": {"count": len(page_studies), "has_next": bool(next_token)}},
+        )
+        if not next_token:
+            break
+        params["pageToken"] = next_token
+
+    trials = [_flatten_trial(s) for s in all_studies]
+    _logger.info("ALS trial fetch complete", extra={"data": {"total": len(trials)}})
+    return trials
+
+
+def _flatten_trial(study: dict) -> dict:
+    proto = study.get("protocolSection", {})
+    id_mod = proto.get("identificationModule", {})
+    desc_mod = proto.get("descriptionModule", {})
+    design_mod = proto.get("designModule", {})
+    sponsor_mod = proto.get("sponsorCollaboratorsModule", {})
+    arms_mod = proto.get("armsInterventionsModule", {})
+    status_mod = proto.get("statusModule", {})
+
+    nct_id = id_mod.get("nctId", "")
+    title = id_mod.get("briefTitle", "")
+    interventions = [
+        {"type": iv.get("type", ""), "name": iv.get("name", "")}
+        for iv in arms_mod.get("interventions", [])
+    ]
+    intervention_names = " ".join(iv["name"] for iv in interventions)
+
+    return {
+        "nct_id": nct_id,
+        "title": title,
+        "phase": ", ".join(design_mod.get("phases", [])) or "N/A",
+        "status": status_mod.get("overallStatus", ""),
+        "sponsor": sponsor_mod.get("leadSponsor", {}).get("name", ""),
+        "summary": desc_mod.get("briefSummary", ""),
+        "interventions": interventions,
+        "start_date": status_mod.get("startDateStruct", {}).get("date", ""),
+        "url": f"https://clinicaltrials.gov/study/{nct_id}" if nct_id else "",
+        "target_entities": extract_target_entities(f"{title} {intervention_names}"),
+    }
+
+
+def extract_target_entities(text: str) -> list[str]:
+    """Scan text for known ALS target names. Returns sorted canonical entity names."""
+    found: set[str] = set()
+    for pattern, canonical in _KNOWN_TARGETS:
+        if pattern.search(text):
+            found.add(canonical)
+    return sorted(found)

ingestion/pmc.py

@@ -0,0 +1,117 @@
+"""PMC XML full-text fetcher for Open Access papers."""
+from __future__ import annotations
+
+import os
+import time
+import xml.etree.ElementTree as ET
+
+from Bio import Entrez
+
+from logging_config import get_logger
+
+_logger = get_logger("ingestion.pmc")
+
+
+def _configure_entrez() -> None:
+    email = os.environ.get("ENTREZ_EMAIL")
+    if not email:
+        raise EnvironmentError("ENTREZ_EMAIL environment variable is required")
+    Entrez.email = email
+    api_key = os.getenv("NCBI_API_KEY")
+    if api_key:
+        Entrez.api_key = api_key
+
+
+def _sleep() -> None:
+    time.sleep(0.1 if os.getenv("NCBI_API_KEY") else 0.4)
+
+
+def get_pmcids(pmids: list[str]) -> dict[str, str]:
+    """
+    Map PubMed IDs to PMC IDs for papers with Open Access full text.
+    Returns {pmid: pmcid}.
+    """
+    _configure_entrez()
+    if not pmids:
+        return {}
+
+    result: dict[str, str] = {}
+    for i in range(0, len(pmids), 200):
+        batch = pmids[i : i + 200]
+        for attempt in range(3):
+            try:
+                handle = Entrez.elink(dbfrom="pubmed", db="pmc", id=",".join(batch))
+                link_sets = Entrez.read(handle)
+                handle.close()
+                break
+            except Exception as exc:
+                if attempt == 2:
+                    _logger.warning(f"elink failed: {exc}")
+                    link_sets = []
+                    break
+                time.sleep(2 ** attempt)
+
+        for link_set in link_sets:
+            source_ids = link_set.get("IdList", [])
+            source_id = str(source_ids[0]) if source_ids else None
+            for db_link in link_set.get("LinkSetDb", []):
+                if db_link.get("DbTo") == "pmc":
+                    links = db_link.get("Link", [])
+                    if links and source_id:
+                        result[source_id] = str(links[0]["Id"])
+                    break
+        _sleep()
+
+    _logger.info("PMC ID lookup", extra={"data": {"pmids": len(pmids), "found": len(result)}})
+    return result
+
+
+def fetch_full_text(pmcid: str) -> str | None:
+    """
+    Fetch PMC XML for a single PMCID and parse into section-labeled text.
+    Returns None if the fetch fails or the article has no body sections.
+    """
+    _configure_entrez()
+    for attempt in range(3):
+        try:
+            handle = Entrez.efetch(db="pmc", id=pmcid, rettype="xml", retmode="xml")
+            xml_data = handle.read()
+            handle.close()
+            break
+        except Exception as exc:
+            if attempt == 2:
+                _logger.warning(f"PMC fetch failed for PMCID {pmcid}: {exc}")
+                return None
+            time.sleep(2 ** attempt)
+    _sleep()
+    return _parse_jats_xml(xml_data)
+
+
+def _parse_jats_xml(xml_data: bytes) -> str | None:
+    """Extract section-labeled text from JATS/PMC XML."""
+    try:
+        root = ET.fromstring(xml_data)
+    except ET.ParseError:
+        return None
+
+    body = root.find(".//body")
+    if body is None:
+        return None
+
+    sections: list[str] = []
+    for sec in body.findall(".//sec"):
+        # Skip nested sections — only top-level sec elements under body
+        if sec in body:
+            title_el = sec.find("title")
+            title = (title_el.text or "Section").strip() if title_el is not None else "Section"
+
+            paragraphs: list[str] = []
+            for p in sec.findall("p"):
+                text = "".join(p.itertext()).strip()
+                if text:
+                    paragraphs.append(text)
+
+            if paragraphs:
+                sections.append(f"[{title}]\n" + "\n".join(paragraphs))
+
+    return "\n\n".join(sections) if sections else None

ingestion/pubmed.py

@@ -0,0 +1,109 @@
+"""PubMed Entrez API client for ALS paper ingestion."""
+from __future__ import annotations
+
+import os
+import time
+
+from Bio import Entrez, Medline
+
+from config import PUBMED_BATCH_SIZE
+from logging_config import get_logger
+from models import ALSPaper
+
+_logger = get_logger("ingestion.pubmed")
+
+
+def _configure_entrez() -> None:
+    email = os.environ.get("ENTREZ_EMAIL")
+    if not email:
+        raise EnvironmentError("ENTREZ_EMAIL environment variable is required by NCBI")
+    Entrez.email = email
+    api_key = os.getenv("NCBI_API_KEY")
+    if api_key:
+        Entrez.api_key = api_key
+
+
+def _sleep() -> None:
+    """Respect NCBI rate limits: 10 req/s with API key, 3 req/s without."""
+    time.sleep(0.1 if os.getenv("NCBI_API_KEY") else 0.4)
+
+
+def search_pmids(query: str, max_results: int = 500) -> list[str]:
+    """Search PubMed with a query string and return a list of PMIDs."""
+    _configure_entrez()
+    handle = Entrez.esearch(db="pubmed", term=query, retmax=max_results)
+    record = Entrez.read(handle)
+    handle.close()
+    pmids = list(record["IdList"])
+    _logger.info("PubMed esearch", extra={"data": {"count": len(pmids), "query": query[:80]}})
+    return pmids
+
+
+def fetch_by_pmids(pmids: list[str]) -> list[ALSPaper]:
+    """Fetch and parse paper records for a list of PMIDs."""
+    _configure_entrez()
+    papers: list[ALSPaper] = []
+
+    for i in range(0, len(pmids), PUBMED_BATCH_SIZE):
+        batch = pmids[i : i + PUBMED_BATCH_SIZE]
+        _logger.debug("Fetching Entrez batch", extra={"data": {"batch": i // PUBMED_BATCH_SIZE + 1, "size": len(batch)}})
+
+        for attempt in range(3):
+            try:
+                handle = Entrez.efetch(db="pubmed", id=",".join(batch), rettype="medline", retmode="text")
+                records = list(Medline.parse(handle))
+                handle.close()
+                break
+            except Exception as exc:
+                if attempt == 2:
+                    raise
+                wait = 2 ** attempt
+                _logger.warning(f"Entrez fetch error (attempt {attempt + 1}): {exc}")
+                time.sleep(wait)
+
+        for record in records:
+            paper = _parse_record(record)
+            if paper:
+                papers.append(paper)
+        _sleep()
+
+    _logger.info("PubMed fetch complete", extra={"data": {"total": len(papers)}})
+    return papers
+
+
+def _parse_record(record: dict) -> ALSPaper | None:
+    """Convert a Biopython Medline record to ALSPaper. Returns None if no abstract."""
+    pmid = record.get("PMID", "")
+    abstract = record.get("AB", "")
+    if not pmid or not abstract:
+        return None
+
+    authors = record.get("FAU", record.get("AU", []))
+    if isinstance(authors, str):
+        authors = [authors]
+
+    # "DP" field: "2023 Jan 15", "2023 Jan", "2023"
+    year = 0
+    date_str = record.get("DP", "")
+    if date_str:
+        try:
+            year = int(date_str.split()[0])
+        except (ValueError, IndexError):
+            pass
+
+    # DOI from AID list: ["10.1093/xxx [doi]", "S0092-8674(23)00001-1 [pii]"]
+    doi = ""
+    for aid in record.get("AID", []):
+        if aid.endswith("[doi]"):
+            doi = aid.replace(" [doi]", "").strip()
+            break
+
+    return ALSPaper(
+        pmid=pmid,
+        title=record.get("TI", ""),
+        abstract=abstract,
+        authors=authors,
+        year=year,
+        doi=doi,
+        mesh_terms=record.get("MH", []),
+    )

ingestion/semantic_scholar.py

@@ -0,0 +1,64 @@
+"""Semantic Scholar API client for citation count enrichment."""
+from __future__ import annotations
+
+import time
+
+import httpx
+
+from config import SEMANTIC_SCHOLAR_BASE
+from logging_config import get_logger
+
+_logger = get_logger("ingestion.semantic_scholar")
+
+_BATCH_SIZE = 500  # Semantic Scholar batch endpoint limit
+
+
+def fetch_citation_counts(pmids: list[str]) -> dict[str, int]:
+    """
+    Fetch citation counts for a list of PMIDs via Semantic Scholar batch API.
+    Returns {pmid: citation_count}. PMIDs with no S2 record are omitted.
+    """
+    if not pmids:
+        return {}
+
+    result: dict[str, int] = {}
+
+    for i in range(0, len(pmids), _BATCH_SIZE):
+        batch = pmids[i : i + _BATCH_SIZE]
+        ids = [f"PMID:{pmid}" for pmid in batch]
+
+        for attempt in range(3):
+            try:
+                resp = httpx.post(
+                    f"{SEMANTIC_SCHOLAR_BASE}/paper/batch",
+                    params={"fields": "citationCount,externalIds"},
+                    json={"ids": ids},
+                    timeout=30,
+                )
+                resp.raise_for_status()
+                papers = resp.json()
+                break
+            except httpx.HTTPError as exc:
+                if attempt == 2:
+                    _logger.warning(f"Semantic Scholar batch failed: {exc}")
+                    papers = []
+                    break
+                time.sleep(2 ** attempt)
+
+        for paper in papers:
+            if paper is None:
+                continue
+            ext = paper.get("externalIds") or {}
+            pmid = ext.get("PubMed")
+            count = paper.get("citationCount")
+            if pmid and count is not None:
+                result[str(pmid)] = int(count)
+
+        # Free tier allows ~100 requests per 5 minutes; 1s delay keeps us safe
+        time.sleep(1.0)
+
+    _logger.info(
+        "Semantic Scholar citation fetch",
+        extra={"data": {"requested": len(pmids), "found": len(result)}},
+    )
+    return result

llm.py

@@ -0,0 +1,87 @@
+import os
+from abc import ABC, abstractmethod
+
+import anthropic
+import openai
+
+from config import SYNTHESIS_MODEL
+
+
+def cached_system(text: str) -> list[dict]:
+    """Wrap a system prompt string for Anthropic prompt caching."""
+    return [{"type": "text", "text": text, "cache_control": {"type": "ephemeral"}}]
+
+
+def cached_tools(tools: list[dict]) -> list[dict]:
+    """Mark the last tool with cache_control so the full tool list is cached."""
+    if not tools:
+        return tools
+    return [*tools[:-1], {**tools[-1], "cache_control": {"type": "ephemeral"}}]
+
+
+def _normalize_messages(messages: list) -> list[dict]:
+    """Convert message objects or dicts to {role, content} dicts."""
+    result = []
+    for msg in messages:
+        if hasattr(msg, "type"):
+            raw_role, content = msg.type, msg.content
+        else:
+            raw_role, content = msg.get("role", "user"), msg.get("content", "")
+
+        if raw_role in ("human", "user"):
+            role = "user"
+        elif raw_role in ("ai", "assistant"):
+            role = "assistant"
+        else:
+            continue
+
+        result.append({"role": role, "content": content})
+    return result
+
+
+class LLMProvider(ABC):
+    @abstractmethod
+    def complete(self, messages: list, system: str = "") -> str: ...
+
+
+class AnthropicProvider(LLMProvider):
+    def __init__(self):
+        self._client = anthropic.Anthropic()
+        self._model = SYNTHESIS_MODEL
+
+    def complete(self, messages: list, system: str = "") -> str:
+        kwargs = dict(
+            model=self._model,
+            max_tokens=8192,
+            messages=_normalize_messages(messages),
+        )
+        if system:
+            kwargs["system"] = cached_system(system)
+        response = self._client.messages.create(**kwargs)
+        return next((b.text for b in response.content if b.type == "text"), "")
+
+
+class OpenAIProvider(LLMProvider):
+    MODEL = "gpt-4o"
+
+    def __init__(self):
+        self._client = openai.OpenAI()
+
+    def complete(self, messages: list, system: str = "") -> str:
+        normalized = _normalize_messages(messages)
+        if system:
+            normalized = [{"role": "system", "content": system}] + normalized
+        response = self._client.chat.completions.create(
+            model=self.MODEL,
+            messages=normalized,
+        )
+        return response.choices[0].message.content or ""
+
+
+def get_provider(name: str | None = None) -> LLMProvider:
+    name = name or os.getenv("LLM_PROVIDER", "anthropic")
+    if name == "openai":
+        return OpenAIProvider()
+    if name == "anthropic":
+        return AnthropicProvider()
+    raise ValueError(f"Unknown LLM provider: {name!r}. Choose 'anthropic' or 'openai'.")

logging_config.py

@@ -0,0 +1,60 @@
+from __future__ import annotations
+
+import json
+import logging
+import logging.handlers
+import os
+from datetime import datetime, timezone
+
+_LOG_LEVEL = os.getenv("CANDLE_LOG_LEVEL", "WARNING").upper()
+_LOG_DIR = os.getenv("CANDLE_LOG_DIR", "logs")
+
+_configured = False
+
+
+class _JSONFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        entry: dict = {
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+            "level": record.levelname,
+            "logger": record.name,
+            "message": record.getMessage(),
+        }
+        if hasattr(record, "data"):
+            entry["data"] = record.data
+        return json.dumps(entry, default=str)
+
+
+def _setup() -> None:
+    global _configured
+    if _configured:
+        return
+    _configured = True
+
+    level = getattr(logging, _LOG_LEVEL, logging.WARNING)
+
+    root = logging.getLogger("candle_fire")
+    root.setLevel(logging.DEBUG)
+    root.propagate = False
+
+    console = logging.StreamHandler()
+    console.setLevel(level)
+    console.setFormatter(logging.Formatter("%(levelname)s:%(name)s:%(message)s"))
+    root.addHandler(console)
+
+    os.makedirs(_LOG_DIR, exist_ok=True)
+    fh = logging.handlers.RotatingFileHandler(
+        os.path.join(_LOG_DIR, "candle_fire.log"),
+        maxBytes=10 * 1024 * 1024,
+        backupCount=5,
+        encoding="utf-8",
+    )
+    fh.setLevel(logging.DEBUG)
+    fh.setFormatter(_JSONFormatter())
+    root.addHandler(fh)
+
+
+def get_logger(name: str) -> logging.Logger:
+    """Return a ``candle_fire.<name>`` logger, configuring handlers on first call."""
+    _setup()
+    return logging.getLogger(f"candle_fire.{name}")

main.py

@@ -0,0 +1,2 @@
+"""CLI entry point for candle-fire."""
+# Stage 3 implementation

models.py

@@ -0,0 +1,126 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Optional
+
+
+@dataclass
+class ALSPaper:
+    pmid: str
+    title: str
+    abstract: str
+    authors: list[str]
+    year: int
+    doi: str
+    mesh_terms: list[str]
+    # Populated after PMC fetch (None if not Open Access)
+    full_text: Optional[str] = None
+    # Populated after Semantic Scholar enrichment
+    citation_count: int = 0
+    # Populated after entity extraction
+    entity_names: list[str] = field(default_factory=list)
+
+    def to_dict(self) -> dict:
+        return {
+            "pmid": self.pmid,
+            "title": self.title,
+            "abstract": self.abstract,
+            "authors": self.authors,
+            "year": self.year,
+            "doi": self.doi,
+            "mesh_terms": self.mesh_terms,
+            "full_text": self.full_text,
+            "citation_count": self.citation_count,
+            "entity_names": self.entity_names,
+        }
+
+    @classmethod
+    def from_dict(cls, d: dict) -> ALSPaper:
+        return cls(
+            pmid=d["pmid"],
+            title=d["title"],
+            abstract=d["abstract"],
+            authors=d.get("authors", []),
+            year=d.get("year", 0),
+            doi=d.get("doi", ""),
+            mesh_terms=d.get("mesh_terms", []),
+            full_text=d.get("full_text"),
+            citation_count=d.get("citation_count", 0),
+            entity_names=d.get("entity_names", []),
+        )
+
+
+@dataclass
+class ExtractedEntity:
+    type: str  # Gene | Protein | Compound | Pathway | Phenotype | Mechanism
+    name: str  # raw name from text
+    canonical_id: str  # normalized canonical identifier
+    confidence: float  # 0.0–1.0
+    mentions: int  # occurrence count in paper
+
+
+@dataclass
+class EntityRelationship:
+    source: str  # canonical_id of source entity
+    target: str  # canonical_id of target entity
+    relation_type: str  # BINDS | INHIBITS | ASSOCIATED_WITH | TESTED_IN | EXPRESSED_IN | CO_OCCURS
+    evidence_pmids: list[str]
+    confidence: float  # average confidence across supporting papers
+    evidence_text: str = ""  # representative excerpt from the paper
+
+
+@dataclass
+class PaperExtractionResult:
+    pmid: str
+    entities: list[ExtractedEntity]
+    relationships: list[EntityRelationship]
+
+    def to_dict(self) -> dict:
+        return {
+            "pmid": self.pmid,
+            "entities": [
+                {
+                    "type": e.type,
+                    "name": e.name,
+                    "canonical_id": e.canonical_id,
+                    "confidence": e.confidence,
+                    "mentions": e.mentions,
+                }
+                for e in self.entities
+            ],
+            "relationships": [
+                {
+                    "source": r.source,
+                    "target": r.target,
+                    "relation_type": r.relation_type,
+                    "evidence_pmids": r.evidence_pmids,
+                    "confidence": r.confidence,
+                    "evidence_text": r.evidence_text,
+                }
+                for r in self.relationships
+            ],
+        }
+
+
+@dataclass
+class TrialSummary:
+    nct_id: str
+    title: str
+    phase: str
+    status: str
+    interventions: list[str]
+    target_entities: list[str]  # canonical_ids of targeted genes/proteins/compounds
+    sponsor: str = ""
+    start_date: str = ""
+    url: str = ""
+
+
+@dataclass
+class ResearchLandscape:
+    query: str
+    mechanisms: list[str]  # 2–3 key mechanism bullet points
+    entities: list[dict]  # {name, type, description, paper_count}
+    papers: list[dict]  # top 5: {pmid, title, year, doi, citation_count}
+    trials: list[TrialSummary]
+    evidence_count: int  # total papers retrieved
+    generated_at: str  # ISO timestamp

project_proposal/proposal.md

@@ -0,0 +1,256 @@
+# Candle-Fire: ALS Research Landscape Tool
+
+## Project Initiative
+
+Candle-fire is a physician-facing ALS research intelligence platform — a sibling to beacon. Where beacon helps patients find clinical trials, candle-fire helps physicians understand the *research evidence behind* those trials and ALS biology more broadly.
+
+**The problem**: A physician encounters a clinical trial for an antisense oligonucleotide targeting SOD1. To evaluate it, they need to know: What is the evidence base for SOD1 as a target? What are the known mechanisms? What else has been tried? Today this requires hours of manual literature review.
+
+**The solution**: A physician types a free-text question ("What's the evidence for tofersen targeting SOD1 in ALS?") and gets a synthesized, cited answer grounded in ~500 curated ALS papers, enriched by a knowledge graph linking genes, proteins, compounds, pathways, and clinical trials — delivered in under 30 seconds.
+
+**End state**: A Gradio web app deployable on HuggingFace Spaces. Two layers of intelligence: (1) a vector RAG layer for semantic paper retrieval, (2) a knowledge graph layer for entity-level relationship traversal. Claude Sonnet synthesizes both into a structured research landscape report with citations, mechanism summaries, and related trial links.
+
+---
+
+## Critique of Original Plan
+
+1. **RAG alone is semantically blind.** Query for "riluzole" must surface "glutamate excitotoxicity" via the KG (riluzole → INHIBITS → glutamate excitotoxicity). KG expansion *precedes* RAG retrieval — not optional.
+
+2. **Entity normalization is a prerequisite for KG construction.** TDP-43 / TARDBP / TDP43 must resolve to one canonical node before graph build. The normalizer runs as part of extraction and writes `canonical_ids.json` as the single source of truth.
+
+3. **All heavy compute is offline batch.** Ingestion, extraction, KG build, RAG indexing — run once. Query time: NetworkX pickle (read) + ChromaDB (read) + Anthropic API only.
+
+4. **Citation counts weight evidence quality.** A paper cited 500 times is stronger evidence than one cited 5 times. Fetch citation counts from Semantic Scholar API (free, no auth). Use to re-rank RAG results and weight KG edge confidence.
+
+5. **PMC XML full text, not raw PDFs.** ~50-60% of recent ALS papers are PMC Open Access. Ingest structured XML (section-labeled: Introduction, Methods, Results, Discussion) via PubMed Entrez. Always fall back to abstract. Avoid PDF parsing — too brittle and legally ambiguous for paywalled papers.
+
+---
+
+## Architecture
+
+### Tech Stack
+- **Language**: Python 3.11+, uv package manager
+- **LLM**: Claude Sonnet 4-6 (entity extraction + research synthesis)
+- **Vector store**: ChromaDB (SQLite-backed, persistent, no separate service)
+- **Graph**: NetworkX DiGraph (pickled, loaded once at startup, scales to 20K+ papers)
+- **Embeddings**: `all-MiniLM-L6-v2` via sentence-transformers (local, no API needed)
+- **UI**: Gradio 6 (HuggingFace Spaces deployment, mirrors beacon)
+- **Paper source**: PubMed Entrez API + PMC XML full text (hybrid)
+- **Citation counts**: Semantic Scholar API (free, by DOI/PMID)
+
+### Data Flow
+
+```
+OFFLINE PIPELINE (run once, in order):
+
+  Stage 1: Ingestion
+    scripts/ingest_papers.py
+      → ingestion/pubmed.py (PubMed Entrez, batch 200 PMIDs, abstract always)
+      → ingestion/pmc.py (PMC XML full text for OA papers, ~50-60% coverage)
+      → ingestion/semantic_scholar.py (citation counts per PMID/DOI)
+      → data/papers/papers.jsonl  [ALSPaper: title, abstract, full_text?, citation_count, ...]
+
+    scripts/ingest_trials.py  [parallel with above]
+      → ingestion/clinicaltrials.py (ClinicalTrials.gov v2, condition=ALS)
+      → data/trials/trials.jsonl
+
+  Stage 2: Entity Extraction
+    scripts/extract_entities.py
+      → extraction/extractor.py (Claude Sonnet, 10 papers/call, retry+backoff, resumable)
+      → extraction/normalizer.py (HGNC alias table + PubChem + HGNC REST fallback)
+      → data/extracted/entities.jsonl
+      → data/extracted/canonical_ids.json
+
+  Stage 3: Knowledge Graph Build
+    scripts/build_graph.py
+      → graph/builder.py (NetworkX DiGraph, upsert nodes+edges, weight by citation_count)
+      → data/graph/als_graph.pkl + als_graph.json
+
+  Stage 4: RAG Index Build
+    scripts/build_index.py
+      → rag/indexer.py (ChromaDB "als_papers", chunk by section if full text, else abstract)
+      → data/chroma/
+
+ONLINE QUERY PIPELINE (per physician request):
+
+  Physician free-text question
+    → agents/research_agent.py
+        1. Claude: extract query entities → ["SOD1", "antisense oligonucleotide"]
+        2. graph/query.py: 1-hop KG expansion → ["SOD1", "TARDBP", "tofersen", "RNA splicing", ...]
+        3. rag/retriever.py: semantic search + entity-filtered search → top 15 papers
+           (re-ranked by: ChromaDB distance × log(citation_count + 1))
+        4. graph/query.py: find_trials_for_target() for each query entity
+        5. Claude Sonnet (streaming): synthesize ResearchLandscape
+    → Gradio UI (streaming response with citations)
+```
+
+---
+
+## File Structure
+
+```
+candle-fire/
+├── pyproject.toml
+├── .env.example             # ANTHROPIC_API_KEY, ENTREZ_EMAIL, NCBI_API_KEY
+├── CLAUDE.md                # Architecture guide (module responsibilities)
+│
+├── config.py                # Model names, paths, ALS seed entities, API endpoints
+├── models.py                # Dataclasses: ALSPaper, ExtractedEntity, EntityRelationship, ResearchLandscape
+├── prompts.py               # System prompts: extraction + synthesis
+├── tools.py                 # Tool schema loader (mirrors beacon/tools.py)
+├── llm.py                   # LLM provider abstraction (copied from beacon/llm.py)
+├── logging_config.py        # Structured JSON logging (adapted from beacon/beacon_logging.py)
+├── app.py                   # Gradio UI entry point (graph + ChromaDB loaded at startup)
+├── main.py                  # CLI entry point (Rich console)
+│
+├── ingestion/
+│   ├── pubmed.py            # PubMed Entrez: fetch abstracts + metadata by query or PMID list
+│   ├── pmc.py               # PMC XML full text: fetch & parse structured sections for OA papers
+│   ├── clinicaltrials.py    # ClinicalTrials.gov v2 (adapted from beacon/trials_api.py, no geo)
+│   └── semantic_scholar.py  # Citation counts by PMID/DOI (batch API, free tier)
+│
+├── extraction/
+│   ├── extractor.py         # Claude Sonnet NER: batch 10 papers, retry+backoff, .progress.json
+│   └── normalizer.py        # Canonical ID mapping (HGNC alias table + PubChem + REST fallback)
+│
+├── graph/
+│   ├── builder.py           # NetworkX DiGraph: upsert nodes+edges, seed ALS entities, citation weighting
+│   ├── query.py             # Traversal: expand_query_entities, find_trials_for_target, get_entity_evidence
+│   └── serializer.py        # Save/load: pickle (fast) + JSON (human-readable)
+│
+├── rag/
+│   ├── indexer.py           # ChromaDB collection builder: section-aware chunking, citation_count metadata
+│   └── retriever.py         # search(), search_by_entities(), citation-weighted re-ranking
+│
+├── agents/
+│   └── research_agent.py    # Multi-step synthesis agent (streaming, mirrors beacon/agents/research.py)
+│
+├── scripts/
+│   ├── ingest_papers.py     # CLI: PubMed + PMC XML + citation counts → papers.jsonl
+│   ├── ingest_trials.py     # CLI: ClinicalTrials.gov → trials.jsonl
+│   ├── extract_entities.py  # CLI: papers.jsonl → entities.jsonl (resumable)
+│   ├── build_graph.py       # CLI: entities.jsonl + trials.jsonl → als_graph.pkl
+│   └── build_index.py       # CLI: papers.jsonl + entities.jsonl → data/chroma/
+│
+├── data/
+│   ├── papers/papers.jsonl
+│   ├── trials/trials.jsonl
+│   ├── extracted/
+│   │   ├── entities.jsonl
+│   │   ├── canonical_ids.json
+│   │   └── .progress.json   # Extraction resumability tracker
+│   ├── graph/
+│   │   ├── als_graph.pkl
+│   │   └── als_graph.json
+│   ├── chroma/              # ChromaDB SQLite store
+│   └── tools/
+│       ├── extract_entities.json
+│       └── search_landscape.json
+│
+└── tests/
+    ├── test_pubmed.py
+    ├── test_pmc.py
+    ├── test_semantic_scholar.py
+    ├── test_extractor.py
+    ├── test_normalizer.py
+    ├── test_graph_builder.py
+    ├── test_graph_query.py
+    ├── test_retriever.py
+    └── test_research_agent.py
+```
+
+---
+
+## Staged Implementation Plan
+
+### Stage 1 — Project Foundation
+**Goal**: Runnable skeleton with all dependencies wired.
+
+- `pyproject.toml` with all dependencies (anthropic, gradio, chromadb, networkx, biopython, httpx, sentence-transformers, rich, python-dotenv)
+- `config.py`, `models.py`, `prompts.py` (stubs), `tools.py`, `llm.py` (copied from beacon), `logging_config.py`
+- `data/tools/extract_entities.json` and `search_landscape.json` tool schemas
+- All `__init__.py` files, `.env.example`, `CLAUDE.md`
+
+**Done when**: `uv run python -c "import anthropic, chromadb, networkx, Bio"` passes with no errors.
+
+---
+
+### Stage 2 — Paper Ingestion Pipeline
+**Goal**: Populate `data/papers/papers.jsonl` with ~500 ALS papers including citation counts.
+
+- `ingestion/pubmed.py`: PubMed Entrez client (fetch by MeSH query or PMID file, batch 200)
+- `ingestion/pmc.py`: PMC XML full-text fetcher for OA papers (parse by section)
+- `ingestion/semantic_scholar.py`: Citation count enrichment (batch by PMID)
+- `scripts/ingest_papers.py`: Orchestrates all three, writes `papers.jsonl`
+- `scripts/ingest_trials.py` + `ingestion/clinicaltrials.py`: ALS trials → `trials.jsonl`
+
+PubMed seed query: `"amyotrophic lateral sclerosis"[MeSH Major Topic] AND ("2018"[PDAT]:"2024"[PDAT]) AND hasabstract[text]`
+
+**Done when**: `papers.jsonl` has 500 lines with `citation_count` populated; `trials.jsonl` has 20+ records.
+
+---
+
+### Stage 3 — RAG Pipeline + Working v0
+**Goal**: End-to-end working query pipeline using RAG only (no KG yet).
+
+- `rag/indexer.py`: ChromaDB collection builder (section-aware chunking, citation_count in metadata)
+- `rag/retriever.py`: `search()`, `search_by_entities()`, citation-weighted re-ranking
+- `scripts/build_index.py`
+- `agents/research_agent.py` (RAG-only version, no KG expansion step yet)
+- `prompts.py` synthesis prompt
+- `main.py`: CLI interface
+
+**Done when**: `uv run python main.py "What compounds target glutamate excitotoxicity in ALS?"` returns a synthesized answer with cited PMIDs.
+
+---
+
+### Stage 4 — Entity Extraction + Knowledge Graph
+**Goal**: Offline pipeline produces a populated KG; query agent upgrades to KG+RAG.
+
+- `extraction/extractor.py`: Claude Sonnet NER, 10 papers/call, resumable via `.progress.json`
+- `extraction/normalizer.py`: HGNC alias table (~50 ALS genes) + PubChem fallback + REST fallback
+- `scripts/extract_entities.py`
+- `graph/builder.py`: NetworkX DiGraph with upsert, citation-weighted edge confidence, seed entities
+- `graph/query.py`: `expand_query_entities()`, `find_trials_for_target()`, `get_entity_evidence()`
+- `graph/serializer.py`
+- `scripts/build_graph.py`
+- Upgrade `agents/research_agent.py` to include KG expansion step before RAG
+
+**Done when**: `G.number_of_nodes() > 100`; query for "tofersen" surfaces SOD1 as the mechanism link (not just literal tofersen matches).
+
+---
+
+### Stage 5 — Gradio UI + Deployment Polish
+**Goal**: Browser-accessible web app ready for HuggingFace Spaces.
+
+- `app.py`: Gradio UI with streaming, example questions sidebar, citation display, disclaimer
+- Module-level graph + ChromaDB client initialization (once at startup)
+- `requirements.txt` (HF Spaces mirror of pyproject.toml)
+- `README.md`: setup instructions, offline pipeline run order, env vars
+- Tests: `uv run pytest` all pass
+
+**Done when**: `uv run gradio app.py` → physician asks "What's the mechanism of tofersen in ALS?" → streaming response includes SOD1 mechanism, 3+ cited papers, at least 1 NCT trial link, and a disclaimer.
+
+---
+
+## Key Reuse from Beacon
+
+| Beacon file | Candle-fire file | Change |
+|---|---|---|
+| `beacon/llm.py` | `llm.py` | Copy verbatim; model constants from `config.py` |
+| `beacon/beacon_logging.py` | `logging_config.py` | Namespace → `candle_fire` |
+| `beacon/trials_api.py` | `ingestion/clinicaltrials.py` | Remove geo/distance; add `extract_target_entities()` |
+| `beacon/tools.py` | `tools.py` | Copy pattern; update tool names |
+| `beacon/agents/research.py` | `agents/research_agent.py` | Adapt streaming loop; replace tool handlers |
+| `beacon/app.py` | `app.py` | Single-turn instead of multi-turn state machine |
+
+---
+
+## Estimated Costs
+
+| Item | Cost |
+|---|---|
+| Entity extraction (500 papers, 50 Sonnet calls ~10K tokens each) | ~$1.50 one-time |
+| Semantic Scholar citation counts | Free |
+| PMC XML full text | Free |
+| Per physician query (Sonnet, ~15K tokens in+out) | ~$0.05/query |

prompts.py

@@ -0,0 +1,60 @@
+"""System prompts for extraction and synthesis agents."""
+
+EXTRACTION_SYSTEM = """\
+You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis) research.
+Your task is to extract biomedical entities and relationships from ALS paper abstracts.
+
+Entity types to extract:
+- Gene: genetic loci (e.g., SOD1, TARDBP, FUS, C9orf72)
+- Protein: protein products (e.g., TDP-43, FUS protein, SOD1 protein)
+- Compound: drugs, small molecules, biologics (e.g., riluzole, tofersen, AMX0035)
+- Pathway: biological pathways or processes (e.g., glutamate excitotoxicity, autophagy)
+- Phenotype: disease features or clinical observations (e.g., bulbar onset, respiratory failure)
+- Mechanism: molecular or cellular mechanisms (e.g., protein aggregation, oxidative stress)
+
+Relationship types to extract:
+- BINDS: compound/protein binds to a target
+- INHIBITS: compound/gene inhibits a target
+- ASSOCIATED_WITH: entity is associated with a disease phenotype or another entity
+- TESTED_IN: compound is tested in a clinical trial or animal model
+- EXPRESSED_IN: gene/protein is expressed in a tissue or cell type
+- CO_OCCURS: entities frequently co-occur in ALS context (weakest relationship)
+
+Be precise. Only extract entities explicitly mentioned. Confidence reflects how clearly
+the entity is identified in the text (1.0 = unambiguous, 0.5 = inferred, 0.3 = uncertain).
+"""
+
+SYNTHESIS_SYSTEM = """\
+You are a clinical research synthesis expert specializing in ALS (amyotrophic lateral sclerosis).
+You help physicians understand the research evidence behind ALS biology, drug targets, and clinical trials.
+
+When answering a physician's question, structure your response as follows:
+
+## Key Mechanisms
+2–3 bullet points summarizing the core biological mechanisms relevant to the query.
+
+## Entities Involved
+Brief descriptions of the key genes, proteins, compounds, or pathways involved,
+with the number of supporting papers where known.
+
+## Evidence Strength
+A short paragraph on the overall strength and consistency of the evidence
+(number of papers, trial phases, consensus vs. controversy).
+
+## Key Citations
+Up to 5 most relevant papers, formatted as:
+- [Title] (Year) — PMID: [number]
+
+## Related Clinical Trials
+Any relevant ALS clinical trials linked to the topic, with NCT ID and status.
+
+---
+*Research synthesis tool. Always verify with primary sources and current clinical evidence.
+Not a substitute for clinical judgment.*
+
+Guidelines:
+- Be precise and cite PMIDs for every factual claim where available
+- Acknowledge uncertainty where evidence is limited or conflicting
+- Use clinical language appropriate for a physician audience
+- If a query falls outside ALS research, note that and answer only from ALS context
+"""

pyproject.toml

@@ -0,0 +1,39 @@
+[project]
+name = "candle-fire"
+version = "0.1.0"
+description = "ALS research landscape tool for physicians"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "anthropic>=0.50.0",
+    "gradio>=6.14.0,<7.0.0",
+    "chromadb>=0.5.0",
+    "networkx>=3.3",
+    "biopython>=1.84",
+    "httpx>=0.27.0",
+    "python-dotenv>=1.2.2",
+    "rich>=13.0.0",
+    "sentence-transformers>=3.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "pytest-mock>=3.14",
+    "pytest-httpx>=0.35",
+    "pytest-cov",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.coverage.run]
+omit = [
+    "app.py",
+    "main.py",
+    "config.py",
+    "tools.py",
+    "logging_config.py",
+    "llm.py",
+    "prompts.py",
+]

rag/indexer.py

@@ -0,0 +1,2 @@
+"""ChromaDB collection builder with section-aware chunking and citation metadata."""
+# Stage 3 implementation

rag/retriever.py

@@ -0,0 +1,2 @@
+"""ChromaDB query interface with citation-weighted re-ranking."""
+# Stage 3 implementation

scripts/build_graph.py

@@ -0,0 +1,7 @@
+"""
+Build the ALS knowledge graph from extracted entities and clinical trials.
+
+Usage:
+    uv run python scripts/build_graph.py
+"""
+# Stage 4 implementation

scripts/build_index.py

@@ -0,0 +1,8 @@
+"""
+Build the ChromaDB vector index from papers and extracted entities.
+
+Usage:
+    uv run python scripts/build_index.py
+    uv run python scripts/build_index.py --reset  # drop and rebuild collection
+"""
+# Stage 3 implementation

scripts/extract_entities.py

@@ -0,0 +1,10 @@
+"""
+Extract biomedical entities from papers using Claude Sonnet.
+Resumable — safe to interrupt and re-run.
+
+Usage:
+    uv run python scripts/extract_entities.py            # all papers
+    uv run python scripts/extract_entities.py --limit 10 # test run
+    uv run python scripts/extract_entities.py --reset    # clear progress and restart
+"""
+# Stage 4 implementation

scripts/ingest_papers.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+"""
+Ingest ALS papers from PubMed + PMC full text + Semantic Scholar citation counts.
+
+Usage:
+    uv run python scripts/ingest_papers.py                         # default query, 500 papers
+    uv run python scripts/ingest_papers.py --max 10                # small test run
+    uv run python scripts/ingest_papers.py --pmid-file pmids.txt   # from curated PMID list
+    uv run python scripts/ingest_papers.py --skip-fulltext         # abstracts only
+    uv run python scripts/ingest_papers.py --skip-citations        # skip Semantic Scholar
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from dotenv import load_dotenv
+
+load_dotenv()
+
+from rich.console import Console
+from rich.progress import track
+
+import ingestion.pmc as pmc
+import ingestion.pubmed as pubmed
+import ingestion.semantic_scholar as ss
+from config import PAPERS_PATH, PUBMED_DEFAULT_MAX, PUBMED_DEFAULT_QUERY
+
+console = Console()
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Ingest ALS papers")
+    parser.add_argument("--query", default=PUBMED_DEFAULT_QUERY, help="PubMed query string")
+    parser.add_argument("--max", type=int, default=PUBMED_DEFAULT_MAX, dest="max_results", help="Max papers to fetch")
+    parser.add_argument("--pmid-file", help="Path to file with one PMID per line")
+    parser.add_argument("--skip-fulltext", action="store_true", help="Skip PMC full text fetch")
+    parser.add_argument("--skip-citations", action="store_true", help="Skip Semantic Scholar citation counts")
+    args = parser.parse_args()
+
+    PAPERS_PATH.parent.mkdir(parents=True, exist_ok=True)
+
+    # Step 1: Obtain PMIDs
+    if args.pmid_file:
+        lines = Path(args.pmid_file).read_text().splitlines()
+        pmids = [line.strip() for line in lines if line.strip()]
+        console.print(f"[cyan]Loaded {len(pmids)} PMIDs from {args.pmid_file}[/cyan]")
+    else:
+        console.print(f"[cyan]Searching PubMed (max {args.max_results})...[/cyan]")
+        pmids = pubmed.search_pmids(args.query, max_results=args.max_results)
+        console.print(f"[green]Found {len(pmids)} PMIDs[/green]")
+
+    if not pmids:
+        console.print("[red]No PMIDs found — check your query or PMID file.[/red]")
+        sys.exit(1)
+
+    # Step 2: Fetch paper records from PubMed
+    console.print("[cyan]Fetching Medline records...[/cyan]")
+    papers = pubmed.fetch_by_pmids(pmids)
+    console.print(f"[green]Parsed {len(papers)} papers with abstracts[/green]")
+
+    if not papers:
+        console.print("[red]No papers with abstracts retrieved.[/red]")
+        sys.exit(1)
+
+    # Step 3: Enrich with PMC full text
+    if not args.skip_fulltext:
+        console.print("[cyan]Looking up PMC IDs for Open Access full text...[/cyan]")
+        all_pmids = [p.pmid for p in papers]
+        pmcid_map = pmc.get_pmcids(all_pmids)
+        console.print(f"[green]{len(pmcid_map)} papers have PMC full text available[/green]")
+
+        pmid_to_paper = {p.pmid: p for p in papers}
+        ft_count = 0
+        for pmid, pmcid in track(pmcid_map.items(), description="Fetching full text...", console=console):
+            if pmid in pmid_to_paper:
+                full_text = pmc.fetch_full_text(pmcid)
+                if full_text:
+                    pmid_to_paper[pmid].full_text = full_text
+                    ft_count += 1
+        console.print(f"[green]Retrieved section-parsed full text for {ft_count} papers[/green]")
+
+    # Step 4: Enrich with citation counts from Semantic Scholar
+    if not args.skip_citations:
+        console.print("[cyan]Fetching citation counts from Semantic Scholar...[/cyan]")
+        citation_map = ss.fetch_citation_counts([p.pmid for p in papers])
+        for paper in papers:
+            paper.citation_count = citation_map.get(paper.pmid, 0)
+        console.print(f"[green]Got citation counts for {len(citation_map)}/{len(papers)} papers[/green]")
+
+    # Step 5: Write to JSONL
+    with open(PAPERS_PATH, "w", encoding="utf-8") as f:
+        for paper in papers:
+            f.write(json.dumps(paper.to_dict()) + "\n")
+
+    has_fulltext = sum(1 for p in papers if p.full_text)
+    has_citations = sum(1 for p in papers if p.citation_count > 0)
+
+    console.print(f"\n[bold green]Done![/bold green] Written to {PAPERS_PATH}")
+    console.print(f"  Papers:         {len(papers)}")
+    console.print(f"  With full text: {has_fulltext} ({has_fulltext * 100 // len(papers)}%)")
+    console.print(f"  With citations: {has_citations} ({has_citations * 100 // len(papers)}%)")
+
+
+if __name__ == "__main__":
+    main()

scripts/ingest_trials.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+"""
+Ingest ALS clinical trials from ClinicalTrials.gov v2 API.
+
+Usage:
+    uv run python scripts/ingest_trials.py
+    uv run python scripts/ingest_trials.py --status RECRUITING
+    uv run python scripts/ingest_trials.py --status COMPLETED
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from dotenv import load_dotenv
+
+load_dotenv()
+
+from rich.console import Console
+
+from config import TRIALS_PATH
+from ingestion.clinicaltrials import fetch_als_trials
+
+console = Console()
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Ingest ALS clinical trials")
+    parser.add_argument(
+        "--status",
+        default="RECRUITING",
+        choices=["RECRUITING", "COMPLETED", "ACTIVE_NOT_RECRUITING", "NOT_YET_RECRUITING"],
+        help="Trial status filter (default: RECRUITING)",
+    )
+    args = parser.parse_args()
+
+    TRIALS_PATH.parent.mkdir(parents=True, exist_ok=True)
+
+    console.print(f"[cyan]Fetching ALS interventional trials (status={args.status})...[/cyan]")
+    trials = fetch_als_trials(status=args.status)
+    console.print(f"[green]Fetched {len(trials)} trials[/green]")
+
+    with open(TRIALS_PATH, "w", encoding="utf-8") as f:
+        for trial in trials:
+            f.write(json.dumps(trial) + "\n")
+
+    with_targets = sum(1 for t in trials if t.get("target_entities"))
+    console.print(f"\n[bold green]Done![/bold green] Written to {TRIALS_PATH}")
+    console.print(f"  Trials:               {len(trials)}")
+    console.print(f"  With entity targets:  {with_targets}")
+
+
+if __name__ == "__main__":
+    main()

tools.py

@@ -0,0 +1,38 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import anthropic
+
+_DATA = Path(__file__).parent / "data" / "tools"
+
+
+def _load(name: str) -> dict:
+    return json.loads((_DATA / f"{name}.json").read_text())
+
+
+EXTRACT_ENTITIES_TOOL: anthropic.types.ToolParam = {
+    "name": "extract_entities",
+    "description": (
+        "Extract biomedical entities and relationships from an ALS paper abstract. "
+        "For each entity, identify its type (Gene, Protein, Compound, Pathway, Phenotype, or Mechanism), "
+        "the exact name as it appears in the text, and the confidence of the identification. "
+        "For relationships, identify the source entity, target entity, and the type of relationship."
+    ),
+    "input_schema": _load("extract_entities"),
+}
+
+SEARCH_LANDSCAPE_TOOL: anthropic.types.ToolParam = {
+    "name": "search_research_landscape",
+    "description": (
+        "Search the ALS research knowledge base by combining knowledge graph traversal and "
+        "vector similarity search. Provide the entities you identified in the physician's query "
+        "and the original query text. Returns ranked papers, related biological entities, "
+        "and linked clinical trials."
+    ),
+    "input_schema": _load("search_landscape"),
+}
+
+EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_ENTITIES_TOOL]
+RESEARCH_TOOLS: list[anthropic.types.ToolParam] = [SEARCH_LANDSCAPE_TOOL]