PROJECT_LOG.md

# ResearchPath — Project Log

A chronological record of decisions, work sessions, issues, and outcomes for the ResearchPath portfolio project.

**Schema for each entry:**

```
## YYYY-MM-DD HH:MM — [type] Title
**Type:** setup | decision | implementation | issue | resolution | discussion
**Status:** in-progress | complete | blocked | reverted
**Duration:** approx time

### Context
### Decisions
### Actions taken
### Issues encountered
### Files touched
### Outcome
### Next
```

---

## 2026-05-06 15:00 — [discussion] Project framing & scope

**Type:** decision
**Status:** complete
**Duration:** ~30 min

### Context
Chetan (~2.5 YOE Data Scientist, Kyndryl + ex-Swiss Re, targeting top product companies) needed a portfolio project. Earlier "Agentic RAG Document Search" on his resume was copied from another GitHub — needed a genuine, defensible project.

### Decisions
- **Rejected** Underwriter Copilot (insurance) and Equity Research Analyst (SEC filings) — too generic / didn't match his preference for research-flavored work.
- **Rejected** generic-RAG-over-papers framing — already exists (Perplexity, Elicit, Consensus); not enough differentiation.
- **Committed**: **ResearchPath** — agentic research-onboarding companion. Killer feature: prerequisite-chain reading-path planning. Genuinely agentic (graph traversal + recursive retrieval + reasoning over user state), not RAG-with-extra-steps.
- Demo domain: **Reinforcement Learning**. Architecture domain-agnostic.
- Stack: Gemini 2.5 Flash Lite + Groq Llama 3.3 70B (free tiers), BAAI/bge-small-en embeddings, FAISS, LangGraph, Streamlit, RAGAS.
- Eval-first methodology: gold dataset + ablation table is the differentiator from generic portfolios.

### Outcome
Project scope locked. 5-week plan in README.

### Next
Day 1 setup: env, deps, API keys, smoke test.

---

## 2026-05-06 15:20 — [setup] Day 1: Environment bootstrap

**Type:** setup
**Status:** complete
**Duration:** ~45 min

### Context
Stand up local dev env so the agent work in Week 2+ has a working LLM substrate.

### Decisions
- Use `uv` for Python + dependency management (already installed on user's machine).
- Default Gemini model: `gemini-2.5-flash-lite` (chosen after `gemini-2.5-flash` returned transient 503s during smoke test).

### Actions taken
- Scaffolded repo files: `.gitignore`, `.env.example`, `README.md`, `pyproject.toml`, `researchpath/__init__.py`, `researchpath/llm.py`, `scripts/smoke_test.py`.
- `uv python install 3.12` → installed CPython 3.12.9.
- `uv sync` → resolved 38 packages including `google-genai`, `groq`, `python-dotenv`, `rich`, `pytest`, `ruff`.
- User created `.env` from template and pasted Gemini + Groq API keys.
- `uv run python scripts/smoke_test.py` → both providers returned valid completions.

### Issues encountered
- **Unicode arrow `→` broke on Windows cp1252 codepage** when `rich` tried to write to stdout. Fix: removed the arrow from `scripts/smoke_test.py`. Also set `PYTHONIOENCODING=utf-8` for the run.
- **Gemini 2.5 Flash returned 503 UNAVAILABLE (transient overload)** despite a valid key. Fix: switched default model in `researchpath/llm.py` from `gemini-2.5-flash` to `gemini-2.5-flash-lite`.

### Files touched
- Created: `.gitignore`, `.env.example`, `README.md`, `pyproject.toml`, `researchpath/__init__.py`, `researchpath/llm.py`, `scripts/smoke_test.py`, `.claude/settings.local.json`, `PROJECT_LOG.md`.
- User-created (gitignored): `.env`.

### Outcome
Both Gemini and Groq APIs verified end-to-end. `.claude/settings.local.json` grants Bash + PowerShell auto-approval for this project (gitignored — personal preference, not committed).

### Next
Day 2: git init → arxiv corpus fetch script → download ~17 canonical RL papers → first commit.

---

## 2026-05-06 16:00 — [implementation] Day 2: Corpus pipeline + first commit

**Type:** implementation
**Status:** complete (with one outstanding follow-up — see "Outcome")
**Duration:** ~75 min

### Context
Build the end-to-end corpus path: fetch canonical RL papers from arXiv → parse PDFs → chunk text into a single JSONL ready for embedding. Land everything as the first git commit.

### Decisions
- **17-paper canon** chosen across 5 eras (value-based, policy-gradient, actor-critic, model-based, RLHF). See `researchpath/corpus.py`.
- **Streaming downloads via `requests`** instead of `arxiv` library's built-in downloader — the latter was truncating large PDFs at exact 1MB / 2MB boundaries on Windows.
- **PDF validation step** (open with pymupdf, read page 1) gates every download; truncated files auto-deleted and retried.
- **Chunking**: 800-char windows, 100-char overlap, sentence-boundary aware, hyphenation-aware.
- **Accept partial corpus (10/17)** rather than grind on rate-limited retries — script is idempotent so missing papers can be picked up later.

### Actions taken
- Added deps: `arxiv`, `pymupdf`. `uv sync` resolved 5 new packages.
- `researchpath/corpus.py` — 17-paper canon with `era` tag.
- `scripts/fetch_corpus.py` — streaming downloader with per-paper validate-and-retry, 12s inter-request spacing, 30s backoff.
- `researchpath/parsing.py` — PDF text extraction + sentence-boundary chunking.
- `scripts/parse_corpus.py` — drives parsing across all valid PDFs into `data/chunks.jsonl`.
- Ran fetcher: 10 papers landed cleanly. Deleted 3 truncated stragglers (`1801.01290`, `1911.08265`, `2301.04104`).
- Ran parser: **1,093 chunks, 778,995 chars** across 10 papers.
- `git init -b main` → first commit `270993c` ("Bootstrap ResearchPath: scaffold + corpus pipeline"). 13 files, 1,611 insertions.

### Issues encountered
- **arXiv HTTP 429 rate-limiting** on PDF endpoint. Tried 3s→8s→12s inter-request delays; eventually too much accumulated state on this IP. Pivoted to "accept partial corpus, document the gap" per user's `feedback_dont_grind.md`.
- **`arxiv` library truncates large PDFs at 1MB/2MB** even on retry — buffer issue specific to its downloader. Switched to direct `requests` streaming. (Confirmed by post-hoc validation: 3 stragglers stuck at exactly 2,097,152 / 1,048,576 bytes.)
- **Unicode `→` in console output** broke on cp1252 (already known from Day 1; reused `PYTHONIOENCODING=utf-8`).

### Files touched
- Added: `researchpath/corpus.py`, `researchpath/parsing.py`, `scripts/fetch_corpus.py`, `scripts/parse_corpus.py`.
- Modified: `pyproject.toml` (+arxiv, +pymupdf).
- Generated (gitignored): `data/papers/*.pdf` (10 files), `data/chunks.jsonl`.

### Outcome
End-to-end corpus path works: arxiv ID → validated PDF → chunked JSONL. Repo under git, first commit landed.

**Outstanding:** 4 papers still missing — `1511.05952` (PER), `1707.06347` (PPO), `1802.01561` (IMPALA), `2106.01345` (Decision Transformer). PPO is the painful gap (canonical paper). Re-run `uv run python scripts/fetch_corpus.py` later (script is idempotent + only retries missing/invalid).

### Next
Day 3: embeddings + FAISS index over `data/chunks.jsonl`. Use `BAAI/bge-small-en-v1.5` (CPU-friendly, ~133MB, strong on academic text). Then a tiny CLI: ask question → top-k chunks → grounded Gemini answer. That's the baseline RAG to measure against.

---

## 2026-05-06 17:00 — [implementation] Day 3: Embeddings, FAISS index, baseline RAG

**Type:** implementation
**Status:** complete
**Duration:** ~50 min

### Context
Stand up the first measurable RAG system: chunks → embeddings → vector search → grounded LLM answer with citations. This becomes the baseline that all future improvements (hybrid retrieval, reranker, agent planning) get measured against.

### Decisions
- **Embedding model: `BAAI/bge-small-en-v1.5`** (384 dim, ~133MB, CPU-fast, strong on academic text). Documents embedded raw; queries prefixed with the BGE-recommended retrieval instruction for measurably better results.
- **FAISS `IndexFlatIP`** with L2-normalized embeddings — exact cosine search, fast enough for ~1k chunks. Will swap to HNSW if corpus grows past ~50k.
- **Citation format `[arxiv_id, p<page>]`** — parseable by future eval code, human-readable in output.
- **System prompt is strict**: only-from-sources, no speculation. Will tighten further once we have eval numbers.

### Actions taken
- Added deps: `sentence-transformers`, `faiss-cpu`, `numpy`. `uv sync` resolved 29 new packages (~330MB inc. torch CPU).
- `researchpath/embeddings.py` — `Embedder` wrapper with separate `embed_documents` / `embed_query` (BGE wants different prefixes).
- `researchpath/index.py` — `build_index`, `load_index`, `search`. `Hit` dataclass carries score + citation metadata.
- `researchpath/rag.py` — `build_prompt`, `answer`. Strict system prompt enforcing citations + grounding.
- `scripts/build_index.py` — drives end-to-end embed + persist.
- `scripts/ask.py` — CLI for question → answer with citations. `--show-sources` flag for debugging retrieval.
- Built index: 1,093 chunks embedded in **148s** on CPU. Persisted `data/index.faiss` + `data/index.chunks.json`.
- **Smoke test query**: *"How does Rainbow combine multiple deep Q-learning improvements like Double DQN, Dueling networks, and prioritized replay?"*
  - Answer correctly cited [1710.02298, p1] for Rainbow components and [1511.06581, p4] for Dueling architecture details.
  - Listed all 6 Rainbow components accurately. No hallucination observed.
  - 1,106 input / 196 output tokens (≈ free under Gemini quota).

### Issues encountered
- **HF Hub symlink warning on Windows** — cache-system can't use symlinks without admin/Developer Mode. Cosmetic only; cache works in degraded mode (more disk, no functional impact).
- **`VIRTUAL_ENV` mismatch warning from uv** — leftover env var from a prior shell session pointing at a different Python. Harmless; `.venv` is used regardless.

### Files touched
- Added: `researchpath/embeddings.py`, `researchpath/index.py`, `researchpath/rag.py`, `scripts/build_index.py`, `scripts/ask.py`.
- Modified: `pyproject.toml` (+sentence-transformers, +faiss-cpu, +numpy), `uv.lock`.
- Generated (gitignored): `data/index.faiss`, `data/index.chunks.json`.

### Outcome
Baseline RAG is **end-to-end working** and producing high-quality, grounded, cited answers on a real RL question. This is the system we'll measure in Day 4.

### Next
Day 4: **eval harness + gold dataset.** Build ~30 (question, expected answer key, expected source citations) tuples sourced from OpenAI Spinning Up problem sets, Sutton & Barto exercises, and 3-5 hand-curated questions per major paper. Then implement metrics (citation faithfulness, retrieval recall@k, answer quality blind-rated 1-5). First numbers go into the README eval table.

---

## 2026-05-06 18:00 — [implementation] Day 4: Eval harness + gold dataset + baseline numbers

**Type:** implementation
**Status:** complete
**Duration:** ~90 min

### Context
Build a rigorous eval harness before optimizing anything. Eval-first is the portfolio differentiator — ablation table beats vibes.

### Decisions
- **30-question gold dataset** across 10 indexed papers (3 per single-paper + 4 cross-paper comparison). Difficulty-stratified: 7 easy / 15 medium / 8 hard. Each example: `{id, question, expected_arxiv_ids, expected_key_claim, difficulty}`.
- **Three metrics** (no citation faithfulness per-claim — too expensive on free tier):
  - `retrieval_recall@k`: fraction of expected arxiv IDs found in top-k hits
  - `citation_presence`: answer cites at least one expected paper (string check)
  - `answer_correctness`: LLM-as-judge (Groq llama-3.3-70b), YES/NO prompt
- **Judge model: Groq** not Gemini — Gemini free tier has 20 req/day limit (hit during initial test run). Groq handles batch eval at no cost.
- **RAG model: Groq** for batch eval. Gemini stays as default for `scripts/ask.py` (the demo CLI). Added `--groq` flag to `run_eval.py` to toggle.
- **Retry logic** added to `gemini_generate()` — parses retry delay from 429 error body, sleeps and retries up to 5 times.

### Actions taken
- `data/gold_dataset.json` — 30 QA triples (hand-authored from Spinning Up + paper reading).
- `researchpath/eval.py` — `GoldExample`, `EvalResult`, `EvalSummary` dataclasses + `evaluate_example`, `summarize`, `results_to_json`.
- `scripts/run_eval.py` — full pipeline: load index → batch retrieve + generate → judge → print table + save JSON.
- Updated `researchpath/rag.py` — added `generate_fn` injection param + `answer_groq()` convenience wrapper.
- Updated `researchpath/llm.py` — retry-with-backoff on Gemini 429 (parses `retryDelay` from error body).
- Ran full 30-question eval (`--groq`). Duration: ~3.5 min. Results saved to `data/eval_results.json`.

### Issues encountered
- **Gemini 20 req/day free-tier limit** — hit on second run (smoke test + partial full run = 20 calls). Fix: route eval to Groq entirely.
- **Retry delay detection**: Gemini's 429 body contains `retryDelay: '10s'` in the Details array. Regex extracts it; if missing, defaults to 15s sleep.

### Files touched
- Added: `data/gold_dataset.json`, `researchpath/eval.py`, `scripts/run_eval.py`.
- Modified: `researchpath/rag.py` (generate_fn injection, answer_groq), `researchpath/llm.py` (retry logic), `README.md` (eval table row 1).
- Generated (gitignored): `data/eval_results.json`.

### Outcome

**Baseline RAG numbers (Groq llama-3.3-70b, k=5, n=30):**

| Metric | Value |
|---|---|
| Retrieval Recall@5 | **90.0%** |
| Citation Presence | **86.7%** |
| Answer Correctness | **36.7%** |
| Avg Latency | **5.07s** |
| RAG Tokens | 33,498 in / 7,000 out |

**Gap analysis:** Retrieval finds the right paper in 27/30 cases — the FAISS index is working well. But answer correctness is only 37%, revealing two failure modes:
1. **Wrong-chunk problem**: Right paper retrieved, but the specific mechanistic claim is in a chunk outside top-5 (e.g., Rainbow's 6 components, distributional RL definition). Fix: hybrid retrieval (BM25 + vector) to catch exact-term hits + smaller chunks.
2. **Precision failure**: Right chunk retrieved, but answer paraphrases around the key claim without naming the specific mechanism (e.g., "stability" instead of "identifiability" for Dueling DQN mean subtraction). Fix: reranker + more directive prompt.

Difficulty breakdown: easy=28.6%, medium=53.3%, hard=12.5% answer correctness — harder questions show compounding retrieval + precision failures.

### Next
Day 5: Optimization round 1 — implement hybrid BM25+vector retrieval and measure improvement on same 30 gold examples. Target: push Answer Correctness from 37% → 55%+.

---

## 2026-05-06 19:30 — [implementation] Day 5: Hybrid BM25+FAISS retrieval (RRF fusion)

**Type:** implementation
**Status:** complete
**Duration:** ~90 min

### Context
Day 4 baseline showed Retrieval Recall@5=90% but Answer Correctness=37%. Root-cause: right paper retrieved but wrong chunks — dense embeddings de-prioritize specific technical terms ("conjugate gradient", "Polyak averaging", "identifiability") relative to surrounding context. BM25 keyword search catches exact-term hits that dense misses.

### Decisions
- **BM25Okapi** (rank-bm25 library) for sparse retrieval over the same 1,093 chunks.
- **Reciprocal Rank Fusion** (RRF, k=60) to combine BM25 and FAISS rankings without normalizing incompatible score spaces.
- **Stop-word filter** in tokenizer: query "What does GAE stand for?" without stop words = `['gae', 'stand', 'fundamental', 'tradeoff']` — much better signal than including "what", "does", "for", "it".
- **fetch_k=20** from each retriever before fusion, then top-k=5 from combined candidate set.

### Actions taken
- `researchpath/retrieval.py` — `BM25Retriever`, `HybridRetriever` with RRF fusion. Stop-word list added to tokenizer.
- `researchpath/index.py` — added `global_idx` field to `Hit` dataclass (FAISS vector position, globally unique across papers). Fixed **critical bug**: original code used `chunk_idx` (within-paper index, 0..N) as RRF dict key, causing collisions across papers with same within-paper index.
- `scripts/ask.py` — added `--hybrid` flag.
- `scripts/run_eval.py` — added `--hybrid` flag, fixed `Path.relative_to()` crash on relative paths.
- Ran full hybrid eval (`--groq --hybrid`): 29/30 questions (cross_04 errored on Groq 100K daily TPD limit).

### Issues encountered
- **global_idx bug**: `chunk_idx` is per-paper, not globally unique. Two chunks from different papers with the same within-paper index collided in the RRF dict, silently corrupting results. Fixed by adding `global_idx` (FAISS position) to Hit and using it in HybridRetriever. Pre-fix: hybrid was WORSE than dense in all spot checks. Post-fix: BETTER or TIED.
- **BM25 stop-word pollution**: Without filtering, "What does GAE manage?" matched InstructGPT appendix QA sections (had literal "What is X? A: ...") scoring 19.9. After adding stop-word filter, query tokens = `['gae', 'stand', 'fundamental', 'tradeoff', 'manage']`, GAE paper correctly ranked first.
- **Groq 100K daily TPD**: Exhausted daily token limit on the 30th question (cross_04). n=29 eval still valid; cross_04 retry deferred.

### Files touched
- Added: `researchpath/retrieval.py`.
- Modified: `researchpath/index.py` (+global_idx field), `scripts/ask.py` (+--hybrid), `scripts/run_eval.py` (+--hybrid, path fix), `README.md` (eval table col 2), `PROJECT_LOG.md`.
- Generated (gitignored): results in memory only (save failed on path bug, now fixed).

### Outcome

**Hybrid BM25+FAISS results (Groq llama-3.3-70b, k=5, n=29):**

| Metric | Baseline | Hybrid | Delta |
|---|---|---|---|
| Retrieval Recall@5 | 90.0% | 89.7% | -0.3 pp |
| Citation Presence | 86.7% | 86.2% | -0.5 pp |
| **Answer Correctness** | **36.7%** | **72.4%** | **+35.7 pp** |
| Avg Latency | 5.07s | 4.58s | -0.5s |

By difficulty: easy 100%, medium 73.3%, hard 42.9%. Hard questions remain the weak spot.

**Why hybrid helped so much:** The global_idx bug fix + stop-word filter together fixed hybrid retrieval so it correctly fuses dense + sparse signals. Many questions where the right paper was retrieved but answer was wrong (dense recall=100%, correct=N in baseline) flipped to correct under hybrid because BM25 surfaced chunks with the specific technical term rather than contextual chunks about the same topic.

### Next
Retry cross_04 when Groq TPD resets (saves the 30/30 result). Then: cross-encoder reranker as column 3 in the ablation table. Target: 75%+ correctness on hard questions.

---

## 2026-05-07 — [implementation] Day 5 cont.: Agentic planning loop (offline prerequisite-chain planner)

**Type:** implementation
**Status:** complete
**Duration:** ~60 min

### Context
The killer differentiator of ResearchPath over generic RAG is the prerequisite-chain reading-path planner: given a target RL paper and what the user already knows, compute the topologically-sorted chain of papers they need to read first. This module runs fully offline — no LLM calls, no API quota burned. It's the deterministic core that an LLM annotation layer will augment later.

### Decisions
- **Static prerequisite graph** as the seed: hand-curate ~10 edges covering canonical chains (DQN→Double DQN→Dueling DQN→Rainbow, TRPO→GAE, DQN→DDPG, DQN→A3C, InstructGPT→DPO). Enough to demo the killer feature; LLM-extracted edges added in a future iteration.
- **BFS backwards** from target, stopping at `known_ids` — avoids pulling in papers the user already understands.
- **Kahn's algorithm** for topological sort with `(year, arxiv_id)` tie-breaking — deterministic + chronologically sensible for RL literature.
- **ReadingStep annotation**: each step carries `bridges_to` (which later papers depend on it), `concepts` (the bridging ideas), and `why` (user-facing rationale) aggregated from the static edge metadata.

### Actions taken
- `researchpath/prerequisites.py` — `Prerequisite` dataclass + `STATIC_PREREQUISITES` tuple (10 hand-curated edges) + `edges_into` / `edges_from` accessors.
- `researchpath/planning.py` — `ReadingStep`, `ReadingPlan` dataclasses + `_bfs_backwards`, `_topo_sort` (Kahn's), `_build_step`, `plan_reading_path`. `ReadingPlan.render_text()` produces human-readable ordered list.
- `scripts/plan.py` — CLI: `--target` (arxiv ID or tag, case-insensitive), `--known` (comma-separated), `--list`. Rich Panel output.
- `tests/test_planning.py` — 7 pytest tests: no-prereq paper (1 step), Rainbow full chain (4 steps in order), known set truncates, known target → empty plan, topo order respects all edges, unknown target raises ValueError, render_text contains target title.

### Smoke tests (pre-pytest, verified manually):
- Rainbow (`1710.02298`, no prior knowledge): `1312.5602 → 1509.06461 → 1511.06581 → 1710.02298` ✓
- Rainbow knowing DQN: `1509.06461 → 1511.06581 → 1710.02298` (3 steps) ✓
- DPO (`2305.18290`) knowing InstructGPT: 1 step (DPO itself) ✓
- DDPG (`1509.02971`): `1312.5602 → 1509.02971` (2 steps) ✓

### Issues encountered
None. BFS + Kahn's on the static graph is deterministic and small enough that correctness is straightforward to verify.

### Files touched
- Added: `researchpath/prerequisites.py`, `researchpath/planning.py`, `scripts/plan.py`, `tests/test_planning.py`.

### Outcome
7/7 pytest tests pass in 0.09s. Planning loop runs offline without any API calls. Demonstrates genuine agentic behavior (graph traversal, state-aware planning) that differentiates ResearchPath from simple RAG demos.

**Sample CLI output** (`uv run python scripts/plan.py --target Rainbow`):
```
1. [1312.5602] DQN (2013) — Playing Atari with Deep Reinforcement Learning
     bridges: -> 1509.06461, 1511.06581, 1710.02298
     concepts: Q-learning, experience replay, target network

2. [1509.06461] DoubleDQN (2015) — Deep Reinforcement Learning with Double Q-learning
     bridges: -> 1511.06581, 1710.02298
     concepts: overestimation bias, decoupled action selection/evaluation, Double Q-learning

3. [1511.06581] DuelingDQN (2015) — Dueling Network Architectures for Deep Reinforcement Learning
     bridges: -> 1710.02298
     concepts: dueling networks

4. [1710.02298] Rainbow (2017) — Rainbow: Combining Improvements in Deep Reinforcement Learning
     bridges: <- target
```

### Next
- Retry cross_04 when Groq TPD resets → update README hybrid result from n=29 to n=30
- Run reranker full eval (column 3 in ablation table): `--hybrid --rerank`
- LLM annotation layer: map free-text user background → `known_ids` set; per-step "what to focus on" rationale
- Fetch 4 missing papers: PER (1511.05952), PPO (1707.06347), IMPALA (1802.01561), Decision Transformer (2106.01345)

---