feat: build retrieval index with llama cpp

Co-authored-by: Codex <noreply@openai.com>

DESIGN.md

@@ -128,8 +128,8 @@ investigate → ideate → score loop — the experience collapses without the m
 | ↳ fallback | `nvidia/parakeet-tdt-0.6b-v3` | 0.6B | **transformers** (no NeMo) | CC-BY-4.0 | 🟩 (Quest brand — verify, §5.1) |
 | LLM brain | **`openbmb/MiniCPM5-1B`** ("OpenCPM5") | 1.08B | **transformers** (self-parse XML) / llama.cpp | **Apache-2.0** | 🏮 OpenBMB |
 | Turn detection (voice-later) | **`pipecat-ai/smart-turn-v3`** | ~8M | ONNX Runtime (browser) | BSD-2 | (natural voice UX) |
-| Embedder | **`google/embeddinggemma-300m`** | ~300M | sentence-transformers / llama.cpp | Gemma (gated) | 🔌 Off the Grid · 🦙 Llama Champion |
-| Fine-tune (to add) | LoRA on MiniCPM5 → published to Hub | — | PEFT / Modal | — | 🎯 Well-Tuned |
+| Embedder | **`ggml-org/embeddinggemma-300M-qat-q4_0-GGUF`** | ~300M | llama.cpp / llama-cpp-python | Gemma | 🔌 Off the Grid · 🦙 Llama Champion · 🟢 Modal |
+| Fine-tune | LoRA on MiniCPM5 → published to Hub | — | PEFT / HF Jobs | — | 🎯 Well-Tuned |
 
 **Total ≈ 1.9B params → ≤4B → 🐜 Tiny Titan eligible.** All open-weight, all runnable locally → 🔌 Off the Grid.
 
@@ -150,8 +150,8 @@ With **text-first + batch ASR**, the old "streaming ASR vs ZeroGPU" Config A/B t
   remains as the Gradio-client contract for external checks.
 - **Voice (later bonus):** push-to-talk records an utterance → POST blob → the same `@spaces.GPU` call also runs
   Nemotron/Parakeet ASR (batch) before the brain. No persistent stream, no WebRTC, **no TURN server**.
-- **Modal (build-time only):** crawl the org + build the EmbeddingGemma index offline; the Space ships with the index
-  artifact. Runtime never calls Modal → 🔌 Off the Grid holds (see §10).
+- **Modal (build-time only):** crawl the org + build the llama.cpp EmbeddingGemma vector index offline; the Space ships
+  with checked-in project vectors. Runtime never calls Modal → 🔌 Off the Grid holds (see §10).
 
 > Off the Grid = no proprietary cloud inference APIs. Open weights on an HF GPU Space / local box / Modal all qualify.
 
@@ -221,33 +221,28 @@ Silero VAD turn detection, FastRTC. Documented but not on the text-first critica
   required before relying on it; fallbacks: port pipecat's numpy-only mel extractor to JS, or do feature-extraction +
   onnx **server-side** per posted blob. Pair with `@ricky0123/vad-web` (Silero) for the speech start/stop gate.
 
-### 5.4 EmbeddingGemma — `google/embeddinggemma-300m`
+### 5.4 EmbeddingGemma GGUF — `ggml-org/embeddinggemma-300M-qat-q4_0-GGUF`
 
-- **Gated** — accept Gemma terms + `HF_TOKEN`. 2048-token ctx, 100+ langs, mean pooling, **fp32/bf16 only (no fp16)**.
-  ```python
-  from sentence_transformers import SentenceTransformer
-  m = SentenceTransformer("google/embeddinggemma-300m", truncate_dim=256)   # Matryoshka 768→512→256→128
-  q = m.encode_query("voice game for kids")          # prefix: "task: search result | query: "
-  d = m.encode_document(project_descriptions)        # prefix: "title: none | text: "
-  ```
-- **Exact prefixes matter:** query → `task: search result | query: `; document → `title: {title} | text: `; whitespace
-  clustering → prompt `Clustering` (`task: clustering | query: `). 256-dim is a good speed/quality tradeoff.
-- Footprint ~1.2 GB fp32 / ~0.6 GB bf16; QAT Q4_0/Q8_0 + ONNX (`onnx-community/embeddinggemma-300m-ONNX`).
+- Active retrieval model: `embeddinggemma-300M-qat-Q4_0.gguf`, 768-dimensional normalized embeddings.
+- Build-time path: Modal remote function runs `llama-cpp-python` with mean pooling and writes `data/project_index.json`.
+- Runtime path: Space embeds each user query through the same GGUF model via llama.cpp, then performs local cosine search
+  over checked-in project vectors.
+- Evidence is recorded in index metadata: model repo, GGUF filename, runtime, dimensions, build source, builder script,
+  llama-cpp-python version, and Modal app name.
 
 ### 5.5 llama.cpp support (🦙 Llama Champion)
 
-The two **language** models run on llama.cpp; the two **audio** models use their own runtimes. Running the core LLM on
-llama.cpp earns the badge.
+The active Llama Champion path is the retrieval model: the project index is built with EmbeddingGemma GGUF through
+llama.cpp on Modal, and runtime query embeddings use the same llama.cpp path.
 
 | Model | llama.cpp? | Runtime | Notes |
 |---|---|---|---|
-| `openbmb/MiniCPM5-1B` | ✅ | llama.cpp / Ollama | `openbmb/MiniCPM5-1B-GGUF` (Q4_K_M 688 MB); standard Llama arch |
-| `google/embeddinggemma-300m` | ✅ | `llama-embedding` | `gemma-embedding` arch (build ≥ b6384); `ggml-org/embeddinggemma-300M-GGUF` |
+| `openbmb/MiniCPM5-1B` | ✅ planned only | llama.cpp / Ollama | Not used for deployed tool-calling; Transformers + LoRA is the deployed brain. |
+| `ggml-org/embeddinggemma-300M-qat-q4_0-GGUF` | ✅ active | llama.cpp / llama-cpp-python | Builds project vectors on Modal and embeds runtime queries in the Space. |
 | ASR (Nemotron / Parakeet) | ❌ | NeMo / transformers | FastConformer-RNNT |
 | `pipecat-ai/smart-turn-v3` | ❌ | ONNX Runtime | Whisper encoder + classifier head |
 
-Verify-before-ship: EmbeddingGemma GGUF quant accuracy drifts ([#19040](https://github.com/ggml-org/llama.cpp/issues/19040))
-→ prefer Q8_0 or keep the embedder on sentence-transformers; MiniCPM5 tool-calling via llama.cpp is a pending PR.
+If retrieval quality becomes the bottleneck, compare Q4_0 against Q8_0, but do not keep two runtime retrieval paths.
 
 ---
 
@@ -336,46 +331,21 @@ score`) into one *code* "research" action the model calls once. The degradation
 
 ## 10. Modal — offline pipeline (build-time only → preserves Off the Grid)
 
-Modal = build-time; runtime never calls it. This is how we claim **both** 🟢 Modal **and** 🔌 Off the Grid. Modal also
-trains the 🎯 Well-Tuned LoRA. Crawl org Spaces → embed with EmbeddingGemma → build vector index → commit to a Volume;
-the Space ships the index artifact and searches locally.
-
-```python
-import modal
-app = modal.App("bsh-advisor-index")
-CACHE = "/cache"
-hf_vol    = modal.Volume.from_name("hf-cache", create_if_missing=True)
-index_vol = modal.Volume.from_name("bsh-index", create_if_missing=True)
-image = (modal.Image.debian_slim("3.12")
-         .pip_install("sentence-transformers", "huggingface_hub", "requests", "numpy", "faiss-cpu")
-         .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HOME": CACHE}))
-
-@app.function(image=image)                                   # CPU: crawl one Space
-def crawl(space_id):
-    import requests
-    m = requests.get(f"https://huggingface.co/api/spaces/{space_id}").json()
-    return {"id": space_id, "text": m.get("cardData", {}).get("short_description", "")}
-
-@app.cls(image=image, gpu="T4", volumes={CACHE: hf_vol}, scaledown_window=120)
-class Embedder:
-    @modal.enter()
-    def load(self):
-        from sentence_transformers import SentenceTransformer
-        self.m = SentenceTransformer("google/embeddinggemma-300m", cache_folder=CACHE, truncate_dim=256)
-    @modal.method()
-    def embed(self, docs): return self.m.encode_document(docs).tolist()
-
-@app.local_entrypoint()
-def main(org="build-small-hackathon"):
-    import requests
-    ids  = [s["id"] for s in requests.get(f"https://huggingface.co/api/spaces?author={org}").json()]
-    docs = [d for d in crawl.map(ids) if d["text"]]
-    vecs = Embedder().embed.remote([d["text"] for d in docs])
-    # build FAISS index → write to index_vol → index_vol.commit()
+Modal = build-time; runtime never calls it. This is how the app claims **both** 🟢 Modal and 🔌 Off the Grid. The
+canonical command is:
+
+```bash
+.venv/bin/modal run scripts/modal_build_project_index.py \
+  --projects data/projects.json \
+  --out data/project_index.json
 ```
 
-- T4/CPU is plenty (pennies; $30/mo free credits). `gpu="T4"`/`"L4"` (note `"A10"`, not `"A10G"`). `volume.commit()`
-  after writing. `HF_TOKEN` via `modal.Secret` for the gated EmbeddingGemma download. Crawl on CPU, embed on GPU.
+The remote function installs `llama-cpp-python`, downloads
+`ggml-org/embeddinggemma-300M-qat-q4_0-GGUF/embeddinggemma-300M-qat-Q4_0.gguf`, embeds every project card through
+llama.cpp, and returns a schema-v2 JSON index. The local entrypoint writes that payload into the repo for Space runtime.
+
+Latest successful run: `hackathon-advisor-llama-index` on Modal, producing a 100-document, 768-dimensional normalized
+index at `2026-06-07T08:16:19+00:00`.
 
 ---
 
@@ -429,7 +399,7 @@ open grimoire as the hero component.
 | 🎨 Off-Brand (badge + $1.5k) | `gr.Server` custom UI is the agent's output surface |
 | 🏮 OpenBMB ($10k) | brain = MiniCPM5-1B ("OpenBMB pick") |
 | 🟩 NVIDIA Quest (2× RTX 5080) | ASR = Nemotron (verify if Parakeet qualifies, §5.1) |
-| 🦙 Llama Champion (badge) | MiniCPM5 + EmbeddingGemma run through llama.cpp (§5.5) |
+| 🦙 Llama Champion (badge) | EmbeddingGemma GGUF retrieval index and runtime query embeddings run through llama.cpp (§5.5) |
 | 📡 Sharing is Caring (badge) | publish the agent's tool-call trace to the Hub |
 | 📓 Field Notes (badge) | this DESIGN.md → a build blog post |
 | 🎖️ Bonus Quest Champion ($2k) | 6/6 badges (needs the Well-Tuned fine-tune) |

README.md

@@ -28,7 +28,7 @@ tags:
 The current milestone is a deployed ZeroGPU + MiniCPM5 LoRA advisor:
 
 - Local snapshot of public `build-small-hackathon` Spaces.
-- Offline search over project titles, tags, models, and descriptions.
+- Modal-built EmbeddingGemma GGUF retrieval index, with runtime query embeddings computed through llama.cpp.
 - Jargon correction for hackathon/model terms.
 - MiniCPM5 tool-call planning with a published PEFT LoRA adapter, plus deterministic local rules for tests and CPU-only
   development.
@@ -52,12 +52,14 @@ Then open <http://127.0.0.1:7860>.
 
 ```bash
 python scripts/crawl_hf_spaces.py --org build-small-hackathon --out data/projects.json
-python scripts/build_project_index.py --projects data/projects.json --out data/project_index.json
+.venv/bin/modal run scripts/modal_build_project_index.py --projects data/projects.json --out data/project_index.json
 python scripts/generate_sample_trace.py --projects data/projects.json --index data/project_index.json --out data/sample_trace.jsonl
 ```
 
 The app uses `data/projects.json` and `data/project_index.json` at runtime. The index validates the snapshot timestamp,
-source, project order, and digest before the app starts.
+source, project order, digest, embedding dimensions, and normalized vector shape before the app starts. The canonical
+index is built on Modal with `ggml-org/embeddinggemma-300M-qat-q4_0-GGUF` through llama.cpp; runtime search embeds the
+user query with the same GGUF model and performs local cosine search over the checked-in vectors.
 
 ## Trace Artifact
 
@@ -136,8 +138,9 @@ depending on browser `localStorage`.
 ## Prize Ledger
 
 `/api/prize-ledger` exposes submission evidence: the documented model stack, total parameter budget, Tiny Titan
-eligibility, runtime backend, and badge readiness. It is kept as an API artifact rather than a primary in-app panel so
-the user-facing app stays centered on idea evaluation. The main `/api/bootstrap` payload does not include the ledger.
+eligibility, runtime backend, retrieval-index metadata, and badge readiness. It is kept as an API artifact rather than a
+primary in-app panel so the user-facing app stays centered on idea evaluation. The main `/api/bootstrap` payload does
+not include the ledger.
 
 ## Wood Map
 
@@ -170,15 +173,19 @@ The deployed Space is configured for ZeroGPU inference with:
 
 ```bash
 ADVISOR_ZERO_GPU=1
-ADVISOR_ZERO_GPU_DURATION=60
+ADVISOR_ZERO_GPU_DURATION=120
 ADVISOR_MODEL_BACKEND=minicpm-transformers
 ADVISOR_MODEL_ID=openbmb/MiniCPM5-1B
 ADVISOR_ADAPTER_ID=build-small-hackathon/hackathon-advisor-minicpm5-lora
 ADVISOR_ADAPTER_REVISION=25de69bcde397e1bcdd852923b56a42f10222650
+ADVISOR_EMBEDDING_MODEL_REPO=ggml-org/embeddinggemma-300M-qat-q4_0-GGUF
+ADVISOR_EMBEDDING_MODEL_FILE=embeddinggemma-300M-qat-Q4_0.gguf
 ```
 
 `agent_turn` wraps the engine call with `spaces.GPU` when `ADVISOR_ZERO_GPU=1`, so model loading and generation run on
-the ZeroGPU allocation. Local tests and CPU-only development still default to `ADVISOR_MODEL_BACKEND=rules`.
+the ZeroGPU allocation. The retrieval query embedder downloads the GGUF model through `huggingface_hub` unless
+`ADVISOR_EMBEDDING_MODEL_PATH` points to a local file. Local tests and CPU-only development still default to
+`ADVISOR_MODEL_BACKEND=rules`.
 
 ## Test
 

app.py

@@ -134,7 +134,7 @@ def runtime() -> dict:
 
 @app.get("/api/prize-ledger")
 def prize_ledger_endpoint() -> dict:
-    return prize_ledger(engine.runtime_status())
+    return prize_ledger(engine.runtime_status(), trace_metadata(index))
 
 
 @app.get("/api/tool-contracts")
@@ -153,7 +153,7 @@ def demo_session() -> dict:
 @app.get("/api/demo-bundle.zip")
 def demo_bundle() -> Response:
     runtime_status = engine.runtime_status()
-    ledger = prize_ledger(runtime_status)
+    ledger = prize_ledger(runtime_status, trace_metadata(index))
     metadata = {
         **trace_metadata(index),
         "project_count": len(index.projects),
@@ -219,7 +219,7 @@ def chapter_api(payload: dict[str, Any] | None = Body(default=None)) -> Response
 @app.get("/api/lora-training-kit.zip")
 def lora_training_kit() -> Response:
     runtime_status = engine.runtime_status()
-    ledger = prize_ledger(runtime_status)
+    ledger = prize_ledger(runtime_status, trace_metadata(index))
     metadata = {
         **trace_metadata(index),
         "project_count": len(index.projects),
@@ -291,7 +291,7 @@ def submission_packet_artifact(session_json: str = "{}") -> str:
             **trace_metadata(index),
             "project_count": len(index.projects),
         },
-        prize_ledger(runtime_status),
+        prize_ledger(runtime_status, trace_metadata(index)),
     )
 
 

data/sample_trace.jsonl

@@ -1,4 +1,4 @@
-{"app": "hackathon-advisor", "generated_at": "2026-06-07T03:39:50+00:00", "idea_count": 2, "index": {"algorithm": "tfidf-sparse-v1", "index_generated_at": "2026-06-06T19:44:42+00:00", "snapshot_digest": "1d59c168b446bc1268ef648897fac523e85b258fd6048a37409b9cfbc5c28329", "snapshot_generated_at": "2026-06-06T19:20:47+00:00"}, "schema_version": 1, "turn_count": 3, "type": "trace_manifest"}
-{"artifact_title": "A local-first archive cartographer for family photos", "input": "A local-first archive cartographer for family photos", "overall": 4.9, "plan_steps": 0, "response": "The ink bleeds around A local-first archive cartographer for family photos. Closest echoes: page 30: Family Bill Assistant; page 45: Kasualdad LFED; page 55: Mycelium. The seal reads ECHO x4 at 4.9/10. Keep the audience, but change the mechanism or artifact so the demo proves a gap instead of joining a cluster.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {"pitch": "A local-first archive cartographer for family photos", "title": "A local-first archive cartographer for family photos"}, "name": "save_idea"}, "errors": [], "status": "valid"}, "tools": [{"name": "save_idea", "summary": "Wrote idea page 'A local-first archive cartographer for family photos'."}, {"name": "search_projects", "summary": "Checked 5 closest project echoes."}, {"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.9/10."}], "turn_index": 1, "type": "agent_turn", "verdict": "ECHO x4"}
-{"artifact_title": "Hands-on science coach", "input": "write bolder and find whitespace", "overall": 6.0, "plan_steps": 0, "response": "Gold gathers on Hands-on science coach. A lab-notebook companion that designs safe experiments from household materials. No close project echoes in the current snapshot. The seal reads UNWRITTEN at 6.0/10. The next move is to make one concrete before/after scene and cite the two weakest nearby echoes in the margin.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {}, "name": "find_whitespace"}, "errors": [], "status": "valid"}, "tools": [{"name": "find_whitespace", "summary": "Ranked 4 under-explored regions."}, {"name": "save_idea", "summary": "Wrote idea page 'Hands-on science coach'."}, {"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.8/10."}], "turn_index": 2, "type": "agent_turn", "verdict": "UNWRITTEN"}
-{"artifact_title": "Hands-on science coach", "input": "make a build plan", "overall": 6.0, "plan_steps": 6, "response": "The wax seal for Hands-on science coach reads 6.0/10, UNWRITTEN. The build path is: 1. Lock a one-sentence promise and one test input that proves what is different. 2. Compare against the nearest echoes, then sharpen the part only this idea can own. 3. Build the smallest happy path: input, nearby project citations, score, and one shareable output. 4. Add one selected-goal feature only after the core loop is smooth enough to explain without narration. 5. Collect successful advisor examples before training a tiny LoRA. 6. Write build notes from the exact decisions, screenshots, and outputs.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {}, "name": "make_plan"}, "errors": [], "status": "valid"}, "tools": [{"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.8/10."}, {"name": "make_plan", "summary": "Drafted 6 build steps."}], "turn_index": 3, "type": "agent_turn", "verdict": "UNWRITTEN"}
+{"app": "hackathon-advisor", "generated_at": "2026-06-07T08:17:29+00:00", "idea_count": 2, "index": {"algorithm": "llama-cpp-embedding-v1", "index_generated_at": "2026-06-07T08:16:19+00:00", "snapshot_digest": "1d59c168b446bc1268ef648897fac523e85b258fd6048a37409b9cfbc5c28329", "snapshot_generated_at": "2026-06-06T19:20:47+00:00"}, "schema_version": 1, "turn_count": 3, "type": "trace_manifest"}
+{"artifact_title": "A local-first archive cartographer for family photos", "input": "A local-first archive cartographer for family photos", "overall": 4.6, "plan_steps": 0, "response": "The ink bleeds around A local-first archive cartographer for family photos. Closest echoes: page 30: Family Bill Assistant; page 50: Local in 30s — Lore Lens; page 41: Investigative News Agent. The seal reads ECHO x4 at 4.6/10. Keep the audience, but change the mechanism or artifact so the demo proves a gap instead of joining a cluster.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {"pitch": "A local-first archive cartographer for family photos", "title": "A local-first archive cartographer for family photos"}, "name": "save_idea"}, "errors": [], "status": "valid"}, "tools": [{"name": "save_idea", "summary": "Wrote idea page 'A local-first archive cartographer for family photos'."}, {"name": "search_projects", "summary": "Checked 5 closest project echoes."}, {"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.6/10."}], "turn_index": 1, "type": "agent_turn", "verdict": "ECHO x4"}
+{"artifact_title": "Personal archive cartographer", "input": "write bolder and find whitespace", "overall": 4.8, "plan_steps": 0, "response": "Gold gathers on Personal archive cartographer. A tiny model that maps a private archive into stories without sending it to cloud APIs. Nearest echoes are weak: Dream Museum, Storybook. The seal reads ECHO x4 at 4.8/10. The next move is to make one concrete before/after scene and cite the two weakest nearby echoes in the margin.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {}, "name": "find_whitespace"}, "errors": [], "status": "valid"}, "tools": [{"name": "find_whitespace", "summary": "Ranked 4 under-explored regions."}, {"name": "save_idea", "summary": "Wrote idea page 'Personal archive cartographer'."}, {"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.8/10."}], "turn_index": 2, "type": "agent_turn", "verdict": "ECHO x4"}
+{"artifact_title": "Personal archive cartographer", "input": "make a build plan", "overall": 4.8, "plan_steps": 6, "response": "The wax seal for Personal archive cartographer reads 4.8/10, ECHO x4. The build path is: 1. Lock a one-sentence promise and one test input that proves what is different. 2. Compare against the nearest echoes, then sharpen the part only this idea can own. 3. Build the smallest happy path: input, nearby project citations, score, and one shareable output. 4. Add one selected-goal feature only after the core loop is smooth enough to explain without narration. 5. Collect successful advisor examples before training a tiny LoRA. 6. Write build notes from the exact decisions, screenshots, and outputs.", "schema_version": 1, "tool_resolution": {"call": {"arguments": {}, "name": "make_plan"}, "errors": [], "status": "valid"}, "tools": [{"name": "score_idea", "summary": "Pressed a five-quadrant seal: 4.8/10."}, {"name": "make_plan", "summary": "Drafted 6 build steps."}], "turn_index": 3, "type": "agent_turn", "verdict": "ECHO x4"}

hackathon_advisor/data.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from collections import Counter
+from collections.abc import Callable, Sequence
 from dataclasses import dataclass
 from datetime import datetime, timezone
 from hashlib import sha256
@@ -8,6 +8,7 @@ import json
 import math
 from pathlib import Path
 import re
+from typing import Any
 
 
 TOKEN_RE = re.compile(r"[a-z0-9][a-z0-9.+_-]*", re.IGNORECASE)
@@ -22,6 +23,15 @@ GENERIC_PUBLIC_SUMMARY_RE = re.compile(
     re.IGNORECASE,
 )
 
+INDEX_SCHEMA_VERSION = 2
+INDEX_ALGORITHM = "llama-cpp-embedding-v1"
+DEFAULT_EMBEDDING_MODEL_REPO = "ggml-org/embeddinggemma-300M-qat-q4_0-GGUF"
+DEFAULT_EMBEDDING_MODEL_FILE = "embeddinggemma-300M-qat-Q4_0.gguf"
+DEFAULT_EMBEDDING_RUNTIME = "llama.cpp via llama-cpp-python"
+
+
+EmbeddingFunction = Callable[[str], Sequence[float]]
+
 
 @dataclass(frozen=True)
 class Project:
@@ -199,47 +209,45 @@ WHITESPACE_SEEDS: tuple[WhitespaceSeed, ...] = (
 )
 
 
-INDEX_ALGORITHM = "tfidf-sparse-v1"
-
-
 class ProjectIndex:
     def __init__(
         self,
         projects: list[Project],
         generated_at: str,
         source: str,
-        index_payload: dict | None = None,
+        index_payload: dict,
+        query_embedder: EmbeddingFunction | None = None,
     ) -> None:
         if not projects:
             raise ValueError("project index requires at least one project")
+        validate_index_payload(index_payload, projects, generated_at, source)
         self.projects = projects
         self.generated_at = generated_at
         self.source = source
-        if index_payload is None:
-            index_payload = build_index_payload(projects, generated_at, source)
-        validate_index_payload(index_payload, projects, generated_at, source)
         self.index_generated_at = str(index_payload["generated_at"])
         self.index_algorithm = str(index_payload["algorithm"])
         self.snapshot_digest = str(index_payload["snapshot_digest"])
-        self._idf = {str(term): float(value) for term, value in index_payload["idf"].items()}
-        self._documents = [
-            Counter({str(term): float(value) for term, value in document["weights"].items()})
+        self.embedding_metadata = dict(index_payload["embedding"])
+        self.embedding_dimensions = int(self.embedding_metadata["dimensions"])
+        self._query_embedder = query_embedder
+        self._vectors = [
+            tuple(float(value) for value in document["vector"])
             for document in index_payload["documents"]
         ]
-        self._norms = [float(document["norm"]) for document in index_payload["documents"]]
 
     @classmethod
-    def from_file(cls, path: Path) -> "ProjectIndex":
+    def from_file(cls, path: Path, query_embedder: EmbeddingFunction | None = None) -> "ProjectIndex":
         data = json.loads(path.read_text(encoding="utf-8"))
         projects = [Project.from_dict(item) for item in data["projects"]]
-        return cls(
-            projects=projects,
-            generated_at=str(data.get("generated_at") or ""),
-            source=str(data.get("source") or ""),
-        )
+        raise ValueError("ProjectIndex.from_file requires a separate embedding index payload")
 
     @classmethod
-    def from_files(cls, project_path: Path, index_path: Path) -> "ProjectIndex":
+    def from_files(
+        cls,
+        project_path: Path,
+        index_path: Path,
+        query_embedder: EmbeddingFunction | None = None,
+    ) -> "ProjectIndex":
         data = json.loads(project_path.read_text(encoding="utf-8"))
         index_payload = json.loads(index_path.read_text(encoding="utf-8"))
         projects = [Project.from_dict(item) for item in data["projects"]]
@@ -248,8 +256,12 @@ class ProjectIndex:
             generated_at=str(data.get("generated_at") or ""),
             source=str(data.get("source") or ""),
             index_payload=index_payload,
+            query_embedder=query_embedder,
         )
 
+    def set_query_embedder(self, embedder: EmbeddingFunction) -> None:
+        self._query_embedder = embedder
+
     def top_projects(self, limit: int = 8) -> list[Project]:
         return sorted(
             self.projects,
@@ -258,35 +270,21 @@ class ProjectIndex:
         )[:limit]
 
     def search(self, query: str, limit: int = 5) -> list[SearchHit]:
-        query_terms = tokenize(query)
+        query_terms = set(tokenize(query))
         if not query_terms:
             return []
-        query_doc = Counter(query_terms)
-        query_norm = self._norm(query_doc)
+        query_vector = normalize_vector(self._embed_query(query))
         hits: list[SearchHit] = []
-        for page_number, (project, doc, doc_norm) in enumerate(
-            zip(self.projects, self._documents, self._norms, strict=True),
+        for page_number, (project, vector) in enumerate(
+            zip(self.projects, self._vectors, strict=True),
             start=1,
         ):
-            if doc_norm == 0.0 or query_norm == 0.0:
-                continue
-            raw = 0.0
-            matched: list[str] = []
-            for term, count in query_doc.items():
-                if term not in doc:
-                    continue
-                raw += (count * self._idf.get(term, 1.0)) * doc[term]
-                matched.append(term)
-            if not matched:
-                continue
-            title_bonus = sum(0.08 for term in matched if term in tokenize(project.title))
-            tag_bonus = sum(0.05 for term in matched if term in tokenize(" ".join(project.tags)))
-            score = raw / (query_norm * doc_norm) + title_bonus + tag_bonus
+            score = max(0.0, min(1.0, (dot_product(query_vector, vector) + 1.0) / 2.0))
             hits.append(
                 SearchHit(
                     project=project,
                     score=score,
-                    matched_terms=tuple(sorted(matched)),
+                    matched_terms=matched_terms(query_terms, project),
                     page_number=page_number,
                 )
             )
@@ -304,7 +302,7 @@ class ProjectIndex:
         for seed in WHITESPACE_SEEDS:
             hits = self.search(seed.query, limit=3)
             saturation = sum(hit.score for hit in hits) / max(len(hits), 1)
-            score = max(0.0, 1.0 - min(saturation, 0.95))
+            score = max(0.0, min(1.0, 1.0 - max(0.0, saturation - 0.35) / 0.60))
             if hits:
                 evidence = f"Nearest echoes are weak: {', '.join(hit.project.title for hit in hits[:2])}."
             else:
@@ -321,47 +319,67 @@ class ProjectIndex:
         items.sort(key=lambda item: item.score, reverse=True)
         return items[:limit]
 
-    def _norm(self, doc: Counter[str]) -> float:
-        return math.sqrt(sum((count * self._idf.get(term, 1.0)) ** 2 for term, count in doc.items()))
+    def _embed_query(self, query: str) -> Sequence[float]:
+        if self._query_embedder is None:
+            from hackathon_advisor.llama_embedding import create_llama_cpp_embedder
+
+            self._query_embedder = create_llama_cpp_embedder(self.embedding_metadata)
+        return self._query_embedder(query)
 
 
 def tokenize(text: str) -> list[str]:
     return [token.lower().strip("._-+") for token in TOKEN_RE.findall(text) if len(token.strip("._-+")) > 1]
 
 
-def build_index_payload(projects: list[Project], snapshot_generated_at: str, source: str) -> dict:
-    documents = [Counter(tokenize(project.searchable_text)) for project in projects]
-    df = Counter(term for document in documents for term in document)
-    idf = {
-        term: math.log((1 + len(documents)) / (1 + freq)) + 1.0
-        for term, freq in sorted(df.items())
+def matched_terms(query_terms: set[str], project: Project) -> tuple[str, ...]:
+    project_terms = set(tokenize(project.searchable_text))
+    return tuple(sorted(query_terms & project_terms)[:8])
+
+
+def build_index_payload(
+    projects: list[Project],
+    snapshot_generated_at: str,
+    source: str,
+    embeddings: Sequence[Sequence[float]],
+    *,
+    embedding_metadata: dict[str, Any] | None = None,
+) -> dict:
+    if len(embeddings) != len(projects):
+        raise ValueError("embedding count must match project count")
+    normalized = [normalize_vector(vector) for vector in embeddings]
+    dimensions = len(normalized[0]) if normalized else 0
+    if dimensions <= 0:
+        raise ValueError("embedding vectors must not be empty")
+    if any(len(vector) != dimensions for vector in normalized):
+        raise ValueError("embedding vectors must have one shared dimension")
+
+    metadata = {
+        "model_repo": DEFAULT_EMBEDDING_MODEL_REPO,
+        "model_file": DEFAULT_EMBEDDING_MODEL_FILE,
+        "runtime": DEFAULT_EMBEDDING_RUNTIME,
+        "dimensions": dimensions,
+        "normalized": True,
+        **(embedding_metadata or {}),
     }
     indexed_documents = []
-    for project, document in zip(projects, documents, strict=True):
-        weights = {
-            term: round(count * idf.get(term, 1.0), 8)
-            for term, count in sorted(document.items())
-        }
-        norm = math.sqrt(sum(value * value for value in weights.values()))
+    for project, vector in zip(projects, normalized, strict=True):
         indexed_documents.append(
             {
                 "project_id": project.id,
-                "tokens": sum(document.values()),
-                "unique_terms": len(document),
-                "norm": round(norm, 8),
-                "weights": weights,
+                "text_digest": sha256(project.searchable_text.encode("utf-8")).hexdigest(),
+                "norm": round(vector_norm(vector), 8),
+                "vector": [round(value, 8) for value in vector],
             }
         )
     return {
-        "schema_version": 1,
+        "schema_version": INDEX_SCHEMA_VERSION,
         "algorithm": INDEX_ALGORITHM,
         "generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
         "snapshot_generated_at": snapshot_generated_at,
         "snapshot_source": source,
         "snapshot_digest": project_snapshot_digest(projects, snapshot_generated_at, source),
         "document_count": len(projects),
-        "vocabulary_size": len(idf),
-        "idf": {term: round(value, 8) for term, value in idf.items()},
+        "embedding": metadata,
         "documents": indexed_documents,
     }
 
@@ -372,7 +390,7 @@ def validate_index_payload(
     snapshot_generated_at: str,
     snapshot_source: str,
 ) -> None:
-    if payload.get("schema_version") != 1:
+    if payload.get("schema_version") != INDEX_SCHEMA_VERSION:
         raise ValueError("unsupported project index schema version")
     if payload.get("algorithm") != INDEX_ALGORITHM:
         raise ValueError(f"unsupported project index algorithm: {payload.get('algorithm')}")
@@ -386,6 +404,16 @@ def validate_index_payload(
         snapshot_source,
     ):
         raise ValueError("project index digest does not match projects snapshot")
+
+    embedding = payload.get("embedding")
+    if not isinstance(embedding, dict):
+        raise ValueError("project index embedding metadata is missing")
+    dimensions = int(embedding.get("dimensions") or 0)
+    if dimensions <= 0:
+        raise ValueError("project index embedding dimensions must be positive")
+    if embedding.get("runtime") != DEFAULT_EMBEDDING_RUNTIME:
+        raise ValueError("project index embedding runtime must be llama.cpp")
+
     documents = payload.get("documents")
     if not isinstance(documents, list) or len(documents) != len(projects):
         raise ValueError("project index document count does not match projects snapshot")
@@ -393,6 +421,31 @@ def validate_index_payload(
     indexed_ids = [document.get("project_id") for document in documents]
     if indexed_ids != project_ids:
         raise ValueError("project index project order does not match projects snapshot")
+    for document in documents:
+        vector = document.get("vector")
+        if not isinstance(vector, list) or len(vector) != dimensions:
+            raise ValueError("project index vector dimensions do not match embedding metadata")
+        norm = vector_norm(float(value) for value in vector)
+        if not 0.99 <= norm <= 1.01:
+            raise ValueError("project index vectors must be normalized")
+
+
+def normalize_vector(vector: Sequence[float]) -> tuple[float, ...]:
+    values = tuple(float(value) for value in vector)
+    norm = vector_norm(values)
+    if norm == 0.0:
+        raise ValueError("embedding vector norm must be non-zero")
+    return tuple(value / norm for value in values)
+
+
+def vector_norm(vector: Sequence[float]) -> float:
+    return math.sqrt(sum(float(value) * float(value) for value in vector))
+
+
+def dot_product(left: Sequence[float], right: Sequence[float]) -> float:
+    if len(left) != len(right):
+        raise ValueError("embedding vectors must have equal dimensions")
+    return sum(float(a) * float(b) for a, b in zip(left, right, strict=True))
 
 
 def project_snapshot_digest(projects: list[Project], generated_at: str, source: str) -> str:

hackathon_advisor/llama_embedding.py

@@ -0,0 +1,108 @@
+from __future__ import annotations
+
+from collections.abc import Sequence
+from pathlib import Path
+import os
+from typing import Any
+
+from hackathon_advisor.data import (
+    DEFAULT_EMBEDDING_MODEL_FILE,
+    DEFAULT_EMBEDDING_MODEL_REPO,
+)
+
+
+TRUE_VALUES = {"1", "true", "yes", "on"}
+DEFAULT_N_CTX = 512
+
+
+class LlamaCppEmbedder:
+    def __init__(
+        self,
+        *,
+        model_repo: str = DEFAULT_EMBEDDING_MODEL_REPO,
+        model_file: str = DEFAULT_EMBEDDING_MODEL_FILE,
+        model_path: str = "",
+        n_ctx: int = DEFAULT_N_CTX,
+        n_threads: int | None = None,
+        n_gpu_layers: int = 0,
+        verbose: bool = False,
+    ) -> None:
+        self.model_repo = model_repo.strip() or DEFAULT_EMBEDDING_MODEL_REPO
+        self.model_file = model_file.strip() or DEFAULT_EMBEDDING_MODEL_FILE
+        self.model_path = model_path.strip()
+        self.n_ctx = n_ctx
+        self.n_threads = n_threads
+        self.n_gpu_layers = n_gpu_layers
+        self.verbose = verbose
+        self._model = None
+
+    def __call__(self, text: str) -> Sequence[float]:
+        return self.embed(text)
+
+    def embed(self, text: str) -> Sequence[float]:
+        model = self._ensure_model()
+        return model.embed(text, normalize=True)
+
+    def _ensure_model(self):
+        if self._model is not None:
+            return self._model
+        from huggingface_hub import hf_hub_download
+        from llama_cpp import LLAMA_POOLING_TYPE_MEAN, Llama
+
+        model_path = self.model_path
+        if not model_path:
+            model_path = hf_hub_download(
+                repo_id=self.model_repo,
+                filename=self.model_file,
+                repo_type="model",
+            )
+        if not Path(model_path).is_file():
+            raise RuntimeError(f"llama.cpp embedding model was not found: {model_path}")
+        self._model = Llama(
+            model_path=model_path,
+            embedding=True,
+            pooling_type=LLAMA_POOLING_TYPE_MEAN,
+            n_ctx=self.n_ctx,
+            n_threads=self.n_threads,
+            n_gpu_layers=self.n_gpu_layers,
+            verbose=self.verbose,
+        )
+        return self._model
+
+
+def create_llama_cpp_embedder(metadata: dict[str, Any]) -> LlamaCppEmbedder:
+    return LlamaCppEmbedder(
+        model_repo=os.environ.get(
+            "ADVISOR_EMBEDDING_MODEL_REPO",
+            str(metadata.get("model_repo") or DEFAULT_EMBEDDING_MODEL_REPO),
+        ),
+        model_file=os.environ.get(
+            "ADVISOR_EMBEDDING_MODEL_FILE",
+            str(metadata.get("model_file") or DEFAULT_EMBEDDING_MODEL_FILE),
+        ),
+        model_path=os.environ.get("ADVISOR_EMBEDDING_MODEL_PATH", ""),
+        n_ctx=_int_env("ADVISOR_EMBEDDING_N_CTX", DEFAULT_N_CTX),
+        n_threads=_optional_int_env("ADVISOR_EMBEDDING_THREADS"),
+        n_gpu_layers=_int_env("ADVISOR_EMBEDDING_GPU_LAYERS", 0),
+        verbose=os.environ.get("ADVISOR_EMBEDDING_VERBOSE", "").strip().lower() in TRUE_VALUES,
+    )
+
+
+def _int_env(name: str, default: int) -> int:
+    raw = os.environ.get(name, "").strip()
+    if not raw:
+        return default
+    value = int(raw)
+    if value < 0:
+        raise RuntimeError(f"{name} must be a non-negative integer.")
+    return value
+
+
+def _optional_int_env(name: str) -> int | None:
+    raw = os.environ.get(name, "").strip()
+    if not raw:
+        return None
+    value = int(raw)
+    if value <= 0:
+        raise RuntimeError(f"{name} must be a positive integer.")
+    return value

hackathon_advisor/prize_ledger.py

@@ -13,18 +13,11 @@ MODEL_STACK = [
         "runtime": "ZeroGPU + transformers + PEFT",
     },
     {
-        "role": "Retriever",
-        "model": "offline TF-IDF snapshot",
-        "params_b": 0.0,
-        "status": "deployed",
-        "runtime": "local sparse index",
-    },
-    {
-        "role": "Planned embedder",
-        "model": "google/embeddinggemma-300m",
+        "role": "Embedding retriever",
+        "model": "ggml-org/embeddinggemma-300M-qat-q4_0-GGUF",
         "params_b": 0.30,
-        "status": "documented build path",
-        "runtime": "sentence-transformers / llama.cpp",
+        "status": "deployed",
+        "runtime": "Modal-built llama.cpp GGUF index + runtime llama.cpp query embeddings",
     },
     {
         "role": "Voice bonus",
@@ -40,7 +33,7 @@ BADGE_LEDGER = [
     {
         "name": "Off the Grid",
         "status": "ready",
-        "evidence": "Runtime uses a checked-in snapshot and local search; no proprietary inference API.",
+        "evidence": "Runtime uses checked-in project vectors and local llama.cpp query embeddings; no proprietary inference API.",
     },
     {
         "name": "Off-Brand",
@@ -69,8 +62,8 @@ BADGE_LEDGER = [
     },
     {
         "name": "Llama Champion",
-        "status": "planned",
-        "evidence": "MiniCPM5 GGUF and EmbeddingGemma GGUF paths are documented; runtime does not depend on them yet.",
+        "status": "ready",
+        "evidence": "Retrieval uses an EmbeddingGemma GGUF index built by llama.cpp on Modal and query embeddings computed through llama.cpp at runtime.",
     },
 ]
 
@@ -94,11 +87,12 @@ TRAINING_ARTIFACTS = [
 ]
 
 
-def prize_ledger(runtime: dict[str, Any]) -> dict[str, Any]:
+def prize_ledger(runtime: dict[str, Any], index_metadata: dict[str, Any] | None = None) -> dict[str, Any]:
     total_params = round(sum(float(item["params_b"]) for item in MODEL_STACK), 2)
     largest = max(MODEL_STACK, key=lambda item: float(item["params_b"]))
     return {
         "runtime": runtime,
+        "retrieval_index": index_metadata or {},
         "model_stack": MODEL_STACK,
         "total_params_b": total_params,
         "largest_model": {

hackathon_advisor/trace_export.py

@@ -47,12 +47,26 @@ def build_trace_jsonl(session: dict[str, Any], metadata: dict[str, Any]) -> str:
 
 
 def trace_metadata(index: Any) -> dict[str, str]:
-    return {
+    metadata = {
         "snapshot_generated_at": index.generated_at,
         "index_generated_at": index.index_generated_at,
         "index_algorithm": index.index_algorithm,
         "snapshot_digest": index.snapshot_digest,
     }
+    embedding = getattr(index, "embedding_metadata", None)
+    if isinstance(embedding, dict):
+        metadata.update(
+            {
+                "embedding_model_repo": str(embedding.get("model_repo") or ""),
+                "embedding_model_file": str(embedding.get("model_file") or ""),
+                "embedding_runtime": str(embedding.get("runtime") or ""),
+                "embedding_build_source": str(embedding.get("build_source") or ""),
+                "embedding_dimensions": str(embedding.get("dimensions") or ""),
+                "embedding_builder": str(embedding.get("builder") or ""),
+                "embedding_modal_app": str(embedding.get("modal_app") or ""),
+            }
+        )
+    return metadata
 
 
 def _tools(event: dict[str, Any]) -> list[dict[str, str]]:

pyproject.toml

@@ -10,6 +10,7 @@ dependencies = [
   "accelerate>=1.0,<2",
   "gradio>=6.16.0,<7",
   "huggingface-hub>=0.36,<1",
+  "llama-cpp-python>=0.3.26,<1",
   "peft>=0.13,<1",
   "pillow>=10,<13",
   "spaces>=0.50,<1",
@@ -23,6 +24,7 @@ dev = [
 ]
 model = [
   "accelerate>=1.0,<2",
+  "llama-cpp-python>=0.3.26,<1",
   "peft>=0.13,<1",
   "spaces>=0.50,<1",
   "torch>=2.8,<3",
@@ -35,6 +37,10 @@ train = [
   "torch>=2.8,<3",
   "transformers>=4.55,<5",
 ]
+index = [
+  "llama-cpp-python>=0.3.26,<1",
+  "modal>=1.4,<2",
+]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

requirements.txt

@@ -1,6 +1,7 @@
 accelerate>=1.0,<2
 gradio>=6.16.0,<7
 huggingface-hub>=0.36,<1
+llama-cpp-python>=0.3.26,<1
 peft>=0.13,<1
 pillow>=10,<13
 spaces>=0.50,<1

scripts/build_project_index.py

@@ -2,38 +2,96 @@
 from __future__ import annotations
 
 import argparse
+import importlib.metadata
 import json
 from pathlib import Path
 import sys
 
 ROOT = Path(__file__).resolve().parents[1]
 sys.path.insert(0, str(ROOT))
-from hackathon_advisor.data import Project, build_index_payload
+
+from hackathon_advisor.data import (
+    DEFAULT_EMBEDDING_MODEL_FILE,
+    DEFAULT_EMBEDDING_MODEL_REPO,
+    Project,
+    build_index_payload,
+)
+from hackathon_advisor.llama_embedding import LlamaCppEmbedder
 
 
 def main() -> None:
-    parser = argparse.ArgumentParser(description="Build the offline project retrieval index.")
+    parser = argparse.ArgumentParser(
+        description="Build the offline project retrieval index with llama.cpp embeddings."
+    )
     parser.add_argument("--projects", default="data/projects.json")
     parser.add_argument("--out", default="data/project_index.json")
+    parser.add_argument("--model-repo", default=DEFAULT_EMBEDDING_MODEL_REPO)
+    parser.add_argument("--model-file", default=DEFAULT_EMBEDDING_MODEL_FILE)
+    parser.add_argument("--model-path", default="")
+    parser.add_argument("--n-ctx", type=int, default=512)
+    parser.add_argument("--n-threads", type=int, default=0)
     args = parser.parse_args()
 
-    project_path = Path(args.projects)
-    data = json.loads(project_path.read_text(encoding="utf-8"))
-    projects = [Project.from_dict(item) for item in data["projects"]]
-    payload = build_index_payload(
-        projects=projects,
-        snapshot_generated_at=str(data.get("generated_at") or ""),
-        source=str(data.get("source") or ""),
+    payload = build_payload(
+        Path(args.projects),
+        model_repo=args.model_repo,
+        model_file=args.model_file,
+        model_path=args.model_path,
+        n_ctx=args.n_ctx,
+        n_threads=args.n_threads or None,
+        build_source="local",
+        builder="scripts/build_project_index.py",
     )
     output = Path(args.out)
     output.parent.mkdir(parents=True, exist_ok=True)
     output.write_text(json.dumps(payload, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
     print(
         "wrote "
-        f"{payload['document_count']} docs, {payload['vocabulary_size']} terms "
+        f"{payload['document_count']} docs, {payload['embedding']['dimensions']} dims "
         f"to {output}"
     )
 
 
+def build_payload(
+    project_path: Path,
+    *,
+    model_repo: str,
+    model_file: str,
+    model_path: str = "",
+    n_ctx: int = 512,
+    n_threads: int | None = None,
+    build_source: str,
+    builder: str,
+    modal_app: str = "",
+) -> dict:
+    data = json.loads(project_path.read_text(encoding="utf-8"))
+    projects = [Project.from_dict(item) for item in data["projects"]]
+    embedder = LlamaCppEmbedder(
+        model_repo=model_repo,
+        model_file=model_file,
+        model_path=model_path,
+        n_ctx=n_ctx,
+        n_threads=n_threads,
+        verbose=False,
+    )
+    embeddings = [embedder.embed(project.searchable_text) for project in projects]
+    metadata = {
+        "model_repo": model_repo,
+        "model_file": model_file,
+        "build_source": build_source,
+        "builder": builder,
+        "llama_cpp_python_version": importlib.metadata.version("llama-cpp-python"),
+    }
+    if modal_app:
+        metadata["modal_app"] = modal_app
+    return build_index_payload(
+        projects=projects,
+        snapshot_generated_at=str(data.get("generated_at") or ""),
+        source=str(data.get("source") or ""),
+        embeddings=embeddings,
+        embedding_metadata=metadata,
+    )
+
+
 if __name__ == "__main__":
     main()

scripts/modal_build_project_index.py

@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+from typing import Any
+
+import modal
+
+
+APP_NAME = "hackathon-advisor-llama-index"
+
+app = modal.App(APP_NAME)
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .pip_install(
+        "huggingface-hub>=0.36,<1",
+        "llama-cpp-python>=0.3.26,<1",
+    )
+    .add_local_python_source("hackathon_advisor", copy=True)
+    .add_local_python_source("scripts", copy=True)
+)
+
+
+@app.function(image=image, cpu=4.0, memory=4096, timeout=1800)
+def build_project_index_remote(
+    project_snapshot: dict[str, Any],
+    model_repo: str,
+    model_file: str,
+) -> dict[str, Any]:
+    from pathlib import Path
+    import tempfile
+
+    from scripts.build_project_index import build_payload
+
+    with tempfile.TemporaryDirectory() as tmpdir:
+        project_path = Path(tmpdir) / "projects.json"
+        project_path.write_text(
+            json.dumps(project_snapshot, ensure_ascii=False),
+            encoding="utf-8",
+        )
+        return build_payload(
+            project_path,
+            model_repo=model_repo,
+            model_file=model_file,
+            build_source="modal remote function",
+            builder="scripts/modal_build_project_index.py",
+            modal_app=APP_NAME,
+        )
+
+
+@app.local_entrypoint()
+def main(
+    projects: str = "data/projects.json",
+    out: str = "data/project_index.json",
+    model_repo: str = "ggml-org/embeddinggemma-300M-qat-q4_0-GGUF",
+    model_file: str = "embeddinggemma-300M-qat-Q4_0.gguf",
+) -> None:
+    project_snapshot = json.loads(Path(projects).read_text(encoding="utf-8"))
+    payload = build_project_index_remote.remote(project_snapshot, model_repo, model_file)
+    output = Path(out)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(payload, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+    print(
+        "wrote "
+        f"{payload['document_count']} docs, {payload['embedding']['dimensions']} dims "
+        f"to {output}"
+    )
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Build the llama.cpp embedding index on Modal.")
+    parser.add_argument("--projects", default="data/projects.json")
+    parser.add_argument("--out", default="data/project_index.json")
+    parser.add_argument("--model-repo", default="ggml-org/embeddinggemma-300M-qat-q4_0-GGUF")
+    parser.add_argument("--model-file", default="embeddinggemma-300M-qat-Q4_0.gguf")
+    args = parser.parse_args()
+    with app.run():
+        payload = build_project_index_remote.remote(
+            json.loads(Path(args.projects).read_text(encoding="utf-8")),
+            args.model_repo,
+            args.model_file,
+        )
+    output = Path(args.out)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(payload, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+    print(
+        "wrote "
+        f"{payload['document_count']} docs, {payload['embedding']['dimensions']} dims "
+        f"to {output}"
+    )

tests/__init__.py

@@ -0,0 +1 @@
+

tests/conftest.py

@@ -0,0 +1,10 @@
+from __future__ import annotations
+
+from tests.helpers import test_query_embedder
+
+
+def pytest_configure() -> None:
+    import app
+
+    app.index.set_query_embedder(test_query_embedder)
+    app.engine.index.set_query_embedder(test_query_embedder)

tests/helpers.py

@@ -0,0 +1,26 @@
+from __future__ import annotations
+
+from hashlib import sha256
+from pathlib import Path
+
+from hackathon_advisor.data import ProjectIndex, normalize_vector, tokenize
+
+
+def load_test_index() -> ProjectIndex:
+    return ProjectIndex.from_files(
+        Path("data/projects.json"),
+        Path("data/project_index.json"),
+        query_embedder=test_query_embedder,
+    )
+
+
+def test_query_embedder(text: str) -> tuple[float, ...]:
+    vector = [0.0] * 768
+    for token in tokenize(text):
+        digest = sha256(token.encode("utf-8")).digest()
+        index = int.from_bytes(digest[:2], "big") % len(vector)
+        sign = 1.0 if digest[2] % 2 == 0 else -1.0
+        vector[index] += sign
+    if not any(vector):
+        vector[0] = 1.0
+    return normalize_vector(vector)

tests/test_agent.py

@@ -1,5 +1,7 @@
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.tool_contracts import ToolCall, ToolResolution
@@ -17,7 +19,7 @@ class StaticPlanner:
 
 
 def test_agent_scores_and_persists_idea() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     result = engine.turn("A local-first archive cartographer for family photos", {})
@@ -42,7 +44,7 @@ def test_agent_scores_and_persists_idea() -> None:
 
 
 def test_agent_finds_whitespace() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     result = engine.turn("write bolder and find whitespace", {})
@@ -54,7 +56,7 @@ def test_agent_finds_whitespace() -> None:
 
 
 def test_gap_command_explores_unused_whitespace() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     first = engine.turn("write bolder and find whitespace", {})
@@ -67,7 +69,7 @@ def test_gap_command_explores_unused_whitespace() -> None:
 
 
 def test_agent_preserves_canonical_jargon_case() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     result = engine.turn("use neutron and mini cpm on zero gpu", {})
@@ -77,7 +79,7 @@ def test_agent_preserves_canonical_jargon_case() -> None:
 
 
 def test_plan_command_uses_current_idea() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     first = engine.turn("A local-first archive cartographer for family photos", {})
@@ -91,7 +93,7 @@ def test_plan_command_uses_current_idea() -> None:
 
 
 def test_non_plan_turns_clear_stale_build_plan() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     first = engine.turn("A local-first archive cartographer for family photos", {})
@@ -105,7 +107,7 @@ def test_non_plan_turns_clear_stale_build_plan() -> None:
 
 
 def test_plan_and_rank_do_not_create_placeholder_ideas() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     planned = engine.turn("make a build plan", {})
@@ -120,7 +122,7 @@ def test_plan_and_rank_do_not_create_placeholder_ideas() -> None:
 
 
 def test_plan_uses_profile_context() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {
         "profile": {
@@ -141,7 +143,7 @@ def test_plan_uses_profile_context() -> None:
 
 
 def test_distinct_idea_turns_append_to_board() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     first = engine.turn("A local-first archive cartographer for family photos", {})
@@ -155,7 +157,7 @@ def test_distinct_idea_turns_append_to_board() -> None:
 
 
 def test_compare_ideas_reranks_board_and_selects_winner() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     first = engine.turn("A local-first archive cartographer for family photos", {})
@@ -173,7 +175,7 @@ def test_compare_ideas_reranks_board_and_selects_winner() -> None:
 
 
 def test_plan_preserves_unwritten_whitespace_verdict() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     whitespace = engine.turn("write bolder and find whitespace", {})
@@ -185,7 +187,7 @@ def test_plan_preserves_unwritten_whitespace_verdict() -> None:
 
 
 def test_planner_get_project_drives_project_response() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index, planner=StaticPlanner(ToolCall("get_project", {"id": "lolaby"})))
 
     result = engine.turn("read lolaby", {})
@@ -196,7 +198,7 @@ def test_planner_get_project_drives_project_response() -> None:
 
 
 def test_rule_project_reference_does_not_create_or_score_idea() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     first = engine.turn("A local-first archive cartographer for family photos", {})
 
@@ -213,7 +215,7 @@ def test_rule_project_reference_does_not_create_or_score_idea() -> None:
 
 
 def test_planner_profile_and_goals_update_state() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     planned = AdvisorEngine(index).turn("A local-first archive cartographer for family photos", {})
     planned = AdvisorEngine(index).turn("make a build plan", planned.state)
     assert planned.state["last_plan"]
@@ -236,7 +238,7 @@ def test_planner_profile_and_goals_update_state() -> None:
 
 
 def test_goal_update_invalidates_current_idea_artifact() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     first = AdvisorEngine(index).turn("A local-first archive cartographer for family photos", {})
     first = AdvisorEngine(index).turn("make a build plan", first.state)
     assert first.state["last_plan"]
@@ -255,7 +257,7 @@ def test_goal_update_invalidates_current_idea_artifact() -> None:
 
 
 def test_session_goals_apply_to_new_and_current_ideas() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {"goals": ["Field Notes"]}
 
@@ -268,7 +270,7 @@ def test_session_goals_apply_to_new_and_current_ideas() -> None:
 
 
 def test_well_tuned_goal_adds_training_step_to_plan() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {"goals": ["Well-Tuned"]}
 
@@ -281,7 +283,7 @@ def test_well_tuned_goal_adds_training_step_to_plan() -> None:
 
 
 def test_planner_score_idea_scores_current_idea() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     first = AdvisorEngine(index).turn("A local-first archive cartographer for family photos", {})
     engine = AdvisorEngine(index, planner=StaticPlanner(ToolCall("score_idea", {})))
 

tests/test_app.py

@@ -39,7 +39,7 @@ def test_health_exposes_index_metadata() -> None:
 
     assert payload["ok"] is True
     assert payload["projects"] == len(index.projects)
-    assert payload["index_algorithm"] == "tfidf-sparse-v1"
+    assert payload["index_algorithm"] == "llama-cpp-embedding-v1"
     assert payload["runtime"]["backend"] == "rules"
     assert len(payload["snapshot_digest"]) == 64
 
@@ -47,7 +47,7 @@ def test_health_exposes_index_metadata() -> None:
 def test_bootstrap_exposes_index_metadata() -> None:
     payload = bootstrap()
 
-    assert payload["index_algorithm"] == "tfidf-sparse-v1"
+    assert payload["index_algorithm"] == "llama-cpp-embedding-v1"
     assert payload["index_generated_at"]
     assert payload["snapshot_digest"]
     assert payload["runtime"]["tool_count"] >= 8
@@ -247,5 +247,8 @@ def test_prize_ledger_endpoint_reports_submission_evidence() -> None:
     assert payload["runtime"]["backend"] == "rules"
     assert payload["tiny_titan_eligible"] is True
     assert any(badge["name"] == "Sharing is Caring" for badge in payload["badges"])
+    assert {badge["name"]: badge["status"] for badge in payload["badges"]}["Llama Champion"] == "ready"
+    assert payload["retrieval_index"]["index_algorithm"] == "llama-cpp-embedding-v1"
+    assert payload["retrieval_index"]["embedding_runtime"] == "llama.cpp via llama-cpp-python"
     assert payload["training_artifacts"][0]["endpoint"] == "lora_dataset"
     assert payload["training_artifacts"][1]["endpoint"] == "/api/lora-training-kit.zip"

tests/test_artifact_bundle.py

@@ -1,6 +1,8 @@
 import json
 from io import BytesIO
 from pathlib import Path
+
+from tests.helpers import load_test_index
 from zipfile import ZipFile
 
 from hackathon_advisor.agent import AdvisorEngine
@@ -12,7 +14,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_demo_bundle_contains_submission_evidence_files() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     metadata = {
         **trace_metadata(index),

tests/test_chapter.py

@@ -1,5 +1,8 @@
+import re
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.chapter import build_chapter_markdown
 from hackathon_advisor.data import ProjectIndex
@@ -7,7 +10,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_chapter_markdown_contains_idea_pages_and_citations() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = engine.turn("A local-first archive cartographer for family photos", {}).state
     state = engine.turn("write bolder and find whitespace", state).state
@@ -26,7 +29,7 @@ def test_chapter_markdown_contains_idea_pages_and_citations() -> None:
     assert "Goals:" in markdown
     assert "Targets:" not in markdown
     assert "Closest cited pages:" in markdown
-    assert "Page 30:" in markdown
+    assert re.search(r"Page \d+:", markdown)
 
 
 def test_empty_chapter_markdown_is_explicit() -> None:

tests/test_data.py

@@ -1,22 +1,24 @@
 from pathlib import Path
+
+from tests.helpers import load_test_index
 import json
 
 from hackathon_advisor.data import Project, ProjectIndex, public_project_summary, public_project_title
 
 
 def test_project_index_searches_snapshot() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
 
     hits = index.search("lullaby children audio", limit=3)
 
     assert hits
     assert hits[0].project.id.startswith("build-small-hackathon/")
     assert hits[0].page_number >= 1
-    assert index.index_algorithm == "tfidf-sparse-v1"
+    assert index.index_algorithm == "llama-cpp-embedding-v1"
 
 
 def test_project_index_whitespace() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
 
     items = index.find_whitespace(limit=3)
 

tests/test_demo_rehearsal.py

@@ -1,12 +1,14 @@
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.demo_rehearsal import DEMO_GOALS, build_demo_rehearsal
 
 
 def test_demo_rehearsal_builds_complete_session() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
 
     payload = build_demo_rehearsal(engine)

tests/test_field_notes.py

@@ -1,5 +1,7 @@
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.field_notes import build_field_notes_markdown
@@ -7,7 +9,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_field_notes_markdown_contains_session_decisions() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {
         "profile": {"skills": "frontend prototyping"},

tests/test_lora_dataset.py

@@ -1,6 +1,8 @@
 import json
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.lora_dataset import BASE_MODEL, build_lora_dataset_jsonl
@@ -8,7 +10,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_lora_dataset_exports_tool_call_and_response_examples() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {"goals": ["Well-Tuned", "Field Notes"]}
     state = engine.turn("A local-first archive cartographer for family photos", state).state
@@ -23,7 +25,7 @@ def test_lora_dataset_exports_tool_call_and_response_examples() -> None:
     assert manifest["record_kinds"] == ["tool_call", "advisor_response"]
     assert manifest["example_count"] == len(examples)
     assert manifest["included_turn_count"] == 2
-    assert manifest["index"]["algorithm"] == "tfidf-sparse-v1"
+    assert manifest["index"]["algorithm"] == "llama-cpp-embedding-v1"
     assert {example["example_kind"] for example in examples} == {"tool_call", "advisor_response"}
     assert examples[0]["messages"][2]["content"].startswith('<function name="save_idea">')
     assert examples[0]["goals"] == ["Well-Tuned", "Field Notes"]
@@ -41,7 +43,7 @@ def test_empty_lora_dataset_only_exports_manifest() -> None:
     payload = build_lora_dataset_jsonl(
         {},
         {
-            "index_algorithm": "tfidf-sparse-v1",
+            "index_algorithm": "llama-cpp-embedding-v1",
             "snapshot_generated_at": "2026-06-06T00:00:00+00:00",
             "index_generated_at": "2026-06-06T01:00:00+00:00",
             "snapshot_digest": "abc",

tests/test_lora_training_kit.py

@@ -3,6 +3,8 @@ import subprocess
 import sys
 from io import BytesIO
 from pathlib import Path
+
+from tests.helpers import load_test_index
 from zipfile import ZipFile
 
 from hackathon_advisor.agent import AdvisorEngine
@@ -18,7 +20,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_lora_training_kit_contains_recipe_and_model_card() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     metadata = {
         **trace_metadata(index),
@@ -67,7 +69,7 @@ def test_parse_lora_dataset_jsonl_rejects_empty_payload() -> None:
 
 
 def test_train_minicpm_lora_dry_run_writes_recipe(tmp_path: Path) -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     metadata = {
         **trace_metadata(index),

tests/test_prize_ledger.py

@@ -2,7 +2,10 @@ from hackathon_advisor.prize_ledger import prize_ledger
 
 
 def test_prize_ledger_tracks_param_budget_and_badges() -> None:
-    payload = prize_ledger({"backend": "rules", "model_id": "deterministic-tool-router"})
+    payload = prize_ledger(
+        {"backend": "rules", "model_id": "deterministic-tool-router"},
+        {"index_algorithm": "llama-cpp-embedding-v1"},
+    )
 
     assert payload["runtime"]["backend"] == "rules"
     assert payload["total_params_b"] <= payload["tiny_titan_limit_b"]
@@ -11,6 +14,8 @@ def test_prize_ledger_tracks_param_budget_and_badges() -> None:
     badges = {badge["name"]: badge["status"] for badge in payload["badges"]}
     assert badges["Off the Grid"] == "ready"
     assert badges["Well-Tuned"] == "ready"
+    assert badges["Llama Champion"] == "ready"
+    assert payload["retrieval_index"]["index_algorithm"] == "llama-cpp-embedding-v1"
     assert payload["training_artifacts"][0]["base_model"] == "openbmb/MiniCPM5-1B"
     assert payload["training_artifacts"][1]["format"] == "zip"
     assert payload["training_artifacts"][1]["adapter_repo"] == "build-small-hackathon/hackathon-advisor-minicpm5-lora"

tests/test_submission_packet.py

@@ -1,5 +1,7 @@
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.prize_ledger import prize_ledger
@@ -8,7 +10,7 @@ from hackathon_advisor.trace_export import trace_metadata
 
 
 def test_submission_packet_contains_demo_and_prize_evidence() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = {"goals": ["Well-Tuned", "Field Notes"]}
     state = engine.turn("A local-first archive cartographer for family photos", state).state
@@ -44,7 +46,7 @@ def test_empty_submission_packet_is_honest_about_missing_session_artifacts() ->
         {
             "snapshot_generated_at": "2026-06-06T00:00:00+00:00",
             "project_count": 100,
-            "index_algorithm": "tfidf-sparse-v1",
+            "index_algorithm": "llama-cpp-embedding-v1",
             "index_generated_at": "2026-06-06T01:00:00+00:00",
             "snapshot_digest": "abc",
         },

tests/test_trace_export.py

@@ -1,13 +1,15 @@
 import json
 from pathlib import Path
 
+from tests.helpers import load_test_index
+
 from hackathon_advisor.agent import AdvisorEngine
 from hackathon_advisor.data import ProjectIndex
 from hackathon_advisor.trace_export import build_trace_jsonl, trace_metadata
 
 
 def test_trace_jsonl_contains_manifest_and_turns() -> None:
-    index = ProjectIndex.from_files(Path("data/projects.json"), Path("data/project_index.json"))
+    index = load_test_index()
     engine = AdvisorEngine(index)
     state = engine.turn("A local-first archive cartographer for family photos", {}).state
     state = engine.turn("make a build plan", state).state
@@ -16,7 +18,7 @@ def test_trace_jsonl_contains_manifest_and_turns() -> None:
 
     assert lines[0]["type"] == "trace_manifest"
     assert lines[0]["turn_count"] == 2
-    assert lines[0]["index"]["algorithm"] == "tfidf-sparse-v1"
+    assert lines[0]["index"]["algorithm"] == "llama-cpp-embedding-v1"
     assert lines[1]["type"] == "agent_turn"
     assert lines[1]["tools"]
     assert lines[1]["tool_resolution"]["call"]["name"] == "save_idea"