---
license: apache-2.0
base_model:
  - Qwen/Qwen3.5-4B
tags:
  - lora
  - knowledge-graph
  - tool-use
  - graph-traversal
  - question-answering
  - scientific-reasoning
---

# Prospector

Prospector is a compact LLM policy post-trained to answer questions over a
scientific knowledge graph. The policy calls a small set of tools to traverse
the graph turn by turn, building its context from what each call returns,
and answers with a chain of edges that is checked against what its tools
retrieved. Prospector is Qwen3.5 at 0.8B, 2B and 4B parameters with LoRA
adapters, trained on one graph built from the semiconductor-manufacturing
literature. With frozen weights, the same adapters transfer to graphs from
three other fields.

This repository holds the weights, the training rows and the stored
evaluation runs of the paper, laid out as the code repository expects them.
The code that serves, evaluates and trains them is at
[github.com/lamm-mit/Prospector](https://github.com/lamm-mit/Prospector).
Its README covers the evaluation harness, serving a policy, training one on
your own graph, and the command behind every number in the paper.

## Get started

The code repository's quick start, on a Mac with Apple silicon
or a Linux machine with an NVIDIA GPU:

```bash
# install uv, unless you have it
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
git clone https://github.com/lamm-mit/Prospector && cd Prospector
# download this repository's adapters and data into the checkout, about 9 GB
uvx --from huggingface_hub hf download lamm-mit/Prospector --local-dir . \
    --include "models/*" --include "data/*"
# build the environment and evaluate Prospector-4B on six questions
./quickstart.sh
```

The first run also downloads the base model from the Hub and merges the
adapters into it. The semiconductor graph is then under `data/raw/`, and the
code repository's `data/raw/README.md` downloads the three other graphs. The
code repository's README, under "Reproducing the paper", gives for each
number in the paper the command that recomputes it and the stored run the
number is computed from.

## Results

The gate accepts a cited chain when every cited edge was returned by a tool
and the answer names a node of the chain. Gated accuracy is the fraction of
questions whose chain passes the gate and whose answer names a gold label.
The table gives means over the seeds of the runs stored here, ± one standard
deviation.

| policy, graph | gated accuracy |
|---|---|
| Prospector-4B, semiconductor, eight seeds (`flash20cap1024-s0` to `-s7`) | 0.839 ± 0.015 |
| Prospector-2B, semiconductor, eight seeds (`q2bd2eval-ck120-s0` to `-s7`) | 0.755 ± 0.016 |
| Prospector-0.8B, semiconductor, eight seeds (`q0.8bd2eval-ck120-s0` to `-s7`) | 0.694 ± 0.021 |
| Prospector-4B, bio-inspired materials, three seeds (`flash20stackbio-s0` to `-s2`) | 0.803 ± 0.017 |
| Prospector-4B, PFAS, three seeds (`stack1kpfas-ctx32k-s0` to `-s2`) | 0.766 ± 0.022 |
| Prospector-4B, materials properties, three seeds (`stack1kmatprop-s0` to `-s2`) | 0.809 ± 0.011 |

## The served policies

The paper's recipe trains each size in four stages: a supervised stage on a
frontier teacher's demonstrations, a GRPO round over whole episodes, a
second supervised stage on the demonstrations blended with the policy's own
kept episodes, and two decision rounds of GRPO that resume stored
mid-episode states. A verifier is then trained on the served policy's own
samples. Each size serves the second supervised stage's adapter with the
checkpoint that each decision round's evaluation grid picked applied on top,
in this load order, over `Qwen/Qwen3.5-<size>`. The policies run with the
`plain_chatml` chat template at a budget of 20 tool calls and 1,024
generated tokens a turn.

| size | policy (`--adapter`, in load order) | verifier |
|---|---|---|
| 4B | `models/traj_sft_4b_flash20g2/final`, `models/traj_grpo_4b_flash20d1/checkpoint-120`, `models/traj_grpo_4b_flash20d2/checkpoint-120` | `models/verifier_sft_4b_flash20v/final` |
| 2B | `models/traj_sft_2b_flash20g2/final`, `models/traj_grpo_2b_flash20d1/checkpoint-100`, `models/traj_grpo_2b_flash20d2/checkpoint-120` | `models/verifier_sft_2b_flash20v/final` |
| 0.8B | `models/traj_sft_0.8b_flash20g2/final`, `models/traj_grpo_0.8b_flash20d1/checkpoint-100`, `models/traj_grpo_0.8b_flash20d2/checkpoint-120` | `models/verifier_sft_0.8b_flash20v/final` |

The adapters are PEFT LoRA adapters in transformers' fused naming for Qwen3.5
(`in_proj_qkv`), which the code repository's pinned `transformers` and `peft`
load. Each `run.json` records the chat template its adapter was trained
under, and each GRPO round's `peak.txt` records the checkpoint its grid
picked.

## What is here

Downloaded into a checkout's root, every file lands where the code expects
it. `MANIFEST` lists every file by pattern with its reason.

```
data/raw/        the semiconductor graph, the paper's own; the code repository's
                 data/raw/README.md downloads the three other graphs from their sources
models/          the recipe's six training runs at each size, each adapter with the
                 run.json that records what it was trained from, and
                 recipe_backfill_v1.json, which records how those run.json files were
                 written and each later change made by hand to a stored file here
data/traj/       the training rows of each stage, each with a .provenance.json sidecar
                 naming what it was built from, where one exists
data/runs/       the stored evaluation runs: every run the paper reads or the code
                 repository's README names, the timing runs behind the cost figure,
                 and the latent-state probe's derived files under _probe/
data/analysis/   inference_speed.json, which prices the timing runs at the posted
                 rates it dates
MANIFEST         the list of what is here and why, pattern by pattern
```

The set is the smallest that serves the three policies, lets `prospector
run` re-evaluate a size over the released adapters without training
anything, and recomputes every figure in the paper from stored runs, with
the probe's hidden states (about 1.7 GB a run) regenerated by its `states`
stage from the stored transcripts.

**The training rows.** `data/traj/` holds the rows each stage trained from,
named as the code derives them. Stage 1's demonstrations
(`tsmc_sft_mix_flash20.jsonl`) are shared by the three sizes. The second
supervised stage's mix, the decision rounds' stored states and the
verifier's rows exist per size. The teacher runs the demonstrations came
from (`flash20-teach-4x-g0` to `-g7` and `-p2`) are under `data/runs/`. The
0.8B's and 2B's mixes and stored states were rebuilt by the code's builders
from the stored runs they were sampled from, and the same builders reproduce
the 4B's files byte for byte. With these adapters in place, `prospector run`
trains nothing. At 0.8B and 2B the command reuses the stored evaluations
here, which carry the names it derives, and evaluates what it does not find.
At 4B, whose stored runs carry older names, the command evaluates afresh
under its own names.

**The stored runs.** A run is a directory `data/runs/<name>--<benchmark>/`
in the shape the code repository's README describes. A closed-book or
retrieved-subgraph run, which has no episodes, holds one reply per question
and, where it was scored, the scorer's output under the same gate.

## License and citation

The adapters, the rows, the runs and the semiconductor graph are released
under the Apache License 2.0.

```bibtex
@article{yang2026prospector,
  title   = {Prospector: learning a compact graph traversal policy for scientific reasoning},
  author  = {Yang, Haiqian and Buehler, Markus J.},
  journal = {arXiv preprint},
  year    = {2026}
}
```