basecode/experiments/reproduction_matrix.md

Reproduction Matrix

This matrix is the execution checklist for source-code reproduction first, then basecode reproduction with matched metrics.

Paper / route	Source code path	Required assets	Source metrics to reproduce	Basecode hook	Status
LLaDA/Dream-style block diffusion	reference/code/Nemotron-Labs-Diffusion, reference/code/mbd-lms	llada2-mini, dream-v0-base-7b, GSM8K/MATH/HumanEval/MBPP	accuracy, NFE, latency, tokens per forward	ConfidenceRemaskSampler, MultiBlockSampler	planned
MBD-LMs	reference/code/mbd-lms	llada2-mini, math/code datasets	multi-block speed/quality tradeoff	MultiBlockSampler	planned
DMax	reference/code/DMax	dmax-16b, GSM8K/MATH/code evals	aggressive parallel decoding accuracy and speed	DMaxSampler	planned
TAD / trajectory self-distillation	reference/code/TAD, reference/code/T3D	trajectory datasets, distilled checkpoints	few-step quality retention	DMaxSampler with distilled adapter	planned
Fast-dLLM	reference/code/Fast-dLLM	discrete dLLM checkpoint	KV-cache speedup, parallel decode latency	adapter cache API	planned
DFlash / PRESTO	reference/code/dflash, Nemotron references	draft model + AR target	acceptance length, speedup, target quality	SpeculativeSampler	planned
dMoE	reference/code/dMoE	dmoe-16b	expert/block routing quality and speed	expert adapter extension	planned
ELF / continuous flows	reference/code/ELF	ELF checkpoints, OpenWebText/LM1B	perplexity, MAUVE/diversity, generation speed	separate continuous adapter	planned
FLM/FMLM one-step	reference/code/flm	manual checkpoints, OpenWebText/LM1B	unconditional generation PPL/time	separate continuous adapter	planned

Run order:

1. Confirm remote assets with python3 scripts/download_assets.py --dry-run.
2. Download core tier only.
3. Run each source repository's smallest documented eval.
4. Normalize metrics into runs/source_reproductions/<paper_id>/metrics.json.
5. Implement/attach matching adapter in basecode/blockdiff/adapters/.
6. Run basecode evals into runs/basecode/<method_id>/metrics.json.
7. Compare source vs basecode metrics in one table.