Memory-LoRA: A Hypernetwork that Writes Repo-Specific Adapters for Gemma-4-E2B
One line: we train a small neural network (a hypernetwork) that reads an embedding of a codebase and emits a LoRA adapter for a frozen
google/gemma-4-E2B. The adapter injects repo-specific knowledge into the model with zero extra tokens at inference time — no RAG, no context stuffing. Everything runs locally on an Apple-Silicon Mac (M4 Pro, 64 GB).
This document is the onboarding bible for the project. It covers what we built, why every non-obvious choice was made, what data we used, every experiment we ran and its result, and the hard-won lessons (the MPS memory leak alone cost us hours). Read it top to bottom once; after that use it as a reference.
Table of Contents
- The idea in 60 seconds
- Origin: the Code2LoRA paper
- Architecture
- The target model: Gemma-4-E2B specifics
- Data pipeline: the 6 views
- Data pipeline: QA generation
- What is learnable — Tier A / B / C
- Datasets inventory
- Experiments & results
- Key decisions & lessons learned
- Repository map
- How to run it
- Evaluation methodology
- Costs & budget discipline
- Current status & roadmap
- Glossary
1. The idea in 60 seconds
A coding agent (Claude Code, Codex, etc.) is great at reasoning but knows nothing about your repo until you paste files into its context — which is slow, expensive, and capped by the context window. The usual fix is RAG (retrieve chunks at query time). We do something different and complementary:
We bake the repo's "personality" directly into the model's weights, once, as a LoRA adapter — and we generate that adapter with a neural network instead of training it.
┌─────────────────────────────┐
repo on disk ──────► │ 6-view extractor + Qwen3 │ ──► 12288-d
│ frozen embedding encoder │ repo embedding
└─────────────────────────────┘ │
▼
┌──────────────────────────┐
│ HYPERNETWORK (our head) │
│ MLP → per-module A,B │
└──────────────────────────┘
│ LoRA weights
▼
Q: "what layer owns auth in this repo?" ──► ┌──────────────────────────────┐
│ FROZEN Gemma-4-E2B + injected │ ──► "the middleware
│ LoRA (zero extra tokens) │ layer, via ..."
└──────────────────────────────┘
The magic: the hypernetwork is trained across hundreds of repos, so it learns
the mapping repo embedding → good adapter. At inference on a brand-new repo it
has never seen, it embeds the repo once and produces an adapter in a single forward
pass. This is the same reason the source paper needed 400+ repos, not 1: breadth is
what makes the mapping generalize.
2. Origin: the Code2LoRA paper
We reverse-engineered Code2LoRA (arXiv 2606.06492v1) and found its released
code (anonymous.4open.science/r/code2lora-6857, MIT). The paper's contribution:
a static hypernetwork that maps a repository embedding → a LoRA adapter for a
frozen code LLM, evaluated on RepoPeftBench with IR (in-repo) / CR (cross-repo)
splits. On a full H100 setup they report 63.8 % cross-repo Exact Match.
Our project is the Doc2LoRA variant the paper itself cites — hypernetwork maps a document/repo view → LoRA — reimplemented against Gemma-4-E2B, trained fully locally on MPS (no CUDA/H100), and extended in two directions the paper does not cover:
- Memory / recall: the adapter should let the model recall facts about the repo, not just complete code.
- Tech-Lead judgment: architecture, data-flow, conventions, contracts, ops — the things a 20-year senior engineer "just knows" about a codebase.
We keep the paper's proven autograd trick almost verbatim (see §3) and change only what Gemma-4 and Apple Silicon force us to change.
3. Architecture
Three frozen/learned pieces. Only the middle one (the head) is trained.
3.1 Frozen encoder — memory_lora/encoder.py
- Qwen3-Embedding-0.6B, frozen, no gradient flows through it.
- Each repo view is chunked into token windows (2048 tokens, 128 overlap), each chunk mean-pooled, then chunks combined with mean + max pooling → a 2048-d vector per view.
- Embeddings are precomputed once and cached to parquet — the encoder never runs during training.
3.2 The hypernetwork head — memory_lora/core.py :: MemoryLoRAHead
The only trained component. Design (kept close to the paper):
- 2-layer GELU MLP trunk (
input_dim → hidden_dim → hidden_dim), followed by L2-normalize + √hidden_dim rescale (stabilizes the magnitude of generated weights). - Per-module-type output heads: for each target module type it emits an
A ∈ [rank, in_features]andB ∈ [out_features, rank]. One (A,B) pair per type, shared across all layers of that type — this is what keeps the head tractable (188.6 M params) instead of exploding per-layer. - Squashing:
tanh(raw) * exp(log_scale)with a learned per-typelog_scale(init -3.5). This starts the generated adapter near-zero (so training begins close to the base model) and lets each type learn its own output scale. - Defaults:
hidden_dim=128,rank=16,dropout=0.1.- Why hidden_dim=128 and not the paper's 512/1024? A 745 M-param head (hidden_dim=512) barely moved eval loss (~1.9 → ~2.7) but was far heavier to train on MPS. 128 cuts head size dramatically with negligible quality loss locally. Bump it later on real GPUs.
3.3 The LoRA injection — memory_lora/core.py :: LoRA
base nn.Linear (FROZEN) hypernetwork output
│ │
x ──►│ Wx ────────────────┐ │
│ (input detached +──► y = Wx + scaling · B (A x)
│ into base) │ ▲ ▲
x ──────────────────────────┘ │ │
A,B are NON-detached tensors so
autograd flows LM-loss → head
Critical detail (straight from the paper's code): A and B are plain, non-buffer
tensor attributes, not nn.Parameter and not detached, so the gradient of the LM
loss flows through the injected weights into the hypernetwork. The base
nn.Linear is frozen and its input is detached. Get this wrong and the head never
learns.
3.4 Shape-qualified module types — the heterogeneity fix
Gemma-4-E2B is not a uniform stack (see §4). Two q_projs can have different
shapes. If you key the head by bare type name (q_proj) you get
"type q_proj inconsistent dims" crashes. Fix: key by shape-qualified type,
e.g. q_proj_1536x2048 vs q_proj_1536x4096. The v2 run discovered 14 shape
types across 205 target modules:
down_proj_12288x1536 down_proj_6144x1536 gate_proj_1536x12288 gate_proj_1536x6144
k_proj_1536x256 k_proj_1536x512 o_proj_2048x1536 o_proj_4096x1536
q_proj_1536x2048 q_proj_1536x4096 up_proj_1536x12288 up_proj_1536x6144
v_proj_1536x256 v_proj_1536x512
get_module_specs(root_prefix="model.language_model.") restricts wrapping to the
text decoder — the vision and audio towers are never touched (not even
inspected), so the multimodal forward path stays intact and they cost only idle RAM.
4. The target model: Gemma-4-E2B specifics
Verified by reading the actual safetensors header, not guessed:
- Real model. Google shipped Gemma 4 in March 2026. Apache-2.0, ungated.
Class
Gemma4ForConditionalGeneration,model_type: "gemma4". Loaded viaAutoModelForImageTextToText. - Requires
transformers >= 5.5.0.dev0— install from themainbranch, not a pinned PyPI release (this is the single biggest environment risk; smoke-test first). - Decoder is nested: layers live at
model.language_model.layers.{i}.*, notmodel.layers.*. The layer-index regex had to change accordingly. - 35 text layers, heterogeneous:
- Aggressive KV sharing — 20 of 35 layers lack their own
k_proj/v_proj(num_kv_shared_layers=20). - Every 5th layer is wider (the
*_4096,*_12288shape variants above).
- Aggressive KV sharing — 20 of 35 layers lack their own
- Device
mps, precision bf16 (fall back to fp16 if unstable). Noflash_attention_2on MPS — usesdpa, fall back toeager.
5. Data pipeline: the 6 views
scripts/build_repo_multiview.py clones a repo and extracts 6 complementary
views, embeds each with Qwen3 → 2048-d, and concatenates to a 12288-d repo
vector. The views encode the different "lenses" a senior engineer uses:
| View | What it captures | Source signals |
|---|---|---|
v_graph |
call / import / dependency structure | AST for Python (memory_lora/codegraph.py); IMPORT_RE/DEF_RE regex fallback for other languages |
v_arch |
architecture & layout | README, folder tree |
v_history |
how the code evolved | git log, recent diffs |
v_contracts |
behavioral contracts | test files |
v_conventions |
idioms & style | representative source files |
v_ops |
build / deploy / runtime | CI config, Dockerfile, build files |
Multi-language from the start. CODE_EXTS + regex fallbacks mean the graph view
works for 9 languages, not just Python (this was a deliberate correction — see §10).
Repos with < 3 code files are skipped. The build is resume-safe (skips repos
already in multiview_sources.jsonl) and flushes the embeddings parquet every 10
repos, so a crash never loses more than 10 repos of work.
6. Data pipeline: QA generation
The repo embedding is the input; the target is repo-scoped Q&A. Two generators,
both parallelized (ThreadPoolExecutor, --workers 10) with a per-prompt disk
cache (idempotent reruns) and a --model flag:
scripts/generate_repo_scoped_qa.py— reads the same 6 views and asks the LLM for 8–12 repo-level judgment questions ("what layer owns X", "what convention does this repo use for Y", "how does data flow through Z", "why is it structured this way"). Target scope = input scope (repo-level embedding ↔ repo-level QA).scripts/generate_commitpack_qa.py— breadth generator: one commit per distinct repo across CommitPackFT (25k+ distinct repos), 3–4 commit-scoped judgment questions (why / conventions / contracts / impact). For a hypernetwork, distinct-repo count is the currency of generalization, so we favor 1 commit × many repos over many commits × one repo.
Discipline (both): answers are short judgment, never file-path/line-number lists. This is deliberate — see Tier A/B/C next.
Models used (OpenRouter, OpenAI-compatible API):
| Model | Role | Notes | Cost |
|---|---|---|---|
google/gemini-3.6-flash |
high-quality QA | reasoning is mandatory → needs generous max_tokens (3000–4000) or it returns empty |
~$0.0021 / QA |
google/gemma-4-31b-it |
bulk / cheap QA | non-reasoning, clean JSON |
The OpenRouter key lives in a git-ignored .env (mode 600) and is never pasted
into a command line.
7. What is learnable — Tier A / B / C
A LoRA adapter has finite capacity. We classify repo knowledge by whether a LoRA can hold it — this drives the entire QA design:
- Tier A — Judgment & conventions (LEARNABLE). "This repo puts business logic in services, not views." Compressible, generalizes. → This is what we train on.
- Tier B — Structural gist (LEARNABLE). "Auth flows through middleware." The kind of thing, not the exact file.
- Tier C — Exact recall & multi-hop (NOT reliably learnable). "Line 412 of
foo.pycallsbar()." This needs retrieval (RAG), not weights.
So Memory-LoRA and RAG are complementary: the adapter carries Tier A/B judgment for free (zero tokens); RAG handles Tier C exact lookups. The QA prompts forbid exact file/line answers precisely so we never ask the LoRA to do a job it structurally can't.
8. Datasets inventory
Everything lives under data/ (git-ignored blobs). Sizes are approximate.
| Path | What | Scale |
|---|---|---|
data/real_code2lora/ |
RepoPeftBench from the code2lora HF org — 500 Python repos, repo-commit embeddings + diffs |
73,849 repo-commit rows; ~1.2 GB |
data/commitpack/multilang_commits.jsonl |
CommitPackFT shards, 9 languages | 25k+ distinct repos |
data/docs/multiview_sources.jsonl |
6-view view_text per repo (input to QA gen) |
growing (1000s of repos) |
data/embeddings/multiview_embeddings.parquet |
12288-d multi-view repo embeddings | 1000+ repos |
data/embeddings/aligned6_embeddings.parquet |
assembled training inputs (repos with ≥1 QA) | 1058 repos (current) |
data/qna/repo_scoped_qa.jsonl |
repo-level judgment QA | 11,232 QA |
data/qna/techlead_qa_commitpack.jsonl |
commit-scoped breadth QA | 9,245 QA |
data/qna/techlead_qa.jsonl |
SWE-bench tech-lead QA | 2,786 QA |
data/qna/aligned6_qna.jsonl |
assembled training targets | 8,540 QA (current) |
data/openrouter_cache/ |
per-prompt response cache | ~19 MB |
Language balancing. SWE-bench is 79 % Django. Left alone, the dataset was 46 %
Django. 2.0 %** while preserving the 2400+ distinct repos'
diversity. scripts/consolidate_qa.py applies a per-repo cap (default 12–15 QA/repo)
which collapses Django to **assemble_6view_dataset.py applies the same cap when building the final
aligned set.
9. Experiments & results
Chronological, with the actual numbers we measured. Two families of runs.
9.1 Reproducing the paper (single-view, real RepoPeftBench)
| Run | What | Result |
|---|---|---|
full1 (early) |
first end-to-end hypernetwork on converted real data | CR EM 0.056–0.083, EditSim ~0.27 — pipeline works, undertrained |
sixview/converted-real (best single-view ckpt) |
after more training | CR EM 0.524, EditSim 0.635 |
| Paper (reference, H100) | their full run | CR EM 0.638 |
Headline: on real code, after only ~2.4 % of one epoch of local MPS training, we reached 52.4 % cross-repo Exact Match vs the paper's 63.8 % on a full H100 setup. The mechanism demonstrably works — the generated adapter recovers repo-specific identifiers the base model does not know.
9.2 The 6-view Tech-Lead model (the current line of work)
Loss is causal-LM cross-entropy on QA targets; lower is better. Three eval suites:
cr_val / cr_test (held-out repos) and ir_test (held-out QA of train repos).
| Run | Dataset | Best held-out cr_test loss |
Notes |
|---|---|---|---|
sixview_v1 |
515 repos / 3,988 QA (415 train repos) | 2.848 (step ~1060) | Overfit afterward: train loss fell to 1.75 while cr_test drifted to 3.35. Classic small-dataset ceiling. |
sixview_v2 |
1,058 repos / 8,540 QA (858 train repos) | in progress | Resumed from sixview_v1/head.best.pt; 2× the data specifically to break v1's ceiling. |
sixview_v1 metrics trajectory (from runs/sixview_v1/metrics.jsonl):
step 1245 cr_test 2.962 ir_test 2.535 (end of epoch 2 — near best)
step 1400 cr_test 3.245 ir_test 2.593 (overfitting begins)
step 1600 cr_test 3.352 ir_test 2.655 (train loss still falling → ceiling hit)
The v1→v2 story is the core empirical lesson: the small aligned set was the bottleneck, not the architecture — hence the push to build 1000+ more repos.
10. Key decisions & lessons learned
The expensive knowledge. Read this section twice.
10.1 ⚠️ The MPS gradient-checkpointing memory leak (the big one)
Symptom: training with gradient_checkpointing_enable() (use_reentrant=False)
leaked ~12 GB per step and OOM'd the whole machine within a few steps.
Diagnosis (scripts/diag_mps_leak.py): forward-only was stable; train + checkpoint
leaked 39 GB → 18 GB free in 2 steps. Isolated the checkpointing path as the cause.
Fix: --no-gradient-checkpointing. We have enough unified memory to hold
activations without it once the multimodal towers sit idle. This is documented as a
standing memory (mps-gradient-checkpointing-leak.md).
10.2 ⚠️ psutil RSS is blind to MPS memory
Our first memory safety-net used psutil RSS / ps -o rss — it reported < 1 GB
while top showed 55–83 GB actually in use. MPS (GPU) allocations don't show up
in process RSS.
Fix: the safety check uses psutil.virtual_memory().available (system-wide)
with a --min-available-gb floor (default 5). To observe MPS memory, use
top -l 1 -pid <PID> -stats mem, not ps.
10.3 Memory competition between concurrent jobs
Three concurrent jobs once pushed available memory under the 10 GB floor and training
self-stopped. Lesson: during MPS training, run data builds/embedding on CPU
(--device cpu) so they don't contend for the GPU/unified memory. We now routinely
run training (MPS) + QA gen (network) + multiview build (CPU) together without
contention.
10.4 Don't lose hours of training
Every long run writes checkpoints every 50 steps (overwriting head.latest.pt),
every 30 minutes (timestamped head.tNNNNm.pt), per-epoch (head.epN.pt),
and a head.best.pt on eval improvement. Runs are launched with nohup … & disown so they survive terminal/session death. sixview_v1 in fact survived a full
session interruption and kept training. Resume with --resume-from <ckpt> (loads head
weights; optimizer restarts fresh).
10.5 Data-quality corrections (user-driven)
- "I still see lots of Django." SWE-bench is Django-dominated. → per-repo cap + multi-language sourcing dropped Django 46 % → 2.0 %.
- "It must be good for any programming language." → 9-language diversity via CommitPackFT and language-agnostic view extraction.
- "Where's the code context in the QA?" → clarified the two-channel design: the
repo embedding is the context channel, the QA is only the target. They are
joined by
doc_idat assembly time.
10.6 OpenRouter gotchas
gemini-3.6-flashreturned empty until we raisedmax_tokens— it's a mandatory-reasoning model that spends tokens on hidden reasoning before content. Reasoning cannot be disabled (400 error).- CommitPackFT's HF loader is deprecated → fetch raw
data.jsonldirectly. global MODELafter use is aSyntaxError→ set viaglobals()["MODEL"] = ....
10.7 Performance fix worth knowing
Loading embeddings was 5+ min because _list_to_f32_array used a Python loop.
Vectorized via col.combine_chunks().flatten().to_numpy() → ~220× faster.
11. Repository map
memory_lora/ # the library (importable package)
core.py # LoRA wrapper, MemoryLoRAHead hypernetwork,
# get_module_specs / replace_with_lora / inject_lora_weights,
# load_doc_rows / load_qna_rows
encoder.py # Qwen3 chunk + embed + mean/max pool (frozen)
codegraph.py # Python AST extractor (imports, sigs, call graph)
data_paths.py # local parquet/jsonl path resolver
scripts/
build_repo_multiview.py # clone → 6 views → 12288-d embeddings (multi-language, resume-safe)
generate_repo_scoped_qa.py # repo-level judgment QA (aligned to the 6 views)
generate_commitpack_qa.py # commit-scoped breadth QA across 1000s of distinct repos
generate_techlead_qa.py # SWE-bench tech-lead QA
generate_synthetic_dataset.py # original synthetic doc + QA generator
consolidate_qa.py # per-repo cap → language/domain balancing
assemble_6view_dataset.py # join embeddings ↔ all QA by repo → aligned6_{embeddings,qna}
augment_paraphrases.py # QA paraphrase augmentation
convert_real_code2lora.py # RepoPeftBench → our schema
build_doc_embeddings.py # encoder pass over documents
merge_corpora.py # combine multiple corpora
train_memory_lora.py # THE trainer (MPS, one-repo-per-step, checkpoints, TensorBoard)
train_direct_lora.py # baseline: plain per-repo LoRA (no hypernetwork)
eval_memory_lora.py # EM / EditSim recall eval on cr/ir splits
show_eval_examples.py # dump concrete base-vs-adapted examples
test_embed_this_repo.py # embed the current repo (pipeline demo)
test_recall_this_repo.py # query the adapted model about this repo
diag_mps_leak.py # the memory-leak isolation harness
data/ # git-ignored: embeddings, qna, sources, caches
runs/ # git-ignored: checkpoints, logs, metrics.jsonl, tb/
requirements.txt # torch 2.13 (MPS), transformers@main, openai, pyarrow, tensorboard…
.env # git-ignored, mode 600: OPENROUTER_API_KEY
12. How to run it
Setup
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt # installs transformers from git main
echo "OPENROUTER_API_KEY=sk-or-..." > .env && chmod 600 .env
Smoke-test the environment first (gates everything): confirm transformers main
loads google/gemma-4-E2B on mps and runs a text-only forward pass.
Build data
# 1) multi-view embeddings for a repo list (CPU to stay off the GPU during training)
python scripts/build_repo_multiview.py \
--repos-file data/multilang_repo_list.txt --max-repos 1000 --device cpu
# 2) repo-scoped QA (cheap model) — appends, resume-safe, cached
./venv/bin/python scripts/generate_repo_scoped_qa.py \
--model google/gemma-4-31b-it --workers 10
# 3) balance + assemble the aligned training set
python scripts/consolidate_qa.py --per-repo-cap 12
python scripts/assemble_6view_dataset.py
# -> data/embeddings/aligned6_embeddings.parquet + data/qna/aligned6_qna.jsonl
Train (the exact sixview_v2 command)
nohup ./venv/bin/python scripts/train_memory_lora.py --output-dir sixview_v2 \
--resume-from runs/sixview_v1/head.best.pt \
--embeddings-path data/embeddings/aligned6_embeddings.parquet \
--qna-path data/qna/aligned6_qna.jsonl --epochs 100 --max-hours 8 \
--checkpoint-every-steps 50 --checkpoint-every-minutes 30 --epoch-ckpt-every 5 \
--eval-every-steps 300 --limit-eval-docs 40 --max-seq-len 512 --fixed-seq-len \
--max-qna-per-doc 12 --lm-micro-batch 2 --device mps --no-gradient-checkpointing \
--rank 16 --head-hidden-dim 128 --head-dropout 0.1 --weight-decay 0.05 \
--early-stop-patience 25 --lr 8e-5 --lr-total-steps 9000 --min-available-gb 5 \
> runs/sixview_v2_train.log 2>&1 &
disown
Flags you must not forget: --no-gradient-checkpointing (the leak),
--device cpu for builds during training (contention), --min-available-gb
(system-wide memory floor).
Watch it
tensorboard --logdir runs/sixview_v2/tb # train/loss, train/lr, eval/{suite}_loss
tail -f runs/sixview_v2_train.log
Evaluate & inspect
python scripts/eval_memory_lora.py --ckpt runs/sixview_v2/head.best.pt # EM / EditSim
python scripts/show_eval_examples.py # base vs adapted
13. Evaluation methodology
- Splits (deterministic, by
md5(repo) % 100): 80 % train / 10 %cr_val/ 10 %cr_testby repo, so cross-repo suites are entirely held-out repositories the hypernetwork never trained on. Within train repos, ~15 % of QA is held out →ir_test(in-repo generalization to unseen questions of seen repos). - Metrics: causal-LM eval loss during training (fast, every N steps on
--limit-eval-docsdocs to stay cheap on CPU), plus generation-time Exact Match (EM) and EditSim for the recall eval. - The proof spot-check: query the adapted model with repo-specific questions and confirm the base (un-adapted) model gets them wrong/vague — proving the adapter, not the base model's pretraining, does the work.
CPU eval of a float32 5B model is slow (~20 min for a full pass) → we cap eval docs (e.g. 10–40) for in-loop evals and run full EM eval separately.
14. Costs & budget discipline
- Spend baseline: $31.00 (
runs/spend_baseline.txt); ~$33.90 total to date; ~$11 remaining. The project is run under explicit budget caps ("spend at most $4 more") with spend-guards. - Unit economics:
gemini-3.6-flash≈ $0.0021/QA;gemma-4-31b-it≈ $0.001/repo (~$1 per 1000 repos) — which is exactly why the 1000-repo expansion uses the gemma model. - Free levers: the per-prompt cache makes reruns free; embedding and training are local (electricity only).
15. Current status & roadmap
Live right now (three jobs in parallel, no contention):
sixview_v2training — resumed fromhead.best.pton the doubled 1,058-repo / 8,540-QA dataset (858 steps/epoch), MPS. First held-out eval at step 300 tells us whether doubling the data broke v1's 2.848 ceiling.- QA generation —
gemma-4-31b-itfilling in all ~1,032 new repos for the complete dataset (next training run). - Multiview build — cloning/embedding toward the full +1,000-new target (CPU).
Roadmap:
- Finish the complete 1000-new-repo dataset (embeddings + QA).
- Assemble the full aligned set (~1,650 repos) and train
sixview_v3on it. - Run generation-time EM/EditSim on the 6-view model (base vs adapted).
- Push
head_hidden_dimback up once on real GPUs; the 128 default was an MPS-locality compromise. - Broaden Tier-A/B QA toward agent-harness use cases (Jira/ticket tracking,
diff/impact reasoning) already scaffolded in
generate_techlead_qa.py.
Open questions:
- Does the 12288-d 6-view embedding actually beat the single 2048-d view on generation EM, or only on loss? (loss says yes; EM eval pending)
- What's the real Tier-B ceiling — how much structural gist fits in rank-16?
- Optimal per-repo QA cap for the breadth/depth trade-off.
16. Glossary
- Hypernetwork — a network that outputs the weights of another network. Here: repo embedding → LoRA (A,B) matrices.
- LoRA — Low-Rank Adaptation:
y = Wx + scaling · B(Ax), withA,Blow-rank (rank 16). We generate A,B instead of training them per-repo. - 6 views — graph / arch / history / contracts / conventions / ops; each 2048-d, concatenated to 12288-d.
- CR / IR — cross-repo (held-out repos) / in-repo (held-out QA of seen repos).
- EM / EditSim — Exact Match / edit-distance similarity of generated vs gold.
- Tier A/B/C — judgment (learnable) / structural gist (learnable) / exact recall (needs RAG).
- MPS — Apple's Metal Performance Shaders GPU backend for PyTorch.
- RepoPeftBench — the paper's benchmark; 500 Python repos, repo-commit embeddings + diffs, IR/CR splits.
Maintained as living documentation. If you change a default, a path, or a flag, update the matching section here — onboarding depends on it.