NYSgpt
/

nsr-encoder

+---
+license: mit
+base_model: BAAI/bge-m3
+library_name: sentence-transformers
+pipeline_tag: feature-extraction
+tags: [nuclear-physics, information-retrieval, NSR, EXFOR, NSR-CPT, bge-m3, dense-retrieval]
+language: [en]
+datasets: [NYSgpt/nsr-retrieval-benchmark]
+---
+# nsr-encoder-bge-m3
+The **NSR-CPT retriever**: `BAAI/bge-m3` fine-tuned on expert-written query→paper
+pairs from the Nuclear Science References (NSR) corpus. Dense, 1024-d, cosine.
+Drop-in replacement for bge-m3 in any nuclear-physics literature retrieval stack.
+**Story in one line:** MedCPT trained on PubMed click logs; NSR-CPT trains on
+something rarer — NSR indexers *hand-wrote* a structured keyword abstract for
+~200k papers, so every training query is a domain expert describing a specific
+paper in canonical terms, plus EXFOR experiment→paper links.
+## Training data
+- Source: `NYSgpt/nsr-training-triplets` (private) — queries are NSR keyword
+  abstracts (KW) and EXFOR entry text (EX); positives are the paper's title +
+  publisher abstract; **hard negatives mined from our own bge-m3 embedding space**
+  (kNN neighbours that are not the annotated paper; MedCPT rule: negatives from
+  the paired retriever's space) plus a lexical (FTS) arm; 8 negatives per query.
+- Split by paper (`sha256` hash), train/val/test 0.8/0.1/0.1; the benchmark's
+  held-out papers are excluded as query sources and positives.
+- This run: **39,568 pairs** (all EX + KW stratified across era / richness /
+  reference-type cells), 1 epoch — the 60-minute training budget from a measured
+  12.6 (rehearsal; 19.7 sustained in the run) pairs/s on one L40S.
+- Disclosure: in this run's negatives, held-out papers could appear as *negatives*
+  for other queries (standard DPR/MedCPT-style mining from the full corpus; the
+  query→positive side is strictly disjoint). A clean re-mined dataset that also
+  excludes held-out papers from negatives is published alongside.
+## Training
+FlagEmbedding `finetune.embedder.encoder_only.m3`, dense-only (no sparse /
+colbert / self-distill), `train_group_size 8`, `query_max_len 128`,
+`passage_max_len 256`, bf16, batch 16, lr 1e-5, temperature 0.02,
+normalized embeddings. One g6e.xlarge (L40S 48 GB), 33.5 min, ≈ $1.9.
+## Evaluation — `nsr-eval-v1` (frozen, held out by paper, full 277,068-doc universe)
+**KW family — R@1 / R@10 / nDCG@10** (A = FTS, B = stock bge-m3, RRF = A+B, **D = nsr-encoder-bge-m3**, RRF-D = A+D)
+| segment | n | A | B | RRF | **D** | RRF-D | D vs B R@10 |
+|---|---:|---|---|---|---|---|---:|
+| blended | 4,998 | 0.161 / 0.165 / 0.163 | 0.080 / 0.171 / 0.121 | 0.212 / 0.282 / 0.244 | **0.252 / 0.487 / 0.363** | 0.344 / 0.542 / 0.437 | +186% |
+| richness:has-abstract | 857 | 0.160 / 0.162 / 0.161 | 0.222 / 0.383 / 0.296 | 0.317 / 0.455 / 0.380 | **0.422 / 0.631 / 0.527** | 0.495 / 0.678 / 0.585 | +65% |
+| richness:title-only | 4,141 | 0.161 / 0.166 / 0.163 | 0.051 / 0.127 / 0.085 | 0.190 / 0.246 / 0.216 | **0.216 / 0.458 / 0.329** | 0.312 / 0.513 / 0.407 | +262% |
+| era:pre1970 | 602 | 0.105 / 0.131 / 0.118 | 0.022 / 0.076 / 0.045 | 0.115 / 0.183 / 0.144 | **0.133 / 0.311 / 0.211** | 0.201 / 0.380 / 0.282 | +307% |
+| era:1970-1999 | 2,686 | 0.181 / 0.183 / 0.182 | 0.067 / 0.152 / 0.105 | 0.220 / 0.281 / 0.249 | **0.259 / 0.507 / 0.376** | 0.363 / 0.565 / 0.458 | +233% |
+| era:2000+ | 1,710 | 0.147 / 0.149 / 0.148 | 0.121 / 0.232 / 0.174 | 0.232 / 0.318 / 0.273 | **0.282 / 0.519 / 0.397** | 0.364 / 0.562 / 0.460 | +123% |
+| reftype:journal | 4,250 | 0.159 / 0.164 / 0.161 | 0.087 / 0.179 / 0.129 | 0.216 / 0.288 / 0.249 | **0.263 / 0.495 / 0.372** | 0.352 / 0.548 / 0.445 | +176% |
+| reftype:other | 748 | 0.168 / 0.174 / 0.171 | 0.039 / 0.120 / 0.075 | 0.189 / 0.246 / 0.217 | **0.187 / 0.447 / 0.310** | 0.297 / 0.503 / 0.396 | +271% |
+**EX family — R@1 / R@10 / nDCG@10** (A = FTS, B = stock bge-m3, RRF = A+B, **D = nsr-encoder-bge-m3**, RRF-D = A+D)
+| segment | n | A | B | RRF | **D** | RRF-D | D vs B R@10 |
+|---|---:|---|---|---|---|---|---:|
+| blended | 4,997 | 0.017 / 0.018 / 0.017 | 0.871 / 0.958 / 0.878 | 0.870 / 0.959 / 0.878 | **0.911 / 0.966 / 0.906** | 0.911 / 0.967 / 0.906 | +1% |
+| richness:has-abstract | 448 | 0.011 / 0.011 / 0.010 | 0.578 / 0.862 / 0.689 | 0.583 / 0.864 / 0.693 | **0.752 / 0.940 / 0.822** | 0.761 / 0.946 / 0.830 | +9% |
+| richness:title-only | 4,549 | 0.017 / 0.018 / 0.017 | 0.899 / 0.968 / 0.896 | 0.899 / 0.968 / 0.896 | **0.927 / 0.969 / 0.914** | 0.926 / 0.969 / 0.914 | +0% |
+| era:pre1970 | 1,112 | 0.014 / 0.015 / 0.015 | 0.862 / 0.963 / 0.894 | 0.862 / 0.963 / 0.894 | **0.910 / 0.970 / 0.923** | 0.909 / 0.970 / 0.923 | +1% |
+| era:1970-1999 | 2,368 | 0.024 / 0.025 / 0.024 | 0.891 / 0.962 / 0.885 | 0.891 / 0.962 / 0.885 | **0.919 / 0.963 / 0.903** | 0.918 / 0.963 / 0.903 | +0% |
+| era:2000+ | 1,517 | 0.007 / 0.009 / 0.008 | 0.844 / 0.949 / 0.855 | 0.844 / 0.950 / 0.855 | **0.900 / 0.969 / 0.899** | 0.901 / 0.970 / 0.900 | +2% |
+| reftype:journal | 4,665 | 0.016 / 0.017 / 0.016 | 0.875 / 0.962 / 0.887 | 0.875 / 0.962 / 0.886 | **0.916 / 0.970 / 0.914** | 0.915 / 0.971 / 0.914 | +1% |
+| reftype:other | 332 | 0.024 / 0.027 / 0.022 | 0.801 / 0.904 / 0.753 | 0.807 / 0.907 / 0.758 | **0.840 / 0.913 / 0.792** | 0.849 / 0.916 / 0.798 | +1% |
+**Gate 3 (pre-registered): D ≥ B on Recall@10 and nDCG@10 in every segment, ≥ 10 % relative on blended KW → PASS** (blended KW: R@10 +186 %, nDCG@10 +200 %; title-only R@10 +262 %). Fusion of FTS with the new encoder (RRF-D) is the strongest serving arm: KW R@1 .344 vs .212 with stock bge-m3.
+Baselines are in the benchmark card. Read per segment; the KW *title-only* segment
+is where stock bge-m3 was weakest (R@1 .05) and is the target of this training.
+## Usage
+```python
+from sentence_transformers import SentenceTransformer
+m = SentenceTransformer("NYSgpt/nsr-encoder-bge-m3")
+q = m.encode(["92Zr(n,γ) cross section, stellar nucleosynthesis"], normalize_embeddings=True)
+```
+Serve with Hugging Face TEI: `--model-id NYSgpt/nsr-encoder-bge-m3`.
+## Provenance
+Corpus: NNDC NSR dump 2026-06-30 (+ arXiv/APS/OpenAlex/S2 enrichment), 277,068
+embeddable documents. Built by NYSgpt, 2026-08-17. Part of **NSR-CPT** with
+`nsr-reranker-gte-modernbert` and `nsr-retrieval-benchmark`.