Upload complete H216 AutoModel checkpoint

9a0a8aa verified 15 days ago

2.29 kB

license: mit
language:
  - en
library_name: transformers
pipeline_tag: feature-extraction
base_model: BAAI/bge-small-en-v1.5
tags:
  - information-retrieval
  - mteb
  - scientific-retrieval

BGE Small Structural Separator

This is the complete Transformers checkpoint for the H216 Shared Structural Separator Field Encoder. It is compatible with AutoModel and AutoTokenizer.

H216 starts from the immutable BAAI/bge-small-en-v1.5@5c38ec7c405ec4b44b94cc5a9bb96e735b38267a checkpoint and learns exactly one 384-value input-embedding row: token [unused2], vocabulary ID 3. All other model parameters are unchanged.

Document formatting inserts [unused2] before a nonempty title and before each punctuation-delimited sentence. The document is encoded once and its normalized CLS vector is stored. Queries use ordinary tokenizer formatting with no instruction. Retrieval is exact cosine over one query vector and one document vector; there is no fusion, routing, reranking, expansion, or protected candidate frontier.

from transformers import AutoModel, AutoTokenizer

model_id = "thu-nmrc/bge-small-structural-separator"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id)

The repository includes separator_row.pt, the deterministic training manifest, and export_manifest.json with SHA-256 hashes for the complete checkpoint. Training used 4,096 S2ORC citation-context/abstract pairs and no BEIR labels, validation selection, benchmark negative mining, or post-training tuning.

The bounded research claim is a learned document-side structural separator adaptation within the dense bi-encoder family. It is not claimed as a replacement for all dense retrievers. Full evaluation details, training code, overlap audits, and disclosed regressions are maintained at https://github.com/thu-nmrc/bge-small-structural-separator.

The frozen BEIR8 gate improves macro NDCG@10 by +0.012705 over the matched plain BGE control. A later preregistered six-task MTEB(eng, v2) Retrieval extension does not support broad transfer: macro NDCG@10 changes by -0.011038 and H216 wins two of six tasks. Across all ten official Retrieval tasks the macro delta is +0.002855, but claims remain bounded to the BEIR8 structure-native evidence.