| license: mit | |
| library_name: sentence-transformers | |
| pipeline_tag: feature-extraction | |
| tags: | |
| - sentence-transformers | |
| - feature-extraction | |
| - sentence-similarity | |
| - scientific-documents | |
| - modernbert | |
| - citation-context | |
| base_model: answerdotai/ModernBERT-base | |
| language: | |
| - en | |
| # SciEmbed-CTX-8192 | |
| Long-context variant (max_seq_length=8192). Recommended for long scientific inputs; best within-recipe scores on the Body-Fact Retrieval probe and LongEmbed. | |
| A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the **SciEmbed** release (paper under double-blind review; author info omitted). | |
| ## Usage | |
| ```python | |
| from sentence_transformers import SentenceTransformer | |
| model = SentenceTransformer("anon-nlp/sciembed-ctx-8192") | |
| emb = model.encode(["citation-context supervision for scientific embeddings"], | |
| normalize_embeddings=True) | |
| ``` | |
| - **Context length:** 8192 tokens | |
| - **Pooling:** mean · **Output dim:** 768 (Matryoshka-truncatable to 512/256/128) | |
| - **License:** MIT | |
| ## Citation | |
| See the repository README. Paper: *SciEmbed: Citation-Context Supervision for Scientific Document Embeddings* (under review). | |