File size: 1,244 Bytes
7b9acab | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | ---
license: mit
library_name: sentence-transformers
pipeline_tag: feature-extraction
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- scientific-documents
- modernbert
- citation-context
base_model: answerdotai/ModernBERT-base
language:
- en
---
# SciEmbed-CTX-8192
Long-context variant (max_seq_length=8192). Recommended for long scientific inputs; best within-recipe scores on the Body-Fact Retrieval probe and LongEmbed.
A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the **SciEmbed** release (Findings of EMNLP 2026).
## Usage
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("J0nasW/sciembed-ctx-8192")
emb = model.encode(["citation-context supervision for scientific embeddings"],
normalize_embeddings=True)
```
- **Context length:** 8192 tokens
- **Pooling:** mean · **Output dim:** 768 (Matryoshka-truncatable to 512/256/128)
- **License:** MIT
## Citation
See the repository README. Paper: *SciEmbed: Citation-Context Supervision for Scientific Document Embeddings*, Findings of the Association for Computational Linguistics: EMNLP 2026.
|