| license: mit | |
| library_name: sentence-transformers | |
| pipeline_tag: feature-extraction | |
| tags: | |
| - sentence-transformers | |
| - feature-extraction | |
| - sentence-similarity | |
| - scientific-documents | |
| - modernbert | |
| - citation-context | |
| base_model: answerdotai/ModernBERT-base | |
| language: | |
| - en | |
| # SciEmbed-CTX-2048 | |
| Intermediate long-context variant (max_seq_length=2048). Point on the 512→2K→8K context-length scan. | |
| A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the **SciEmbed** release (paper under double-blind review; author info omitted). | |
| ## Usage | |
| ```python | |
| from sentence_transformers import SentenceTransformer | |
| model = SentenceTransformer("anon-nlp/sciembed-ctx-2048") | |
| emb = model.encode(["citation-context supervision for scientific embeddings"], | |
| normalize_embeddings=True) | |
| ``` | |
| - **Context length:** 2048 tokens | |
| - **Pooling:** mean · **Output dim:** 768 (Matryoshka-truncatable to 512/256/128) | |
| - **License:** MIT | |
| ## Citation | |
| See the repository README. Paper: *SciEmbed: Citation-Context Supervision for Scientific Document Embeddings* (under review). | |