SciEmbed-CTX-8192

Long-context variant (max_seq_length=8192). Recommended for long scientific inputs; best within-recipe scores on the Body-Fact Retrieval probe and LongEmbed.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-ctx-8192")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)
  • Context length: 8192 tokens
  • Pooling: mean ยท Output dim: 768 (Matryoshka-truncatable to 512/256/128)
  • License: MIT

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).

Downloads last month
37
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for anon-nlp/sciembed-ctx-8192

Finetuned
(1,619)
this model