SciEmbed-CTX-2048

Intermediate long-context variant (max_seq_length=2048). Point on the 512โ†’2Kโ†’8K context-length scan.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-ctx-2048")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)
  • Context length: 2048 tokens
  • Pooling: mean ยท Output dim: 768 (Matryoshka-truncatable to 512/256/128)
  • License: MIT

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).

Downloads last month
45
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for anon-nlp/sciembed-ctx-2048

Finetuned
(1,619)
this model