---
license: mit
library_name: sentence-transformers
pipeline_tag: feature-extraction
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- scientific-documents
- modernbert
- citation-context
base_model: answerdotai/ModernBERT-base
language:
- en
---

# SciEmbed-CTX

Signal A+B on a 7M-pair subsample (3 epochs). Best ablation; the FULL model is this recipe scaled to the full pool.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the **SciEmbed** release (paper under double-blind review; author info omitted).

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-ctx")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)
```

- **Context length:** 512 tokens
- **Pooling:** mean · **Output dim:** 768 (Matryoshka-truncatable to 512/256/128)
- **License:** MIT

## SciRepEval (4-category macro)

| Classif. | Regr. | Prox. | Search | Overall |
|---|---|---|---|---|
| 75.5 | **28.3** | 80.9 | 82.5 | 66.8 ± 0.02 |

## Citation

See the repository README. Paper: *SciEmbed: Citation-Context Supervision for Scientific Document Embeddings* (under review).