cadence-specter2

Fine-tuned version of SPECTER2 for personalized research paper recommendation, built for Cadence — an AI-powered research discovery platform.

Datasets used (metadata: paper, abstract, etc)

S2ORC ( S3 bucket via API key), OpenAlex, Arxiv

Performance

Paper Pair	Base SPECTER2	Cadence Fine-tuned
Attention is All You Need ↔ BERT	0.833	0.871
Attention is All You Need ↔ Cancer Research Paper	0.852	-0.044
Similarity gap	-0.019 ❌	+0.914 ✅

The base model was confused — it rated a cancer paper as more similar to a transformer paper than BERT was. After fine-tuning, the model has crystal clear domain separation with a gap of +0.914.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("rohan5manza/cadence-specter2")

embeddings = model.encode(
    ["Attention Is All You Need [SEP] The dominant sequence transduction models..."],
    normalize_embeddings=True
)

Important: Use [SEP] to separate title from abstract. Always set normalize_embeddings=True.

Training Details

• Base model: allenai/specter2_base
• Training pairs: 267,841 triplets (anchor, positive, negative)
• Signals: bibliographic coupling + category co-occurrence
• Epochs: 3
• Final loss: 0.042
• Hardware: NVIDIA RTX 4060 Ti 16GB
• Training time: ~3.6 hours

About Cadence

Cadence is a personalized research discovery app. Try it at cadence.rohanmarar.com. Source code at github.com/Rohan5manza/cadence-backend.