sciseek-nano

sciseek-nano is a highly efficient embedding model for scientific literature search. It is fine-tuned specifically for embedding search queries and title + abstract descriptions of research papers. We release it alongside the larger sciseek-micro; both are trained on the sciseek-train dataset.

sciseek-nano powers the semantic literature search of refract.science.

On SciSeek, both models are Pareto efficient regarding their model and embedding size, outperforming much larger models. With 47M parameters and 384 dimensions, sciseek-nano ranks third of all models we evaluated on the SciSeek benchmark, behind sciseek-micro and Qwen3-Embedding-4B (85× its size).

model base model params dim SciSeek LitSearch 8-bit ONNX
sciseek-nano (this model) granite-embedding-small-english-r2 47M 384 0.646 0.512 53 MB
sciseek-micro granite-embedding-english-r2 149M 768 0.715 0.584 167 MB

nDCG@10 on SciSeek (3,010 machine-generated queries over 200,000 arXiv papers) and LitSearch (597 real researcher questions).

nDCG@10 against parameter count on SciSeek and LitSearch

  • SciSeek nDCG@10 0.646, +0.059 [+0.054, +0.064] over its base model. Only sciseek-micro and Qwen3-Embedding-4B score significantly higher; every other model we tested scores significantly lower.
  • LitSearch nDCG@10 0.512, +0.026 [+0.012, +0.040] over its base model. Here, sciseek-micro and four larger general-purpose models (Qwen3-Embedding-4B and -0.6B, jina-v5-small and -nano) score significantly higher; granite-r2, gte-modernbert and leaf-ir are tied, and the rest score lower.
  • At equal index size it leads SciSeek: truncated to 384 dimensions, the best Matryoshka model reaches 0.612 (Qwen3-Embedding-4B), significantly below sciseek-nano.

Usage

The models run in sentence-transformers, ONNX Runtime and transformers.js, and we provide docker compose files for deploying the models with TEI and vLLM.

Embed papers as "Title:\n{title}\nAbstract:\n{abstract}"; queries need no prefix or instruction. The CLS token is used for pooling; L2-normalise and compare with the dot product. The model is specialised for literature search; for other tasks, use a general model such as granite-embedding-small-english-r2, the base model of sciseek-nano.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("daniel-gomm/sciseek-nano")

query = "methods for compressing large language models via knowledge distillation"
papers = [
    "Title:\nTinyBERT: Distilling BERT for Natural Language Understanding\n"
    "Abstract:\nWe propose a novel Transformer distillation method...",
    "Title:\nObservations of Type Ia supernovae in distant galaxies\n"
    "Abstract:\nWe report photometric measurements...",
]
q = model.encode(query, normalize_embeddings=True)
d = model.encode(papers, normalize_embeddings=True)
print(q @ d.T)  # cosine similarity

In the browser or Node with transformers.js:

import { pipeline } from '@huggingface/transformers'

const extractor = await pipeline('feature-extraction', 'daniel-gomm/sciseek-nano', {
  device: 'webgpu', dtype: 'fp16',   // or device: 'wasm', dtype: 'q8' (53 MB)
})
const q = await extractor('methods for compressing large language models via knowledge distillation',
                          { pooling: 'cls', normalize: true })   // q.dims = [1, 384]

Not every browser exposes WebGPU with fp16, so fall back to WASM if loading fails, and run the extractor in a Web Worker. For a server, deploy/ has docker compose files for Text Embeddings Inference and vLLM; vLLM returns unnormalised embeddings, so normalise them yourself.

Results

We evaluated 14 open embedding models (23M to 4B parameters) and BM25 in the same setup. We use the query/document prompts and document format specified in each of the model cards. Search is implemented as the exact cosine over the full corpus. No setting was tuned on SciSeek or LitSearch, and every top-10 result of every run on SciSeek has a relevance label.

model params dim SciSeek LitSearch
sciseek-micro 149M 768 0.715 0.584
Qwen3-Embedding-4B (8-bit) 4.0B 2560 0.682 0.598
sciseek-nano (this model) 47M 384 0.646 0.512
granite-embedding-english-r2 149M 768 0.623 0.497
jina-embeddings-v5-text-small 596M 1024 0.620 0.556
Qwen3-Embedding-0.6B 596M 1024 0.597 0.538
jina-embeddings-v5-text-nano 239M 768 0.587 0.541
granite-embedding-small-english-r2 47M 384 0.587 0.486
gte-modernbert-base 149M 768 0.581 0.501
mdbr-leaf-ir 23M 768 0.554 0.490
embeddinggemma-2 270M 768 0.550 0.481
bge-small-en-v1.5 33M 384 0.499 0.428
BM25 — — 0.471 0.394
all-MiniLM-L6-v2 23M 384 0.468 0.355
SPECTER2 110M 768 0.297 0.248
queries 3,010 597

nDCG@10 at each model's native embedding size. Qwen3-Embedding-4B was run with 8-bit weights to fit an 8 GB GPU. Exact settings and run files can be found in the SciSeek benchmark under baselines/systems.json.

Effect of fine-tuning

nDCG@10 per query type, base model and fine-tuned model

Compared to the base model, fine-tuning has the largest effect on descriptive queries (questions, problem descriptions, citation contexts, literature reviews) where the semantic context is more relevant for determining relevance than direct overlap. Looking up the titles of known items, which BM25 already solves, drops slightly for both models.

Other scientific retrieval sets. The fine-tuning keeps the base model's general ability on most sets (nDCG@10, change against the base model with 95% CI; * = CI excludes 0):

benchmark what it tests n sciseek-nano sciseek-micro
DORIS-MAE complex multi-aspect literature queries 100 0.651 (+0.003) 0.630 (−0.028*)
BEIR TREC-COVID biomedical literature search 50 0.630 (−0.035*) 0.712 (−0.017)
BEIR SciFact claim verification 300 0.741 (+0.001) 0.748 (−0.001)
BEIR NFCorpus lay-language nutrition/medical queries 323 0.366 (−0.004) 0.370 (−0.008)
BEIR SciDocs a paper title as the query, citing papers as targets 1,000 0.229 (−0.010*) 0.240 (−0.011*)

Formats and quantizations

build file sciseek-nano sciseek-micro
PyTorch model.safetensors 95 MB 298 MB
ONNX fp32 onnx/model.onnx 191 MB 597 MB
ONNX fp16 (WebGPU) onnx/model_fp16.onnx 96 MB 298 MB
ONNX 8-bit (WASM, q8) onnx/model_quantized.onnx 53 MB 167 MB

The fp32 and fp16 ONNX builds give the same embeddings as PyTorch (minimum cosine 1.0000 on 256 texts). The 8-bit builds do not significantly decrease performance and achieve the same test results on both benchmarks. On SciSeek, nDCG@10 changes by −0.000 [−0.001, +0.001] (nano) and −0.001 [−0.001, +0.000] (micro), on LitSearch it changes by +0.004 [+0.000, +0.007] and +0.001 [−0.002, +0.004].

Training

sciseek-nano is fine-tuned from granite-embedding-small-english-r2 using the sciseek-train dataset, which consists of semi-synthetic training data sourced from arXiv publications and synthetic search queries. We train with InfoNCE with in-batch and mined hard negatives and false-negative masking, with 512 papers per step for one epoch. We select the best performing model from a separate validation set and blend the fine-tuned and base weights with WiSE-FT (1 - α) * base + α * fine-tuned with α = 0.45 at step 1,000 of 3,248.

We ensure that the training data is entirely independent of all papers from the evaluation benchmarks SciSeek, LitSearch, and DORIS-MAE to ensure there is no leakage.

We will provide full details of the training process in an upcoming paper.

Limitations

The SciSeek embedding models are trained for literature search, thus the performance for generic retrieval tasks is not sufficiently assessed. Our training and evaluation are also limited to papers sourced from arXiv, which limits the domain of documents the model is trained on, primarily consisting of physics, mathematics, and computer science papers. Other research domains are covered to a lesser degree. Additionally, the training data is semi-synthetic and queries are generated. While evaluation results suggest generalization to human-written queries, we expect a narrower query distribution in our training data than in real search.

License and citation

Apache-2.0. The model builds on IBM Granite Embedding R2. Thank you to arXiv for use of its open access interoperability.

@misc{gomm2026sciseek,
  title  = {SciSeek: Benchmarking and Training Embedding Models for Scientific Literature Search},
  author = {Gomm, Daniel},
  year   = {2026},
  url    = {https://huggingface.co/datasets/daniel-gomm/SciSeek}
}
Downloads last month
25
Safetensors
Model size
47.7M params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for daniel-gomm/sciseek-nano

Quantized
(16)
this model

Dataset used to train daniel-gomm/sciseek-nano

Collection including daniel-gomm/sciseek-nano

Evaluation results