license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3-Embedding-4B
library_name: sentence-transformers
Description
This is one CSRv2 model finetuned on MTEB reranking datasets with Qwen3-Embedding-4B as backbone.
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our Github.
Sentence Transformer Usage
You can evaluate this model loaded by Sentence Transformers with the following code snippet (take SciDocsRR as one example):
import mteb
from sentence_transformers import SparseEncoder
model = SparseEncoder(
"Y-Research-Group/CSRv2-reranking",
trust_remote_code=True
)
model.prompts = {
"SciDocsRR": "Instruct: Given a title of a scientific paper, retrieve the titles of other relevant papers\n Query:"
}
task = mteb.get_tasks(tasks=["SciDocsRR"])
evaluation = mteb.MTEB(tasks=task)
evaluation.run(
model,
eval_splits=["test"],
output_folder="./results/SciDocsRR",
show_progress_bar=True
encode_kwargs={"convert_to_sparse_tensor": False, "batch_size": 8}
) # MTEB don't support sparse tensors yet, so we need to convert to dense tensors
It is suggested that you use our default prompts in evaluation.
Multi-TopK Support
Our model supports different sparsity levels due to the utilization of Multi-TopK loss in training.
You can change sparsity model by adjusting the k parameterin the file3_SparseAutoEncoder/config.json`.
We set sparsity level to 2 by default.
For instance, if you want to evaluate with sparsity level $K=8$ (which means there are 8 activated neurons in
each embedding vector), the 3_SparseAutoEncoder/config.json should look like this:
{
"input_dim": 2560,
"hidden_dim": 10240,
"k": 8,
"k_aux": 1024,
"normalize": false,
"dead_threshold": 30
}
CSRv2 Qwen Series
We will release a series of CSRv2 models finetuned on common tasks in MTEB with Qwen3-Embedding-4B as backbone. These tasks are:
Citation
@inproceedings{guo2026csrv2,
title={{CSR}v2: Unlocking Ultra-sparse Embeddings},
author={Guo, Lixuan and Wang, Yifei and Wen, Tiansheng and Wang, Yifan and Feng, Aosong and Chen, Bo and Jegelka, Stefanie and You, Chenyu},
booktitle={International Conference on Learning Representations (ICLR)},
year={2026}
}