| --- |
| language: en |
| license: apache-2.0 |
| tags: |
| - bioasq |
| - biomedical |
| - reranking |
| - information-retrieval |
| --- |
| |
| # BioASQ Phase A Reranker Models |
|
|
| Fine-tuned rerankers for biomedical document retrieval, trained on BioASQ data. |
| All models are cross-encoders fine-tuned from publicly available base models. |
|
|
| ## Loading a model |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| |
| subfolder = "nvidia-llama-nemotron-rerank-1b-v2-E2-S4-Mmulti_neg_pairwise-Linfonce-FullData" |
| tokenizer = AutoTokenizer.from_pretrained("IEETA/BioASQ-14B", subfolder=subfolder) |
| model = AutoModelForSequenceClassification.from_pretrained("IEETA/BioASQ-14B", subfolder=subfolder) |
| ``` |
|
|
| For nvidia/llama-nemotron variants, also copy `llama_bidirectional_model.py` from the subfolder |
| and pass `trust_remote_code=True`. |
|
|
| --- |
|
|
| ## outputs-E5-Pairwise — Shifter sampler, 5 epochs, pairwise |
|
|
| | Model | Path | map-bioasq@10 | |
| |---|---|---| |
| | nvidia/llama-nemotron-rerank-1b-v2 | `nvidia-llama-nemotron-rerank-1b-v2-E5-Pairwise` | 0.9970 | |
| | BAAI/bge-reranker-v2-m3 | `BAAI-bge-reranker-v2-m3-E5-Pairwise` | 0.6824 | |
| | BAAI/bge-reranker-base | `BAAI-bge-reranker-base-E5-Pairwise` | 0.6686 | |
| | nboost/pt-biobert-base-msmarco | `nboost-pt-biobert-base-msmarco-E5-Pairwise` | 0.6608 | |
| | cross-encoder/ms-marco-MiniLM-L-6-v2 | `cross-encoder-ms-marco-MiniLM-L-6-v2-E5-Pairwise` | 0.6373 | |
| | ncbi/MedCPT-Cross-Encoder | `ncbi-MedCPT-Cross-Encoder-E5-Pairwise` | 0.6404 | |
| | michiyasunaga/BioLinkBERT-base | `michiyasunaga-BioLinkBERT-base-E5-Pairwise` | 0.6403 | |
| | monologg/biobert_v1.1_pubmed | `monologg-biobert_v1.1_pubmed-E5-Pairwise` | 0.6346 | |
| | microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext | `microsoft-BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext-E5-Pairwise` | 0.6291 | |
| | pritamdeka/S-PubMedBert-MS-MARCO | `pritamdeka-S-PubMedBert-MS-MARCO-E5-Pairwise` | 0.5985 | |
| | allenai/specter2_base | `allenai-specter2_base-E5-Pairwise` | 0.5912 | |
| | dmis-lab/biobert-base-cased-v1.2 | `dmis-lab-biobert-base-cased-v1.2-E5-Pairwise` | 0.5848 | |
| | cross-encoder/ms-marco-electra-base | `cross-encoder-ms-marco-electra-base-E5-Pairwise` | 0.5654 | |
| | emilyalsentzer/Bio_ClinicalBERT | `emilyalsentzer-Bio_ClinicalBERT-E5-Pairwise` | 0.4587 | |
| | cambridgeltl/SapBERT-from-PubMedBERT-fulltext | `cambridgeltl-SapBERT-from-PubMedBERT-fulltext-E5-Pairwise` | 0.2594 | |
|
|
| --- |
|
|
| ## outputs — Experiments |
|
|
| | Model | Characteristics | Path | map-bioasq@10 | |
| |---|---|---|---| |
| | nvidia/llama-nemotron-rerank-1b-v2 | E2-S4, multi_neg_pairwise, InfoNCE, FullData | `nvidia-llama-nemotron-rerank-1b-v2-E2-S4-Mmulti_neg_pairwise-Linfonce-FullData` | 0.9995 | |
| | nvidia/llama-nemotron-rerank-1b-v2 | E2, pairwise (13B1+13B2) | `nvidia-llama-nemotron-rerank-1b-v2_llama-E2-Pairwise` | 0.9970 | |
| | BAAI/bge-reranker-v2-m3 | E2-S1, pairwise, FullData, shifter | `BAAI-bge-reranker-v2-m3-E2-S1-Mpairwise-FullDataTrue` | 0.6705 | |
| | BAAI/bge-reranker-base | E2-S1, pairwise, FullData, shifter | `BAAI-bge-reranker-base-E2-S1-Mpairwise-FullDataTrue` | 0.6489 | |
| | nboost/pt-biobert-base-msmarco | E2-S1, pairwise, FullData, shifter | `nboost-pt-biobert-base-msmarco-E2-S1-Mpairwise-FullDataTrue` | 0.6274 | |
| | ncbi/MedCPT-Cross-Encoder | E2-S1, pairwise, FullData, shifter | `ncbi-MedCPT-Cross-Encoder-E2-S1-Mpairwise-FullDataTrue` | 0.6251 | |
| | michiyasunaga/BioLinkBERT-base | E2-S1, pairwise, FullData, shifter | `michiyasunaga-BioLinkBERT-base-E2-S1-Mpairwise-FullDataTrue` | 0.6178 | |
| | microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext | E2-S1, pairwise, FullData, shifter | `microsoft-BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext-E2-S1-Mpairwise-FullDataTrue` | 0.6153 | |
| | cross-encoder/ms-marco-MiniLM-L-6-v2 | E3-S8, multi_neg_pairwise | `cross-encoder-ms-marco-MiniLM-L-6-v2-E3-S8-Mmulti_neg_pairwise` | 0.6098 | |
| | monologg/biobert_v1.1_pubmed | E2-S1, pairwise, FullData, shifter | `monologg-biobert_v1.1_pubmed-E2-S1-Mpairwise-FullDataTrue` | 0.6053 | |
| | cross-encoder/ms-marco-MiniLM-L-6-v2 | E2-S1, pairwise, FullData, shifter | `cross-encoder-ms-marco-MiniLM-L-6-v2-E2-S1-Mpairwise-FullDataTrue` | 0.5944 | |
| | pritamdeka/S-PubMedBert-MS-MARCO | E2-S1, pairwise, FullData, shifter | `pritamdeka-S-PubMedBert-MS-MARCO-E2-S1-Mpairwise-FullDataTrue` | 0.5839 | |
| | michiyasunaga/BioLinkBERT-large | E2-S1, pairwise, FullData, shifter | `michiyasunaga-BioLinkBERT-large-E2-S1-Mpairwise-FullDataTrue` | 0.5781 | |
| | ncbi/MedCPT-Cross-Encoder | E3-S1, pairwise, FullData, shifter | `ncbi-MedCPT-Cross-Encoder-E3-S1-Mpairwise-FullDataTrue` | 0.5766 | |
|
|
