File size: 4,997 Bytes
ccb8b71 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | ---
base_model: Qwen/Qwen3-8B
library_name: peft
pipeline_tag: text-generation
tags:
- qwen
- qwen3
- lora
- peft
- biomedical-entity-linking
- clinical-nlp
- concept-normalization
- reranking
- candidate-ranking
- reasoning
- reinforcement-learning
license: other
---
# Qwen3-8B-LoRA-ContextBioEL-Reranker-RL
This repository provides a LoRA adapter for Qwen3-8B for the reranker stage of a clinical biomedical entity linking pipeline.
This model reranks a top-10 candidate list using the rewritten term, marked note context, and candidate semantic tags, and outputs the best concept_id. It was further optimized with reinforcement learning (RL) for entity-linking-oriented reranking behavior.
## Model type
- Base model: Qwen/Qwen3-8B
- Adapter type: LoRA
- Stage: Reranker
- Training: RL
- Task: Context-aware biomedical entity linking reranking
## Intended use
Inputs:
- `rewritten_term`
- `context_marked`, where the target mention is explicitly enclosed by `<mention>...</mention>`
- `candidates`, a top-10 candidate list containing:
- `concept_id`
- `concept_name`
- `semantic_tag`
Output:
- exactly one selected `concept_id` in the `<answer>...</answer>` block
This model is intended for research use in biomedical entity linking pipelines.
## Important decoding note
This adapter was trained with reasoning-style outputs.
Please:
- enable thinking
- do not use greedy decoding
Recommended decoding:
- `do_sample=True`
- non-greedy decoding such as temperature/top-p sampling
- parse the final prediction from the `<answer>...</answer>` span
## Usage example
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
import json
base_model_path = "Qwen/Qwen3-8B"
adapter_path = "Tao-AI-Informatics/Qwen3-8B-LoRA-ContextBioEL-Reranker-RL"
tokenizer = AutoTokenizer.from_pretrained(base_model_path, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_path,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_path)
cands_json = json.dumps([
{"concept_id": "22298006", "concept_name": "myocardial infarction", "semantic_tag": "disorder"},
{"concept_id": "57054005", "concept_name": "acute myocardial infarction", "semantic_tag": "disorder"}
], indent=2)
messages = [
{
"role": "system",
"content": (
"You are a clinical concept normalization model that reranks a top-10 candidate list using context and semantic tags.\n\n"
"Inputs you will receive:\n"
"- rewritten_term\n"
"- context_marked with <mention>...</mention>\n"
"- candidates: top-10 items (concept_id, concept_name, semantic_tag)\n\n"
"Think before answer\n\n"
"Output ONLY:\n"
"<think>...</think>\n"
"<answer>...</answer>\n\n"
"In <think>, write a detailed reasoning with these parts:\n"
"1) Context interpretation: what the mention means in this note (section cues, negation, experiencer, temporality).\n"
"2) Type inference: what semantic type/tag is expected (and why other tags are wrong).\n"
"3) Candidate comparison: evaluate multiple candidates. Note over-specific vs too-general, added qualifiers, and tag alignment.\n"
"4) Decision: justify the final choice.\n\n"
"In <answer>, use exactly one of:\n"
"- <answer><concept_id></answer>\n"
),
},
{
"role": "user",
"content": (
"Task: Choose the best concept_id from candidates.\n\n"
"rewritten_term:\nacute myocardial infarction\n\n"
"context_marked:\n"
"The patient was admitted for <mention>heart attack</mention> yesterday.\n\n"
f"candidates (top10; no scores):\n{cands_json}"
),
},
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=True,
temperature=0.6,
top_p=0.95,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
```
## Notes
- This is a LoRA adapter, not a standalone full model.
- The adapter is designed for the rewriting stage, not retrieval by itself.
- In downstream pipelines, the rewritten term is typically passed to a retriever or reranker.
## Limitations
- This model is intended for research use only.
- Performance may vary across ontologies, institutions, and note styles.
- The model should be evaluated carefully before any real-world deployment.
- The final normalized term should be extracted from the <answer>...</answer> block.
## Citation
If you use this model, please cite the associated paper when available. |