Qwen-3-8B-TP

This model is fine-tuned from Qwen/Qwen3-8B by using TrialPanorama dataset for clinical trials.

Model Details

• Base Model: Qwen/Qwen3-8B
• Fine-tuning Method: Two-stage training
  • Stage 1: Supervised Fine-Tuning (SFT) for knowledge injection
  • Stage 2: RLVR (Reinforcement Learning with Verifiable Reward)

Usage

Basic Usage with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "linjc16/Qwen-3-8B-TP"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Prepare input (a toy example)
prompt = """Given the following clinical trial information, search for relevant studies:

[Input Information]

Please provide relevant clinical trials and reasoning."""

# Generate response
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **model_inputs,
    max_new_tokens=512,
    temperature=0.6,
    top_p=0.95,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Usage with vLLM (Recommended for Production)

from vllm import LLM, SamplingParams

# Initialize vLLM
llm = LLM(
    model="linjc16/Qwen-3-8B-TP",
    tensor_parallel_size=1,
    dtype="bfloat16",
    enable_reasoning=True,
    reasoning_parser="deepseek_r1"
)

# Set sampling parameters
sampling_params = SamplingParams(
    temperature=0.6,
    top_p=0.95,
    max_tokens=512
)

# Generate
prompts = ["Your study search prompt here"]
outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    print(output.outputs[0].text)

Usage with SGLang

from sglang import Runtime, Engine, ModelConfig, SamplingParams

# Initialize SGLang
runtime = Runtime(
    model_path="linjc16/Qwen-3-8B-TP",
    reasoning_parser="qwen3"
)

# Set sampling parameters
sampling_params = SamplingParams(
    temperature=0.6,
    top_p=0.95,
    max_new_tokens=512
)

# Generate
prompts = ["Your study search prompt here"]
outputs = runtime.generate(prompts, sampling_params)

for output in outputs:
    print(output.text)

Citation

@article{wang2025trialpanorama,
  title     = {Developing Large Language Models for Clinical Research Using One Million Clinical Trials},
  author    = {Wang, Zifeng and Lin, Jiacheng and Jin, Qiao and Gao, Junyi and Pradeepkumar, Jathurshan and Jiang, Pengcheng and Lu, Zhiyong and Sun, Jimeng},
  journal   = {arXiv preprint arXiv:2505.16097},
  year      = {2025},
  url       = {https://arxiv.org/abs/2505.16097}
}