metadata
library_name: peft
base_model: Qwen/Qwen3-8B
tags:
- peft
- lora
- text-generation
- question-answering
- rag
license: apache-2.0
language:
- en
datasets:
- DinoStackAI/narrativeqa-rag
Qwen3-8b-lora-narrativeqa
LoRA adapter for Qwen/Qwen3-8B fine-tuned on the narrativeqa RAG generative dataset (DinoStackAI/narrativeqa-rag).
- Best dev metric:
eval_loss= 0.9738
Load with PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
torch_dtype="auto",
device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "DinoStackAI/Qwen3-8b-lora-narrativeqa")
tokenizer = AutoTokenizer.from_pretrained("DinoStackAI/Qwen3-8b-lora-narrativeqa")
Load with vLLM (LoRA)
from vllm import LLM
from vllm.lora.request import LoRARequest
llm = LLM(
model="Qwen/Qwen3-8B",
enable_lora=True,
max_lora_rank=16,
)
outputs = llm.generate(
prompts,
lora_request=LoRARequest("narrativeqa", 1, "DinoStackAI/Qwen3-8b-lora-narrativeqa"),
)
Use this adapter with scripts/generation/run_rag_generation.py --lora-path DinoStackAI/Qwen3-8b-lora-narrativeqa.
Training details
- Base model:
Qwen/Qwen3-8B - Fine-tuning dataset:
DinoStackAI/narrativeqa-rag - Method: LoRA (
r=16,lora_alpha=32,lora_dropout=0.05) - Target modules:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj - Loss: SFT with completion-only masking (
assistant_only_loss=True) - Best checkpoint selection: dev
eval_loss