File size: 1,737 Bytes
dbad0a1 36c24d8 dbad0a1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
library_name: peft
base_model: Qwen/Qwen3-8B
tags:
- peft
- lora
- text-generation
- question-answering
- rag
license: apache-2.0
language:
- en
datasets:
- DinoStackAI/narrativeqa-rag
---
# Qwen3-8b-lora-narrativeqa
LoRA adapter for [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) fine-tuned on the **narrativeqa** RAG generative dataset ([DinoStackAI/narrativeqa-rag](https://huggingface.co/datasets/DinoStackAI/narrativeqa-rag)).
- **Best dev metric:** `eval_loss` = 0.9738
## Load with PEFT
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
torch_dtype="auto",
device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "DinoStackAI/Qwen3-8b-lora-narrativeqa")
tokenizer = AutoTokenizer.from_pretrained("DinoStackAI/Qwen3-8b-lora-narrativeqa")
```
## Load with vLLM (LoRA)
```python
from vllm import LLM
from vllm.lora.request import LoRARequest
llm = LLM(
model="Qwen/Qwen3-8B",
enable_lora=True,
max_lora_rank=16,
)
outputs = llm.generate(
prompts,
lora_request=LoRARequest("narrativeqa", 1, "DinoStackAI/Qwen3-8b-lora-narrativeqa"),
)
```
Use this adapter with `scripts/generation/run_rag_generation.py --lora-path DinoStackAI/Qwen3-8b-lora-narrativeqa`.
## Training details
- **Base model:** `Qwen/Qwen3-8B`
- **Fine-tuning dataset:** `DinoStackAI/narrativeqa-rag`
- **Method:** LoRA (`r=16`, `lora_alpha=32`, `lora_dropout=0.05`)
- **Target modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- **Loss:** SFT with completion-only masking (`assistant_only_loss=True`)
- **Best checkpoint selection:** dev `eval_loss`
|