README.md

---
library_name: peft
base_model: Qwen/Qwen3-8B
tags:
- peft
- lora
- text-generation
- question-answering
- rag
license: apache-2.0
language:
- en
datasets:
- DinoStackAI/narrativeqa-rag
---

# Qwen3-8b-lora-narrativeqa

LoRA adapter for [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) fine-tuned on the **narrativeqa** RAG generative dataset ([DinoStackAI/narrativeqa-rag](https://huggingface.co/datasets/DinoStackAI/narrativeqa-rag)).
- **Best dev metric:** `eval_loss` = 0.9738

## Load with PEFT

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "DinoStackAI/Qwen3-8b-lora-narrativeqa")
tokenizer = AutoTokenizer.from_pretrained("DinoStackAI/Qwen3-8b-lora-narrativeqa")
```

## Load with vLLM (LoRA)

```python
from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-8B",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.generate(
    prompts,
    lora_request=LoRARequest("narrativeqa", 1, "DinoStackAI/Qwen3-8b-lora-narrativeqa"),
)
```

Use this adapter with `scripts/generation/run_rag_generation.py --lora-path DinoStackAI/Qwen3-8b-lora-narrativeqa`.

## Training details

- **Base model:** `Qwen/Qwen3-8B`
- **Fine-tuning dataset:** `DinoStackAI/narrativeqa-rag`
- **Method:** LoRA (`r=16`, `lora_alpha=32`, `lora_dropout=0.05`)
- **Target modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- **Loss:** SFT with completion-only masking (`assistant_only_loss=True`)
- **Best checkpoint selection:** dev `eval_loss`