File size: 1,724 Bytes
4d51a12 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
library_name: sentence-transformers
base_model: Qwen/Qwen3-Embedding-4B
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- lora
- peft
- embedding
- retrieval
- rag
license: apache-2.0
language:
- en
datasets:
- DinoStackAI/qasper-rag
---
# Qwen3-Emb-4b-lora-qasper
LoRA adapter for [Qwen/Qwen3-Embedding-4B](https://huggingface.co/Qwen/Qwen3-Embedding-4B) fine-tuned on the **qasper** RAG retrieval dataset ([DinoStackAI/qasper-rag](https://huggingface.co/datasets/DinoStackAI/qasper-rag)).
- **Best dev metric:** `eval_qasper-dev_cosine_ndcg@10` = 0.1366
## Load with Sentence Transformers
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("DinoStackAI/Qwen3-Emb-4b-lora-qasper")
embeddings = model.encode(["Instruct: ...\nQuery:your query", "document text"])
```
Or load the base model and adapter explicitly:
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Qwen/Qwen3-Embedding-4B")
model.load_adapter("DinoStackAI/Qwen3-Emb-4b-lora-qasper")
```
## Load with vLLM (LoRA)
```python
from vllm import LLM
from vllm.lora.request import LoRARequest
llm = LLM(
model="Qwen/Qwen3-Embedding-4B",
task="embed",
enable_lora=True,
max_lora_rank=16,
)
outputs = llm.embed(
["Instruct: ...\nQuery:your query"],
lora_request=LoRARequest("qasper", 1, "DinoStackAI/Qwen3-Emb-4b-lora-qasper"),
)
```
## Training details
- **Base model:** `Qwen/Qwen3-Embedding-4B`
- **Fine-tuning dataset:** `DinoStackAI/qasper-rag`
- **Method:** LoRA (`r=16`, `lora_alpha=32`, targets `q_proj` / `v_proj`)
- **Loss:** CachedMultipleNegativesRankingLoss
- **Best checkpoint selection:** dev IR NDCG@10
|