README.md

metadata

library_name: sentence-transformers
base_model: Qwen/Qwen3-Embedding-4B
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - lora
  - peft
  - embedding
  - retrieval
  - rag
license: apache-2.0
language:
  - en
datasets:
  - DinoStackAI/qasper-rag

Qwen3-Emb-4b-lora-qasper

LoRA adapter for Qwen/Qwen3-Embedding-4B fine-tuned on the qasper RAG retrieval dataset (DinoStackAI/qasper-rag).

• Best dev metric: eval_qasper-dev_cosine_ndcg@10 = 0.1366

Load with Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DinoStackAI/Qwen3-Emb-4b-lora-qasper")
embeddings = model.encode(["Instruct: ...\nQuery:your query", "document text"])

Or load the base model and adapter explicitly:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Qwen/Qwen3-Embedding-4B")
model.load_adapter("DinoStackAI/Qwen3-Emb-4b-lora-qasper")

Load with vLLM (LoRA)

from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-Embedding-4B",
    task="embed",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.embed(
    ["Instruct: ...\nQuery:your query"],
    lora_request=LoRARequest("qasper", 1, "DinoStackAI/Qwen3-Emb-4b-lora-qasper"),
)

Training details

• Base model: Qwen/Qwen3-Embedding-4B
• Fine-tuning dataset: DinoStackAI/qasper-rag
• Method: LoRA (r=16, lora_alpha=32, targets q_proj / v_proj)
• Loss: CachedMultipleNegativesRankingLoss
• Best checkpoint selection: dev IR NDCG@10