Qwen3-Emb-4b-lora-telco-dpr

LoRA adapter for Qwen/Qwen3-Embedding-4B fine-tuned on the telco-dpr RAG retrieval dataset (DinoStackAI/telco-dpr-rag).

• Best dev metric: eval_telco-dpr-dev_cosine_ndcg@10 = 0.6732

Load with Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DinoStackAI/Qwen3-Emb-4b-lora-telco-dpr")
embeddings = model.encode(["Instruct: ...\nQuery:your query", "document text"])

Or load the base model and adapter explicitly:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Qwen/Qwen3-Embedding-4B")
model.load_adapter("DinoStackAI/Qwen3-Emb-4b-lora-telco-dpr")

Load with vLLM (LoRA)

from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-Embedding-4B",
    task="embed",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.embed(
    ["Instruct: ...\nQuery:your query"],
    lora_request=LoRARequest("telco-dpr", 1, "DinoStackAI/Qwen3-Emb-4b-lora-telco-dpr"),
)

Training details

• Base model: Qwen/Qwen3-Embedding-4B
• Fine-tuning dataset: DinoStackAI/telco-dpr-rag
• Method: LoRA (r=16, lora_alpha=32, targets q_proj / v_proj)
• Loss: CachedMultipleNegativesRankingLoss
• Best checkpoint selection: dev IR NDCG@10