factuality-dpr-context

Fine-tuned Dense Passage Retriever (Context Encoder) trained on positive (question, supporting passage) pairs from HotpotQA to improve factual grounding in Retrieval-Augmented Generation (RAG).

This encoder forms part of a hybrid re-ranking system to reduce hallucinations in LLM-based QA.

---

Model Architecture

• Base Model: facebook/dpr-ctx_encoder-single-nq-base
• Output Format: Dense embedding vector (768-dim)
• Objective: Contrastive learning with InfoNCE-style loss
• Aligns question and passage representations for strong semantic retrieval

---

Training Details

Property	Value
Dataset	HotpotQA (fullwiki) – first 3,000 training samples
Positive Labels	Supporting passage titles
Epochs	2
Effective Batch Size	128
Optimizer	AdamW (lr=2e-5)
GPU	Kaggle T4
Random Seed	42 (fully reproducible)

Training Loss Progress:

• Epoch 1 → 0.1298
• Epoch 2 → 0.0528

The fine-tuning improves DPR passage alignment, directly boosting downstream RAG relevance.

---

Intended Use

This model is not a standalone text generator.

It must be paired with: ✔ DPR Question Encoder → Anshul017/factuality-dpr-question
✔ RAG Generator (e.g., FLAN-T5-Large)
✔ Optional hybrid re-ranker (BM25 + NLI)

Primary use cases:

• Open-domain QA
• Retrieval-Augmented Generation
• Hallucination-aware evaluation research

---

Performance

Using HotpotQA 1k validation split:

Metric	Score
Recall@1	0.5720
Recall@5	0.7420
Recall@20	0.8090

This represents a noticeable improvement over zero-shot DPR recall quality for multi-hop queries.

---

Related Models

• Question Encoder: https://huggingface.co/Anshul017/factuality-dpr-question

These two models must be used together for correct dense retrieval.

---

How to Use

from transformers import DPRContextEncoder, DPRContextEncoderTokenizer

ctx_model = DPRContextEncoder.from_pretrained("Anshul017/factuality-dpr-context")
ctx_tokenizer = DPRContextEncoderTokenizer.from_pretrained("Anshul017/factuality-dpr-context")

Author

Anshul Thakur M.Tech C.S.E N.I.T Hamirpur