|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- RAG |
|
|
- retrieval-augmented-generation |
|
|
- document-qa |
|
|
- pdf-processing |
|
|
- hybrid-retrieval |
|
|
- cross-encoder |
|
|
- langchain |
|
|
- chromadb |
|
|
- bm25 |
|
|
- semantic-chunking |
|
|
- multi-document |
|
|
- question-answering |
|
|
library_name: langchain |
|
|
pipeline_tag: question-answering |
|
|
datasets: [] |
|
|
metrics: |
|
|
- accuracy |
|
|
base_model: |
|
|
- BAAI/bge-large-en-v1.5 |
|
|
- BAAI/bge-reranker-v2-m3 |
|
|
- sentence-transformers/all-MiniLM-L6-v2 |
|
|
--- |
|
|
|
|
|
# Multi-Document RAG System |
|
|
|
|
|
A production-ready **Retrieval-Augmented Generation (RAG)** system for intelligent question-answering over multiple PDF documents. Features hybrid retrieval (vector + keyword search), cross-encoder re-ranking, semantic chunking, and a Gradio web interface. |
|
|
|
|
|
 |
|
|
 |
|
|
 |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This system implements an advanced RAG pipeline that combines multiple state-of-the-art techniques for optimal document retrieval and question answering: |
|
|
|
|
|
### Core Models Used |
|
|
|
|
|
| Component | Model | Purpose | |
|
|
|-----------|-------|---------| |
|
|
| **Embeddings** | `BAAI/bge-large-en-v1.5` | 1024-dim normalized embeddings for semantic search | |
|
|
| **Re-ranker** | `BAAI/bge-reranker-v2-m3` | Cross-encoder neural re-ranking for precision | |
|
|
| **Chunker** | `sentence-transformers/all-MiniLM-L6-v2` | Semantic similarity for intelligent chunking | |
|
|
| **LLM** | Llama 3.3 70B (via Groq API) | Generation with inline citations | |
|
|
|
|
|
### Architecture |
|
|
|
|
|
``` |
|
|
User Query |
|
|
│ |
|
|
├── Query Classification (factoid/summary/comparison/extraction/reasoning) |
|
|
├── Multi-Query Expansion (3 alternative phrasings) |
|
|
└── HyDE Generation (hypothetical answer document) |
|
|
│ |
|
|
▼ |
|
|
┌──────────────────────────────────────┐ |
|
|
│ Hybrid Retrieval │ |
|
|
│ ┌─────────────┐ ┌─────────────┐ │ |
|
|
│ │ ChromaDB │ │ BM25 │ │ |
|
|
│ │ (Vector) │ │ (Keyword) │ │ |
|
|
│ └─────────────┘ └─────────────┘ │ |
|
|
│ │ │ │ |
|
|
│ └──────┬───────┘ │ |
|
|
│ ▼ │ |
|
|
│ RRF Fusion + Deduplication │ |
|
|
└──────────────────────────────────────┘ |
|
|
│ |
|
|
▼ |
|
|
Cross-Encoder Re-ranking |
|
|
(BAAI/bge-reranker-v2-m3) |
|
|
│ |
|
|
▼ |
|
|
LLM Generation (Llama 3.3 70B) |
|
|
with inline source citations |
|
|
│ |
|
|
▼ |
|
|
Answer Verification (for complex queries) |
|
|
``` |
|
|
|
|
|
## Key Features |
|
|
|
|
|
### Hybrid Retrieval |
|
|
- **Vector Search (MMR)**: Semantic similarity with diversity via ChromaDB |
|
|
- **Keyword Search (BM25)**: Exact term matching for rare words |
|
|
- **Reciprocal Rank Fusion**: Combines multiple ranked lists optimally |
|
|
|
|
|
### Semantic Chunking |
|
|
Documents are split based on sentence embedding similarity rather than fixed character counts, preserving coherent ideas within chunks. |
|
|
|
|
|
### Intelligent Query Classification |
|
|
Automatically classifies queries into 5 types with adaptive retrieval: |
|
|
|
|
|
| Query Type | Retrieval Depth (k) | Answer Style | |
|
|
|------------|---------------------|--------------| |
|
|
| Factoid | 6 | Direct | |
|
|
| Summary | 10 | Bullets | |
|
|
| Comparison | 12 | Bullets | |
|
|
| Extraction | 8 | Direct | |
|
|
| Reasoning | 10 | Steps | |
|
|
|
|
|
### Multi-Document Support |
|
|
- Upload multiple PDFs to build a combined knowledge base |
|
|
- Automatic PDF diversity enforcement for cross-document queries |
|
|
- Clear source attribution with document name and page number |
|
|
|
|
|
### Query Enhancement |
|
|
- **HyDE**: Generates hypothetical answer documents for better retrieval |
|
|
- **Multi-Query Expansion**: Creates 3 alternative phrasings for broader coverage |
|
|
|
|
|
### Answer Verification |
|
|
Self-verification step for complex queries ensures answers are direct, structured, and grounded in sources. |
|
|
|
|
|
## Intended Uses |
|
|
|
|
|
### Primary Use Cases |
|
|
- **Academic Research**: Analyze and compare research papers |
|
|
- **Document Q&A**: Answer questions over technical documentation |
|
|
- **Literature Review**: Synthesize information across multiple sources |
|
|
- **Knowledge Extraction**: Extract specific facts, methodologies, or findings |
|
|
|
|
|
### Out-of-Scope Uses |
|
|
- Real-time streaming applications (latency-sensitive) |
|
|
- Non-English documents (optimized for English) |
|
|
- Image/table-heavy PDFs (text extraction only) |
|
|
|
|
|
## How to Use |
|
|
|
|
|
### Requirements |
|
|
- Python 3.10+ |
|
|
- Groq API key (free at [console.groq.com](https://console.groq.com)) |
|
|
- GPU recommended but not required |
|
|
|
|
|
### Installation |
|
|
|
|
|
```bash |
|
|
pip install numpy==1.26.4 pandas==2.2.2 scipy==1.13.1 |
|
|
pip install langchain-core==0.2.40 langchain-community==0.2.16 langchain==0.2.16 |
|
|
pip install langchain-groq==0.1.9 langchain-text-splitters==0.2.4 |
|
|
pip install chromadb==0.5.5 sentence-transformers==3.0.1 |
|
|
pip install pypdf==4.3.1 rank-bm25==0.2.2 gradio torch |
|
|
``` |
|
|
|
|
|
### Quick Start |
|
|
|
|
|
1. Open `rag.ipynb` in Jupyter Notebook or Google Colab |
|
|
2. Run all cells sequentially |
|
|
3. Enter your Groq API key in the Setup tab |
|
|
4. Upload PDF documents |
|
|
5. Ask questions in the Chat tab |
|
|
|
|
|
### Example Queries |
|
|
|
|
|
```python |
|
|
# Single Document Analysis |
|
|
"What is the main contribution of this paper?" |
|
|
"Explain the methodology in detail" |
|
|
"What are the limitations mentioned by the authors?" |
|
|
|
|
|
# Multi-Document Comparison |
|
|
"Compare the approaches discussed in these papers" |
|
|
"What are the key differences between the methodologies?" |
|
|
``` |
|
|
|
|
|
## Technical Specifications |
|
|
|
|
|
### Performance Benchmarks |
|
|
|
|
|
| Operation | Typical Duration | |
|
|
|-----------|------------------| |
|
|
| Model initialization | 30-60 seconds | |
|
|
| PDF ingestion (per doc) | 10-30 seconds | |
|
|
| Simple queries | 5-8 seconds | |
|
|
| Complex queries | 10-15 seconds | |
|
|
| Full document summary | 30-90 seconds | |
|
|
|
|
|
### Configuration Parameters |
|
|
|
|
|
| Parameter | Default | Description | |
|
|
|-----------|---------|-------------| |
|
|
| `max_chunk_size` | 1000 | Maximum characters per semantic chunk | |
|
|
| `similarity_threshold` | 0.5 | Cosine similarity for chunk grouping | |
|
|
| `chunk_size` | 800 | Fallback text splitter chunk size | |
|
|
| `chunk_overlap` | 150 | Character overlap between chunks | |
|
|
| `fetch_factor` | 2 | Multiplier for initial retrieval pool | |
|
|
| `lambda_mult` | 0.6 | MMR diversity parameter | |
|
|
| `cache_max_size` | 100 | Maximum cached query responses | |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- Requires active internet connection for Groq API calls |
|
|
- PDF quality affects text extraction accuracy |
|
|
- Large documents may take longer to process |
|
|
- Query cache does not persist between sessions |
|
|
- Optimized for English language documents |
|
|
|
|
|
## Training Details |
|
|
|
|
|
This is a **retrieval system**, not a trained model. It orchestrates pre-trained models: |
|
|
|
|
|
- **Embeddings**: Uses pre-trained `BAAI/bge-large-en-v1.5` without fine-tuning |
|
|
- **Re-ranker**: Uses pre-trained `BAAI/bge-reranker-v2-m3` without fine-tuning |
|
|
- **LLM**: Uses Llama 3.3 70B via Groq API with zero-shot prompting |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
The system was evaluated qualitatively on academic papers and technical documents for: |
|
|
- Answer relevance and accuracy |
|
|
- Source attribution correctness |
|
|
- Cross-document comparison quality |
|
|
- Response structure and readability |
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
- **Hardware**: Developed and tested on Google Colab (NVIDIA T4 GPU) |
|
|
- **Inference**: Primary compute via Groq API (cloud-hosted) |
|
|
- Local model loading: ~2GB VRAM for embeddings + re-ranker |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@software{multi_doc_rag_system, |
|
|
title = {Multi-Document RAG System}, |
|
|
year = {2024}, |
|
|
description = {Production-ready RAG system with hybrid retrieval and cross-encoder re-ranking}, |
|
|
url = {https://huggingface.co/your-username/your-repo} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Acknowledgements |
|
|
|
|
|
This project builds upon: |
|
|
- [LangChain](https://github.com/langchain-ai/langchain) for RAG orchestration |
|
|
- [ChromaDB](https://github.com/chroma-core/chroma) for vector storage |
|
|
- [Sentence Transformers](https://www.sbert.net/) for embeddings |
|
|
- [BAAI](https://huggingface.co/BAAI) for BGE models |
|
|
- [Groq](https://groq.com/) for fast LLM inference |
|
|
|
|
|
## Contact |
|
|
|
|
|
For questions or feedback, please open an issue on the repository. |
|
|
|
