--- title: SmartRAG emoji: ๐Ÿง  colorFrom: blue colorTo: purple sdk: docker app_port: 7860 pinned: true --- # ๐Ÿง  SmartRAG โ€” Production AI Assistant for Programmers > QLoRA Fine-Tuning ยท Hybrid Search (BM25+Dense) ยท Cross-Encoder Reranking ยท ReAct Agent ยท Embedding Cache ยท Rate Limiting ยท Ablation Study ยท FastAPI ยท React --- ## ๐ŸŽฏ Use Case: AI Assistant for Programmers Answers developer questions grounded in real documentation โ€” no hallucinated APIs. Answers developer questions grounded in real documentation โ€” no hallucinated APIs. - *"How do I handle exceptions in asyncio?"* - *"What is the GIL and how does it affect threading?"* - *"How do I optimize a slow SQL query with N+1 problems?"* --- ## ๐Ÿ—๏ธ Architecture ``` User Query โ”‚ โ–ผ Rate Limiter (Token Bucket, per-IP) โ”‚ โ–ผ ReAct Agent (Thought โ†’ Tool โ†’ Observe โ†’ Repeat) Tools: vector_search | hybrid_search | code_executor | calculator | web_search โ”‚ โ–ผ Hybrid Retrieval Dense (ChromaDB) + Sparse (BM25) โ†’ RRF Fusion โ†’ Cross-Encoder Reranker โ”‚ โ–ผ Embedding Cache (LRU / Redis) โ€” 350ร— faster on cache hits โ”‚ โ–ผ Fine-Tuned LLM (Mistral-7B + QLoRA, programming domain) โ”‚ โ–ผ RAGAS Evaluation + Ablation Study + MLflow ``` --- ## ๐Ÿ“Š Ablation Results (Before vs After) | System | Keyword Coverage | Faithfulness | Failure Rate | |---|---|---|---| | A: Base Model + Dense RAG | 0.41 | 0.38 | 67% | | B: Fine-Tuned + Dense RAG | 0.78 | 0.71 | 17% | | C: Fine-Tuned + Hybrid | 0.84 | 0.76 | 17% | | **D: Full Pipeline** | **0.84** | **0.76** | **17%** | Fine-tuning improved keyword coverage **+90% relative**. Hybrid search captured exact API names that dense-only missed. --- ## โš™๏ธ System Design | Decision | Choice | Reason | |---|---|---| | Hybrid fusion | RRF (ฮฑ=0.7) | Robust to different score scales | | Reranker | ms-marco-MiniLM-L-6-v2 | +27% MRR vs dense-only | | Cache | LRU โ†’ Redis | 350ร— latency on hits | | Rate limit | Token bucket | Handles burst traffic | | HNSW tuning | ef=100, M=16 | Quality/latency tradeoff | | Quantization | NF4 double-quant | 7B fits in 4GB VRAM | --- ## ๐Ÿ“ Files ``` smartrag/ โ”œโ”€โ”€ config.py โ† All configs (model, hybrid, cache, agent, system) โ”œโ”€โ”€ data/ โ”‚ โ”œโ”€โ”€ prepare_dataset.py โ† Generic dataset pipeline โ”‚ โ””โ”€โ”€ code_dataset.py โ† Programming domain (18K Python QA pairs) โ”œโ”€โ”€ training/finetune.py โ† QLoRA fine-tuning โ”œโ”€โ”€ rag/ โ”‚ โ”œโ”€โ”€ vectorstore.py โ† ChromaDB dense retrieval โ”‚ โ”œโ”€โ”€ hybrid_search.py โ† BM25 + dense + RRF fusion โ”‚ โ”œโ”€โ”€ reranker.py โ† Cross-encoder reranking โ”‚ โ”œโ”€โ”€ cache.py โ† Embedding cache (memory/disk/Redis) โ”‚ โ”œโ”€โ”€ pipeline.py โ† Single-pass RAG โ”‚ โ””โ”€โ”€ agent.py โ† ReAct multi-step agent โ”œโ”€โ”€ evaluation/ โ”‚ โ”œโ”€โ”€ evaluate.py โ† RAGAS metrics โ”‚ โ””โ”€โ”€ ablation.py โ† Before/after comparison + failure analysis โ”œโ”€โ”€ api/ โ”‚ โ”œโ”€โ”€ app.py โ† FastAPI โ”‚ โ””โ”€โ”€ rate_limiter.py โ† Token bucket rate limiting โ”œโ”€โ”€ ui/app.py โ† Streamlit UI โ”œโ”€โ”€ frontend/src/App.jsx โ† React frontend โ””โ”€โ”€ tests/test_smartrag.py โ† 18 unit + API tests ``` --- ## ๐Ÿš€ Quick Start ```bash pip install -r requirements.txt python -m data.code_dataset # 1. Prepare programming dataset python -m training.finetune # 2. Fine-tune (needs GPU) python -m evaluation.ablation # 3. Run ablation study uvicorn api.app:app --port 8000 # 4. Launch API streamlit run ui/app.py # 5. Launch UI mlflow ui --port 5000 # 6. View experiments ``` ======= title: Smartrag emoji: ๐Ÿš€ colorFrom: gray colorTo: pink sdk: docker pinned: false license: mit short_description: SmartRAG โ€” Production LLM System for Programmers --- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference >>>>>>> 8aefd1e917c894217d33d6ee1fe21b4d2668ceb9