JimmyBhoy commited on
Commit
0ef94af
·
verified ·
1 Parent(s): 069c0ce

Upload 7 files

Browse files
Files changed (7) hide show
  1. README.md +206 -11
  2. app.py +115 -0
  3. config.py +139 -0
  4. requirements.txt +44 -0
  5. retriever.py +117 -0
  6. setup.py +102 -0
  7. tools.py +219 -0
README.md CHANGED
@@ -1,13 +1,208 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
- title: Production RAG Agent
3
- emoji: 📉
4
- colorFrom: yellow
5
- colorTo: red
6
- sdk: gradio
7
- sdk_version: 5.43.1
8
- app_file: app.py
9
- pinned: false
10
- short_description: Production-ready Agentic RAG system based on Unit_3_Agentic_
11
- ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🚀 Production RAG Agent - Complete Setup
2
+
3
+ ## 🎯 What You Have
4
+
5
+ I've created a complete, production-ready RAG (Retrieval-Augmented Generation) agent system with:
6
+
7
+ ### Core Files Created
8
+ - ✅ `app.py` - Main Gradio application with professional UI
9
+ - ✅ `config.py` - Centralized configuration management
10
+ - ✅ `retriever.py` - Advanced document retrieval system
11
+ - ✅ `tools.py` - Multiple custom tools for the agent
12
+ - ✅ `requirements.txt` - All necessary dependencies
13
+ - ✅ `setup.py` - Automated setup script
14
+ - ✅ `knowledge_base/` - Folder for your documents
15
+ - ✅ `vector_store/` - For AI embeddings storage
16
+ - ✅ `logs/` - For system logs
17
+
18
+ ### Key Features
19
+ 🤖 **Agentic RAG** using Hugging Face's smolagents
20
+ 🔍 **Hybrid Search** (semantic + keyword)
21
+ 🌐 **Web Search** for current information
22
+ 📚 **Knowledge Base** support
23
+ 🛠️ **Multi-tool Support** (weather, math, time, etc.)
24
+ 💬 **Professional Chat Interface**
25
+
26
+ ## 🚀 Quick Start Options
27
+
28
+ ### Option 1: Local Testing (Recommended First)
29
+
30
+ 1. **Open Terminal/Command Prompt** in the project folder:
31
+ ```bash
32
+ cd "C:\Users\j_car\Desktop\Production_RAG_Agent"
33
+ ```
34
+
35
+ 2. **Run the setup script**:
36
+ ```bash
37
+ python setup.py
38
+ ```
39
+
40
+ 3. **Set your Hugging Face token**:
41
+ ```bash
42
+ # Windows Command Prompt:
43
+ set HF_TOKEN=your_hugging_face_token_here
44
+
45
+ # Windows PowerShell:
46
+ $env:HF_TOKEN="your_hugging_face_token_here"
47
+ ```
48
+
49
+ 4. **Launch the application**:
50
+ ```bash
51
+ python app.py
52
+ ```
53
+
54
+ 5. **Open your browser** to `http://localhost:7860`
55
+
56
+ ### Option 2: Deploy to Hugging Face Spaces (Production)
57
+
58
+ 1. **Go to your space**: https://huggingface.co/spaces/JimmyBhoy/Production_RAG_Agent
59
+
60
+ 2. **Upload files** (drag and drop or use the file upload):
61
+ - `app.py`
62
+ - `config.py`
63
+ - `retriever.py`
64
+ - `tools.py`
65
+ - `requirements.txt`
66
+
67
+ 3. **Set environment variables** in Space Settings:
68
+ - `HF_TOKEN` = your_hugging_face_token
69
+
70
+ 4. **Upload documents** to `knowledge_base/` folder (optional)
71
+
72
+ 5. **Space will automatically build and deploy!**
73
+
74
+ ## 🔧 Configuration
75
+
76
+ ### Required Environment Variables
77
+ ```bash
78
+ HF_TOKEN=your_hugging_face_token_here
79
+ ```
80
+
81
+ ### Optional API Keys (for enhanced features)
82
+ ```bash
83
+ OPENWEATHER_API_KEY=your_weather_api_key
84
+ SERPER_API_KEY=your_search_api_key
85
+ ```
86
+
87
+ ## 📚 Adding Your Knowledge Base
88
+
89
+ 1. **Add documents** to the `knowledge_base/` folder:
90
+ - Text files (`.txt`, `.md`)
91
+ - PDFs (`.pdf`)
92
+ - Data files (`.json`, `.csv`)
93
+
94
+ 2. **Restart the application** - it will automatically:
95
+ - Process your documents
96
+ - Create embeddings
97
+ - Build a searchable vector database
98
+
99
+ ## 🎯 Usage Examples
100
+
101
+ ### Basic Queries
102
+ - "What is machine learning?"
103
+ - "Tell me about recent AI developments"
104
+ - "What's the weather in London?"
105
+
106
+ ### Knowledge Base Queries (after adding documents)
107
+ - "Summarize our company policy document"
108
+ - "What does the API documentation say about authentication?"
109
+
110
+ ### Complex Multi-step Queries
111
+ - "Search for recent news about electric vehicles, then analyze it"
112
+ - "What's the current weather and how might it affect operations?"
113
+
114
+ ## 🛠️ Customization
115
+
116
+ ### Adding More Tools
117
+ Edit `tools.py` to add custom functions:
118
+ ```python
119
+ @tool
120
+ def my_custom_tool(parameter: str) -> str:
121
+ """Your custom tool description"""
122
+ # Your logic here
123
+ return "Tool result"
124
+ ```
125
+
126
+ ### Modifying the Interface
127
+ Edit `app.py` to customize the Gradio interface:
128
+ - Change themes and styling
129
+ - Add more examples
130
+ - Modify the layout
131
+
132
+ ### Adjusting AI Behavior
133
+ Edit `config.py` to tune:
134
+ - Model selection
135
+ - Response length
136
+ - Search parameters
137
+ - Chunk sizes
138
+
139
+ ## 🔍 Troubleshooting
140
+
141
+ ### Common Issues & Solutions
142
+
143
+ 1. **Import Errors**:
144
+ ```bash
145
+ pip install -r requirements.txt
146
+ ```
147
+
148
+ 2. **No HF Token Error**:
149
+ - Get token from: https://huggingface.co/settings/tokens
150
+ - Set as environment variable
151
+
152
+ 3. **No Search Results**:
153
+ - Add documents to `knowledge_base/` folder
154
+ - Restart the application
155
+
156
+ 4. **Web Search Not Working**:
157
+ ```bash
158
+ pip install duckduckgo-search
159
+ ```
160
+
161
+ ### Performance Tips
162
+ - **Faster responses**: Reduce `TOP_K_RETRIEVAL` in config
163
+ - **Better accuracy**: Increase `CHUNK_SIZE` in config
164
+ - **More context**: Increase `MAX_TOKENS` in config
165
+
166
+ ## 📈 Production Deployment
167
+
168
+ ### For Hugging Face Spaces
169
+ - ✅ Automatic scaling
170
+ - ✅ HTTPS enabled
171
+ - ✅ Global CDN
172
+ - ✅ Easy sharing
173
+ - ✅ Version control
174
+
175
+ ### For Custom Deployment
176
+ - Use Docker for containerization
177
+ - Set up reverse proxy (nginx)
178
+ - Configure SSL certificates
179
+ - Monitor with logging
180
+
181
+ ## 🆘 Getting Help
182
+
183
+ 1. **Check the logs** in the `logs/` folder
184
+ 2. **Review error messages** in the terminal
185
+ 3. **Test individual components** with the setup script
186
+ 4. **Check Hugging Face Space discussions** for community help
187
+
188
+ ## 🎉 Next Steps
189
+
190
+ 1. **Test locally first** to understand the system
191
+ 2. **Add your own documents** to the knowledge base
192
+ 3. **Deploy to Hugging Face Spaces** for production use
193
+ 4. **Customize tools and interface** for your specific needs
194
+ 5. **Share with your team** and gather feedback
195
+
196
  ---
 
 
 
 
 
 
 
 
 
 
197
 
198
+ **You now have a complete, production-ready RAG system!** 🚀
199
+
200
+ This system combines the latest AI techniques with practical deployment options. Whether you use it locally or deploy to Hugging Face Spaces, you'll have a powerful AI assistant that can:
201
+
202
+ - Answer questions using multiple information sources
203
+ - Search the web for current information
204
+ - Process your own documents and data
205
+ - Perform complex multi-step reasoning
206
+ - Provide cited, accurate responses
207
+
208
+ **Happy RAGging!** 🤖✨
app.py ADDED
@@ -0,0 +1,115 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ **Model**: {self.config.MODEL_NAME}
3
+ **Available Tools**: {', '.join(tool_names)}
4
+ **Max Iterations**: {self.config.MAX_ITERATIONS}
5
+ **Status**: ✅ Ready
6
+
7
+ **Capabilities**:
8
+ - Web search for current information
9
+ - Document retrieval from knowledge base
10
+ - Weather information
11
+ - Multi-step reasoning and planning
12
+ - Source attribution
13
+ """
14
+
15
+ # Initialize the RAG agent
16
+ rag_agent = ProductionRAGAgent()
17
+
18
+ def chat_interface(message: str, history: List) -> str:
19
+ """Gradio chat interface function"""
20
+ return rag_agent.query(message, history)
21
+
22
+ def get_info() -> str:
23
+ """Get agent information"""
24
+ return rag_agent.get_agent_info()
25
+
26
+ # Create Gradio interface
27
+ def create_interface():
28
+ """Create the Gradio web interface"""
29
+
30
+ with gr.Blocks(
31
+ title="Production RAG Agent",
32
+ theme=gr.themes.Soft(),
33
+ css="""
34
+ .container { max-width: 1200px; margin: auto; }
35
+ .header { text-align: center; margin-bottom: 30px; }
36
+ .info-box { background: #f8f9fa; padding: 20px; border-radius: 10px; margin: 10px 0; }
37
+ """
38
+ ) as demo:
39
+
40
+ # Header
41
+ with gr.Row():
42
+ gr.HTML("""
43
+ <div class="header">
44
+ <h1>🚀 Production RAG Agent</h1>
45
+ <p>Advanced Retrieval-Augmented Generation with Multi-Tool Support</p>
46
+ </div>
47
+ """)
48
+
49
+ # Main interface
50
+ with gr.Row():
51
+ with gr.Column(scale=3):
52
+ # Chat interface
53
+ chatbot = gr.ChatInterface(
54
+ fn=chat_interface,
55
+ title="💬 Chat with RAG Agent",
56
+ description="Ask questions and get answers from multiple information sources",
57
+ examples=[
58
+ "What are the latest developments in AI?",
59
+ "Explain how transformer models work",
60
+ "What's the weather like today?",
61
+ "Compare different machine learning algorithms",
62
+ "Tell me about recent research in computer vision"
63
+ ],
64
+ retry_btn="🔄 Retry",
65
+ undo_btn="↶ Undo",
66
+ clear_btn="🗑️ Clear",
67
+ submit_btn="📤 Send"
68
+ )
69
+
70
+ with gr.Column(scale=1):
71
+ # Agent information panel
72
+ with gr.Group():
73
+ gr.Markdown("### 🤖 Agent Status")
74
+ info_display = gr.Markdown(value=get_info())
75
+ refresh_btn = gr.Button("🔄 Refresh Status", size="sm")
76
+ refresh_btn.click(fn=get_info, outputs=info_display)
77
+
78
+ # Usage tips
79
+ with gr.Group():
80
+ gr.Markdown("""
81
+ ### 💡 Usage Tips
82
+
83
+ **Best Practices:**
84
+ - Be specific in your questions
85
+ - Ask for sources when needed
86
+ - Use follow-up questions for clarification
87
+
88
+ **Available Features:**
89
+ - 🌐 Web search for current info
90
+ - 📚 Document knowledge retrieval
91
+ - 🌤️ Weather information
92
+ - 🧠 Multi-step reasoning
93
+ """)
94
+
95
+ # Footer
96
+ with gr.Row():
97
+ gr.HTML("""
98
+ <div style="text-align: center; margin-top: 30px; padding: 20px; background: #f1f3f4; border-radius: 10px;">
99
+ <p><strong>Production RAG Agent</strong> - Powered by Hugging Face 🤗</p>
100
+ <p>Built with smolagents, Gradio, and advanced retrieval techniques</p>
101
+ </div>
102
+ """)
103
+
104
+ return demo
105
+
106
+ # Create and launch the interface
107
+ if __name__ == "__main__":
108
+ demo = create_interface()
109
+ demo.queue()
110
+ demo.launch(
111
+ share=True,
112
+ server_name="0.0.0.0",
113
+ server_port=7860,
114
+ show_error=True
115
+ )
config.py ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from pathlib import Path
3
+
4
+ class RAGConfig:
5
+ """Configuration settings for the RAG Agent"""
6
+
7
+ # Model settings
8
+ MODEL_NAME = "microsoft/DialoGPT-medium" # Default model, can be changed
9
+ EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
10
+
11
+ # Agent settings
12
+ MAX_ITERATIONS = 5
13
+ TEMPERATURE = 0.7
14
+ MAX_TOKENS = 2048
15
+
16
+ # Retrieval settings
17
+ CHUNK_SIZE = 512
18
+ CHUNK_OVERLAP = 50
19
+ TOP_K_RETRIEVAL = 5
20
+ SIMILARITY_THRESHOLD = 0.7
21
+
22
+ # Paths
23
+ BASE_DIR = Path(__file__).parent
24
+ KNOWLEDGE_BASE_PATH = BASE_DIR / "knowledge_base"
25
+ VECTOR_STORE_PATH = BASE_DIR / "vector_store"
26
+ LOGS_PATH = BASE_DIR / "logs"
27
+
28
+ # API Keys (set as environment variables)
29
+ HF_TOKEN = os.getenv("HF_TOKEN")
30
+ OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
31
+ GROQ_API_KEY = os.getenv("GROQ_API_KEY")
32
+
33
+ # Web search settings
34
+ MAX_SEARCH_RESULTS = 5
35
+ SEARCH_TIMEOUT = 10
36
+
37
+ # Vector database settings
38
+ VECTOR_DB_TYPE = "faiss" # Options: faiss, chroma, pinecone
39
+ PERSIST_DIRECTORY = str(VECTOR_STORE_PATH)
40
+
41
+ # Gradio settings
42
+ GRADIO_SHARE = True
43
+ GRADIO_PORT = 7860
44
+ GRADIO_HOST = "0.0.0.0"
45
+
46
+ # Supported file types for knowledge base
47
+ SUPPORTED_EXTENSIONS = ['.txt', '.md', '.pdf', '.docx', '.json', '.csv']
48
+
49
+ # Advanced settings
50
+ USE_RERANKING = True
51
+ RERANKER_MODEL = "cross-encoder/ms-marco-MiniLM-L-2-v2"
52
+
53
+ # Logging
54
+ LOG_LEVEL = "INFO"
55
+ LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
56
+
57
+ def __init__(self):
58
+ """Initialize configuration and create necessary directories"""
59
+ self._create_directories()
60
+ self._validate_config()
61
+
62
+ def _create_directories(self):
63
+ """Create necessary directories if they don't exist"""
64
+ directories = [
65
+ self.KNOWLEDGE_BASE_PATH,
66
+ self.VECTOR_STORE_PATH,
67
+ self.LOGS_PATH
68
+ ]
69
+
70
+ for directory in directories:
71
+ directory.mkdir(parents=True, exist_ok=True)
72
+
73
+ def _validate_config(self):
74
+ """Validate configuration settings"""
75
+ # Check if HF token is available
76
+ if not self.HF_TOKEN:
77
+ print("⚠️ Warning: HF_TOKEN not set. Some features may not work.")
78
+
79
+ # Validate paths
80
+ if not self.BASE_DIR.exists():
81
+ raise ValueError(f"Base directory does not exist: {self.BASE_DIR}")
82
+
83
+ def get_model_config(self) -> dict:
84
+ """Get model configuration dictionary"""
85
+ return {
86
+ "model_name": self.MODEL_NAME,
87
+ "temperature": self.TEMPERATURE,
88
+ "max_tokens": self.MAX_TOKENS,
89
+ "token": self.HF_TOKEN
90
+ }
91
+
92
+ def get_retrieval_config(self) -> dict:
93
+ """Get retrieval configuration dictionary"""
94
+ return {
95
+ "chunk_size": self.CHUNK_SIZE,
96
+ "chunk_overlap": self.CHUNK_OVERLAP,
97
+ "top_k": self.TOP_K_RETRIEVAL,
98
+ "similarity_threshold": self.SIMILARITY_THRESHOLD,
99
+ "embedding_model": self.EMBEDDING_MODEL,
100
+ "reranker_model": self.RERANKER_MODEL if self.USE_RERANKING else None
101
+ }
102
+
103
+ def get_gradio_config(self) -> dict:
104
+ """Get Gradio configuration dictionary"""
105
+ return {
106
+ "share": self.GRADIO_SHARE,
107
+ "server_port": self.GRADIO_PORT,
108
+ "server_name": self.GRADIO_HOST
109
+ }
110
+
111
+ @classmethod
112
+ def from_env(cls):
113
+ """Create configuration from environment variables"""
114
+ instance = cls()
115
+
116
+ # Override with environment variables if available
117
+ env_mappings = {
118
+ "RAG_MODEL_NAME": "MODEL_NAME",
119
+ "RAG_MAX_ITERATIONS": "MAX_ITERATIONS",
120
+ "RAG_TEMPERATURE": "TEMPERATURE",
121
+ "RAG_CHUNK_SIZE": "CHUNK_SIZE",
122
+ "RAG_TOP_K": "TOP_K_RETRIEVAL"
123
+ }
124
+
125
+ for env_var, attr_name in env_mappings.items():
126
+ env_value = os.getenv(env_var)
127
+ if env_value:
128
+ # Convert to appropriate type
129
+ if attr_name in ["MAX_ITERATIONS", "CHUNK_SIZE", "TOP_K_RETRIEVAL"]:
130
+ setattr(instance, attr_name, int(env_value))
131
+ elif attr_name in ["TEMPERATURE"]:
132
+ setattr(instance, attr_name, float(env_value))
133
+ else:
134
+ setattr(instance, attr_name, env_value)
135
+
136
+ return instance
137
+
138
+ # Global config instance
139
+ config = RAGConfig.from_env()
requirements.txt ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Core dependencies
2
+ gradio>=4.0.0
3
+ smolagents>=0.1.0
4
+ transformers>=4.35.0
5
+ torch>=2.0.0
6
+ huggingface_hub>=0.20.0
7
+
8
+ # RAG and retrieval
9
+ langchain>=0.1.0
10
+ langchain-community>=0.0.10
11
+ langchain-huggingface>=0.0.3
12
+ sentence-transformers>=2.2.2
13
+ faiss-cpu>=1.7.4
14
+
15
+ # Document processing
16
+ pypdf>=3.17.0
17
+ python-docx>=1.1.0
18
+ openpyxl>=3.1.0
19
+
20
+ # Data processing
21
+ pandas>=2.0.0
22
+ numpy>=1.24.0
23
+
24
+ # Web search and APIs
25
+ duckduckgo-search>=3.9.0
26
+ requests>=2.31.0
27
+ beautifulsoup4>=4.12.0
28
+
29
+ # Time and timezone handling
30
+ pytz>=2023.3
31
+
32
+ # Logging and utilities
33
+ python-dotenv>=1.0.0
34
+ pydantic>=2.5.0
35
+
36
+ # Optional: Advanced features
37
+ # rank_bm25>=0.2.2
38
+ # chromadb>=0.4.0
39
+ # openai>=1.0.0
40
+
41
+ # Development (optional)
42
+ # black>=23.0.0
43
+ # pytest>=7.0.0
44
+ # jupyter>=1.0.0
retriever.py ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ def add_document(self, content: str, metadata: Dict[str, Any] = None) -> bool:
3
+ """Add a new document to the knowledge base"""
4
+ try:
5
+ # Create document
6
+ doc = Document(page_content=content, metadata=metadata or {})
7
+
8
+ # Add to documents list
9
+ self.documents.append(doc)
10
+
11
+ # Update vector store if it exists
12
+ if self.vector_store and self.embeddings:
13
+ chunks = self.text_splitter.split_documents([doc])
14
+ self.vector_store.add_documents(chunks)
15
+
16
+ # Save updated vector store
17
+ vector_store_path = self.config.VECTOR_STORE_PATH / "faiss_index"
18
+ self.vector_store.save_local(str(vector_store_path))
19
+
20
+ logger.info("Successfully added document to knowledge base")
21
+ return True
22
+
23
+ except Exception as e:
24
+ logger.error(f"Error adding document: {e}")
25
+ return False
26
+
27
+ def get_stats(self) -> Dict[str, Any]:
28
+ """Get statistics about the knowledge base"""
29
+ return {
30
+ 'total_documents': len(self.documents),
31
+ 'vector_store_available': self.vector_store is not None,
32
+ 'embedding_model': self.config.EMBEDDING_MODEL,
33
+ 'knowledge_base_path': str(self.knowledge_base_path),
34
+ 'supported_extensions': self.config.SUPPORTED_EXTENSIONS
35
+ }
36
+
37
+ @tool
38
+ def search_knowledge_base(query: str, search_type: str = "hybrid", max_results: int = 5) -> str:
39
+ """
40
+ Search the knowledge base for relevant information.
41
+
42
+ Args:
43
+ query: The search query
44
+ search_type: Type of search ('similarity', 'keyword', or 'hybrid')
45
+ max_results: Maximum number of results to return
46
+
47
+ Returns:
48
+ Formatted search results with sources
49
+ """
50
+ try:
51
+ # Initialize retriever (global instance would be better in production)
52
+ retriever = DocumentRetriever()
53
+
54
+ # Perform search based on type
55
+ if search_type == "similarity":
56
+ results = retriever.similarity_search(query, max_results)
57
+ elif search_type == "keyword":
58
+ results = retriever.keyword_search(query, max_results)
59
+ else: # hybrid
60
+ results = retriever.hybrid_search(query, max_results)
61
+
62
+ if not results:
63
+ return f"No relevant information found for query: '{query}'"
64
+
65
+ # Format results
66
+ formatted_results = []
67
+ for i, result in enumerate(results, 1):
68
+ content = result['content'][:500] + "..." if len(result['content']) > 500 else result['content']
69
+ source = result.get('source', 'Unknown source')
70
+ score = result.get('similarity_score', result.get('keyword_score', result.get('combined_score', 0)))
71
+
72
+ formatted_results.append(f"""
73
+ **Result {i}** (Score: {score:.3f})
74
+ Source: {source}
75
+ Content: {content}
76
+ """)
77
+
78
+ return "\n".join(formatted_results)
79
+
80
+ except Exception as e:
81
+ return f"Error searching knowledge base: {str(e)}"
82
+
83
+ def create_retriever_tool(retriever: DocumentRetriever):
84
+ """Create a tool function for the retriever"""
85
+
86
+ @tool
87
+ def retriever_tool(query: str) -> str:
88
+ """
89
+ Retrieve relevant information from the knowledge base.
90
+
91
+ Args:
92
+ query: The search query
93
+
94
+ Returns:
95
+ Relevant information from the knowledge base
96
+ """
97
+ try:
98
+ results = retriever.hybrid_search(query)
99
+
100
+ if not results:
101
+ return f"No relevant information found for: {query}"
102
+
103
+ # Format results for the agent
104
+ formatted_output = []
105
+ for result in results:
106
+ content = result['content'][:800] + "..." if len(result['content']) > 800 else result['content']
107
+ source = Path(result.get('source', 'unknown')).name
108
+
109
+ formatted_output.append(f"Source: {source}\n{content}")
110
+
111
+ return "\n\n---\n\n".join(formatted_output)
112
+
113
+ except Exception as e:
114
+ logger.error(f"Retriever tool error: {e}")
115
+ return f"Error retrieving information: {str(e)}"
116
+
117
+ return retriever_tool
setup.py ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Setup script for Production RAG Agent
4
+ Run this script to install dependencies and set up the environment
5
+ """
6
+
7
+ import subprocess
8
+ import sys
9
+ import os
10
+ from pathlib import Path
11
+
12
+ def run_command(command):
13
+ """Run a shell command and return the result"""
14
+ try:
15
+ result = subprocess.run(command, shell=True, check=True, capture_output=True, text=True)
16
+ return True, result.stdout
17
+ except subprocess.CalledProcessError as e:
18
+ return False, e.stderr
19
+
20
+ def main():
21
+ print("🚀 Setting up Production RAG Agent...")
22
+ print("=" * 50)
23
+
24
+ # Check Python version
25
+ print(f"✓ Python version: {sys.version}")
26
+
27
+ # Install dependencies
28
+ print("\n📦 Installing dependencies...")
29
+ success, output = run_command("pip install -r requirements.txt")
30
+
31
+ if success:
32
+ print("✓ Dependencies installed successfully")
33
+ else:
34
+ print(f"❌ Error installing dependencies: {output}")
35
+ return
36
+
37
+ # Check environment variables
38
+ print("\n🔑 Checking environment variables...")
39
+
40
+ required_vars = ["HF_TOKEN"]
41
+ optional_vars = ["OPENWEATHER_API_KEY", "SERPER_API_KEY"]
42
+
43
+ for var in required_vars:
44
+ if os.getenv(var):
45
+ print(f"✓ {var} is set")
46
+ else:
47
+ print(f"⚠️ {var} is not set (required for full functionality)")
48
+
49
+ for var in optional_vars:
50
+ if os.getenv(var):
51
+ print(f"✓ {var} is set")
52
+ else:
53
+ print(f"ℹ️ {var} is not set (optional)")
54
+
55
+ # Check directories
56
+ print("\n📁 Checking directories...")
57
+ base_dir = Path(__file__).parent
58
+
59
+ directories = [
60
+ base_dir / "knowledge_base",
61
+ base_dir / "vector_store",
62
+ base_dir / "logs"
63
+ ]
64
+
65
+ for directory in directories:
66
+ if directory.exists():
67
+ print(f"✓ {directory.name} directory exists")
68
+ else:
69
+ directory.mkdir(parents=True, exist_ok=True)
70
+ print(f"✓ Created {directory.name} directory")
71
+
72
+ # Test imports
73
+ print("\n🧪 Testing imports...")
74
+
75
+ test_imports = [
76
+ "gradio",
77
+ "transformers",
78
+ "torch",
79
+ "sentence_transformers",
80
+ "langchain"
81
+ ]
82
+
83
+ for module in test_imports:
84
+ try:
85
+ __import__(module)
86
+ print(f"✓ {module}")
87
+ except ImportError:
88
+ print(f"❌ {module} (install failed)")
89
+
90
+ print("\n" + "=" * 50)
91
+ print("🎉 Setup complete!")
92
+ print("\nNext steps:")
93
+ print("1. Set your HF_TOKEN environment variable")
94
+ print("2. Add documents to the knowledge_base folder")
95
+ print("3. Run: python app.py")
96
+ print("\nFor Hugging Face Spaces:")
97
+ print("1. Upload all files to your space")
98
+ print("2. Set HF_TOKEN in space settings")
99
+ print("3. Your space will automatically deploy!")
100
+
101
+ if __name__ == "__main__":
102
+ main()
tools.py ADDED
@@ -0,0 +1,219 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ @tool
3
+ def get_weather_info(location: str = "New York") -> str:
4
+ """
5
+ Get current weather information for a location.
6
+
7
+ Args:
8
+ location: Location to get weather for (default: New York)
9
+
10
+ Returns:
11
+ Current weather information
12
+ """
13
+ try:
14
+ # Using a free weather API (OpenWeatherMap)
15
+ api_key = os.getenv('OPENWEATHER_API_KEY')
16
+
17
+ if not api_key:
18
+ # Fallback to mock data for demo
19
+ return f"""
20
+ 🌤️ **Weather for {location}** (Demo Data)
21
+ Temperature: 22°C (72°F)
22
+ Condition: Partly Cloudy
23
+ Humidity: 65%
24
+ Wind: 8 mph NW
25
+
26
+ *Note: This is demo data. Set OPENWEATHER_API_KEY for real weather data.*
27
+ """
28
+
29
+ # Real API call
30
+ base_url = "https://api.openweathermap.org/data/2.5/weather"
31
+ params = {
32
+ 'q': location,
33
+ 'appid': api_key,
34
+ 'units': 'metric'
35
+ }
36
+
37
+ response = requests.get(base_url, params=params, timeout=5)
38
+ response.raise_for_status()
39
+
40
+ data = response.json()
41
+
42
+ weather_info = f"""
43
+ 🌤️ **Weather for {data['name']}, {data['sys']['country']}**
44
+ Temperature: {data['main']['temp']:.1f}°C ({data['main']['temp'] * 9/5 + 32:.1f}°F)
45
+ Feels like: {data['main']['feels_like']:.1f}°C
46
+ Condition: {data['weather'][0]['description'].title()}
47
+ Humidity: {data['main']['humidity']}%
48
+ Pressure: {data['main']['pressure']} hPa
49
+ Wind: {data['wind']['speed']} m/s
50
+
51
+ Last updated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
52
+ """
53
+
54
+ return weather_info
55
+
56
+ except requests.exceptions.RequestException as e:
57
+ logger.error(f"Weather API error: {e}")
58
+ return f"Unable to fetch weather data for {location}. API error: {str(e)}"
59
+ except Exception as e:
60
+ logger.error(f"Weather tool error: {e}")
61
+ return f"Error getting weather information: {str(e)}"
62
+
63
+ @tool
64
+ def calculate_math(expression: str) -> str:
65
+ """
66
+ Safely evaluate mathematical expressions.
67
+
68
+ Args:
69
+ expression: Mathematical expression to evaluate (e.g., "2 + 2 * 3")
70
+
71
+ Returns:
72
+ Result of the mathematical calculation
73
+ """
74
+ try:
75
+ # Simple evaluation for basic math operations
76
+ # In production, you might want to use a more sophisticated math parser
77
+
78
+ # Remove any potentially dangerous characters
79
+ allowed_chars = "0123456789+-*/()., "
80
+ cleaned_expression = ''.join(c for c in expression if c in allowed_chars)
81
+
82
+ if cleaned_expression != expression:
83
+ return f"Expression contains invalid characters. Cleaned: {cleaned_expression}"
84
+
85
+ # Evaluate the expression
86
+ result = eval(cleaned_expression)
87
+
88
+ return f"""
89
+ 🧮 **Mathematical Calculation**
90
+ Expression: {expression}
91
+ Result: {result}
92
+
93
+ Calculated at: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
94
+ """
95
+
96
+ except ZeroDivisionError:
97
+ return "Error: Division by zero is not allowed."
98
+ except Exception as e:
99
+ logger.error(f"Math calculation error: {e}")
100
+ return f"Error calculating expression '{expression}': {str(e)}"
101
+
102
+ @tool
103
+ def get_current_time(timezone: str = "UTC") -> str:
104
+ """
105
+ Get current time for a specific timezone.
106
+
107
+ Args:
108
+ timezone: Timezone (default: UTC)
109
+
110
+ Returns:
111
+ Current date and time information
112
+ """
113
+ try:
114
+ from datetime import datetime
115
+ try:
116
+ import pytz
117
+
118
+ if timezone == "UTC":
119
+ current_time = datetime.utcnow()
120
+ tz_info = "UTC"
121
+ else:
122
+ try:
123
+ tz = pytz.timezone(timezone)
124
+ current_time = datetime.now(tz)
125
+ tz_info = timezone
126
+ except:
127
+ # Fallback to UTC
128
+ current_time = datetime.utcnow()
129
+ tz_info = "UTC (fallback - invalid timezone provided)"
130
+ except ImportError:
131
+ # Fallback without pytz
132
+ current_time = datetime.now()
133
+ tz_info = "Local System Time"
134
+
135
+ time_info = f"""
136
+ 🕒 **Current Time Information**
137
+ Date: {current_time.strftime('%A, %B %d, %Y')}
138
+ Time: {current_time.strftime('%H:%M:%S')}
139
+ Timezone: {tz_info}
140
+ ISO Format: {current_time.isoformat()}
141
+ """
142
+
143
+ return time_info
144
+
145
+ except Exception as e:
146
+ logger.error(f"Time tool error: {e}")
147
+ return f"Error getting current time: {str(e)}"
148
+
149
+ @tool
150
+ def text_summarizer(text: str, max_sentences: int = 3) -> str:
151
+ """
152
+ Summarize a given text to specified number of sentences.
153
+
154
+ Args:
155
+ text: Text to summarize
156
+ max_sentences: Maximum number of sentences in summary (default: 3)
157
+
158
+ Returns:
159
+ Summarized text
160
+ """
161
+ try:
162
+ import re
163
+
164
+ if not text.strip():
165
+ return "No text provided to summarize."
166
+
167
+ # Simple extractive summarization
168
+ # Split into sentences
169
+ sentences = re.split(r'[.!?]+', text.strip())
170
+ sentences = [s.strip() for s in sentences if s.strip()]
171
+
172
+ if len(sentences) <= max_sentences:
173
+ return f"**Summary:** {text[:500]}..." if len(text) > 500 else text
174
+
175
+ # Simple scoring based on sentence length and position
176
+ scored_sentences = []
177
+ for i, sentence in enumerate(sentences):
178
+ # Prefer sentences of medium length and earlier position
179
+ length_score = min(len(sentence.split()) / 15, 1.0) # Normalize to words
180
+ position_score = 1.0 - (i / len(sentences)) # Earlier sentences get higher score
181
+
182
+ total_score = length_score * 0.7 + position_score * 0.3
183
+ scored_sentences.append((sentence, total_score))
184
+
185
+ # Sort by score and take top sentences
186
+ scored_sentences.sort(key=lambda x: x[1], reverse=True)
187
+ summary_sentences = [s[0] for s in scored_sentences[:max_sentences]]
188
+
189
+ # Maintain original order
190
+ final_summary = []
191
+ for sentence in sentences:
192
+ if sentence in summary_sentences:
193
+ final_summary.append(sentence)
194
+ if len(final_summary) == max_sentences:
195
+ break
196
+
197
+ summary = '. '.join(final_summary) + '.'
198
+
199
+ return f"""
200
+ 📄 **Text Summary**
201
+ Original length: {len(text)} characters
202
+ Summary length: {len(summary)} characters
203
+ Sentences: {max_sentences}
204
+
205
+ **Summary:** {summary}
206
+ """
207
+
208
+ except Exception as e:
209
+ logger.error(f"Text summarization error: {e}")
210
+ return f"Error summarizing text: {str(e)}"
211
+
212
+ # List of all available tools for easy import
213
+ AVAILABLE_TOOLS = [
214
+ web_search,
215
+ get_weather_info,
216
+ calculate_math,
217
+ get_current_time,
218
+ text_summarizer
219
+ ]