Add comprehensive logging and debug endpoints for tool calling

README_old.md

@@ -1,431 +0,0 @@
----
-title: Web3 Research Co-Pilot
-emoji: 🚀
-colorFrom: blue
-colorTo: purple
-sdk: docker
-sdk_version: "latest"
-app_file: app.py
-pinned: false
----
-
-# 🚀 Web3 Research Co-Pilot
-
-AI-powered cryptocurrency research assistant with real-time market data and professional analysis.
-
-**🌐 Live Demo**: https://archcoder-web3-copilot.hf.space
-
-## ⚡ Quick Start
-
-### 1. Get API Key
-Get your **Google Gemini API key** from [Google AI Studio](https://aistudio.google.com/)
-
-### 2. Run Locally
-```bash
-git clone https://github.com/Transcendental-Programmer/web3-research-agent.git
-cd web3-research-agent
-pip install -r requirements.txt
-
-# Create .env file
-echo "GEMINI_API_KEY=your_api_key_here" > .env
-
-# Run the app
-python app.py
-```
-
-### 3. Access
-Open http://localhost:7860 in your browser
-
-## 🔧 Optional Setup
-
-Add these to your `.env` for enhanced features:
-```bash
-COINGECKO_API_KEY=your_coingecko_key    # Higher rate limits
-ETHERSCAN_API_KEY=your_etherscan_key    # Blockchain data
-```
-
-## 🐳 Deploy to HuggingFace Spaces
-
-1. Fork this repo
-2. Create a new [HuggingFace Space](https://huggingface.co/spaces) with Docker SDK
-3. Set environment variables in Space settings
-4. Connect your repo and deploy
-
-## ✨ Features
-
-- **AI Analysis**: Google Gemini-powered insights
-- **Live Data**: CoinGecko, DeFiLlama, Etherscan APIs  
-- **Visualizations**: Interactive charts and graphs
-- **Professional UI**: Clean, responsive interface
-- **Multi-chain**: Ethereum, DeFi, Layer 2 support
-
-## 🎯 Usage Examples
-
-- "What's Bitcoin's price and market sentiment?"
-- "Show me top DeFi protocols by TVL"
-- "Analyze Ethereum gas prices"
-- "Compare BTC vs ETH performance"
-
-## � Troubleshooting
-
-**Can't access localhost:7860?**
-- Check if app is running: `ps aux | grep "python app.py"`
-- Try your local IP: `hostname -I` then http://YOUR_IP:7860
-- In cloud environments, use provided forwarded URL
-
-**API errors?**
-- Verify GEMINI_API_KEY in .env file
-- Check API key permissions
-
-## 📁 Project Structure
-
-```
-web3-research-agent/
-├── app.py              # Main FastAPI app
-├── requirements.txt    # Dependencies
-├── Dockerfile         # Container config
-├── test_suite.py      # Tests
-└── src/               # Source code
-    ├── research_agent.py
-    ├── visualizations.py
-    └── ...
-```
-
----
-
-**Built for the Web3 community** ❤️
-   pip install -r requirements.txt
-   ```
-
-3. **Set up environment variables**
-   
-   Create a `.env` file in the project root:
-   ```bash
-   cp .env.example .env
-   ```
-   
-   Edit `.env` with your API keys:
-   ```properties
-   # Required
-   GEMINI_API_KEY=your_gemini_api_key_here
-   
-   # Optional (for enhanced functionality)
-   COINGECKO_API_KEY=your_coingecko_api_key_here
-   ETHERSCAN_API_KEY=your_etherscan_api_key_here
-   ```
-
-4. **Run the application**
-   ```bash
-   python app.py
-   ```
-
-5. **Access the interface**
-   - **Local Development**: http://localhost:7860
-   - **Production/Cloud**: Check your hosting platform's port forwarding
-   - **Docker**: http://localhost:7860
-
-## 🚀 HuggingFace Spaces Deployment
-
-This project is configured for HuggingFace Spaces deployment with Docker SDK.
-
-### Quick Deploy to HF Spaces
-
-1. **Fork/Clone this repository**
-2. **Create a new HuggingFace Space**:
-   - Go to [HuggingFace Spaces](https://huggingface.co/spaces)
-   - Click "Create new Space"
-   - Choose "Docker" as the SDK
-   - Upload your repository files
-
-3. **Configure Environment Variables**:
-   In your Space settings, add:
-   ```
-   GEMINI_API_KEY=your_gemini_api_key
-   COINGECKO_API_KEY=your_coingecko_api_key  
-   ETHERSCAN_API_KEY=your_etherscan_api_key
-   ```
-
-4. **Deploy via Git**:
-   ```bash
-   git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
-   git push hf main
-   ```
-
-### HF Spaces Configuration
-
-The project includes a proper `README.md` header for HF Spaces:
-```yaml
-title: Web3 Research Co-Pilot
-emoji: 🚀
-colorFrom: blue
-colorTo: purple
-sdk: docker
-app_file: app.py
-```
-
-## 🐳 Docker Deployment
-
-### Local Docker
-
-```bash
-# Build the image
-docker build -t web3-research-copilot .
-
-# Run the container
-docker run -p 7860:7860 \
-  -e GEMINI_API_KEY=your_key_here \
-  -e COINGECKO_API_KEY=your_key_here \
-  -e ETHERSCAN_API_KEY=your_key_here \
-  web3-research-copilot
-```
-
-### HuggingFace Spaces
-
-This project is optimized for HuggingFace Spaces deployment:
-
-1. **Create a new Space**:
-   - Go to [HuggingFace Spaces](https://huggingface.co/spaces)
-   - Click "Create new Space"
-   - Choose "Docker" as the SDK
-   - Set visibility to "Public" or "Private"
-
-2. **Configure Environment Variables**:
-   In your Space settings, add:
-   ```
-   GEMINI_API_KEY=your_gemini_api_key
-   COINGECKO_API_KEY=your_coingecko_api_key  
-   ETHERSCAN_API_KEY=your_etherscan_api_key
-   ```
-
-3. **Deploy**:
-   ```bash
-   git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
-   git push hf main
-   ```
-
-**Live Example**: https://archcoder-web3-copilot.hf.space
-
-## 🔄 Development Setup
-
-### Adding HuggingFace Remote
-
-To deploy to your HuggingFace Space:
-
-```bash
-# Add HuggingFace remote
-git remote add hf https://huggingface.co/spaces/ArchCoder/web3-copilot
-
-# Or for your own space
-git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
-
-# Deploy to HF Spaces
-git push hf main
-```
-
-## 📊 Testing
-
-Run the comprehensive test suite:
-
-```bash
-python test_suite.py
-```
-
-Expected output:
-```
-🚀 Web3 Research Co-Pilot - Test Suite
-==================================================
-✅ All imports successful
-✅ Configuration validated
-✅ Visualizations working
-✅ Tools initialized
-✅ Service functional
-✅ API endpoints healthy
-✅ Performance acceptable
-
-Tests passed: 7/7
-Success rate: 100.0%
-🎉 All tests passed!
-```
-
-## 🏗️ Project Structure
-
-```
-web3-research-agent/
-├── app.py                      # Main FastAPI application
-├── requirements.txt            # Python dependencies
-├── pyproject.toml             # Project configuration
-├── uv.lock                    # Dependency lock file
-├── test_suite.py              # Comprehensive test suite
-├── .env.example               # Environment template
-├── src/                       # Source code
-│   ├── __init__.py
-│   ├── research_agent.py      # Main agent logic
-│   ├── enhanced_agent.py      # Enhanced agent features
-│   ├── config.py              # Configuration management
-│   ├── cache_manager.py       # Response caching
-│   ├── api_clients.py         # External API clients
-│   ├── defillama_client.py    # DeFiLlama integration
-│   ├── news_aggregator.py     # News and social data
-│   ├── portfolio_analyzer.py  # Portfolio analysis
-│   └── visualizations.py      # Chart generation
-└── README.md                  # This file
-```
-
-## 🔧 Configuration
-
-### Required Environment Variables
-
-| Variable | Description | Required |
-|----------|-------------|----------|
-| `GEMINI_API_KEY` | Google Gemini API key for AI analysis | ✅ |
-
-### Optional Environment Variables
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `COINGECKO_API_KEY` | CoinGecko API key for enhanced rate limits | None |
-| `ETHERSCAN_API_KEY` | Etherscan API key for blockchain data | None |
-| `AIRAA_WEBHOOK_URL` | AIRAA integration webhook URL | None |
-| `AIRAA_API_KEY` | AIRAA API authentication key | None |
-
-### Getting API Keys
-
-1. **Google Gemini API**: 
-   - Go to [Google AI Studio](https://aistudio.google.com/)
-   - Create a new API key
-   - Copy the key to your `.env` file
-
-2. **CoinGecko API** (optional):
-   - Sign up at [CoinGecko](https://www.coingecko.com/api)
-   - Get your free API key
-   - Provides higher rate limits
-
-3. **Etherscan API** (optional):
-   - Register at [Etherscan](https://etherscan.io/apis)
-   - Create a free API key
-   - Enables blockchain data queries
-
-## 🎯 Usage Examples
-
-### Market Analysis
-```
-"Analyze Bitcoin price trends and institutional adoption patterns"
-```
-
-### DeFi Research
-```
-"Compare top DeFi protocols by TVL, yield, and risk metrics"
-```
-
-### Layer 2 Analysis
-```
-"Evaluate Ethereum Layer 2 scaling solutions and adoption metrics"
-```
-
-### Yield Optimization
-```
-"Identify optimal yield farming strategies across multiple chains"
-```
-
-## 🌐 API Endpoints
-
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/` | GET | Web interface |
-| `/status` | GET | System status and configuration |
-| `/query` | POST | Process research queries |
-| `/health` | GET | Health check |
-
-### Query API Example
-
-```bash
-curl -X POST "http://localhost:7860/query" \
-  -H "Content-Type: application/json" \
-  -d '{"query": "What is the current Bitcoin price?"}'
-```
-
-## 🐛 Troubleshooting
-
-### Port Access Issues
-
-**Problem**: Can't access the app on http://localhost:7860 or http://0.0.0.0:7860
-
-**Solutions**:
-
-1. **Check if the app is running**:
-   ```bash
-   ps aux | grep "python app.py"
-   ```
-
-2. **Verify port binding**:
-   ```bash
-   netstat -tlnp | grep :7860
-   ```
-
-3. **For Development Environments (VS Code, etc.)**:
-   - Look for port forwarding notifications
-   - Check your IDE's "Ports" or "Forwarded Ports" tab
-   - Use the forwarded URL provided by your development environment
-
-4. **For Cloud/Remote Environments**:
-   - The app binds to `0.0.0.0:7860` for external access
-   - Use your platform's provided URL (not localhost)
-   - Check firewall rules if on a VPS/server
-
-5. **Local Network Access**:
-   ```bash
-   # Find your local IP
-   hostname -I
-   # Access via: http://YOUR_IP:7860
-   ```
-
-### Common Issues
-
-1. **"GEMINI_API_KEY not configured"**
-   - Ensure you've set the API key in your `.env` file
-   - Verify the key is valid and has proper permissions
-
-2. **"Connection refused" on port 7860**
-   - Check if another process is using port 7860: `lsof -i :7860`
-   - Ensure the app started successfully: `python app.py`
-
-3. **Import errors**
-   - Ensure all dependencies are installed: `pip install -r requirements.txt`
-   - Check Python version compatibility (3.11+)
-
-4. **Slow visualization loading**
-   - Check your internet connection
-   - API rate limits may be affecting data retrieval
-
-### Getting Help
-
-- Check the test suite: `python test_suite.py`
-- Review logs in the terminal output
-- Verify API keys are configured correctly
-- Open an issue on GitHub with error details
-
-## 🤝 Contributing
-
-1. Fork the repository
-2. Create a feature branch
-3. Make your changes
-4. Add tests for new functionality
-5. Run the test suite
-6. Submit a pull request
-
-## 📄 License
-
-This project is licensed under the MIT License - see the LICENSE file for details.
-
-## 🙏 Acknowledgments
-
-- **Google Gemini** for AI capabilities
-- **CoinGecko** for comprehensive market data
-- **DeFiLlama** for DeFi protocol analytics
-- **Etherscan** for blockchain data
-- **FastAPI** for the web framework
-- **Plotly** for interactive visualizations
-
----
-
-**Built with ❤️ for the Web3 community**

app.py

@@ -71,9 +71,10 @@ class Web3CoPilotService:
     
     async def process_query(self, query: str) -> QueryResponse:
         """Process research query with visualizations"""
-        logger.info(f"Processing query: {query[:100]}...")
+        logger.info(f"🔍 Processing query: {query[:100]}...")
         
         if not query.strip():
+            logger.warning("⚠️ Empty query received")
             return QueryResponse(
                 success=False, 
                 response="Please provide a research query.", 
@@ -82,6 +83,7 @@ class Web3CoPilotService:
         
         try:
             if not self.enabled:
+                logger.info("ℹ️ Processing in limited mode (no GEMINI_API_KEY)")
                 response = """**Research Assistant - Limited Mode**
 
 API access available for basic cryptocurrency data:
@@ -92,28 +94,35 @@ API access available for basic cryptocurrency data:
 Configure GEMINI_API_KEY environment variable for full AI analysis."""
                 return QueryResponse(success=True, response=response, sources=["System"])
             
-            logger.info("Processing with AI research agent...")
+            logger.info("🤖 Processing with AI research agent...")
+            logger.info(f"🛠️ Available tools: {[tool.name for tool in self.agent.tools] if self.agent else []}")
+            
             result = await self.agent.research_query(query)
+            logger.info(f"🔄 Agent research completed: success={result.get('success')}")
             
             if result.get("success"):
                 response = result.get("result", "No analysis generated")
                 sources = result.get("sources", [])
                 metadata = result.get("metadata", {})
                 
+                logger.info(f"📊 Response generated: {len(response)} chars, {len(sources)} sources")
+                
                 # Generate visualizations if relevant data is available
                 visualizations = []
                 if metadata:
+                    logger.info("📈 Checking for visualization data...")
                     vis_html = await self._generate_visualizations(metadata, query)
                     if vis_html:
                         visualizations.append(vis_html)
+                        logger.info("✅ Visualization generated")
                 
                 # Send to AIRAA if enabled
                 if self.airaa and self.airaa.enabled:
                     try:
                         await self.airaa.send_research_data(query, response)
-                        logger.info("Data sent to AIRAA")
+                        logger.info("📤 Data sent to AIRAA")
                     except Exception as e:
-                        logger.warning(f"AIRAA integration failed: {e}")
+                        logger.warning(f"⚠️ AIRAA integration failed: {e}")
                 
                 return QueryResponse(
                     success=True, 
@@ -124,11 +133,11 @@ Configure GEMINI_API_KEY environment variable for full AI analysis."""
                 )
             else:
                 error_msg = result.get("error", "Research analysis failed")
-                logger.error(f"Research failed: {error_msg}")
+                logger.error(f"❌ Research failed: {error_msg}")
                 return QueryResponse(success=False, response=error_msg, error=error_msg)
             
         except Exception as e:
-            logger.error(f"Query processing error: {e}")
+            logger.error(f"💥 Query processing error: {e}", exc_info=True)
             error_msg = f"Processing error: {str(e)}"
             return QueryResponse(success=False, response=error_msg, error=error_msg)
     
@@ -677,34 +686,60 @@ async def get_homepage(request: Request):
                 const sendBtn = document.getElementById('sendBtn');
                 const query = input.value.trim();
 
-                if (!query) return;
+                if (!query) {
+                    console.log('❌ Empty query, not sending');
+                    return;
+                }
 
+                console.log('📤 Sending query:', query);
                 addMessage('user', query);
                 input.value = '';
 
+                // Update button state
                 sendBtn.disabled = true;
                 sendBtn.innerHTML = '<span class="loading">Processing</span>';
 
                 try {
+                    console.log('🔄 Making API request...');
+                    const requestStart = Date.now();
+                    
                     const response = await fetch('/query', {
                         method: 'POST',
                         headers: { 'Content-Type': 'application/json' },
                         body: JSON.stringify({ query, chat_history: chatHistory })
                     });
 
+                    const requestTime = Date.now() - requestStart;
+                    console.log(`⏱️ Request completed in ${requestTime}ms`);
+
+                    if (!response.ok) {
+                        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+                    }
+
                     const result = await response.json();
+                    console.log('📥 Response received:', {
+                        success: result.success,
+                        responseLength: result.response?.length || 0,
+                        sources: result.sources?.length || 0,
+                        visualizations: result.visualizations?.length || 0
+                    });
 
                     if (result.success) {
                         addMessage('assistant', result.response, result.sources, result.visualizations);
+                        console.log('✅ Message added successfully');
                     } else {
-                        addMessage('assistant', result.response || 'Analysis failed. Please try again.');
+                        console.error('❌ Query failed:', result.error);
+                        addMessage('assistant', result.response || 'Analysis failed. Please try again.', [], []);
                     }
                 } catch (error) {
-                    addMessage('assistant', 'Connection error. Please check your network and try again.');
+                    console.error('💥 Request error:', error);
+                    addMessage('assistant', `Connection error: ${error.message}. Please check your network and try again.`);
                 } finally {
+                    // Reset button state
                     sendBtn.disabled = false;
                     sendBtn.innerHTML = 'Research';
                     input.focus();
+                    console.log('🔄 Button state reset');
                 }
             }
 
@@ -790,8 +825,40 @@ async def get_status():
 
 @app.post("/query", response_model=QueryResponse)
 async def process_query(request: QueryRequest):
-    """Process research query"""
-    return await service.process_query(request.query)
+    """Process research query with detailed logging"""
+    # Log incoming request
+    logger.info(f"📥 Query received: {request.query[:100]}...")
+    logger.info(f"📊 Chat history length: {len(request.chat_history) if request.chat_history else 0}")
+    
+    start_time = datetime.now()
+    
+    try:
+        # Process the query
+        result = await service.process_query(request.query)
+        
+        # Log result
+        processing_time = (datetime.now() - start_time).total_seconds()
+        logger.info(f"✅ Query processed in {processing_time:.2f}s - Success: {result.success}")
+        
+        if result.success:
+            logger.info(f"📤 Response length: {len(result.response)} chars")
+            logger.info(f"🔗 Sources: {result.sources}")
+            if result.visualizations:
+                logger.info(f"📈 Visualizations: {len(result.visualizations)} charts")
+        else:
+            logger.error(f"❌ Query failed: {result.error}")
+        
+        return result
+        
+    except Exception as e:
+        processing_time = (datetime.now() - start_time).total_seconds()
+        logger.error(f"💥 Query processing exception after {processing_time:.2f}s: {e}")
+        
+        return QueryResponse(
+            success=False,
+            response=f"System error: {str(e)}",
+            error=str(e)
+        )
 
 @app.get("/health")
 async def health_check():
@@ -803,6 +870,54 @@ async def health_check():
         "version": "2.0.0"
     }
 
+@app.get("/debug/tools")
+async def debug_tools():
+    """Debug endpoint to test tool availability and functionality"""
+    try:
+        if not service.enabled or not service.agent:
+            return {
+                "success": False,
+                "error": "AI agent not enabled",
+                "tools_available": False,
+                "gemini_configured": bool(config.GEMINI_API_KEY)
+            }
+        
+        tools_info = []
+        for tool in service.agent.tools:
+            tools_info.append({
+                "name": tool.name,
+                "description": getattr(tool, 'description', 'No description'),
+                "enabled": getattr(tool, 'enabled', True)
+            })
+        
+        # Test a simple API call
+        test_result = None
+        try:
+            test_result = await service.process_query("What is the current Bitcoin price?")
+        except Exception as e:
+            test_result = {"error": str(e)}
+        
+        return {
+            "success": True,
+            "tools_count": len(service.agent.tools),
+            "tools_info": tools_info,
+            "test_query_result": {
+                "success": test_result.success if hasattr(test_result, 'success') else False,
+                "response_length": len(test_result.response) if hasattr(test_result, 'response') else 0,
+                "sources": test_result.sources if hasattr(test_result, 'sources') else [],
+                "error": test_result.error if hasattr(test_result, 'error') else None
+            },
+            "gemini_configured": bool(config.GEMINI_API_KEY),
+            "timestamp": datetime.now().isoformat()
+        }
+    except Exception as e:
+        logger.error(f"Debug tools error: {e}")
+        return {
+            "success": False,
+            "error": str(e),
+            "timestamp": datetime.now().isoformat()
+        }
+
 if __name__ == "__main__":
     import uvicorn
     logger.info("Starting Web3 Research Co-Pilot...")

src/tools/base_tool.py

@@ -75,6 +75,28 @@ class BaseWeb3Tool(BaseTool, ABC):
     def _run(self, query: str, filters: Optional[Dict[str, Any]] = None) -> str:
         return asyncio.run(self._arun(query, filters))
     
+    def run(self, tool_input: Dict[str, Any]) -> str:
+        """Handle both direct calls and LangChain tool calling format"""
+        if isinstance(tool_input, dict):
+            query = tool_input.get('query', '')
+            filters = tool_input.get('filters')
+        else:
+            # Fallback for string input
+            query = str(tool_input)
+            filters = None
+        return self._run(query, filters)
+    
+    async def arun(self, tool_input: Dict[str, Any]) -> str:
+        """Async version of run method"""
+        if isinstance(tool_input, dict):
+            query = tool_input.get('query', '')
+            filters = tool_input.get('filters')
+        else:
+            # Fallback for string input
+            query = str(tool_input)
+            filters = None
+        return await self._arun(query, filters)
+
     @abstractmethod
     async def _arun(self, query: str, filters: Optional[Dict[str, Any]] = None) -> str:
         pass