Update

PLAN.md

@@ -1,21 +1,254 @@
-# Implementation Plan
+# Implementation Plan - Stage 4: MVP - Real Integration
 
-**Date:** [YYYY-MM-DD]
-**Dev Record:** [link to dev/dev_YYMMDD_##_concise_title.md]
-**Status:** [Planning | In Progress | Completed]
+**Date:** 2026-01-02
+**Dev Record:** dev/dev_260103_15_stage4_mvp_integration.md
+**Status:** Planning
 
 ## Objective
 
-[Clear goal statement]
+Fix integration issues to achieve MVP: Agent answers real GAIA questions using real APIs (Gemini, Claude, Tavily), even if accuracy is low. Target: Get from 0/20 → at least 5/20 questions correct.
+
+## Current Problem Analysis
+
+**HuggingFace Result:** 0/20 correct, all answers = "Unable to answer: No evidence collected"
+
+**Root Causes Identified:**
+
+1. **API Keys Issue:** Environment variables may not be set in HuggingFace Space
+2. **Silent Failures:** LLM function calling fails but errors are swallowed
+3. **No Evidence Collection:** Tool execution broken, evidence list stays empty
+4. **Poor Error Visibility:** User sees "Unable to answer" with no diagnostic info
 
 ## Steps
 
-[Implementation steps]
+### 1. Add Comprehensive Debug Logging
+
+**File:** `src/agent/graph.py`
+
+**Changes:**
+
+- Add detailed logging in each node (plan/execute/answer)
+- Log LLM responses, tool calls, evidence collected
+- Log errors with full stack traces
+- Add state inspection logging
+
+**Purpose:** Understand where exactly the integration fails
+
+### 2. Improve Error Messages
+
+**File:** `src/agent/graph.py` - `answer_node`
+
+**Current:**
+
+```python
+state["answer"] = "Unable to answer: No evidence collected"
+```
+
+**New:**
+
+```python
+if not evidence:
+    error_summary = "; ".join(state["errors"]) if state["errors"] else "No errors logged"
+    state["answer"] = f"ERROR: No evidence. Errors: {error_summary}"
+```
+
+**Purpose:** Show WHY it failed (API key missing? Tool failed? LLM failed?)
+
+### 3. Add Graceful Degradation in LLM Client
+
+**File:** `src/agent/llm_client.py`
+
+**Changes:**
+
+- Better exception handling with specific error types
+- Distinguish between: API key missing, rate limit, network error, API error
+- Log which provider failed and why
+- Add fallback messages instead of re-raising
+
+**Example:**
+
+```python
+try:
+    return plan_question_gemini(...)
+except ValueError as e:
+    if "GOOGLE_API_KEY" in str(e):
+        logger.error("Gemini API key not set")
+        # Try Claude fallback
+except Exception as e:
+    logger.error(f"Gemini failed: {type(e).__name__}: {e}")
+```
+
+### 4. Add API Key Validation Check
+
+**File:** `src/agent/graph.py` - Add validation before execution
+
+**New function:**
+
+```python
+def validate_environment() -> List[str]:
+    """Check which API keys are available."""
+    missing = []
+    if not os.getenv("GOOGLE_API_KEY"):
+        missing.append("GOOGLE_API_KEY (Gemini)")
+    if not os.getenv("ANTHROPIC_API_KEY"):
+        missing.append("ANTHROPIC_API_KEY (Claude)")
+    if not os.getenv("TAVILY_API_KEY"):
+        missing.append("TAVILY_API_KEY (Search)")
+    return missing
+```
+
+Call at agent initialization to warn early.
+
+### 5. Fix Tool Execution Error Handling
+
+**File:** `src/agent/graph.py` - `execute_node`
+
+**Issue:** If LLM function calling returns empty tool_calls, execution continues silently
+
+**Fix:**
+
+```python
+tool_calls = select_tools_with_function_calling(...)
+
+if not tool_calls:
+    logger.error("LLM returned no tool calls - check LLM integration")
+    state["errors"].append("Tool selection failed: LLM returned no tools")
+    return state  # Early return instead of continuing
+```
+
+### 6. Add Fallback to Direct Tool Execution (MVP Hack)
+
+**File:** `src/agent/graph.py` - `execute_node`
+
+**If LLM function calling fails completely, use rule-based fallback:**
+
+```python
+# If LLM function calling fails, try simple heuristics
+if not tool_calls and "search" in question.lower():
+    logger.warning("LLM tool selection failed, using fallback: search")
+    tool_calls = [{"tool": "search", "params": {"query": question}}]
+```
+
+**Purpose:** Get SOMETHING working even if LLM fails (this is MVP - quality doesn't matter)
+
+### 7. Test with Mock-Free Integration Tests
+
+**File:** `test/test_integration_real_apis.py` (NEW)
+
+**Tests:**
+
+- Test with real GOOGLE_API_KEY (if available)
+- Test with real ANTHROPIC_API_KEY (if available)
+- Test with real TAVILY_API_KEY (if available)
+- Skip tests if API keys not available (not fail)
+
+**Purpose:** Validate real API integration works locally before deploying
+
+### 8. Add Gradio UI Error Display
+
+**File:** `app.py`
+
+**Current:** Shows only answer
+
+**New:** Show diagnostic info in UI
+
+```python
+def answer_question(question):
+    agent = GAIAAgent()
+    answer = agent(question)
+
+    # Show errors if present
+    if hasattr(agent, 'last_state'):
+        errors = agent.last_state.get('errors', [])
+        if errors:
+            return f"{answer}\n\nDIAGNOSTICS:\n" + "\n".join(errors)
+
+    return answer
+```
+
+### 9. Update HuggingFace Space Configuration
+
+**Action Items:**
+
+1. Add environment variables in Space Settings:
+   - `GOOGLE_API_KEY` (for Gemini - primary)
+   - `ANTHROPIC_API_KEY` (for Claude - fallback)
+   - `TAVILY_API_KEY` (for web search)
+2. Set to "Public" visibility if needed
+3. Verify build succeeds after adding keys
+
+### 10. Deploy and Test Real Questions
+
+**Actions:**
+
+- Commit all changes
+- Push to HuggingFace Spaces
+- Wait for build
+- Test with 5 simple GAIA questions manually
+- Verify at least 1-2 work (doesn't need to be correct, just collect evidence)
 
 ## Files to Modify
 
-[List of files]
+1. `src/agent/graph.py` - Add logging, improve error handling, add validation
+2. `src/agent/llm_client.py` - Better exception handling, specific error types
+3. `app.py` - Show diagnostics in UI
+4. `test/test_integration_real_apis.py` - NEW - Real API integration tests
+5. `README.md` - Document required API keys
 
 ## Success Criteria
 
-[Completion criteria]
+**MVP Definition:** Agent runs real APIs and collects evidence (even if answers wrong)
+
+- [ ] Agent attempts real LLM calls (Gemini or Claude)
+- [ ] Agent attempts real tool calls (Tavily search)
+- [ ] Evidence is collected (not empty list)
+- [ ] Errors are visible and actionable
+- [ ] At least 1/20 GAIA questions collects evidence (even if answer wrong)
+- [ ] Target: 5/20 questions answered (quality doesn't matter, just not "Unable to answer")
+
+**Non-Goals for MVP:**
+
+- ❌ High accuracy (not needed for MVP)
+- ❌ Optimal tool selection (can be random/fallback)
+- ❌ Perfect error recovery (basic is enough)
+- ❌ Performance optimization (Stage 5)
+
+## Debug Strategy
+
+**If still failing after fixes:**
+
+1. **Check logs in HuggingFace Space container logs**
+2. **Add print statements** (not just logger) to see output
+3. **Test locally first** with real API keys
+4. **Simplify to single tool** (just search, no LLM function calling)
+5. **Hardcode a simple question** to verify basic flow works
+
+## Risk Analysis
+
+**High Risk Issues:**
+
+1. **Gemini function calling API complex** - May fail even with correct implementation
+   - **Mitigation:** Claude fallback + hardcoded tool selection fallback
+2. **API keys not propagating** to container
+   - **Mitigation:** Add validation at startup, fail fast with clear message
+3. **Tool execution fails silently**
+   - **Mitigation:** Explicit error logging, return partial results
+
+**Medium Risk Issues:**
+
+1. **Rate limits** on free tier APIs
+   - **Mitigation:** Retry with exponential backoff (already in tools)
+2. **Network timeouts** in HuggingFace environment
+   - **Mitigation:** Increase timeout settings, add timeout logging
+
+## Next Stage Preview
+
+**Stage 5: Production Quality (After MVP Works)**
+
+- Performance optimization (reduce latency)
+- Accuracy improvements (15/20 target)
+- GAIA benchmark validation
+- Cost optimization
+- Caching strategies
+
+**But first:** Get to MVP (5/20 working, real APIs connected)

dev/dev_260103_16_huggingface_integration.md

@@ -154,12 +154,14 @@ All tests passing with new 3-tier fallback architecture.
 - **Status:** Agent is functional and operational
 
 **What worked:**
+
 - ✅ Question 1: "How many studio albums were published by Mercedes Sosa between 2000 and 2009?" → Answer: "3" (CORRECT)
 - ✅ HuggingFace LLM (Qwen 2.5 72B) successfully used for planning and tool selection
 - ✅ Web search tool executed successfully
 - ✅ Evidence collection and answer synthesis working
 
 **What failed:**
+
 - ❌ Question 2: YouTube video analysis (vision tool) - "Tool vision failed: Exception: Vision analysis failed - Gemini and Claude both failed"
 - Issue: Vision tool requires multimodal LLM access (quota-limited or needs configuration)