Add read_webpage tool for reliable page content extraction

DuckDuckGo snippets cause hallucination and inconsistent answers.
read_webpage fetches the full page text, enabling agent to read:
- Wikipedia articles directly
- Sports stats pages (Baseball Reference)
- Scientific papers and LibreTexts exercises
- News articles with specific award numbers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

app.py

@@ -1,10 +1,12 @@
 import os
+import re
 import gradio as gr
 import requests
 import pandas as pd
 from langchain_openai import ChatOpenAI
 from langchain_community.tools import DuckDuckGoSearchRun
 from langchain_experimental.tools import PythonREPLTool
+from langchain_core.tools import tool
 from langchain_core.messages import SystemMessage, HumanMessage, ToolMessage
 
 # --- Constants ---
@@ -12,26 +14,44 @@ DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
 
 SYSTEM_PROMPT = """You are a general AI assistant. Answer GAIA benchmark questions accurately.
 
-Tool usage:
-- duckduckgo_search: look up facts, names, numbers, events. Use specific queries. Try multiple searches with different terms.
-- Python_REPL: calculations, data analysis, string manipulation. ALWAYS end with print() to output results.
-
-Special cases:
-- Reversed/encoded text: decode it yourself without tools.
-- YouTube video questions: search the video URL or title + key terms from the question.
-- Questions mentioning attached files/Excel/images: file is NOT available. Use web search to find the answer.
-- Wikipedia questions: search "site:en.wikipedia.org [topic]" for precise results.
-- GAIA benchmark questions: search the exact question phrasing to find known solutions.
-
-When you have your final answer, output ONLY this line:
+Available tools:
+- duckduckgo_search: find URLs and snippets. Use for initial discovery.
+- read_webpage: read the FULL content of a specific URL. Use this after finding a relevant URL via search to get precise details.
+- Python_REPL: calculations, data analysis. ALWAYS use print() to show results.
+
+Research strategy:
+1. Search to find the relevant page/source.
+2. Use read_webpage on the specific URL to read actual content — do not rely only on search snippets.
+3. For Wikipedia questions: read the Wikipedia page directly.
+4. For sports stats: read Baseball Reference, Sports Reference, or similar stat pages.
+5. For reversed/encoded text: decode yourself, no tools needed.
+6. For YouTube videos: search for the video title + key question terms.
+7. For questions mentioning attached files: file is unavailable — search the web for the answer.
+
+When you have your final answer, output ONLY:
 FINAL ANSWER: [your answer]
 
-STRICT format rules (answers scored by exact match):
-- Numbers: digits only, no currency symbols, no commas, no units unless explicitly requested
-- Country names: full name (e.g. "Colombia" not "COL" or "COL (Colombia)")
+STRICT format rules (exact match scoring):
+- Numbers: digits only, no $, no commas, no units unless asked
+- Country names: full name (e.g. "Malta" not "MLT" or "Malta (MLT)")
 - Strings: no surrounding quotes, no trailing punctuation, no articles (a/an/the)
 - Lists: comma-separated, no spaces after commas (e.g. "a,b,c" not "a, b, c")
-- If you cannot find the answer, give your best guess — never return placeholder text like [answer]"""
+- Best guess required — never output "No answer found" or placeholder text"""
+
+
+@tool
+def read_webpage(url: str) -> str:
+    """Read the full text content of a webpage. Use after finding a relevant URL via search to get precise information."""
+    try:
+        headers = {"User-Agent": "Mozilla/5.0 (compatible; research-agent/1.0)"}
+        resp = requests.get(url, headers=headers, timeout=15, allow_redirects=True)
+        if resp.status_code != 200:
+            return f"Could not fetch page: HTTP {resp.status_code}"
+        text = re.sub(r"<[^>]+>", " ", resp.text)
+        text = re.sub(r"\s+", " ", text).strip()
+        return text[:6000]
+    except Exception as e:
+        return f"Error reading page: {e}"
 
 
 class BasicAgent:
@@ -39,6 +59,7 @@ class BasicAgent:
         self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
         self.tools = [
             DuckDuckGoSearchRun(),
+            read_webpage,
             PythonREPLTool(),
         ]
         self.tools_map = {t.name: t for t in self.tools}