Punit1 commited on
Commit
52e9d16
Β·
0 Parent(s):

Switch to HF Space README

Browse files
.gitignore ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ src/__pycache__/
2
+ __pycache__/
3
+ *.pyc
4
+ *.pyo
5
+ *.pyd
6
+ .Python
7
+ venv/
8
+ env/
9
+ ENV/
10
+ .venv
11
+ *.sqlite
12
+ *.sqlite-shm
13
+ *.sqlite-wal
14
+ .env
15
+ .DS_Store
16
+ *.log
17
+ .idea/
18
+ .vscode/
19
+ *.swp
20
+ *.swo
21
+ *~
22
+ .ipynb_checkpoints/
.streamlit/config.toml ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [theme]
2
+ primaryColor="#FF4B4B"
3
+ backgroundColor="#FFFFFF"
4
+ secondaryBackgroundColor="#F0F2F6"
5
+ textColor="#262730"
6
+ font="sans serif"
7
+
8
+ [server]
9
+ headless = true
10
+ port = 7860
11
+ enableCORS = false
12
+ enableXsrfProtection = false
DEPLOY_TO_HF.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Quick Deployment to Hugging Face Spaces
2
+
3
+ ## TL;DR - Fast Deployment Steps
4
+
5
+ ### 1. Get API Keys
6
+ - Groq: https://console.groq.com/
7
+ - Tavily: https://tavily.com/
8
+
9
+ ### 2. Create HF Space
10
+ 1. Go to: https://huggingface.co/new-space
11
+ 2. Choose: **Streamlit** SDK
12
+ 3. Name it: `research-agent`
13
+ 4. Create Space
14
+
15
+ ### 3. Upload Files
16
+
17
+ **Using Web Interface:**
18
+ - Upload: `main.py`, `requirements.txt`, entire `src/` folder, `.streamlit/` folder
19
+ - **Rename** `HF_README.md` to `README.md` before uploading
20
+
21
+ **Using Git:**
22
+ ```bash
23
+ git init
24
+ git add .
25
+ git commit -m "Deploy to HF Spaces"
26
+ git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
27
+ git push hf main
28
+ ```
29
+
30
+ ### 4. Add Secrets
31
+ In your Space β†’ Settings β†’ Repository secrets:
32
+ - `GROQ_API_KEY` = your Groq API key
33
+ - `TAVILY_API_KEY` = your Tavily API key
34
+
35
+ ### 5. Done!
36
+ Your app will be live at: `https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME`
37
+
38
+ ---
39
+
40
+ ## Files Checklist
41
+
42
+ βœ… All files are ready in your project:
43
+
44
+ - [x] `main.py` - Main app
45
+ - [x] `requirements.txt` - Dependencies
46
+ - [x] `src/` - Source code
47
+ - [x] `.streamlit/config.toml` - HF configuration
48
+ - [x] `HF_README.md` - Space README (rename to README.md)
49
+ - [x] `.gitignore` - Ignore unnecessary files
50
+
51
+ **Your project is deployment-ready!** πŸš€
52
+
53
+ For detailed instructions, see: `hf_deployment_guide.md`
PROJECT_README.md ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Autonomous Research Agent with LangGraph, Groq, and Streamlit
2
+
3
+ This repository contains the complete source code for an **autonomous AI research agent**. The agent takes a user-defined topic, performs web searches to gather information, evaluates and summarizes relevant sources, and compiles the findings into a comprehensive report.
4
+
5
+ The project is built using a modern AI stack, showcasing a stateful, cyclic architecture that enables complex, multi-step reasoning and execution, all presented through an interactive web interface.
6
+
7
+ ---
8
+
9
+ ## Core Technologies
10
+
11
+ - **Orchestration:** `LangGraph` – Build stateful, multi-actor applications with cycles, enabling complex agentic behaviors.
12
+ - **LLM:** `Groq (Llama 3.3 70B)` – High-speed inference using a Language Processing Unit (LPU) for fast and responsive AI reasoning.
13
+ - **Web Interface:** `Streamlit` – Interactive and user-friendly chat-based web application built entirely in Python.
14
+ - **Search Tool:** `Tavily AI` – AI-optimized search engine to gather accurate and relevant information from the web.
15
+ - **Core Framework:** `LangChain` – Provides foundational components, tools, and integrations.
16
+
17
+ ---
18
+
19
+ ## Key Features
20
+
21
+ - **Stateful, Cyclic Architecture:**
22
+ Uses LangGraph loops to iteratively search, evaluate, and decide whether to continue researching or compile findings, mimicking a human research process.
23
+
24
+ - **High-Performance LLM:**
25
+ Leverages Groq LPU with Llama 3.3 70B for reasoning and content generation at extremely high speeds for a seamless user experience.
26
+
27
+ - **Fault Tolerance and Persistence:**
28
+ Saves the agent's state at every step using `SqliteSaver` checkpointer, allowing long-running tasks to resume from the exact point of failure.
29
+
30
+ - **Interactive Web UI:**
31
+ Streamlit-based chat interface lets users input topics, monitor progress in real-time, and receive the final report directly in the app.
32
+
33
+ - **Deep Observability with LangSmith:**
34
+ Provides detailed traces of every agent step for debugging and understanding complex behavior (optional).
35
+
36
+ ---
37
+
38
+ ## Setup Instructions
39
+
40
+ ### Prerequisites
41
+
42
+ You will need two API keys:
43
+
44
+ 1. **Groq API Key** - Sign up at [console.groq.com](https://console.groq.com/)
45
+ 2. **Tavily API Key** - Sign up at [tavily.com](https://tavily.com/)
46
+
47
+ ### Installation
48
+
49
+ 1. **Clone the repository** (or navigate to the project directory)
50
+
51
+ ```bash
52
+ cd "Research Agent with LangGraph"
53
+ ```
54
+
55
+ 2. **Create and activate a virtual environment**
56
+
57
+ ```powershell
58
+ # Create virtual environment
59
+ python -m venv venv
60
+
61
+ # Activate it (Windows PowerShell)
62
+ .\venv\Scripts\Activate.ps1
63
+ ```
64
+
65
+ 3. **Install dependencies**
66
+
67
+ ```powershell
68
+ .\venv\Scripts\python.exe -m pip install --upgrade pip setuptools wheel
69
+ .\venv\Scripts\python.exe -m pip install -r requirements.txt
70
+ ```
71
+
72
+ 4. **Configure environment variables**
73
+
74
+ Create or edit the `.env` file in the root directory and add your API keys:
75
+
76
+ ```env
77
+ GROQ_API_KEY=your_groq_api_key_here
78
+ TAVILY_API_KEY=your_tavily_api_key_here
79
+ ```
80
+
81
+ You can use `.env.example` as a template.
82
+
83
+ ---
84
+
85
+ ## Running the Application
86
+
87
+ Run the Streamlit app with:
88
+
89
+ ```powershell
90
+ .\venv\Scripts\python.exe -m streamlit run main.py
91
+ ```
92
+
93
+ The app will automatically open in your browser at `http://localhost:8501`
94
+
95
+ ---
96
+
97
+ ## Usage
98
+
99
+ 1. Open the application in your browser
100
+ 2. Enter a research topic in the chat input (e.g., "Recent advances in AI agents")
101
+ 3. Watch the agent work:
102
+ - πŸ” Search for relevant articles
103
+ - πŸ“„ Scrape content from URLs
104
+ - πŸ€– Evaluate relevance using the LLM
105
+ - πŸ“ Summarize useful information
106
+ - πŸ“Š Compile a comprehensive report
107
+ 4. Review the final research report
108
+
109
+ ---
110
+
111
+ ## Project Structure
112
+
113
+ ```
114
+ Research Agent with LangGraph/
115
+ β”œβ”€β”€ main.py # Streamlit UI and application entry point
116
+ β”œβ”€β”€ src/
117
+ β”‚ β”œβ”€β”€ graph.py # LangGraph workflow and node definitions
118
+ β”‚ β”œβ”€β”€ agent_state.py # Agent state schema
119
+ β”‚ └── tools.py # Search and scraping tools
120
+ β”œβ”€β”€ requirements.txt # Python dependencies
121
+ β”œβ”€β”€ .env # API keys (create this file)
122
+ β”œβ”€β”€ .env.example # Template for environment variables
123
+ └── checkpoints.sqlite # SQLite database for state persistence
124
+ ```
125
+
126
+ ---
127
+
128
+ ## Troubleshooting
129
+
130
+ ### Issue: "streamlit.exe not found" or Import Errors
131
+
132
+ **Solution:** Recreate the virtual environment from scratch:
133
+
134
+ ```powershell
135
+ # Delete old venv
136
+ Remove-Item -Recurse -Force venv
137
+
138
+ # Create fresh venv
139
+ python -m venv venv
140
+
141
+ # Upgrade pip
142
+ .\venv\Scripts\python.exe -m pip install --upgrade pip setuptools wheel
143
+
144
+ # Install dependencies
145
+ .\venv\Scripts\python.exe -m pip install -r requirements.txt
146
+ ```
147
+
148
+ ### Issue: API Key Errors
149
+
150
+ **Solution:** Ensure your `.env` file contains valid API keys and is in the project root directory.
151
+
152
+ ---
153
+
154
+ ## How It Works
155
+
156
+ The agent uses a **cyclic LangGraph workflow**:
157
+
158
+ 1. **Search Node** β†’ Searches web using Tavily API
159
+ 2. **Scrape & Summarize Node** β†’ Scrapes URLs one by one, evaluates relevance, and summarizes
160
+ 3. **Router** β†’ Decides to continue scraping or compile report
161
+ 4. **Compile Report Node** β†’ Synthesizes all summaries into a final report
162
+
163
+ Each step's state is saved to SQLite, enabling fault tolerance.
164
+
165
+ ---
166
+
167
+ ## Optional: LangSmith Tracing
168
+
169
+ To enable detailed tracing and debugging, add to your `.env`:
170
+
171
+ ```env
172
+ LANGCHAIN_TRACING_V2=true
173
+ LANGCHAIN_API_KEY=your_langsmith_api_key
174
+ LANGCHAIN_PROJECT=research-agent
175
+ ```
176
+
177
+ ---
178
+
179
+ ## License
180
+
181
+ MIT License - Feel free to use and modify this project.
182
+
183
+ ---
184
+
185
+ ## Contributing
186
+
187
+ Contributions are welcome! Feel free to open issues or submit pull requests.
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Autonomous Research Agent
3
+ emoji: πŸ€–
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: streamlit
7
+ sdk_version: 1.49.1
8
+ app_file: main.py
9
+ pinned: false
10
+ license: mit
11
+ ---
12
+
13
+ # Autonomous Research Agent πŸ€–
14
+
15
+ An intelligent research assistant that autonomously searches the web, evaluates sources, and compiles comprehensive research reports using LangGraph and Groq.
16
+
17
+ ## Features
18
+
19
+ - πŸ” **Autonomous Web Search** - Uses Tavily AI to find relevant articles
20
+ - 🧠 **Smart Evaluation** - LLM-powered relevance filtering
21
+ - πŸ“ **Automatic Summarization** - Extracts key insights from sources
22
+ - πŸ“Š **Report Compilation** - Synthesizes findings into cohesive reports
23
+ - πŸ”„ **Stateful Architecture** - Uses LangGraph for complex agentic workflows
24
+ - ⚑ **High-Speed Inference** - Powered by Groq's LPU (Llama 3.3 70B)
25
+
26
+ ## How to Use
27
+
28
+ 1. Enter a research topic in the chat input
29
+ 2. Watch the agent autonomously:
30
+ - Search for relevant articles
31
+ - Scrape and evaluate content
32
+ - Summarize useful information
33
+ - Compile a comprehensive report
34
+ 3. Review your personalized research report!
35
+
36
+ ## Configuration
37
+
38
+ This Space requires two API keys to function (set in Settings β†’ Repository Secrets):
39
+
40
+ - `GROQ_API_KEY` - Get from [console.groq.com](https://console.groq.com/)
41
+ - `TAVILY_API_KEY` - Get from [tavily.com](https://tavily.com/)
42
+
43
+ ## Technology Stack
44
+
45
+ - **LangGraph** - Stateful agent orchestration
46
+ - **Groq (Llama 3.3 70B)** - High-speed LLM inference
47
+ - **Tavily AI** - AI-optimized search
48
+ - **Streamlit** - Interactive UI
49
+ - **SQLite** - Persistent checkpointing
50
+
51
+ ## Source Code
52
+
53
+ Full source code available at: [GitHub Repository](https://github.com/yourusername/research-agent)
54
+
55
+ ---
56
+
57
+ Built with ❀️ using LangGraph, Groq, and Streamlit
main.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import streamlit as st
2
+ import uuid
3
+ from dotenv import load_dotenv
4
+ from src.graph import app
5
+
6
+ # Load environment variables from.env file
7
+ load_dotenv()
8
+
9
+ # --- Streamlit UI Configuration ---
10
+ st.set_page_config(page_title="Autonomous Research Agent", page_icon="πŸ€–", layout="wide")
11
+ st.title("Autonomous Research Agent")
12
+
13
+ # --- Session State Management ---
14
+ # This ensures that each user session has a unique thread_id
15
+ # and that the message history is maintained across reruns.
16
+ if "thread_id" not in st.session_state:
17
+ st.session_state.thread_id = str(uuid.uuid4())
18
+ # FIX 1: Initialize with an empty list
19
+ st.session_state.messages =[]
20
+
21
+ # Display the chat history
22
+ for message in st.session_state.messages:
23
+ with st.chat_message(message["role"]):
24
+ st.markdown(message["content"])
25
+
26
+ # --- Main Application Logic ---
27
+ if prompt := st.chat_input("What topic should I research for you?"):
28
+ # Add user's message to session state and display it
29
+ st.session_state.messages.append({"role": "user", "content": prompt})
30
+ with st.chat_message("user"):
31
+ st.markdown(prompt)
32
+
33
+ # Prepare to display the agent's response
34
+ with st.chat_message("assistant"):
35
+ # Use a status container to show the agent's progress
36
+ with st.status("Researching...", expanded=True) as status:
37
+ final_report = ""
38
+
39
+ # LangGraph configuration for the specific session
40
+ config = {"configurable": {"thread_id": st.session_state.thread_id}}
41
+ # FIX 2: Initialize summaries with an empty list
42
+ initial_state = {"topic": prompt, "summaries":[]}
43
+
44
+ # Stream events from the LangGraph agent
45
+ for event in app.stream(initial_state, config=config):
46
+ for key, value in event.items():
47
+ if key == "search":
48
+ status.write("Searching for relevant articles...")
49
+ elif key == "scrape_and_evaluate":
50
+ if value.get("scraped_content"):
51
+ url = value['scraped_content'].get('url', 'Unknown URL')
52
+ is_relevant = value['scraped_content'].get('is_relevant', 'Unknown')
53
+ status.write(f"Evaluating URL: {url} - Relevant: {is_relevant}")
54
+ elif key == "summarize":
55
+ status.write("Summarizing relevant content...")
56
+ elif key == "compile_report":
57
+ status.write("Compiling the final report...")
58
+ if value.get("report"):
59
+ final_report = value["report"]
60
+
61
+ # Update the status to "complete" when done
62
+ status.update(label="Research complete!", state="complete", expanded=False)
63
+
64
+ # Display the final report
65
+ st.markdown(final_report)
66
+
67
+ # Add the final report to the session state
68
+ st.session_state.messages.append({"role": "assistant", "content": final_report})
requirements.txt ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ langgraph
2
+ langchain-core
3
+ langchain-openai
4
+ pydantic
5
+ tavily-python
6
+ langchain-groq
7
+ groq
8
+ python-dotenv
9
+ langchain-community
10
+ lxml
11
+ beautifulsoup4
12
+ langgraph-checkpoint-sqlite
13
+ streamlit
src/___init__.py ADDED
File without changes
src/agent_state.py ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import operator
2
+ from typing import TypedDict, Annotated, List, Dict
3
+
4
+ class AgentState(TypedDict):
5
+ """
6
+ Represents the state of our research agent.
7
+ """
8
+ topic: str
9
+ urls: List[str]
10
+ scraped_content: Dict # Changed from List[dict] to Dict for consistency
11
+ summaries: Annotated[List[str], operator.add]
12
+ report: str
13
+ error: str
src/graph.py ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import sqlite3
3
+ from langchain_groq import ChatGroq
4
+ from langchain_core.prompts import ChatPromptTemplate
5
+ from langgraph.graph import StateGraph, END
6
+ from langgraph.checkpoint.sqlite import SqliteSaver
7
+ from.agent_state import AgentState
8
+ from.tools import search_tool, scrape_tool
9
+
10
+ from dotenv import load_dotenv
11
+
12
+ load_dotenv() # Load environment variables from .env
13
+
14
+ llm = ChatGroq(
15
+ model="llama-3.3-70b-versatile",
16
+ temperature=0,
17
+ api_key=os.getenv("GROQ_API_KEY")
18
+ )
19
+
20
+
21
+ # --- Node Functions ---
22
+
23
+ def search_node(state: AgentState):
24
+ """
25
+ Searches for articles on the given topic and updates the state with a list of URLs.
26
+ """
27
+ print("--- Searching for articles ---")
28
+ results = search_tool.invoke(state['topic'])
29
+ urls = [res['url'] for res in results if res and 'url' in res]
30
+ return {"urls": urls}
31
+
32
+ def scrape_and_summarize_node(state: AgentState):
33
+ """
34
+ Scrapes a URL, and if the content is relevant, summarizes it and adds it to the state.
35
+ If not relevant, it discards the content and moves to the next URL.
36
+ """
37
+ print("--- Scraping and summarizing content ---")
38
+ urls = state.get('urls',)
39
+ if not urls:
40
+ return {"error": "No URLs to process."}
41
+
42
+ # Take the next URL from the list
43
+ url_to_scrape = urls.pop(0)
44
+
45
+ content = scrape_tool.invoke({"url": url_to_scrape})
46
+
47
+ if not content or content.startswith("Error"):
48
+ print(f"URL: {url_to_scrape} - Failed to scrape or no content.")
49
+ return {"urls": urls, "error": content}
50
+
51
+ # This prompt asks the LLM to summarize ONLY if the content is relevant.
52
+ # This is more robust than a simple 'yes'/'no' check.
53
+ prompt = ChatPromptTemplate.from_template(
54
+ "You are a research assistant. Your task is to summarize the following content about the topic: {topic}. "
55
+ "If the content is NOT relevant to the topic, respond with only the single word 'IRRELEVANT'. "
56
+ "Otherwise, provide a concise summary of the relevant information."
57
+ "\n\nContent:\n{content}"
58
+ )
59
+
60
+ chain = prompt | llm
61
+ summary_result = chain.invoke({"topic": state['topic'], "content": content[:8000]}).content
62
+
63
+ # If the model returns "IRRELEVANT", we discard it. Otherwise, we add the summary.
64
+ if "IRRELEVANT" in summary_result.upper():
65
+ print(f"URL: {url_to_scrape} - Not relevant.")
66
+ return {"urls": urls}
67
+ else:
68
+ print(f"URL: {url_to_scrape} - Summarized.")
69
+ return {"urls": urls, "summaries": [summary_result]}
70
+
71
+ def compile_report_node(state: AgentState):
72
+ """
73
+ Takes all the collected summaries and synthesizes them into a final report.
74
+ """
75
+ print("--- Compiling final report ---")
76
+ summaries = state.get('summaries',)
77
+ if not summaries:
78
+ return {"report": "No relevant information found to compile a report."}
79
+
80
+ prompt = ChatPromptTemplate.from_template(
81
+ "You are a research report writer. Synthesize the following summaries into a coherent and well-structured research report on the topic: {topic}."
82
+ "\n\nSummaries:\n{summaries}"
83
+ )
84
+
85
+ chain = prompt | llm
86
+ report = chain.invoke({"topic": state['topic'], "summaries": "\n\n---\n\n".join(summaries)}).content
87
+ return {"report": report}
88
+
89
+ # --- Edge Logic ---
90
+
91
+ def should_continue_router(state: AgentState):
92
+ """
93
+ Determines whether the research loop should continue or end.
94
+ """
95
+ if state.get('urls'):
96
+ return "scrape_and_summarize" # Continue if there are more URLs
97
+ else:
98
+ return "compile_report" # End the loop if all URLs are processed
99
+
100
+ # --- Graph Definition ---
101
+
102
+ workflow = StateGraph(AgentState)
103
+
104
+ # Add the nodes to the graph
105
+ workflow.add_node("search", search_node)
106
+ workflow.add_node("scrape_and_summarize", scrape_and_summarize_node)
107
+ workflow.add_node("compile_report", compile_report_node)
108
+
109
+ # Set the entry point and define the flow
110
+ workflow.set_entry_point("search")
111
+ workflow.add_edge("search", "scrape_and_summarize")
112
+ workflow.add_conditional_edges(
113
+ "scrape_and_summarize",
114
+ should_continue_router,
115
+ {
116
+ "scrape_and_summarize": "scrape_and_summarize",
117
+ "compile_report": "compile_report"
118
+ }
119
+ )
120
+ workflow.add_edge("compile_report", END)
121
+
122
+ # --- Compile with Checkpointer for Fault Tolerance ---
123
+ conn = sqlite3.connect("checkpoints.sqlite", check_same_thread=False)
124
+ memory = SqliteSaver(conn=conn)
125
+ app = workflow.compile(checkpointer=memory)
src/tools.py ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import List
2
+ import requests
3
+ from bs4 import BeautifulSoup
4
+ from langchain_core.tools import tool
5
+ from langchain_community.retrievers import TavilySearchAPIRetriever
6
+
7
+ @tool
8
+ def search_tool(query: str) -> List[dict]:
9
+ """Searches the web for a given query using Tavily and returns a list of search results."""
10
+ try:
11
+ retriever = TavilySearchAPIRetriever(k=5)
12
+ results = retriever.invoke(query)
13
+ return [{"url": doc.metadata["source"], "content": doc.page_content} for doc in results]
14
+ except Exception as e:
15
+ # Return an empty list or handle the error as appropriate
16
+ return
17
+
18
+ @tool
19
+ def scrape_tool(url: str) -> str:
20
+ """Scrapes the text content of a given URL."""
21
+ try:
22
+ response = requests.get(url, timeout=10, headers={'User-Agent': 'Mozilla/5.0'})
23
+ response.raise_for_status()
24
+ soup = BeautifulSoup(response.content, "lxml")
25
+ for script_or_style in soup(["script", "style"]):
26
+ script_or_style.decompose()
27
+ text = "\n".join(chunk for chunk in (phrase.strip() for line in (line.strip() for line in soup.get_text().splitlines()) for phrase in line.split(" ")) if chunk)
28
+ return text if text else "No content found."
29
+ except requests.RequestException as e:
30
+ return f"Error scraping URL {url}: {e}"
walkthrough.md.resolved ADDED
@@ -0,0 +1,256 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Research Agent Project - Analysis & Setup Guide
2
+
3
+ ## What This Project Does
4
+
5
+ This is an **Autonomous Research Agent** built with a modern AI stack that:
6
+
7
+ 1. πŸ” **Searches** the web for articles on a given topic (using Tavily AI)
8
+ 2. πŸ“„ **Scrapes** content from the discovered URLs
9
+ 3. πŸ€– **Evaluates** each article for relevance using an LLM
10
+ 4. πŸ“ **Summarizes** relevant content
11
+ 5. πŸ“Š **Compiles** a comprehensive research report
12
+
13
+ ### Architecture
14
+
15
+ The agent uses **LangGraph** to create a stateful, cyclic workflow:
16
+
17
+ ```mermaid
18
+ graph LR
19
+ A[User Input Topic] --> B[Search Node]
20
+ B --> C[Scrape & Summarize Node]
21
+ C --> D{More URLs?}
22
+ D -->|Yes| C
23
+ D -->|No| E[Compile Report Node]
24
+ E --> F[Final Report]
25
+ ```
26
+
27
+ ### Technology Stack
28
+
29
+ - **LangGraph**: Orchestration of the stateful workflow
30
+ - **Groq**: High-speed LLM inference (Llama 3.3 70B)
31
+ - **Streamlit**: Interactive web interface
32
+ - **Tavily AI**: AI-optimized web search
33
+ - **SQLite Checkpointer**: Fault-tolerant state persistence
34
+
35
+ ---
36
+
37
+ ## Enhancements Made
38
+
39
+ Since this project was built 5 months ago, I made the following updates:
40
+
41
+ ### 1. Updated LLM Model
42
+ **[src/graph.py](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py#L14-L18)**
43
+
44
+ Changed from `openai/gpt-oss-120b` (outdated/unavailable) to `llama-3.3-70b-versatile`:
45
+
46
+ ```diff
47
+ llm = ChatGroq(
48
+ - model="openai/gpt-oss-120b",
49
+ + model="llama-3.3-70b-versatile",
50
+ temperature=0,
51
+ api_key=os.getenv("GROQ_API_KEY")
52
+ )
53
+ ```
54
+
55
+ ### 2. Created Environment Configuration Template
56
+ **[.env.example](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env.example)**
57
+
58
+ Added a template to help configure the required API keys.
59
+
60
+ ### 3. Fixed Dependency Installation Issues
61
+
62
+ **Problem:** The initial virtual environment had corrupted dependencies causing import errors.
63
+
64
+ **Solution:** Recreated the virtual environment from scratch:
65
+ 1. Deleted old `venv` folder
66
+ 2. Created fresh virtual environment
67
+ 3. Upgraded pip, setuptools, and wheel
68
+ 4. Installed all dependencies from [requirements.txt](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/requirements.txt)
69
+
70
+ ---
71
+
72
+ ## How to Run
73
+
74
+ ### Prerequisites
75
+
76
+ You need two API keys:
77
+ 1. **Groq API Key** - Get from [console.groq.com](https://console.groq.com/)
78
+ 2. **Tavily API Key** - Get from [tavily.com](https://tavily.com/)
79
+
80
+ ### Step 1: Configure API Keys
81
+
82
+ Edit your [.env](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env) file and add:
83
+
84
+ ```env
85
+ GROQ_API_KEY=your_groq_api_key_here
86
+ TAVILY_API_KEY=your_tavily_api_key_here
87
+ ```
88
+
89
+ > [!IMPORTANT]
90
+ > The [.env](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env) file already exists in the project but needs to be configured with valid API keys.
91
+
92
+ ### Step 2: Run the Application
93
+
94
+ Use this command to run the application:
95
+
96
+ ```powershell
97
+ .\venv\Scripts\python.exe -m streamlit run main.py
98
+ ```
99
+
100
+ > [!TIP]
101
+ > **Alternative command** (if the above doesn't work):
102
+ > ```powershell
103
+ > python -m streamlit run main.py
104
+ > ```
105
+
106
+ The app will start and automatically open in your browser at `http://localhost:8501`
107
+
108
+ ### Step 3: Use the Agent
109
+
110
+ 1. Enter a research topic (e.g., "LangGraph features" or "AI agents in 2026")
111
+ 2. Watch the agent:
112
+ - Search for articles
113
+ - Evaluate each URL for relevance
114
+ - Summarize relevant content
115
+ - Compile the final report
116
+ 3. Review the comprehensive research report
117
+
118
+ ---
119
+
120
+ ## Troubleshooting
121
+
122
+ ### Issue: "streamlit.exe not found"
123
+
124
+ **Cause:** Dependencies weren't properly installed in the virtual environment.
125
+
126
+ **Solution:** Recreate the virtual environment:
127
+
128
+ ```powershell
129
+ # Delete old venv
130
+ Remove-Item -Recurse -Force venv
131
+
132
+ # Create new venv
133
+ python -m venv venv
134
+
135
+ # Upgrade pip
136
+ .\venv\Scripts\python.exe -m pip install --upgrade pip setuptools wheel
137
+
138
+ # Install dependencies
139
+ .\venv\Scripts\python.exe -m pip install -r requirements.txt
140
+ ```
141
+
142
+ ### Issue: Import errors (pydantic, zstandard, etc.)
143
+
144
+ **Cause:** Corrupted package installations.
145
+
146
+ **Solution:** Follow the steps above to recreate the virtual environment completely.
147
+
148
+ ### Issue: "GROQ_API_KEY not set"
149
+
150
+ **Cause:** Missing or improperly configured [.env](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env) file.
151
+
152
+ **Solution:** Ensure your [.env](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env) file contains valid API keys.
153
+
154
+ ---
155
+
156
+ ## Project Evaluation
157
+
158
+ ### βœ… Strengths
159
+
160
+ - **Well-architected**: Clean separation of concerns (state, graph, tools)
161
+ - **Fault-tolerant**: SQLite checkpointer saves state at every step
162
+ - **Modern stack**: Uses cutting-edge tools (LangGraph, Groq LPU)
163
+ - **User-friendly**: Streamlit provides excellent UX with real-time progress tracking
164
+
165
+ ### πŸ”„ Potential Enhancements
166
+
167
+ While the project is solid, here are some optional improvements:
168
+
169
+ 1. **Error Handling**
170
+ - Add retry logic for failed web requests
171
+ - Handle rate limits from Groq/Tavily APIs
172
+
173
+ 2. **Content Quality**
174
+ - Implement a scoring system for source credibility
175
+ - Add citation tracking in the final report
176
+
177
+ 3. **Performance**
178
+ - Parallelize URL scraping (currently sequential)
179
+ - Add caching for previously scraped URLs
180
+
181
+ 4. **Features**
182
+ - Export reports to PDF/Markdown
183
+ - Save research history
184
+ - Allow users to specify number of sources to research
185
+
186
+ 5. **Observability**
187
+ - Enable LangSmith tracing for debugging (already supported, just needs env vars)
188
+ - Add metrics dashboard (search count, success rate, etc.)
189
+
190
+ 6. **Testing**
191
+ - Add unit tests for individual nodes
192
+ - Create integration tests for the full workflow
193
+
194
+ ---
195
+
196
+ ## Technical Deep Dive
197
+
198
+ ### Key Files
199
+
200
+ | File | Purpose |
201
+ |------|---------|
202
+ | [main.py](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/main.py) | Streamlit UI and session management |
203
+ | [src/graph.py](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py) | LangGraph workflow definition and node functions |
204
+ | [src/agent_state.py](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/agent_state.py) | TypedDict defining the agent's state schema |
205
+ | [src/tools.py](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/tools.py) | Search and scraping tools |
206
+
207
+ ### How the Workflow Works
208
+
209
+ 1. **Search Node** ([graph.py:L23-L30](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py#L23-L30))
210
+ - Invokes Tavily search
211
+ - Extracts URLs from results
212
+ - Updates state with URLs list
213
+
214
+ 2. **Scrape & Summarize Node** ([graph.py:L32-L69](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py#L32-L69))
215
+ - Pops one URL from the list
216
+ - Scrapes content using BeautifulSoup
217
+ - Asks LLM to summarize if relevant (or return "IRRELEVANT")
218
+ - Adds summary to state if relevant
219
+
220
+ 3. **Routing Logic** ([graph.py:L91-L98](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py#L91-L98))
221
+ - If URLs remain β†’ loop back to scrape another
222
+ - If no URLs β†’ proceed to compile report
223
+
224
+ 4. **Compile Report Node** ([graph.py:L71-L87](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/src/graph.py#L71-L87))
225
+ - Takes all summaries
226
+ - Synthesizes into a coherent report
227
+ - Returns final report to user
228
+
229
+ ---
230
+
231
+ ## Example Usage
232
+
233
+ **Topic:** "Benefits of LangGraph"
234
+
235
+ **Agent Process:**
236
+ 1. Searches Tavily β†’ finds 5 relevant articles
237
+ 2. Scrapes Article 1 β†’ relevant β†’ summarizes
238
+ 3. Scrapes Article 2 β†’ not relevant β†’ skips
239
+ 4. Scrapes Article 3 β†’ relevant β†’ summarizes
240
+ 5. Scrapes Article 4 β†’ relevant β†’ summarizes
241
+ 6. Scrapes Article 5 β†’ relevant β†’ summarizes
242
+ 7. Compiles final report from 4 summaries
243
+
244
+ **Result:** A comprehensive report covering LangGraph's benefits, compiled from 4 high-quality sources.
245
+
246
+ ---
247
+
248
+ ## Summary
249
+
250
+ βœ… **Project is now fully functional!**
251
+
252
+ - Updated LLM model to `llama-3.3-70b-versatile`
253
+ - Fixed all dependency installation issues
254
+ - Application running successfully on `http://localhost:8501`
255
+
256
+ **Next steps:** Configure your API keys in the [.env](file:///c:/Users/punit/Desktop/project/GenAI/Research%20Agent%20with%20LangGraph/.env) file and start researching!