wu981526092 commited on
Commit
beb0a01
·
1 Parent(s): 32fecea

🔧 Optimize Tool Definition: RAG Search vs Language Model

Browse files

- ✨ Updated tool_001 from 'GPT-4o Language Model' to 'Oxford Economics Knowledge Search'
- 🔍 Enhanced RAG-powered knowledge retrieval capabilities in sample data
- 📚 Updated descriptions to reflect proper tool vs agent distinction
- ⚡ Modified interaction prompts to show knowledge base querying workflow
- 🎯 Updated optimization recommendations for RAG tool enhancement
- 📋 Refined documentation to highlight RAG search capabilities

Key improvements:
- Tools now represent external knowledge systems (RAG search)
- Language models are implicit agent capabilities, not separate tools
- Better alignment with real-world AI assistant architectures
- More accurate representation of retrieval-augmented generation flows

backend/database/README_sample_data.md CHANGED
@@ -19,7 +19,7 @@ The system includes 2 carefully selected examples showcasing AgentGraph's advanc
19
  1. **Oxford Economics AI Assistant** (Enhanced)
20
 
21
  - Type: `aif_inference`
22
- - Example: AI assistant processing economic inquiry with failure detection
23
  - 6 entities, 5 relations, 1 failure, 2 optimizations
24
  - Features: Content references, quality scoring, system summary
25
 
@@ -37,6 +37,7 @@ Each trace comes with a pre-generated knowledge graph showcasing AgentGraph's co
37
  - **Agent interactions and roles** with detailed prompts and content references
38
  - **Task decomposition** with clear importance levels
39
  - **Information flow** with specific interaction prompts
 
40
  - **Failure detection** identifying real issues (spelling errors, system gaps)
41
  - **Optimization recommendations** providing actionable improvements
42
  - **Quality assessment** with confidence scores and metadata
@@ -122,7 +123,7 @@ To disable automatic sample data insertion, modify `init_db.py`:
122
  ## Benefits for Users
123
 
124
  1. **Immediate Value**: New users see AgentGraph's complete capabilities immediately
125
- 2. **Learning**: Examples demonstrate failure detection, optimization suggestions, and advanced features
126
  3. **Testing**: Users can test all AgentGraph features including quality assessment and content referencing
127
  4. **Reference**: Examples serve as high-quality templates showcasing best practices
128
  5. **Feature Discovery**: Users understand the full potential of knowledge graph enhancement
@@ -133,6 +134,7 @@ To disable automatic sample data insertion, modify `init_db.py`:
133
  - All sample traces are realistic and demonstrate real-world scenarios
134
  - Knowledge graphs are hand-crafted to showcase AgentGraph's complete feature set
135
  - Examples include actual failure detection (spelling errors, system gaps)
 
136
  - Optimization recommendations are practical and actionable
137
  - Content references are accurate and support proper traceability
138
  - Quality scores reflect realistic assessment metrics
 
19
  1. **Oxford Economics AI Assistant** (Enhanced)
20
 
21
  - Type: `aif_inference`
22
+ - Example: RAG-powered assistant processing economic inquiry with knowledge search and failure detection
23
  - 6 entities, 5 relations, 1 failure, 2 optimizations
24
  - Features: Content references, quality scoring, system summary
25
 
 
37
  - **Agent interactions and roles** with detailed prompts and content references
38
  - **Task decomposition** with clear importance levels
39
  - **Information flow** with specific interaction prompts
40
+ - **RAG-powered knowledge search** retrieving relevant documents and context
41
  - **Failure detection** identifying real issues (spelling errors, system gaps)
42
  - **Optimization recommendations** providing actionable improvements
43
  - **Quality assessment** with confidence scores and metadata
 
123
  ## Benefits for Users
124
 
125
  1. **Immediate Value**: New users see AgentGraph's complete capabilities immediately
126
+ 2. **Learning**: Examples demonstrate RAG search, failure detection, optimization suggestions, and advanced features
127
  3. **Testing**: Users can test all AgentGraph features including quality assessment and content referencing
128
  4. **Reference**: Examples serve as high-quality templates showcasing best practices
129
  5. **Feature Discovery**: Users understand the full potential of knowledge graph enhancement
 
134
  - All sample traces are realistic and demonstrate real-world scenarios
135
  - Knowledge graphs are hand-crafted to showcase AgentGraph's complete feature set
136
  - Examples include actual failure detection (spelling errors, system gaps)
137
+ - RAG search capabilities demonstrate knowledge retrieval workflows
138
  - Optimization recommendations are practical and actionable
139
  - Content references are accurate and support proper traceability
140
  - Quality scores reflect realistic assessment metrics
backend/database/sample_data.py CHANGED
@@ -15,10 +15,10 @@ SAMPLE_TRACES = [
15
  {
16
  "filename": "oxford_economics_inquiry.json",
17
  "title": "Oxford Economics AI Assistant Demo",
18
- "description": "Enhanced example showing AI assistant processing economic inquiry with failure detection and optimization suggestions",
19
  "trace_type": "aif_inference",
20
  "trace_source": "sample_data",
21
- "tags": ["economics", "ai_assistant", "enhanced", "failure_detection", "optimization"],
22
  "content": """{
23
  "id": "aif_trace_demo_001",
24
  "timestamp": "2025-01-27T00:00:00",
@@ -29,14 +29,14 @@ SAMPLE_TRACES = [
29
  },
30
  "data": {
31
  "total_observations": 1,
32
- "summary": "Oxford Economics inquiry with AI assistant response"
33
  },
34
  "observations": [
35
  {
36
  "id": "demo_obs_001",
37
  "type": "inference",
38
  "timestamp": "2025-01-27T00:00:00",
39
- "input": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on the provided context. Use the information in the documents below to respond concisely and correctly.\\n\\n### Context:\\n['Oxford Economics contact: Mateusz Urban - murban@oxfordeconomics.com', 'Oxford Economics research and analysis services', 'Economic forecasting and consultancy expertise']\\n\\n### Question:\\nwhat does oxford economics present?\\n\\n### Answer:",
40
  "output": "Oxford Economics provides economic analysis, forecasting, and consultancy services.",
41
  "metadata": {
42
  "request_date": "2025-01-27T00:00:00",
@@ -100,14 +100,14 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
100
  "trace_index": 0, # Links to first trace
101
  "graph_data": {
102
  "system_name": "Oxford Economics AI Assistant",
103
- "system_summary": "This intelligent assistant processes user inquiries about Oxford Economics through a streamlined workflow. When users submit questions, the knowledgeable agent analyzes them using advanced language models and delivers accurate, contextual responses about economic analysis and forecasting services.",
104
  "entities": [
105
  {
106
  "id": "agent_001",
107
  "type": "Agent",
108
  "name": "Oxford Economics Knowledge Agent",
109
  "importance": "HIGH",
110
- "raw_prompt": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on the provided context. Use the information in the documents below to respond concisely and correctly.",
111
  "raw_prompt_ref": [
112
  {
113
  "line_start": 31,
@@ -170,9 +170,9 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
170
  {
171
  "id": "tool_001",
172
  "type": "Tool",
173
- "name": "GPT-4o Language Model",
174
  "importance": "HIGH",
175
- "raw_prompt": "Advanced AI language model with economic domain knowledge and structured response capabilities.",
176
  "raw_prompt_ref": [
177
  {
178
  "line_start": 49,
@@ -244,7 +244,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
244
  "target": "tool_001",
245
  "type": "USES",
246
  "importance": "HIGH",
247
- "interaction_prompt": "Agent leverages language model for natural language understanding and generation",
248
  "interaction_prompt_ref": [
249
  {
250
  "line_start": 49,
@@ -284,7 +284,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
284
  {
285
  "id": "opt_002",
286
  "recommendation_type": "TOOL_ENHANCEMENT",
287
- "description": "Integrate Oxford Economics knowledge base with the language model to provide more specific and detailed responses about services and capabilities.",
288
  "affected_ids": ["tool_001"],
289
  "raw_text_ref": [
290
  {
@@ -334,7 +334,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
334
  "type": "Agent",
335
  "name": "Q&A Assistant",
336
  "importance": "HIGH",
337
- "raw_prompt": "Helpful assistant specialized in providing information about Oxford Economics services and capabilities.",
338
  "raw_prompt_ref": [
339
  {
340
  "line_start": 3,
@@ -559,6 +559,6 @@ def get_sample_data_info():
559
  "knowledge_graphs_count": len(SAMPLE_KNOWLEDGE_GRAPHS),
560
  "trace_types": list(set(t["trace_type"] for t in SAMPLE_TRACES)),
561
  "complexity_levels": ["enhanced", "simple"],
562
- "features": ["failure_detection", "optimization_recommendations", "content_references", "quality_scoring"],
563
- "description": "Enhanced AgentGraph examples showcasing Oxford Economics AI assistant with complete failure detection, optimization suggestions, and advanced knowledge graph features"
564
  }
 
15
  {
16
  "filename": "oxford_economics_inquiry.json",
17
  "title": "Oxford Economics AI Assistant Demo",
18
+ "description": "Enhanced example showing RAG-powered AI assistant processing economic inquiry with knowledge search, failure detection and optimization suggestions",
19
  "trace_type": "aif_inference",
20
  "trace_source": "sample_data",
21
+ "tags": ["economics", "rag_assistant", "knowledge_search", "failure_detection", "optimization"],
22
  "content": """{
23
  "id": "aif_trace_demo_001",
24
  "timestamp": "2025-01-27T00:00:00",
 
29
  },
30
  "data": {
31
  "total_observations": 1,
32
+ "summary": "Oxford Economics inquiry with RAG-powered assistant response"
33
  },
34
  "observations": [
35
  {
36
  "id": "demo_obs_001",
37
  "type": "inference",
38
  "timestamp": "2025-01-27T00:00:00",
39
+ "input": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on the provided context. Use the information retrieved from the knowledge base below to respond concisely and correctly.\\n\\n### Retrieved Documents:\\n['Oxford Economics contact: Mateusz Urban - murban@oxfordeconomics.com', 'Oxford Economics research and analysis services', 'Economic forecasting and consultancy expertise']\\n\\n### Question:\\nwhat does oxford economics present?\\n\\n### Answer:",
40
  "output": "Oxford Economics provides economic analysis, forecasting, and consultancy services.",
41
  "metadata": {
42
  "request_date": "2025-01-27T00:00:00",
 
100
  "trace_index": 0, # Links to first trace
101
  "graph_data": {
102
  "system_name": "Oxford Economics AI Assistant",
103
+ "system_summary": "This intelligent assistant processes user inquiries about Oxford Economics through a streamlined workflow. When users submit questions, the knowledgeable agent analyzes them using RAG-powered knowledge search and delivers accurate, contextual responses about economic analysis and forecasting services.",
104
  "entities": [
105
  {
106
  "id": "agent_001",
107
  "type": "Agent",
108
  "name": "Oxford Economics Knowledge Agent",
109
  "importance": "HIGH",
110
+ "raw_prompt": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on retrieved knowledge base context. Use the search results to provide precise responses.",
111
  "raw_prompt_ref": [
112
  {
113
  "line_start": 31,
 
170
  {
171
  "id": "tool_001",
172
  "type": "Tool",
173
+ "name": "Oxford Economics Knowledge Search",
174
  "importance": "HIGH",
175
+ "raw_prompt": "Retrieval-Augmented Generation (RAG) system that searches Oxford Economics knowledge base for relevant documents, contact information, and service details to provide contextual information.",
176
  "raw_prompt_ref": [
177
  {
178
  "line_start": 49,
 
244
  "target": "tool_001",
245
  "type": "USES",
246
  "importance": "HIGH",
247
+ "interaction_prompt": "Agent queries knowledge search system to retrieve relevant Oxford Economics documents and contextual information",
248
  "interaction_prompt_ref": [
249
  {
250
  "line_start": 49,
 
284
  {
285
  "id": "opt_002",
286
  "recommendation_type": "TOOL_ENHANCEMENT",
287
+ "description": "Expand knowledge search capabilities to include real-time market data, recent publications, and dynamic economic indicators alongside static company information.",
288
  "affected_ids": ["tool_001"],
289
  "raw_text_ref": [
290
  {
 
334
  "type": "Agent",
335
  "name": "Q&A Assistant",
336
  "importance": "HIGH",
337
+ "raw_prompt": "RAG-powered assistant specialized in searching knowledge base and providing accurate information about Oxford Economics services and capabilities.",
338
  "raw_prompt_ref": [
339
  {
340
  "line_start": 3,
 
559
  "knowledge_graphs_count": len(SAMPLE_KNOWLEDGE_GRAPHS),
560
  "trace_types": list(set(t["trace_type"] for t in SAMPLE_TRACES)),
561
  "complexity_levels": ["enhanced", "simple"],
562
+ "features": ["rag_search", "failure_detection", "optimization_recommendations", "content_references", "quality_scoring"],
563
+ "description": "Enhanced AgentGraph examples showcasing Oxford Economics RAG-powered assistant with knowledge search, failure detection, optimization suggestions, and advanced knowledge graph features"
564
  }