Spaces:
Running
Running
Commit
·
beb0a01
1
Parent(s):
32fecea
🔧 Optimize Tool Definition: RAG Search vs Language Model
Browse files- ✨ Updated tool_001 from 'GPT-4o Language Model' to 'Oxford Economics Knowledge Search'
- 🔍 Enhanced RAG-powered knowledge retrieval capabilities in sample data
- 📚 Updated descriptions to reflect proper tool vs agent distinction
- ⚡ Modified interaction prompts to show knowledge base querying workflow
- 🎯 Updated optimization recommendations for RAG tool enhancement
- 📋 Refined documentation to highlight RAG search capabilities
Key improvements:
- Tools now represent external knowledge systems (RAG search)
- Language models are implicit agent capabilities, not separate tools
- Better alignment with real-world AI assistant architectures
- More accurate representation of retrieval-augmented generation flows
backend/database/README_sample_data.md
CHANGED
|
@@ -19,7 +19,7 @@ The system includes 2 carefully selected examples showcasing AgentGraph's advanc
|
|
| 19 |
1. **Oxford Economics AI Assistant** (Enhanced)
|
| 20 |
|
| 21 |
- Type: `aif_inference`
|
| 22 |
-
- Example:
|
| 23 |
- 6 entities, 5 relations, 1 failure, 2 optimizations
|
| 24 |
- Features: Content references, quality scoring, system summary
|
| 25 |
|
|
@@ -37,6 +37,7 @@ Each trace comes with a pre-generated knowledge graph showcasing AgentGraph's co
|
|
| 37 |
- **Agent interactions and roles** with detailed prompts and content references
|
| 38 |
- **Task decomposition** with clear importance levels
|
| 39 |
- **Information flow** with specific interaction prompts
|
|
|
|
| 40 |
- **Failure detection** identifying real issues (spelling errors, system gaps)
|
| 41 |
- **Optimization recommendations** providing actionable improvements
|
| 42 |
- **Quality assessment** with confidence scores and metadata
|
|
@@ -122,7 +123,7 @@ To disable automatic sample data insertion, modify `init_db.py`:
|
|
| 122 |
## Benefits for Users
|
| 123 |
|
| 124 |
1. **Immediate Value**: New users see AgentGraph's complete capabilities immediately
|
| 125 |
-
2. **Learning**: Examples demonstrate failure detection, optimization suggestions, and advanced features
|
| 126 |
3. **Testing**: Users can test all AgentGraph features including quality assessment and content referencing
|
| 127 |
4. **Reference**: Examples serve as high-quality templates showcasing best practices
|
| 128 |
5. **Feature Discovery**: Users understand the full potential of knowledge graph enhancement
|
|
@@ -133,6 +134,7 @@ To disable automatic sample data insertion, modify `init_db.py`:
|
|
| 133 |
- All sample traces are realistic and demonstrate real-world scenarios
|
| 134 |
- Knowledge graphs are hand-crafted to showcase AgentGraph's complete feature set
|
| 135 |
- Examples include actual failure detection (spelling errors, system gaps)
|
|
|
|
| 136 |
- Optimization recommendations are practical and actionable
|
| 137 |
- Content references are accurate and support proper traceability
|
| 138 |
- Quality scores reflect realistic assessment metrics
|
|
|
|
| 19 |
1. **Oxford Economics AI Assistant** (Enhanced)
|
| 20 |
|
| 21 |
- Type: `aif_inference`
|
| 22 |
+
- Example: RAG-powered assistant processing economic inquiry with knowledge search and failure detection
|
| 23 |
- 6 entities, 5 relations, 1 failure, 2 optimizations
|
| 24 |
- Features: Content references, quality scoring, system summary
|
| 25 |
|
|
|
|
| 37 |
- **Agent interactions and roles** with detailed prompts and content references
|
| 38 |
- **Task decomposition** with clear importance levels
|
| 39 |
- **Information flow** with specific interaction prompts
|
| 40 |
+
- **RAG-powered knowledge search** retrieving relevant documents and context
|
| 41 |
- **Failure detection** identifying real issues (spelling errors, system gaps)
|
| 42 |
- **Optimization recommendations** providing actionable improvements
|
| 43 |
- **Quality assessment** with confidence scores and metadata
|
|
|
|
| 123 |
## Benefits for Users
|
| 124 |
|
| 125 |
1. **Immediate Value**: New users see AgentGraph's complete capabilities immediately
|
| 126 |
+
2. **Learning**: Examples demonstrate RAG search, failure detection, optimization suggestions, and advanced features
|
| 127 |
3. **Testing**: Users can test all AgentGraph features including quality assessment and content referencing
|
| 128 |
4. **Reference**: Examples serve as high-quality templates showcasing best practices
|
| 129 |
5. **Feature Discovery**: Users understand the full potential of knowledge graph enhancement
|
|
|
|
| 134 |
- All sample traces are realistic and demonstrate real-world scenarios
|
| 135 |
- Knowledge graphs are hand-crafted to showcase AgentGraph's complete feature set
|
| 136 |
- Examples include actual failure detection (spelling errors, system gaps)
|
| 137 |
+
- RAG search capabilities demonstrate knowledge retrieval workflows
|
| 138 |
- Optimization recommendations are practical and actionable
|
| 139 |
- Content references are accurate and support proper traceability
|
| 140 |
- Quality scores reflect realistic assessment metrics
|
backend/database/sample_data.py
CHANGED
|
@@ -15,10 +15,10 @@ SAMPLE_TRACES = [
|
|
| 15 |
{
|
| 16 |
"filename": "oxford_economics_inquiry.json",
|
| 17 |
"title": "Oxford Economics AI Assistant Demo",
|
| 18 |
-
"description": "Enhanced example showing AI assistant processing economic inquiry with failure detection and optimization suggestions",
|
| 19 |
"trace_type": "aif_inference",
|
| 20 |
"trace_source": "sample_data",
|
| 21 |
-
"tags": ["economics", "
|
| 22 |
"content": """{
|
| 23 |
"id": "aif_trace_demo_001",
|
| 24 |
"timestamp": "2025-01-27T00:00:00",
|
|
@@ -29,14 +29,14 @@ SAMPLE_TRACES = [
|
|
| 29 |
},
|
| 30 |
"data": {
|
| 31 |
"total_observations": 1,
|
| 32 |
-
"summary": "Oxford Economics inquiry with
|
| 33 |
},
|
| 34 |
"observations": [
|
| 35 |
{
|
| 36 |
"id": "demo_obs_001",
|
| 37 |
"type": "inference",
|
| 38 |
"timestamp": "2025-01-27T00:00:00",
|
| 39 |
-
"input": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on the provided context. Use the information
|
| 40 |
"output": "Oxford Economics provides economic analysis, forecasting, and consultancy services.",
|
| 41 |
"metadata": {
|
| 42 |
"request_date": "2025-01-27T00:00:00",
|
|
@@ -100,14 +100,14 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
|
|
| 100 |
"trace_index": 0, # Links to first trace
|
| 101 |
"graph_data": {
|
| 102 |
"system_name": "Oxford Economics AI Assistant",
|
| 103 |
-
"system_summary": "This intelligent assistant processes user inquiries about Oxford Economics through a streamlined workflow. When users submit questions, the knowledgeable agent analyzes them using
|
| 104 |
"entities": [
|
| 105 |
{
|
| 106 |
"id": "agent_001",
|
| 107 |
"type": "Agent",
|
| 108 |
"name": "Oxford Economics Knowledge Agent",
|
| 109 |
"importance": "HIGH",
|
| 110 |
-
"raw_prompt": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on
|
| 111 |
"raw_prompt_ref": [
|
| 112 |
{
|
| 113 |
"line_start": 31,
|
|
@@ -170,9 +170,9 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
|
|
| 170 |
{
|
| 171 |
"id": "tool_001",
|
| 172 |
"type": "Tool",
|
| 173 |
-
"name": "
|
| 174 |
"importance": "HIGH",
|
| 175 |
-
"raw_prompt": "
|
| 176 |
"raw_prompt_ref": [
|
| 177 |
{
|
| 178 |
"line_start": 49,
|
|
@@ -244,7 +244,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
|
|
| 244 |
"target": "tool_001",
|
| 245 |
"type": "USES",
|
| 246 |
"importance": "HIGH",
|
| 247 |
-
"interaction_prompt": "Agent
|
| 248 |
"interaction_prompt_ref": [
|
| 249 |
{
|
| 250 |
"line_start": 49,
|
|
@@ -284,7 +284,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
|
|
| 284 |
{
|
| 285 |
"id": "opt_002",
|
| 286 |
"recommendation_type": "TOOL_ENHANCEMENT",
|
| 287 |
-
"description": "
|
| 288 |
"affected_ids": ["tool_001"],
|
| 289 |
"raw_text_ref": [
|
| 290 |
{
|
|
@@ -334,7 +334,7 @@ SAMPLE_KNOWLEDGE_GRAPHS = [
|
|
| 334 |
"type": "Agent",
|
| 335 |
"name": "Q&A Assistant",
|
| 336 |
"importance": "HIGH",
|
| 337 |
-
"raw_prompt": "
|
| 338 |
"raw_prompt_ref": [
|
| 339 |
{
|
| 340 |
"line_start": 3,
|
|
@@ -559,6 +559,6 @@ def get_sample_data_info():
|
|
| 559 |
"knowledge_graphs_count": len(SAMPLE_KNOWLEDGE_GRAPHS),
|
| 560 |
"trace_types": list(set(t["trace_type"] for t in SAMPLE_TRACES)),
|
| 561 |
"complexity_levels": ["enhanced", "simple"],
|
| 562 |
-
"features": ["failure_detection", "optimization_recommendations", "content_references", "quality_scoring"],
|
| 563 |
-
"description": "Enhanced AgentGraph examples showcasing Oxford Economics
|
| 564 |
}
|
|
|
|
| 15 |
{
|
| 16 |
"filename": "oxford_economics_inquiry.json",
|
| 17 |
"title": "Oxford Economics AI Assistant Demo",
|
| 18 |
+
"description": "Enhanced example showing RAG-powered AI assistant processing economic inquiry with knowledge search, failure detection and optimization suggestions",
|
| 19 |
"trace_type": "aif_inference",
|
| 20 |
"trace_source": "sample_data",
|
| 21 |
+
"tags": ["economics", "rag_assistant", "knowledge_search", "failure_detection", "optimization"],
|
| 22 |
"content": """{
|
| 23 |
"id": "aif_trace_demo_001",
|
| 24 |
"timestamp": "2025-01-27T00:00:00",
|
|
|
|
| 29 |
},
|
| 30 |
"data": {
|
| 31 |
"total_observations": 1,
|
| 32 |
+
"summary": "Oxford Economics inquiry with RAG-powered assistant response"
|
| 33 |
},
|
| 34 |
"observations": [
|
| 35 |
{
|
| 36 |
"id": "demo_obs_001",
|
| 37 |
"type": "inference",
|
| 38 |
"timestamp": "2025-01-27T00:00:00",
|
| 39 |
+
"input": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on the provided context. Use the information retrieved from the knowledge base below to respond concisely and correctly.\\n\\n### Retrieved Documents:\\n['Oxford Economics contact: Mateusz Urban - murban@oxfordeconomics.com', 'Oxford Economics research and analysis services', 'Economic forecasting and consultancy expertise']\\n\\n### Question:\\nwhat does oxford economics present?\\n\\n### Answer:",
|
| 40 |
"output": "Oxford Economics provides economic analysis, forecasting, and consultancy services.",
|
| 41 |
"metadata": {
|
| 42 |
"request_date": "2025-01-27T00:00:00",
|
|
|
|
| 100 |
"trace_index": 0, # Links to first trace
|
| 101 |
"graph_data": {
|
| 102 |
"system_name": "Oxford Economics AI Assistant",
|
| 103 |
+
"system_summary": "This intelligent assistant processes user inquiries about Oxford Economics through a streamlined workflow. When users submit questions, the knowledgeable agent analyzes them using RAG-powered knowledge search and delivers accurate, contextual responses about economic analysis and forecasting services.",
|
| 104 |
"entities": [
|
| 105 |
{
|
| 106 |
"id": "agent_001",
|
| 107 |
"type": "Agent",
|
| 108 |
"name": "Oxford Economics Knowledge Agent",
|
| 109 |
"importance": "HIGH",
|
| 110 |
+
"raw_prompt": "You are a knowledgeable assistant on Oxford Economics designed to answer questions accurately based on retrieved knowledge base context. Use the search results to provide precise responses.",
|
| 111 |
"raw_prompt_ref": [
|
| 112 |
{
|
| 113 |
"line_start": 31,
|
|
|
|
| 170 |
{
|
| 171 |
"id": "tool_001",
|
| 172 |
"type": "Tool",
|
| 173 |
+
"name": "Oxford Economics Knowledge Search",
|
| 174 |
"importance": "HIGH",
|
| 175 |
+
"raw_prompt": "Retrieval-Augmented Generation (RAG) system that searches Oxford Economics knowledge base for relevant documents, contact information, and service details to provide contextual information.",
|
| 176 |
"raw_prompt_ref": [
|
| 177 |
{
|
| 178 |
"line_start": 49,
|
|
|
|
| 244 |
"target": "tool_001",
|
| 245 |
"type": "USES",
|
| 246 |
"importance": "HIGH",
|
| 247 |
+
"interaction_prompt": "Agent queries knowledge search system to retrieve relevant Oxford Economics documents and contextual information",
|
| 248 |
"interaction_prompt_ref": [
|
| 249 |
{
|
| 250 |
"line_start": 49,
|
|
|
|
| 284 |
{
|
| 285 |
"id": "opt_002",
|
| 286 |
"recommendation_type": "TOOL_ENHANCEMENT",
|
| 287 |
+
"description": "Expand knowledge search capabilities to include real-time market data, recent publications, and dynamic economic indicators alongside static company information.",
|
| 288 |
"affected_ids": ["tool_001"],
|
| 289 |
"raw_text_ref": [
|
| 290 |
{
|
|
|
|
| 334 |
"type": "Agent",
|
| 335 |
"name": "Q&A Assistant",
|
| 336 |
"importance": "HIGH",
|
| 337 |
+
"raw_prompt": "RAG-powered assistant specialized in searching knowledge base and providing accurate information about Oxford Economics services and capabilities.",
|
| 338 |
"raw_prompt_ref": [
|
| 339 |
{
|
| 340 |
"line_start": 3,
|
|
|
|
| 559 |
"knowledge_graphs_count": len(SAMPLE_KNOWLEDGE_GRAPHS),
|
| 560 |
"trace_types": list(set(t["trace_type"] for t in SAMPLE_TRACES)),
|
| 561 |
"complexity_levels": ["enhanced", "simple"],
|
| 562 |
+
"features": ["rag_search", "failure_detection", "optimization_recommendations", "content_references", "quality_scoring"],
|
| 563 |
+
"description": "Enhanced AgentGraph examples showcasing Oxford Economics RAG-powered assistant with knowledge search, failure detection, optimization suggestions, and advanced knowledge graph features"
|
| 564 |
}
|