Design/phase2_planner_agent_design.md

Phase 2: PlannerAgent & 4-Agent MVP System - Design Document

Date: January 2025
Author: BeatDebate Team
Status: Revised (to match implementation) Review Status: Approved

---

0. References

• Main Project Design: Plans/beatdebate-design-doc.md
• Implemented Agent: src/agents/planner_agent.py
• State Model: src/models/agent_models.py (MusicRecommenderState)

---

1. Problem Statement

Objective: Implement a 4-agent MVP system with sophisticated planning capabilities to demonstrate agentic behavior for the AgentX competition.

Current State: We have validated Last.fm data sources and a working API client. Now we need to build the core agent system that showcases strategic planning and coordination.

Value Proposition:

• Demonstrate advanced agentic planning behavior (AgentX core requirement)
• Show sophisticated multi-agent coordination
• Provide transparent reasoning chains for all decisions
• Create a working MVP that can be extended in later phases

---

2. Goals & Non-Goals

✅ In Scope (Phase 2)

• PlannerAgent: Strategic query analysis, coordination planning, and definition of evaluation framework. Relies on an LLM for generation with fallback mechanisms.
• GenreMoodAgent: Strategy-guided genre/mood search implementation (primarily using Last.fm data for MVP)
• DiscoveryAgent: Strategy-guided similarity/discovery search implementation (primarily using Last.fm similarity and data for MVP)
• JudgeAgent: Strategy-informed multi-criteria decision making
• LangGraph Workflow: Complete orchestration of 4-agent process
• State Management: Shared state across agents
• Basic Testing: Unit tests for each agent

❌ Out of Scope (Phase 2)

• Frontend implementation (Phase 3)
• Complex UI/UX features (Phase 3)
• Advanced caching optimization (Phase 3)
• Performance monitoring (Phase 3)
• User authentication (Post-MVP)
• Full playlist generation (Post-MVP)

---

3. Technical Architecture

3.1 Agent Architecture Overview

User Query: "I need focus music for coding"
     ↓
🧠 PlannerAgent: Strategic Analysis & Coordination Planning
├─ Task Analysis: Analyzes query for intent, complexity, mood, genres (LLM-driven)
├─ Agent Coordination Strategy: Defines search parameters for advocates (LLM-driven)
├─ Evaluation Framework: Sets evaluation criteria & diversity targets for judge (LLM-driven)
└─ Execution Monitoring Plan: Defines basic monitoring protocols (LLM-driven)
     ↓
Coordinated Advocate Execution (Parallel):
├─ 🎸 GenreMoodAgent: Genre/mood-based search with strategy
└─ 🔍 DiscoveryAgent: Similarity/discovery search with strategy
     ↓
⚖️ JudgeAgent: Strategy-Informed Decision Making
├─ Applies PlannerAgent's evaluation criteria
├─ Multi-criteria analysis of candidate tracks
└─ Generates explanations with reasoning transparency
     ↓
🎵 Response: 3 strategically selected tracks with full reasoning chains

3.2 LangGraph State Management

from typing import Dict, List, Any, Optional
from pydantic import BaseModel

class MusicRecommenderState(BaseModel):
    """Shared state across all agents in the workflow"""
    user_query: str
    user_profile: Optional[Dict[str, Any]] = None
    
    # Planning phase
    planning_strategy: Optional[Dict[str, Any]] = None # Generated by PlannerAgent
    # execution_plan: Optional[Dict[str, Any]] = None # This is now part of planning_strategy
    
    # Advocate phase  
    genre_mood_recommendations: List[Dict] = [] # List of track data dicts
    discovery_recommendations: List[Dict] = [] # List of track data dicts
    
    # Judge phase
    final_recommendations: List[Any] = [] # List[TrackRecommendation] after JudgeAgent processing
    
    # Reasoning transparency
    reasoning_log: List[str] = []
    agent_deliberations: List[Dict] = [] # Potentially for more detailed logs from each agent
    
    # Metadata
    processing_start_time: Optional[float] = None
    total_processing_time: Optional[float] = None

3.3 Agent Specifications

3.3.1 PlannerAgent

Responsibility: Strategic coordination and planning engine. Uses an LLM to generate planning components and has fallback mechanisms if LLM calls fail.

Key Methods:

class PlannerAgent(BaseAgent):
    def __init__(self, config: AgentConfig, gemini_client=None):
        # ... initialization ...

    async def process(self, state: MusicRecommenderState) -> MusicRecommenderState:
        """Main method to create the comprehensive music discovery strategy.
           Updates state.planning_strategy and returns the updated state.
        """
        # ... orchestrates calls to internal helpers ...
        pass

    async def _analyze_user_query(self, user_query: str) -> Dict[str, Any]:
        """Analyzes user query for complexity, intent, context, mood, genres using LLM or fallback.
        """
        pass

    async def _plan_agent_coordination(self, user_query: str, task_analysis: Dict[str, Any]) -> Dict[str, Any]:
        """Plans coordination strategy for GenreMoodAgent and DiscoveryAgent using LLM or fallback.
        """
        pass

    async def _create_evaluation_framework(self, user_query: str, task_analysis: Dict[str, Any]) -> Dict[str, Any]:
        """Creates evaluation framework (weights, diversity, explanations) for JudgeAgent using LLM or fallback.
        """
        pass

    async def _setup_execution_monitoring(self, task_analysis: Dict[str, Any]) -> Dict[str, Any]:
        """Sets up basic execution monitoring protocols based on task analysis using LLM or fallback.
        """
        pass
    
    # Other helpers: _parse_json_response, _enhance_*, _fallback_*, _initialize_*, call_llm

Strategy Output Format (state.planning_strategy):

{
    "task_analysis": {
        "primary_goal": "concentration_music",
        "complexity_level": "medium", 
        "context_factors": ["work", "focus", "instrumental_preference"],
        "mood_indicators": ["calm", "focused"],
        "genre_hints": ["ambient", "instrumental"]
    },
    "coordination_strategy": {
        "genre_mood_agent": {
            "focus_areas": ["instrumental", "ambient", "post-rock"],
            "energy_level": "medium-low",
            "search_tags": ["focus", "study", "instrumental"],
            "mood_priority": "focused_calm",
            "genre_constraints": ["no_vocals"]
        },
        "discovery_agent": {
            "novelty_priority": "high",
            "similarity_base": "coding_music_archetypes", 
            "underground_bias": 0.7,
            "discovery_scope": "medium",
            "exploration_strategy": "prioritize lesser-known artists similar to ambient focus music"
        }
    },
    "evaluation_framework": {
        "primary_weights": { // Criteria for JudgeAgent to score TrackRecommendation attributes
            "relevance_score": 0.3,       // General relevance to query
            "novelty_score": 0.2,
            "quality_score": 0.2,         // e.g., production, artist recognition (if available)
            "mood_match_score": 0.2,    // How well it matches mood_indicators
            "concentration_friendliness_score": 0.1 // Specific to focus goal
        },
        "diversity_targets": { // For JudgeAgent's _ensure_diversity method
            "attributes": ["genres", "era"], // Attributes on TrackRecommendation model
            "genres": 2, 
            "era": 1,
            "artist": 3 // Example: Target 3 unique artists
        },
        "explanation_style": "concise" // For JudgeAgent's _generate_final_explanations
    },
    "execution_monitoring": {
        "max_advocate_candidates": 20, // Max tracks each advocate should aim for
        "timeout_seconds_advocates": 120,
        "fallback_threshold_judge": 1 // Min candidates before judge uses simplified logic
    }
}

3.3.2 GenreMoodAgent

Responsibility: Strategy-guided genre and mood-based search. Receives its part of coordination_strategy.

Key Methods:

class GenreMoodAgent(BaseAgent):
    # Updated to reflect process(state) pattern
    async def process(self, state: MusicRecommenderState) -> MusicRecommenderState:
        """Executes strategy to find genre/mood tracks, updates state.genre_mood_recommendations.
        """
        # ... extracts its strategy from state.planning_strategy ...
        # ... calls internal methods like _search_by_mood_tags, _apply_genre_filters ...
        # ... appends List[Dict] to state.genre_mood_recommendations ...
        pass
    # async def _search_by_mood_tags(self, strategy_params: Dict) -> List[Dict]
    # async def _apply_genre_filters(self, tracks: List[Dict], strategy_params: Dict) -> List[Dict]
    # ... other internal helpers ...

3.3.3 DiscoveryAgent

Responsibility: Strategy-guided similarity and underground discovery. Receives its part of coordination_strategy.

Key Methods:

class DiscoveryAgent(BaseAgent):
    # Updated to reflect process(state) pattern
    async def process(self, state: MusicRecommenderState) -> MusicRecommenderState:
        """Executes strategy to find similar/novel tracks, updates state.discovery_recommendations.
        """
        # ... extracts its strategy from state.planning_strategy ...
        # ... calls internal methods like _find_similar_underground_tracks, _apply_novelty_scoring ...
        # ... appends List[Dict] to state.discovery_recommendations ...
        pass
    # async def _find_similar_underground_tracks(self, strategy_params: Dict) -> List[Dict]
    # async def _apply_novelty_scoring(self, tracks: List[Dict], strategy_params: Dict) -> List[Dict]
    # ... other internal helpers ...

3.3.4 JudgeAgent

Responsibility: Strategy-informed multi-criteria decision making. Uses evaluation_framework from planning_strategy.

Key Methods (refer to Design/phase2_judge_agent_design.md for updated details):

class JudgeAgent(BaseAgent):
    async def evaluate_and_select(self, state: MusicRecommenderState) -> MusicRecommenderState:
        # ... (updates state.final_recommendations with List[TrackRecommendation]) ...
        pass 
    # def _apply_evaluation_framework(self, candidates: List[Dict], primary_weights: Dict[str, float]) -> List[TrackRecommendation]:
    # def _ensure_diversity(self, candidates: List[TrackRecommendation], diversity_targets: Dict[str, int], num_recommendations: int) -> List[TrackRecommendation]:
    # async def _generate_final_explanations(self, selections: List[TrackRecommendation], evaluation_framework: Dict) -> List[TrackRecommendation]:

---

4. Implementation Plan

4.1 Development Phases

Week 2 Phase 2.1: Core Agent Infrastructure (Days 1-2)

• [✅] Create base agent class with common functionality
• [✅] Implement PlannerAgent with strategic planning logic (LLM-driven with fallbacks)
• [✅] Set up LangGraph state management (MusicRecommenderState)
• [✅] Create agent factory and configuration system
• [✅] Basic unit tests for PlannerAgent

Week 2 Phase 2.2: Advocate Agents (Days 3-4)

• [✅] Implement GenreMoodAgent with strategy execution
• [✅] Implement DiscoveryAgent with novelty scoring
• [✅] Integration with Last.fm API client
• [✅] Unit tests for advocate agents.

Week 2 Phase 2.3: Judge Agent & Workflow (Days 5-7)

• [✅] Implement JudgeAgent with multi-criteria evaluation
• [✅] Complete LangGraph workflow orchestration (src/services/recommendation_engine.py)
• [✅] End-to-end integration testing (ongoing, some tests skipped)
• [✅] Error handling and recovery mechanisms (basic)
• [✅] Performance optimization and logging (basic logging implemented)

4.2 File Structure

src/
├── agents/
│   ├── __init__.py
│   ├── base_agent.py           # Common agent functionality
│   ├── planner_agent.py        # Strategic planning and coordination (LLM-driven)
│   ├── genre_mood_agent.py     # Genre/mood-based search
│   ├── discovery_agent.py      # Similarity/discovery search  
│   └── judge_agent.py          # Multi-criteria decision making
├── services/
│   ├── recommendation_engine.py # LangGraph workflow orchestration
│   ├── __init__.py
│   # ├── state_manager.py        # (State is managed via LangGraph and Pydantic models directly)
│   # └── reasoning_chain.py      # (Reasoning is appended to state.reasoning_log)
└── models/
    ├── agent_models.py         # Pydantic models for state (MusicRecommenderState) and agent configs
    └── recommendation_models.py # Pydantic model for TrackRecommendation

4.3 Testing Strategy

• Unit Tests: Each agent class with mocked dependencies (✅ Implemented for core logic)
• Integration Tests: Full workflow with test data (✅ Some implemented, some skipped due to mocking complexities)
• Performance Tests: Response time and resource usage (Future phase)
• Demo Scenarios: Curated examples for AgentX presentation (Future phase)

---

5. Success Criteria

5.1 Functional Requirements

• [✅] PlannerAgent creates sophisticated strategy objects (LLM-generated or fallback)
• [✅] Advocate agents execute strategies and populate their respective recommendation lists in the state.
• [✅] JudgeAgent applies evaluation frameworks effectively to TrackRecommendation objects.
• [✅] Complete workflow processes queries end-to-end.
• [✅] All agent interactions are logged for transparency (state.reasoning_log).

5.2 Performance Requirements

• End-to-end response time: 2-5 minutes (acceptable for demo quality)
• Strategy generation: < 30 seconds (LLM dependent)
• Each advocate search: < 60 seconds
• Judge evaluation: < 30 seconds
• 95% success rate on test queries (requires comprehensive test suite)
• Memory usage: < 1GB during operation
• API rate limit compliance: Stay within Last.fm (3 req/sec) and Gemini (15 req/min) limits (requires careful management)

5.3 AgentX Competition Requirements

• [✅] Demonstrates sophisticated agentic planning behavior (via PlannerAgent's LLM use and structured output)
• [✅] Shows clear agent coordination and communication (via planning_strategy and shared state)
• [✅] Provides transparent reasoning chains for all decisions (state.reasoning_log)
• [✅] Handles complex queries with strategic decomposition (PlannerAgent's task_analysis)
• [✅] Showcases innovation in multi-agent orchestration (LangGraph usage)

---

6. Risk Mitigation

6.1 Technical Risks

Risk: API rate limiting during agent coordination
Mitigation: Implemented basic asyncio.sleep in some API-heavy loops. More robust queuing/caching for future phases.

Risk: LangGraph state management complexity
Mitigation: Using Pydantic models for state clarity (✅ Implemented).

Risk: Agent reasoning quality inconsistency (especially LLM-dependent parts)
Mitigation: Comprehensive prompt engineering. Fallback mechanisms implemented in PlannerAgent (✅ Implemented). Continuous testing and refinement needed.

6.2 Timeline Risks

Risk: Agent coordination more complex than expected
Mitigation: Incremental workflow development was followed (✅).

Risk: LLM response latency impacting demo
Mitigation: Fallback mechanisms in PlannerAgent help. Response caching and async optimization are future considerations.

---

7. Next Steps

1. Immediate: Review and merge updated design documents.
2. Post-Review: Address skipped integration tests, particularly mocking for RecommendationEngine.
3. Week 3 Start (Conceptual): Move to Phase 3 frontend implementation (if applicable).
4. Ongoing: Document all design decisions for AgentX submission. Continue refining tests and prompts.

This design positions us to showcase sophisticated agentic planning behavior while maintaining a clear implementation timeline for the AgentX competition deadline.

---

8. Future Considerations (Post-MVP)

• ChromaDB Integration and Custom Embedding Implementation:
  • Integrate ChromaDB as a local vector store.
  • Transition DiscoveryAgent to use custom embeddings (e.g., based on genre + tags + mood + similar_artists as per main design document) with ChromaDB as the primary source for similarity searches.
• Advanced Caching: Implement more sophisticated caching beyond basic API client caching.
• Performance Monitoring: Integrate detailed performance monitoring tools.
• Frontend Enhancements: Develop complex UI/UX features.
• Expanded Agent Capabilities: Further refine agent reasoning, negotiation, and learning.