| # SPEC 01: Demo Termination & Timing Fix | |
| ## Priority: P0 (Hackathon Blocker) | |
| ## Problem Statement | |
| Advanced (Magentic) mode runs indefinitely from user perspective. The demo was manually terminated after ~10 minutes without reaching synthesis. | |
| **Root Cause Hypothesis**: We're trusting `agent_framework.MagenticBuilder.max_round_count` to enforce termination, but: | |
| 1. We don't know how the framework counts "rounds" | |
| 2. Our `iteration` counter only tracks `MagenticAgentMessageEvent`, not all framework rounds | |
| 3. Manager coordination messages (JUDGING) happen between rounds and don't count | |
| ## Investigation Required | |
| ### Question 1: Does max_round_count actually work? | |
| ```python | |
| # Current code (src/orchestrator_magentic.py:111) | |
| .with_standard_manager( | |
| chat_client=manager_client, | |
| max_round_count=self._max_rounds, # Default: 10 | |
| max_stall_count=3, | |
| max_reset_count=2, | |
| ) | |
| ``` | |
| **Test**: Set `max_round_count=2` and verify termination. | |
| ### Question 2: What counts as a "round"? | |
| From demo output: | |
| - `JUDGING` (Manager) - many of these | |
| - `SEARCH_COMPLETE` (Agent) | |
| - `HYPOTHESIZING` (Agent) | |
| - `JUDGE_COMPLETE` (Agent) | |
| - `STREAMING` (Delta events) | |
| Is one "round" = one full cycle of all agents? Or one agent message? | |
| ### Question 3: Why no final synthesis? | |
| The demo showed lots of evidence gathering but never reached `ReportAgent`. Either: | |
| 1. JudgeAgent never said "sufficient=True" | |
| 2. Framework terminated before synthesis (unlikely given time) | |
| 3. Something else broke the flow | |
| ## Proposed Solutions | |
| ### Option A: Add Hard Timeout (Recommended for Hackathon) | |
| ```python | |
| # src/orchestrator_magentic.py | |
| import asyncio | |
| async def run(self, query: str) -> AsyncGenerator[AgentEvent, None]: | |
| # ...existing setup... | |
| DEMO_TIMEOUT_SECONDS = 300 # 5 minutes max | |
| try: | |
| async with asyncio.timeout(DEMO_TIMEOUT_SECONDS): | |
| async for event in workflow.run_stream(task): | |
| # ...existing processing... | |
| except TimeoutError: | |
| yield AgentEvent( | |
| type="complete", | |
| message="Research timed out. Synthesizing available evidence...", | |
| data={"reason": "timeout", "iterations": iteration}, | |
| iteration=iteration, | |
| ) | |
| # Attempt to synthesize whatever we have | |
| ``` | |
| ### Option B: Reduce max_rounds AND Add Progress | |
| ```python | |
| # Lower the round count AND show which round we're on | |
| max_round_count=5, # Was 10 | |
| ``` | |
| Plus yield round number: | |
| ```python | |
| yield AgentEvent( | |
| type="progress", | |
| message=f"Round {round_num}/{max_rounds}...", | |
| iteration=round_num, | |
| ) | |
| ``` | |
| ### Option C: Force Synthesis After N Evidence Items | |
| ```python | |
| # In judge logic | |
| if len(evidence) >= 20: | |
| return "synthesize" # We have enough, stop searching | |
| ``` | |
| ## Acceptance Criteria | |
| - [x] Demo completes in <5 minutes with visible progress | |
| - [x] User sees round count (e.g., "Round 3/5") | |
| - [x] Always produces SOME output (even if partial) | |
| - [x] Timeout prevents infinite running | |
| **Status: IMPLEMENTED** (commit b1d094d) | |
| ## Test Plan | |
| ```python | |
| @pytest.mark.asyncio | |
| async def test_magentic_terminates_within_timeout(): | |
| """Verify demo completes in reasonable time.""" | |
| orchestrator = MagenticOrchestrator(max_rounds=3) | |
| events = [] | |
| start = time.time() | |
| async for event in orchestrator.run("simple test query"): | |
| events.append(event) | |
| if time.time() - start > 120: # 2 min max for test | |
| pytest.fail("Orchestrator did not terminate") | |
| # Must have a completion event | |
| assert any(e.type == "complete" for e in events) | |
| ``` | |
| ## Related Issues | |
| - #65: P1: Advanced Mode takes too long for hackathon demo | |
| - #47: E2E Testing | |
| ## Files to Modify | |
| 1. `src/orchestrator_magentic.py` - Add timeout and progress | |
| 2. `src/app.py` - Display round progress in UI | |
| 3. `tests/unit/test_magentic_termination.py` - Add timeout test | |