Merge pull request #124 from The-Obstacle-Is-The-Way/fix/p1-forced-synthesis

SPEC_ARCHITECTURAL_DEBT.md

@@ -0,0 +1,355 @@
+# Architectural Debt Specification
+
+> **Status**: IMPLEMENTED (Phase 1 Complete)
+> **Date**: 2025-12-04
+> **Scope**: `src/orchestrators/advanced.py` (Primary), System-Wide (Secondary)
+> **Purpose**: Roadmap for "DeepMind Status" Code Quality
+> **Author**: Claude (Senior Review Incorporated)
+
+---
+
+## Executive Summary
+
+The P1/P2 bug fixes in PR #124 introduced technical debt that must be addressed before the PR is considered "done". This spec documents three immediate priorities (DRY violation, redundant imports, magic strings) and five medium-term system-wide improvements.
+
+---
+
+## Part 1: Immediate Cleanup (MUST Complete Before PR Merge)
+
+### Priority 1: DRY Violation - Synthesis Methods
+
+**Problem**: `_handle_timeout()` (lines 201-248) and `_force_synthesis()` (lines 250-297) are **95% identical**.
+
+| `_handle_timeout()` | `_force_synthesis()` |
+|---------------------|----------------------|
+| Lines 201-248 (47 lines) | Lines 250-297 (47 lines) |
+| Yields "Workflow timed out. Synthesizing..." | Yields "Synthesizing research findings..." |
+| Error data: `timeout_synthesis` | Error data: `forced_synthesis` |
+| **Everything else is identical** | **Everything else is identical** |
+
+**SOLID Violation**: **DRY (Don't Repeat Yourself)**. Changes to synthesis logic must be made in two places. This is a maintenance nightmare and a source of future bugs.
+
+**Fix**: Extract unified method `_synthesize_fallback(iteration: int, reason: str)`:
+
+```python
+async def _synthesize_fallback(
+    self, iteration: int, reason: str
+) -> AsyncGenerator[AgentEvent, None]:
+    """
+    Unified fallback synthesis for all termination scenarios.
+
+    Args:
+        iteration: Current workflow iteration
+        reason: Why synthesis is being forced ("timeout", "no_reporter", "max_rounds")
+    """
+    status_messages = {
+        "timeout": "Workflow timed out. Synthesizing available evidence...",
+        "no_reporter": "Synthesizing research findings...",
+        "max_rounds": "Max rounds reached. Synthesizing findings...",
+    }
+
+    try:
+        state = get_magentic_state()
+        evidence_summary = await state.memory.get_context_summary()
+        report_agent = create_report_agent(self._chat_client, domain=self.domain)
+
+        yield AgentEvent(
+            type="synthesizing",
+            message=status_messages.get(reason, "Synthesizing..."),
+            iteration=iteration,
+        )
+
+        synthesis_result = await report_agent.run(
+            f"Synthesize research report from this evidence. "
+            f"If evidence is sparse, say so.\n\n{evidence_summary}"
+        )
+
+        yield AgentEvent(
+            type="complete",
+            message=synthesis_result.text,
+            data={"reason": f"{reason}_synthesis", "iterations": iteration},
+            iteration=iteration,
+        )
+    except Exception as synth_error:
+        logger.error(f"{reason} synthesis failed", error=str(synth_error))
+        yield AgentEvent(
+            type="complete",
+            message=f"Research completed. Synthesis failed: {synth_error}",
+            data={"reason": f"{reason}_synthesis_failed", "iterations": iteration},
+            iteration=iteration,
+        )
+```
+
+**Call Sites to Update**:
+1. Line 447: `async for event in self._handle_timeout(iteration):` → `self._synthesize_fallback(iteration, "timeout")`
+2. Line 412: `async for synth_event in self._force_synthesis(iteration):` → `self._synthesize_fallback(iteration, "no_reporter")`
+3. Line 432: `async for synth_event in self._force_synthesis(iteration):` → `self._synthesize_fallback(iteration, "max_rounds")`
+
+**Delete After Refactor**:
+- `_handle_timeout()` method (lines 201-248)
+- `_force_synthesis()` method (lines 250-297)
+
+---
+
+### Priority 2: Redundant Imports
+
+**Problem**: Imports inside methods that already exist at module level.
+
+| Location | Import | Already At |
+|----------|--------|------------|
+| Line 207 | `from src.agents.magentic_agents import create_report_agent` | Line 35 |
+| Line 208 | `from src.agents.state import get_magentic_state` | Missing! |
+| Line 257 | `from src.agents.magentic_agents import create_report_agent` | Line 35 |
+| Line 258 | `from src.agents.state import get_magentic_state` | Missing! |
+
+**SOLID Violation**: **SRP (Single Responsibility)**. Import management is scattered across the file instead of centralized at the top.
+
+**Fix**:
+1. Add to module-level imports (around line 38):
+   ```python
+   from src.agents.state import get_magentic_state, init_magentic_state
+   ```
+   Note: `init_magentic_state` is already imported at line 38. Add `get_magentic_state` to that import.
+
+2. Remove redundant imports from:
+   - Lines 207-208 (inside `_handle_timeout`)
+   - Lines 257-258 (inside `_force_synthesis`)
+
+---
+
+### Priority 3: Magic Strings
+
+**Problem**: Agent detection relies on string literals that break silently if agents are renamed.
+
+**Current Code** (line 385):
+```python
+agent_name = (event.agent_id or "").lower()
+if "report" in agent_name:  # FRAGILE: Breaks if agent renamed
+    reporter_ran = True
+```
+
+**Also in** `_get_event_type_for_agent()` (lines 593-602):
+```python
+if "search" in agent_lower:    # Magic string
+if "judge" in agent_lower:     # Magic string
+if "hypothes" in agent_lower:  # Magic string
+if "report" in agent_lower:    # Magic string
+```
+
+**SOLID Violation**: **OCP (Open/Closed Principle)**. Renaming an agent requires changes in multiple locations.
+
+**Fix Option A** - Constants:
+```python
+# At module level (after imports)
+REPORTER_AGENT_ID = "reporter"
+SEARCHER_AGENT_ID = "searcher"
+JUDGE_AGENT_ID = "judge"
+HYPOTHESIZER_AGENT_ID = "hypothesizer"
+```
+
+**Fix Option B** - Agent Name Attribute (Preferred):
+```python
+# In magentic_agents.py, ensure each agent has a .name attribute
+# Then in advanced.py:
+if event.agent_id == report_agent.name:
+    reporter_ran = True
+```
+
+**Recommendation**: Option A is simpler and doesn't require modifying agent factory. Use constants.
+
+---
+
+## Part 2: System-Wide Issues (Future PRs)
+
+These are valid concerns identified during code review but are NOT blockers for the current PR.
+
+### Priority 4: Dead Config
+
+**Location**: `src/utils/config.py`
+**Issue**: Zombie configuration values that are never used or raise NotImplemented.
+- `magentic_timeout`: Deprecated, never read
+- `anthropic_api_key`: Config exists but factory raises NotImplemented
+
+**Fix**: Audit config.py, remove dead settings, add deprecation warnings for transitional settings.
+
+---
+
+### Priority 5: Prompt Unification
+
+**Location**: `src/prompts/` vs `src/config/domain.py`
+**Issue**: Two sources of truth for prompts. `src/prompts/` files exist but are ignored. System uses hardcoded strings in `domain.py`.
+
+**Fix**: Pick ONE source of truth. Recommendation: Delete `src/prompts/` if unused, or migrate `domain.py` prompts there.
+
+---
+
+### Priority 6: Factory Monolith
+
+**Location**: `src/clients/factory.py`
+**Issue**: Hardcoded logic for detecting API key prefixes (`sk-` → OpenAI, `sk-ant-` → Anthropic error).
+
+**SOLID Violation**: OCP. Adding a new provider requires modifying the factory.
+
+**Fix**: Provider registry pattern with auto-registration, or strategy pattern with key prefix handlers.
+
+---
+
+### Priority 7: State Class
+
+**Location**: `src/orchestrators/advanced.py` `run()` method
+**Issue**: Method manages 6+ loose variables (`iteration`, `reporter_ran`, `buffer`, `current_agent_id`, `last_streamed_length`, `final_event_received`).
+
+**Fix**: Extract to `WorkflowState` dataclass:
+```python
+@dataclass
+class WorkflowState:
+    iteration: int = 0
+    reporter_ran: bool = False
+    current_message_buffer: str = ""
+    current_agent_id: str | None = None
+    last_streamed_length: int = 0
+    final_event_received: bool = False
+```
+
+---
+
+### Priority 8: Real Integration Tests
+
+**Location**: `tests/e2e/`
+**Issue**: We deleted flaky integration tests. Now we have ZERO automated tests against real APIs.
+
+**Fix**: Create stable `make test-live` suite with:
+- Real HuggingFace Free Tier test
+- Real OpenAI BYOK test
+- Proper timeout handling
+- Skip markers for CI (run manually or on schedule)
+
+---
+
+## Regression Prevention Strategy
+
+**CRITICAL**: Each phase MUST pass smoke tests before merge. Unit tests alone are insufficient.
+
+### Smoke Test Infrastructure
+
+Add to `Makefile`:
+```makefile
+# Smoke tests - run against real APIs (slow, not for CI)
+smoke-free:
+	@echo "Running Free Tier smoke test..."
+	uv run python -m pytest tests/e2e/test_smoke.py::test_free_tier_synthesis -v -s --timeout=600
+
+smoke-paid:
+	@echo "Running Paid Tier smoke test (requires OPENAI_API_KEY)..."
+	uv run python -m pytest tests/e2e/test_smoke.py::test_paid_tier_synthesis -v -s --timeout=300
+
+smoke: smoke-free  # Default to free tier
+```
+
+### Smoke Test Implementation
+
+Create `tests/e2e/test_smoke.py`:
+```python
+"""
+Smoke tests for regression prevention.
+
+These tests run against REAL APIs and verify end-to-end functionality.
+They are slow (2-5 minutes) and should NOT run in CI.
+
+Usage:
+    make smoke-free   # Test Free Tier (HuggingFace)
+    make smoke-paid   # Test Paid Tier (OpenAI BYOK)
+"""
+import pytest
+from src.orchestrators.advanced import AdvancedOrchestrator
+
+@pytest.mark.e2e
+@pytest.mark.timeout(600)  # 10 minute timeout for Free Tier
+async def test_free_tier_synthesis():
+    """Verify Free Tier produces actual synthesis (not just 'Research complete.')"""
+    orch = AdvancedOrchestrator(max_rounds=2)
+
+    events = []
+    async for event in orch.run("What is libido?"):
+        events.append(event)
+
+    # MUST have a complete event
+    complete_events = [e for e in events if e.type == "complete"]
+    assert len(complete_events) >= 1, "No complete event received"
+
+    # Complete event MUST have substantive content (not just signal)
+    final = complete_events[-1]
+    assert len(final.message) > 100, f"Synthesis too short: {len(final.message)} chars"
+    assert "Research complete." not in final.message or len(final.message) > 50, \
+        "Got empty synthesis signal instead of actual report"
+
+    # Should NOT have duplicate content
+    messages = [e.message for e in events if e.message]
+    # Check for exact duplicates of long content
+    long_messages = [m for m in messages if len(m) > 200]
+    assert len(long_messages) == len(set(long_messages)), "Duplicate content detected"
+
+@pytest.mark.e2e
+@pytest.mark.timeout(300)  # 5 minute timeout for Paid Tier
+async def test_paid_tier_synthesis():
+    """Verify Paid Tier (BYOK) produces synthesis."""
+    import os
+    api_key = os.environ.get("OPENAI_API_KEY")
+    if not api_key:
+        pytest.skip("OPENAI_API_KEY not set")
+
+    orch = AdvancedOrchestrator(max_rounds=2, api_key=api_key)
+
+    events = []
+    async for event in orch.run("What is libido?"):
+        events.append(event)
+
+    complete_events = [e for e in events if e.type == "complete"]
+    assert len(complete_events) >= 1
+    assert len(complete_events[-1].message) > 100
+```
+
+### Phase Gate Checklist
+
+Before merging ANY refactoring PR:
+
+```text
+[ ] make check          # All 318+ unit tests pass
+[ ] make smoke-free     # Free Tier produces real synthesis
+[ ] make smoke-paid     # Paid Tier works (if you have key)
+[ ] CodeRabbit approved # No blocking issues
+```
+
+---
+
+## Execution Strategy
+
+### Phase 1: Current PR (REQUIRED)
+Implement **Priority 1, 2, and 3** before merging PR #124.
+
+**Definition of Done**:
+- [x] `_synthesize_fallback(iteration, reason)` implemented
+- [x] `_handle_timeout()` and `_force_synthesis()` deleted
+- [x] All synthesis call sites updated
+- [x] Redundant imports removed
+- [x] `get_magentic_state` added to module-level imports
+- [x] Magic strings replaced with constants
+- [x] All tests pass (`make check`)
+
+### Phase 2: Future PRs (Separate Tickets)
+Create GitHub issues for Priority 4-8. Do NOT bloat the current bug fix PR.
+
+---
+
+## Appendix: Line Number Reference
+
+| Item | Current Location |
+|------|------------------|
+| `_handle_timeout()` | Lines 201-248 |
+| `_force_synthesis()` | Lines 250-297 |
+| Redundant imports (timeout) | Lines 207-208 |
+| Redundant imports (force) | Lines 257-258 |
+| Magic string detection | Line 385 |
+| `_get_event_type_for_agent()` | Lines 582-602 |
+| Module imports | Lines 18-48 |
+| `run()` method | Lines 299-456 |

docs/bugs/ACTIVE_BUGS.md

@@ -57,6 +57,7 @@ All resolved bugs have been moved to `docs/bugs/archive/`. Summary:
 - **P0 Advanced Mode Timeout No Synthesis** - FIXED, actual synthesis on timeout
 
 ### P1 Bugs (All FIXED)
+- **P1 No Synthesis Free Tier** - FIXED in PR fix/p1-forced-synthesis, forced synthesis safety net when ReportAgent doesn't run
 - **P1 Free Tier Tool Execution Failure** - FIXED in PR fix/P1-free-tier-tool-execution, removed premature marker
 - **P1 Gradio Example Click Auto-Submits** - FIXED in PR #120, prevents auto-submit on example click
 - **P1 HuggingFace Router 401 Hyperbolic** - FIXED, invalid token was root cause

docs/bugs/P1_NO_SYNTHESIS_FREE_TIER.md

@@ -0,0 +1,165 @@
+# P1 Bug: No Synthesis Report in Free Tier (Premature Workflow Termination)
+
+**Date**: 2025-12-04
+**Status**: FIXED (PR fix/p1-forced-synthesis)
+**Severity**: P1 (Critical UX - No usable output from research)
+**Component**: `src/orchestrators/advanced.py`
+**Affects**: Free Tier (HuggingFace) primarily, potentially Paid Tier
+
+---
+
+## Executive Summary
+
+The workflow terminates without the ReportAgent ever producing a synthesis report. Users see search results and hypotheses streaming, but the final output is just "Research complete." with no actual research report. This is caused by the 7B Manager model failing to properly delegate to ReportAgent before workflow termination.
+
+---
+
+## Symptom
+
+```text
+📚 **SEARCH_COMPLETE**: searcher: [search results]
+⏱️ **PROGRESS**: Round 1/5 (~3m 0s remaining)
+🔬 **HYPOTHESIZING**: hypothesizer: [hypotheses]
+⏱️ **PROGRESS**: Round 2/5 (~2m 15s remaining)
+✅ **JUDGE_COMPLETE**: judge: [asks for more evidence]
+⏱️ **PROGRESS**: Round 4/5 (~45s remaining)
+Research complete.
+Research complete.   ← NO SYNTHESIS REPORT!
+```
+
+The workflow runs through multiple agents (Search, Hypothesis, Judge) but never reaches the ReportAgent. The user receives no usable research report.
+
+---
+
+## Root Cause Analysis
+
+### Primary Issue: Manager Model Failure
+
+The `with_standard_manager()` in Microsoft Agent Framework uses the provided chat client (HuggingFace 7B model) to coordinate agents. The 7B model:
+
+1. **Cannot follow complex multi-step instructions** - The manager prompt instructs: "When JudgeAgent says SUFFICIENT EVIDENCE → delegate to ReportAgent." The 7B model doesn't reliably follow this.
+
+2. **Triggers premature termination** - The framework has `max_stall_count=3` and `max_reset_count=2`. If the manager keeps making the same delegation or gets confused, the workflow terminates.
+
+3. **Emits final event without synthesis** - The framework sends `MagenticFinalResultEvent` or `WorkflowOutputEvent` without ReportAgent ever running.
+
+### Secondary Issue: Duplicate Complete Events
+
+Both `MagenticFinalResultEvent` and `WorkflowOutputEvent` are emitted when the workflow ends. The previous code handled both, yielding "Research complete." twice.
+
+---
+
+## The Fix
+
+### 1. Track ReportAgent Execution (Forced Synthesis)
+
+Add a `reporter_ran` flag that tracks whether ReportAgent produced output:
+
+```python
+reporter_ran = False  # P1 FIX: Track if ReportAgent produced output
+
+# In MagenticAgentMessageEvent handler:
+agent_name = (event.agent_id or "").lower()
+if "report" in agent_name:
+    reporter_ran = True
+```
+
+### 2. Force Synthesis on Final Event
+
+If the workflow ends without ReportAgent running, force synthesis:
+
+```python
+if isinstance(event, (MagenticFinalResultEvent, WorkflowOutputEvent)):
+    if not reporter_ran:
+        logger.warning("ReportAgent never ran - forcing synthesis")
+        async for synth_event in self._force_synthesis(iteration):
+            yield synth_event
+    else:
+        yield self._handle_final_event(event, iteration, last_streamed_length)
+```
+
+### 3. `_force_synthesis()` Method
+
+Similar to `_handle_timeout()`, invokes ReportAgent directly:
+
+```python
+async def _force_synthesis(self, iteration: int) -> AsyncGenerator[AgentEvent, None]:
+    """Force synthesis when workflow ends without ReportAgent running."""
+    state = get_magentic_state()
+    evidence_summary = await state.memory.get_context_summary()
+    report_agent = create_report_agent(self._chat_client, domain=self.domain)
+
+    yield AgentEvent(type="synthesizing", message="Synthesizing research findings...")
+
+    synthesis_result = await report_agent.run(
+        f"Synthesize research report from this evidence.\n\n{evidence_summary}"
+    )
+
+    yield AgentEvent(type="complete", message=synthesis_result.text)
+```
+
+### 4. Skip Duplicate Final Events
+
+Prevent "Research complete." appearing twice:
+
+```python
+if isinstance(event, (MagenticFinalResultEvent, WorkflowOutputEvent)):
+    if final_event_received:
+        continue  # Skip duplicate final events
+    final_event_received = True
+```
+
+---
+
+## Why This Is The Correct Architecture
+
+| Alternative | Why Wrong |
+|-------------|-----------|
+| Improve manager prompt | 7B models have fundamental reasoning limitations |
+| Use larger model for manager | Defeats "free tier" purpose |
+| Wait for upstream fix | Framework may never change; we control our code |
+| **Forced synthesis safety net** | ✅ Guarantees output regardless of manager behavior |
+
+The `_force_synthesis()` pattern is a **defensive architecture**. It guarantees users always get a research report, even if:
+- The manager model fails to delegate properly
+- The workflow hits stall/reset limits
+- Any unexpected termination occurs
+
+---
+
+## Files Modified
+
+| File | Change |
+|------|--------|
+| `src/orchestrators/advanced.py` | Added `reporter_ran` tracking |
+| `src/orchestrators/advanced.py` | Added `_force_synthesis()` method |
+| `src/orchestrators/advanced.py` | Added duplicate final event skipping |
+| `src/orchestrators/advanced.py` | Added forced synthesis in final event handler |
+| `src/orchestrators/advanced.py` | Added forced synthesis in max rounds fallback |
+
+---
+
+## Test Plan
+
+1. **Free Tier**: Run query, verify synthesis report is always generated
+2. **Paid Tier**: Run query, verify no regression in OpenAI behavior
+3. **Timeout**: Verify existing timeout synthesis still works
+4. **Max Rounds**: Verify synthesis happens even at max rounds
+
+---
+
+## Related
+
+- P2 Duplicate Report Bug (separate issue, also fixed in this PR)
+- P2 First Turn Timeout Bug (previously fixed)
+- Manager model limitations are fundamental to 7B models
+- OpenAI tier works because GPT-5 follows instructions better
+
+---
+
+## Lessons Learned
+
+1. **Defensive architecture** - Don't trust upstream components to always behave correctly
+2. **Tracking flags** - Simple boolean flags can enable powerful safety nets
+3. **AI-native challenges** - When using AI models as infrastructure components, build in fallbacks for model failures
+4. **Regression prevention** - This bug was likely introduced when we unified the architecture; comprehensive test coverage is critical

src/orchestrators/advanced.py

@@ -35,7 +35,7 @@ from src.agents.magentic_agents import (
     create_report_agent,
     create_search_agent,
 )
-from src.agents.state import init_magentic_state
+from src.agents.state import get_magentic_state, init_magentic_state
 from src.clients.base import BaseChatClient
 from src.clients.factory import get_chat_client
 from src.config.domain import ResearchDomain, get_domain_config
@@ -49,6 +49,12 @@ if TYPE_CHECKING:
 
 logger = structlog.get_logger()
 
+# Agent ID constants - prevents silent breakage if agents are renamed
+REPORTER_AGENT_ID = "reporter"
+SEARCHER_AGENT_ID = "searcher"
+JUDGE_AGENT_ID = "judge"
+HYPOTHESIZER_AGENT_ID = "hypothesizer"
+
 
 class AdvancedOrchestrator(OrchestratorProtocol):
     """
@@ -198,32 +204,39 @@ The final output should be a structured research report."""
             iteration=0,
         )
 
-    async def _handle_timeout(self, iteration: int) -> AsyncGenerator[AgentEvent, None]:
-        """Handle workflow timeout by attempting synthesis."""
-        logger.warning("Workflow timed out", iterations=iteration)
-
-        # ACTUALLY synthesize from gathered evidence
-        try:
-            from src.agents.magentic_agents import create_report_agent
-            from src.agents.state import get_magentic_state
+    async def _synthesize_fallback(
+        self, iteration: int, reason: str
+    ) -> AsyncGenerator[AgentEvent, None]:
+        """
+        Unified fallback synthesis for all termination scenarios.
 
-            state = get_magentic_state()
-            memory = state.memory
+        This method handles synthesis when the workflow terminates without
+        a proper report from ReportAgent. It's a safety net for:
+        - Timeout scenarios
+        - Manager model failing to delegate to ReportAgent (7B model limitation)
+        - Max rounds reached without synthesis
 
-            # Get evidence summary from memory
-            evidence_summary = await memory.get_context_summary()
+        Args:
+            iteration: Current workflow iteration count
+            reason: Why synthesis is being forced ("timeout", "no_reporter", "max_rounds")
+        """
+        status_messages = {
+            "timeout": "Workflow timed out. Synthesizing available evidence...",
+            "no_reporter": "Synthesizing research findings...",
+            "max_rounds": "Max rounds reached. Synthesizing findings...",
+        }
 
-            # Create and invoke ReportAgent for synthesis
+        try:
+            state = get_magentic_state()
+            evidence_summary = await state.memory.get_context_summary()
             report_agent = create_report_agent(self._chat_client, domain=self.domain)
 
             yield AgentEvent(
                 type="synthesizing",
-                message="Workflow timed out. Synthesizing available evidence...",
+                message=status_messages.get(reason, "Synthesizing..."),
                 iteration=iteration,
             )
 
-            # Invoke ReportAgent directly
-            # Note: ChatAgent.run() returns AgentRunResponse; access text via .text
             synthesis_result = await report_agent.run(
                 "Synthesize research report from this evidence. "
                 f"If evidence is sparse, say so.\n\n{evidence_summary}"
@@ -232,22 +245,21 @@ The final output should be a structured research report."""
             yield AgentEvent(
                 type="complete",
                 message=synthesis_result.text,
-                data={"reason": "timeout_synthesis", "iterations": iteration},
+                data={"reason": f"{reason}_synthesis", "iterations": iteration},
                 iteration=iteration,
             )
         except Exception as synth_error:
-            logger.error("Timeout synthesis failed", error=str(synth_error))
+            logger.error(f"{reason} synthesis failed", error=str(synth_error))
             yield AgentEvent(
                 type="complete",
-                message=(
-                    f"Research timed out after {iteration} rounds. "
-                    f"Evidence gathered but synthesis failed: {synth_error}"
-                ),
-                data={"reason": "timeout_synthesis_failed", "iterations": iteration},
+                message=f"Research completed. Synthesis failed: {synth_error}",
+                data={"reason": f"{reason}_synthesis_failed", "iterations": iteration},
                 iteration=iteration,
             )
 
-    async def run(self, query: str) -> AsyncGenerator[AgentEvent, None]:
+    async def run(  # noqa: PLR0915 - Complex but necessary for event stream handling
+        self, query: str
+    ) -> AsyncGenerator[AgentEvent, None]:
         """
         Run the workflow.
 
@@ -295,6 +307,7 @@ The final output should be a structured research report."""
 
         iteration = 0
         final_event_received = False
+        reporter_ran = False  # P1 FIX: Track if ReportAgent produced output
 
         # ACCUMULATOR PATTERN: Track streaming content to bypass upstream Repr Bug
         # Upstream bug in _magentic.py flattens message.contents and sets message.text
@@ -328,6 +341,11 @@ The final output should be a structured research report."""
                     if isinstance(event, MagenticAgentMessageEvent):
                         iteration += 1
 
+                        # P1 FIX: Track if ReportAgent produced output
+                        agent_name = (event.agent_id or "").lower()
+                        if REPORTER_AGENT_ID in agent_name:
+                            reporter_ran = True
+
                         comp_event, prog_event = self._handle_completion_event(
                             event, current_message_buffer, iteration
                         )
@@ -340,10 +358,24 @@ The final output should be a structured research report."""
                         current_message_buffer = ""
                         continue
 
-                    # 3. Handle Final Events Inline (P2 Duplicate Report Fix)
+                    # 3. Handle Final Events Inline (P2 Duplicate Report Fix + P1 Forced Synthesis)
                     if isinstance(event, (MagenticFinalResultEvent, WorkflowOutputEvent)):
+                        if final_event_received:
+                            continue  # Skip duplicate final events
                         final_event_received = True
-                        yield self._handle_final_event(event, iteration, last_streamed_length)
+
+                        # P1 FIX: Force synthesis if ReportAgent never ran
+                        if not reporter_ran:
+                            logger.warning(
+                                "ReportAgent never ran - forcing synthesis",
+                                iterations=iteration,
+                            )
+                            async for synth_event in self._synthesize_fallback(
+                                iteration, "no_reporter"
+                            ):
+                                yield synth_event
+                        else:
+                            yield self._handle_final_event(event, iteration, last_streamed_length)
                         continue
 
                     # 4. Handle other events normally
@@ -358,19 +390,24 @@ The final output should be a structured research report."""
                     "Workflow ended without final event",
                     iterations=iteration,
                 )
-                yield AgentEvent(
-                    type="complete",
-                    message=(
-                        f"Research completed after {iteration} agent rounds. "
-                        "Max iterations reached - results may be partial. "
-                        "Try a more specific query for better results."
-                    ),
-                    data={"iterations": iteration, "reason": "max_rounds_reached"},
-                    iteration=iteration,
-                )
+                # P1 FIX: Force synthesis if ReportAgent never ran
+                if not reporter_ran:
+                    async for synth_event in self._synthesize_fallback(iteration, "max_rounds"):
+                        yield synth_event
+                else:
+                    yield AgentEvent(
+                        type="complete",
+                        message=(
+                            f"Research completed after {iteration} agent rounds. "
+                            "Max iterations reached - results may be partial. "
+                            "Try a more specific query for better results."
+                        ),
+                        data={"iterations": iteration, "reason": "max_rounds_reached"},
+                        iteration=iteration,
+                    )
 
         except TimeoutError:
-            async for event in self._handle_timeout(iteration):
+            async for event in self._synthesize_fallback(iteration, "timeout"):
                 yield event
 
         except Exception as e:
@@ -517,13 +554,13 @@ The final output should be a structured research report."""
             Event type string matching AgentEvent.type Literal
         """
         agent_lower = agent_name.lower()
-        if "search" in agent_lower:
+        if SEARCHER_AGENT_ID in agent_lower:
             return "search_complete"
-        if "judge" in agent_lower:
+        if JUDGE_AGENT_ID in agent_lower:
             return "judge_complete"
-        if "hypothes" in agent_lower:
+        if HYPOTHESIZER_AGENT_ID in agent_lower:
             return "hypothesizing"
-        if "report" in agent_lower:
+        if REPORTER_AGENT_ID in agent_lower:
             return "synthesizing"
         return "judging"  # Default for unknown agents
 

tests/unit/orchestrators/test_advanced_timeout.py

@@ -27,11 +27,13 @@ async def test_timeout_synthesizes_evidence():
     mock_workflow.run_stream = slow_stream
 
     # Mock dependencies used inside the timeout block
+    # Note: get_magentic_state and create_report_agent are imported at module level in advanced.py
+    # so we must patch them in that module's namespace, not their original location
     with (
         patch.object(orchestrator, "_build_workflow", return_value=mock_workflow),
         patch("src.orchestrators.advanced.init_magentic_state"),
-        patch("src.agents.state.get_magentic_state") as mock_get_state,
-        patch("src.agents.magentic_agents.create_report_agent") as mock_create_agent,
+        patch("src.orchestrators.advanced.get_magentic_state") as mock_get_state,
+        patch("src.orchestrators.advanced.create_report_agent") as mock_create_agent,
     ):
         # Setup mock state and memory
         mock_memory = AsyncMock()