fix(P0): Implement SPEC_06 - Simple Mode Synthesis Fix (#71)

* feat: implement SPEC_06 fixes for simple mode synthesis

Implements research-backed architecture changes for Simple mode:
- Diverse evidence selection with lost-in-the-middle mitigation
- Debiased judge prompt (scoring only, no decision making)
- Code-enforced termination criteria (prevents infinite loops)
- Real partial synthesis with drug candidates when max iterations reached
- Unit and integration tests for all new logic

* fix: address CodeRabbit review feedback

- Fix type annotation: recommendation str -> Literal["continue", "synthesize"]
- Add

@pytest
.mark.unit markers to unit tests
- Add

@pytest
.mark.integration markers to integration tests
- Simplify early/late evidence assertions using set intersection
- Replace next() with explicit filtering for clearer test diagnostics
- Extract hardcoded thresholds (50, 30) to TERMINATION_CRITERIA
- Fix e2e mock to accept iteration/max_iterations parameters

All 177 tests pass, mypy clean, ruff clean.

* docs: add P3 architectural gap report for ephemeral memory

- Introduced a new document outlining the architectural gap related to the use of ephemeral memory in the EmbeddingService.
- Highlights the lack of persistence in the current implementation, leading to no long-term learning and redundant costs.
- Provides technical details and recommendations for addressing the issue in future product iterations.

docs/bugs/P3_ARCHITECTURAL_GAP_EPHEMERAL_MEMORY.md

@@ -0,0 +1,23 @@
+# P3: Ephemeral Memory Architecture (No Persistence)
+
+**Status:** OPEN
+**Priority:** P3 (Feature/Architecture Gap)
+**Found By:** Codebase Investigation
+**Date:** 2025-11-29
+
+## Description
+The current `EmbeddingService` (`src/services/embeddings.py`) initializes an **in-memory** ChromaDB client (`chromadb.Client()`) and creates a random UUID-based collection for every new session.
+
+While `src/utils/config.py` defines a `chroma_db_path` for persistence, it is currently **ignored**.
+
+## Impact
+1.  **No Long-Term Learning:** The agent cannot "remember" research from previous runs. Every time you restart the app, it starts from zero.
+2.  **Redundant Costs:** If a user researches "Diabetes" twice, the agent re-searches and re-embeds the same papers, wasting tokens and compute time.
+
+## Technical Details
+- **Current:** `self._client = chromadb.Client()` (In-Memory)
+- **Required:** `self._client = chromadb.PersistentClient(path=settings.chroma_db_path)`
+
+## Recommendation
+For a "Hackathon Demo," this is **low priority** (ephemeral is fine).
+For a "Real Product," this is **critical** (users expect a library of research).

src/agent_factory/judges.py

@@ -19,6 +19,7 @@ from src.prompts.judge import (
     SYSTEM_PROMPT,
     format_empty_evidence_prompt,
     format_user_prompt,
+    select_evidence_for_judge,
 )
 from src.utils.config import settings
 from src.utils.models import AssessmentDetails, Evidence, JudgeAssessment
@@ -102,6 +103,8 @@ class JudgeHandler:
         self,
         question: str,
         evidence: list[Evidence],
+        iteration: int = 0,
+        max_iterations: int = 10,
     ) -> JudgeAssessment:
         """
         Assess evidence and determine if it's sufficient.
@@ -109,6 +112,8 @@ class JudgeHandler:
         Args:
             question: The user's research question
             evidence: List of Evidence objects from search
+            iteration: Current iteration number
+            max_iterations: Maximum allowed iterations
 
         Returns:
             JudgeAssessment with evaluation results
@@ -120,11 +125,20 @@ class JudgeHandler:
             "Starting evidence assessment",
             question=question[:100],
             evidence_count=len(evidence),
+            iteration=iteration,
         )
 
         # Format the prompt based on whether we have evidence
         if evidence:
-            user_prompt = format_user_prompt(question, evidence)
+            # Select diverse evidence using embeddings (if available)
+            selected_evidence = await select_evidence_for_judge(evidence, question)
+            user_prompt = format_user_prompt(
+                question,
+                selected_evidence,
+                iteration,
+                max_iterations,
+                total_evidence_count=len(evidence),
+            )
         else:
             user_prompt = format_empty_evidence_prompt(question)
 
@@ -218,6 +232,8 @@ class HFInferenceJudgeHandler:
         self,
         question: str,
         evidence: list[Evidence],
+        iteration: int = 0,
+        max_iterations: int = 10,
     ) -> JudgeAssessment:
         """
         Assess evidence using HuggingFace Inference API.
@@ -246,7 +262,14 @@ class HFInferenceJudgeHandler:
 
         # Format the user prompt
         if evidence:
-            user_prompt = format_user_prompt(question, evidence)
+            selected_evidence = await select_evidence_for_judge(evidence, question)
+            user_prompt = format_user_prompt(
+                question,
+                selected_evidence,
+                iteration,
+                max_iterations,
+                total_evidence_count=len(evidence),
+            )
         else:
             user_prompt = format_empty_evidence_prompt(question)
 
@@ -535,6 +558,8 @@ class MockJudgeHandler:
         self,
         question: str,
         evidence: list[Evidence],
+        iteration: int = 0,
+        max_iterations: int = 10,
     ) -> JudgeAssessment:
         """Return assessment based on actual evidence (demo mode)."""
         self.call_count += 1

src/orchestrators/base.py

@@ -40,12 +40,20 @@ class JudgeHandlerProtocol(Protocol):
     and MockJudgeHandler.
     """
 
-    async def assess(self, question: str, evidence: list[Evidence]) -> JudgeAssessment:
+    async def assess(
+        self,
+        question: str,
+        evidence: list[Evidence],
+        iteration: int = 0,
+        max_iterations: int = 10,
+    ) -> JudgeAssessment:
         """Assess whether collected evidence is sufficient.
 
         Args:
             question: The original research question
             evidence: List of evidence items to assess
+            iteration: Current iteration number
+            max_iterations: Maximum allowed iterations
 
         Returns:
             JudgeAssessment with sufficiency determination and next steps

src/orchestrators/simple.py

@@ -12,7 +12,7 @@ from __future__ import annotations
 
 import asyncio
 from collections.abc import AsyncGenerator
-from typing import TYPE_CHECKING, Any
+from typing import TYPE_CHECKING, Any, ClassVar
 
 import structlog
 
@@ -42,6 +42,18 @@ class Orchestrator:
     Microsoft Agent Framework.
     """
 
+    # Termination thresholds (code-enforced, not LLM-decided)
+    TERMINATION_CRITERIA: ClassVar[dict[str, float]] = {
+        "min_combined_score": 12.0,  # mechanism + clinical >= 12
+        "min_score_with_volume": 10.0,  # >= 10 if 50+ sources
+        "min_evidence_for_volume": 50.0,  # Priority 3: evidence count threshold
+        "late_iteration_threshold": 8.0,  # >= 8 in iterations 8+
+        "max_evidence_threshold": 100.0,  # Force synthesis with 100+ sources
+        "emergency_iteration": 8.0,  # Last 2 iterations = emergency mode
+        "min_confidence": 0.5,  # Minimum confidence for emergency synthesis
+        "min_evidence_for_emergency": 30.0,  # Priority 6: min evidence for emergency
+    }
+
     def __init__(
         self,
         search_handler: SearchHandlerProtocol,
@@ -100,6 +112,7 @@ class Orchestrator:
         try:
             # Deduplicate using semantic similarity
             unique_evidence: list[Evidence] = await embeddings.deduplicate(evidence, threshold=0.85)
+
             logger.info(
                 "Deduplicated evidence",
                 before=len(evidence),
@@ -153,6 +166,65 @@ class Orchestrator:
                 iteration=iteration,
             )
 
+    def _should_synthesize(
+        self,
+        assessment: JudgeAssessment,
+        iteration: int,
+        max_iterations: int,
+        evidence_count: int,
+    ) -> tuple[bool, str]:
+        """
+        Code-enforced synthesis decision.
+
+        Returns (should_synthesize, reason).
+        """
+        combined_score = (
+            assessment.details.mechanism_score + assessment.details.clinical_evidence_score
+        )
+        has_drug_candidates = len(assessment.details.drug_candidates) > 0
+        confidence = assessment.confidence
+
+        # Priority 1: LLM explicitly says sufficient with good scores
+        if assessment.sufficient and assessment.recommendation == "synthesize":
+            if combined_score >= 10:
+                return True, "judge_approved"
+
+        # Priority 2: High scores with drug candidates
+        if (
+            combined_score >= self.TERMINATION_CRITERIA["min_combined_score"]
+            and has_drug_candidates
+        ):
+            return True, "high_scores_with_candidates"
+
+        # Priority 3: Good scores with high evidence volume
+        if (
+            combined_score >= self.TERMINATION_CRITERIA["min_score_with_volume"]
+            and evidence_count >= self.TERMINATION_CRITERIA["min_evidence_for_volume"]
+        ):
+            return True, "good_scores_high_volume"
+
+        # Priority 4: Late iteration with acceptable scores (diminishing returns)
+        is_late_iteration = iteration >= max_iterations - 2
+        if (
+            is_late_iteration
+            and combined_score >= self.TERMINATION_CRITERIA["late_iteration_threshold"]
+        ):
+            return True, "late_iteration_acceptable"
+
+        # Priority 5: Very high evidence count (enough to synthesize something)
+        if evidence_count >= self.TERMINATION_CRITERIA["max_evidence_threshold"]:
+            return True, "max_evidence_reached"
+
+        # Priority 6: Emergency synthesis (avoid garbage output)
+        if (
+            is_late_iteration
+            and evidence_count >= self.TERMINATION_CRITERIA["min_evidence_for_emergency"]
+            and confidence >= self.TERMINATION_CRITERIA["min_confidence"]
+        ):
+            return True, "emergency_synthesis"
+
+        return False, "continue_searching"
+
     async def run(self, query: str) -> AsyncGenerator[AgentEvent, None]:  # noqa: PLR0915
         """
         Run the agent loop for a query.
@@ -252,7 +324,9 @@ class Orchestrator:
             )
 
             try:
-                assessment = await self.judge.assess(query, all_evidence)
+                assessment = await self.judge.assess(
+                    query, all_evidence, iteration, self.config.max_iterations
+                )
 
                 yield AgentEvent(
                     type="judge_complete",
@@ -279,15 +353,37 @@ class Orchestrator:
                     }
                 )
 
-                # === DECISION PHASE ===
-                if assessment.sufficient and assessment.recommendation == "synthesize":
+                # === DECISION PHASE (Code-Enforced) ===
+                should_synth, reason = self._should_synthesize(
+                    assessment=assessment,
+                    iteration=iteration,
+                    max_iterations=self.config.max_iterations,
+                    evidence_count=len(all_evidence),
+                )
+
+                logger.info(
+                    "Synthesis decision",
+                    should_synthesize=should_synth,
+                    reason=reason,
+                    iteration=iteration,
+                    combined_score=assessment.details.mechanism_score
+                    + assessment.details.clinical_evidence_score,
+                    evidence_count=len(all_evidence),
+                    confidence=assessment.confidence,
+                )
+
+                if should_synth:
+                    # Log synthesis trigger reason for debugging
+                    if reason != "judge_approved":
+                        logger.info(f"Code-enforced synthesis triggered: {reason}")
+
                     # Optional Analysis Phase
                     async for event in self._run_analysis_phase(query, all_evidence, iteration):
                         yield event
 
                     yield AgentEvent(
                         type="synthesizing",
-                        message="Evidence sufficient! Preparing synthesis...",
+                        message=f"Evidence sufficient ({reason})! Preparing synthesis...",
                         iteration=iteration,
                     )
 
@@ -300,6 +396,7 @@ class Orchestrator:
                         data={
                             "evidence_count": len(all_evidence),
                             "iterations": iteration,
+                            "synthesis_reason": reason,
                             "drug_candidates": assessment.details.drug_candidates,
                             "key_findings": assessment.details.key_findings,
                         },
@@ -317,10 +414,11 @@ class Orchestrator:
                     yield AgentEvent(
                         type="looping",
                         message=(
-                            f"Need more evidence. "
-                            f"Next searches: {', '.join(current_queries[:2])}..."
+                            f"Gathering more evidence (scores: {assessment.details.mechanism_score}"
+                            f"+{assessment.details.clinical_evidence_score}). "
+                            f"Next: {', '.join(current_queries[:2])}..."
                         ),
-                        data={"next_queries": current_queries},
+                        data={"next_queries": current_queries, "reason": reason},
                         iteration=iteration,
                     )
 
@@ -410,36 +508,93 @@ class Orchestrator:
         evidence: list[Evidence],
     ) -> str:
         """
-        Generate a partial synthesis when max iterations reached.
+        Generate a REAL synthesis when max iterations reached.
 
-        Args:
-            query: The original question
-            evidence: All collected evidence
+        Even when forced to stop, we should provide:
+        - Drug candidates (if any were found)
+        - Key findings
+        - Assessment scores
+        - Actionable citations
 
-        Returns:
-            Formatted partial synthesis as markdown
+        This is still better than a citation dump.
         """
+        # Extract data from last assessment if available
+        last_assessment = self.history[-1]["assessment"] if self.history else {}
+        details = last_assessment.get("details", {})
+
+        drug_candidates = details.get("drug_candidates", [])
+        key_findings = details.get("key_findings", [])
+        mechanism_score = details.get("mechanism_score", 0)
+        clinical_score = details.get("clinical_evidence_score", 0)
+        reasoning = last_assessment.get("reasoning", "Analysis incomplete due to iteration limit.")
+
+        # Format drug candidates
+        if drug_candidates:
+            drug_list = "\n".join([f"- **{d}**" for d in drug_candidates[:5]])
+        else:
+            drug_list = (
+                "- *No specific drug candidates identified in evidence*\n"
+                "- *Try a more specific query or add an API key for better analysis*"
+            )
+
+        # Format key findings
+        if key_findings:
+            findings_list = "\n".join([f"- {f}" for f in key_findings[:5]])
+        else:
+            findings_list = (
+                "- *Key findings require further analysis*\n"
+                "- *See citations below for relevant sources*"
+            )
+
+        # Format citations (top 10)
         citations = "\n".join(
             [
-                f"{i + 1}. [{e.citation.title}]({e.citation.url}) ({e.citation.source.upper()})"
+                f"{i + 1}. [{e.citation.title}]({e.citation.url}) "
+                f"({e.citation.source.upper()}, {e.citation.date})"
                 for i, e in enumerate(evidence[:10])
             ]
         )
 
-        return f"""## Partial Analysis (Max Iterations Reached)
+        combined_score = mechanism_score + clinical_score
+        mech_strength = (
+            "Strong" if mechanism_score >= 7 else "Moderate" if mechanism_score >= 4 else "Limited"
+        )
+        clin_strength = (
+            "Strong" if clinical_score >= 7 else "Moderate" if clinical_score >= 4 else "Limited"
+        )
+        comb_strength = "Sufficient" if combined_score >= 12 else "Partial"
 
-### Question
+        return f"""## Drug Repurposing Analysis
+
+### Research Question
 {query}
 
 ### Status
-Maximum search iterations reached. The evidence gathered may be incomplete.
+Analysis based on {len(evidence)} sources across {len(self.history)} iterations.
+Maximum iterations reached - results may be incomplete.
+
+### Drug Candidates Identified
+{drug_list}
+
+### Key Findings
+{findings_list}
+
+### Evidence Quality Scores
+| Criterion | Score | Interpretation |
+|-----------|-------|----------------|
+| Mechanism | {mechanism_score}/10 | {mech_strength} mechanistic evidence |
+| Clinical | {clinical_score}/10 | {clin_strength} clinical support |
+| Combined | {combined_score}/20 | {comb_strength} for synthesis |
 
-### Evidence Collected
-Found {len(evidence)} sources. Consider refining your query for more specific results.
+### Analysis Summary
+{reasoning}
 
-### Citations
+### Top Citations ({len(evidence)} sources total)
 {citations}
 
 ---
-*Consider searching with more specific terms or drug names.*
+*For more complete analysis:*
+- *Add an OpenAI or Anthropic API key for enhanced LLM analysis*
+- *Try a more specific query (e.g., include drug names)*
+- *Use Advanced mode for multi-agent research*
 """

src/prompts/judge.py

@@ -4,10 +4,16 @@ from src.utils.models import Evidence
 
 SYSTEM_PROMPT = """You are an expert drug repurposing research judge.
 
-Your task is to evaluate evidence from biomedical literature and determine if it's sufficient to
-recommend drug candidates for a given condition.
+Your task is to SCORE evidence from biomedical literature. You do NOT decide whether to
+continue searching or synthesize - that decision is made by the orchestration system
+based on your scores.
 
-## Evaluation Criteria
+## Your Role: Scoring Only
+
+You provide objective scores. The system decides next steps based on explicit thresholds.
+This separation prevents bias in the decision-making process.
+
+## Scoring Criteria
 
 1. **Mechanism Score (0-10)**: How well does the evidence explain the biological mechanism?
    - 0-3: No clear mechanism, speculative
@@ -19,59 +25,123 @@ recommend drug candidates for a given condition.
    - 4-6: Preclinical or early clinical data
    - 7-10: Strong clinical evidence (trials, meta-analyses)
 
-3. **Sufficiency**: Evidence is sufficient when:
-   - Combined scores >= 12 AND
-   - At least one specific drug candidate identified AND
-   - Clear mechanistic rationale exists
-
-## Output Rules
-
-- Always output valid JSON matching the schema
-- Be conservative: only recommend "synthesize" when truly confident
-- If continuing, suggest specific, actionable search queries
-- Never hallucinate drug names or findings not in the evidence
+3. **Drug Candidates**: List SPECIFIC drug names mentioned in the evidence
+   - Only include drugs explicitly mentioned
+   - Do NOT hallucinate or infer drug names
+   - Include drug class if specific names aren't available (e.g., "SSRI antidepressants")
+
+4. **Key Findings**: Extract 3-5 key findings from the evidence
+   - Focus on findings relevant to the research question
+   - Include mechanism insights and clinical outcomes
+
+5. **Confidence (0.0-1.0)**: Your confidence in the scores
+   - Based on evidence quality and relevance
+   - Lower if evidence is tangential or low-quality
+
+## Output Format
+
+Return valid JSON with these fields:
+- details.mechanism_score (int 0-10)
+- details.mechanism_reasoning (string)
+- details.clinical_evidence_score (int 0-10)
+- details.clinical_reasoning (string)
+- details.drug_candidates (list of strings)
+- details.key_findings (list of strings)
+- sufficient (boolean) - TRUE if scores suggest enough evidence
+- confidence (float 0-1)
+- recommendation ("continue" or "synthesize") - Your suggestion (system may override)
+- next_search_queries (list) - If continuing, suggest FOCUSED queries
+- reasoning (string)
+
+## CRITICAL: Search Query Rules
+
+When suggesting next_search_queries:
+- STAY FOCUSED on the original research question
+- Do NOT drift to tangential topics
+- If question is about "female libido", do NOT suggest "bone health" or "muscle mass"
+- Refine existing terms, don't explore random medical associations
 """
 
+MAX_EVIDENCE_FOR_JUDGE = 30  # Keep under token limits
+
 
-def format_user_prompt(question: str, evidence: list[Evidence]) -> str:
+async def select_evidence_for_judge(
+    evidence: list[Evidence],
+    query: str,
+    max_items: int = MAX_EVIDENCE_FOR_JUDGE,
+) -> list[Evidence]:
     """
-    Format the user prompt with question and evidence.
+    Select diverse, relevant evidence for judge evaluation.
 
-    Args:
-        question: The user's research question
-        evidence: List of Evidence objects from search
+    Implements RAG best practices:
+    - Diversity selection over recency-only
+    - Lost-in-the-middle mitigation
+    - Relevance re-ranking
+    """
+    if len(evidence) <= max_items:
+        return evidence
+
+    try:
+        from src.utils.text_utils import select_diverse_evidence
+
+        # Use embedding-based diversity selection
+        return await select_diverse_evidence(evidence, n=max_items, query=query)
+    except ImportError:
+        # Fallback: mix of recent + early (lost-in-the-middle mitigation)
+        early = evidence[: max_items // 3]  # First third
+        recent = evidence[-(max_items * 2 // 3) :]  # Last two-thirds
+        return early + recent
+
+
+def format_user_prompt(
+    question: str,
+    evidence: list[Evidence],
+    iteration: int = 0,
+    max_iterations: int = 10,
+    total_evidence_count: int | None = None,
+) -> str:
+    """
+    Format user prompt with selected evidence and iteration context.
 
-    Returns:
-        Formatted prompt string
+    NOTE: Evidence should be pre-selected using select_evidence_for_judge().
+    This function assumes evidence is already capped.
     """
+    total_count = total_evidence_count or len(evidence)
     max_content_len = 1500
 
     def format_single_evidence(i: int, e: Evidence) -> str:
         content = e.content
         if len(content) > max_content_len:
             content = content[:max_content_len] + "..."
-
         return (
             f"### Evidence {i + 1}\n"
             f"**Source**: {e.citation.source.upper()} - {e.citation.title}\n"
             f"**URL**: {e.citation.url}\n"
-            f"**Date**: {e.citation.date}\n"
             f"**Content**:\n{content}"
         )
 
     evidence_text = "\n\n".join([format_single_evidence(i, e) for i, e in enumerate(evidence)])
 
-    return f"""## Research Question
+    # Lost-in-the-middle mitigation: put critical context at START and END
+    return f"""## Research Question (IMPORTANT - stay focused on this)
 {question}
 
-## Available Evidence ({len(evidence)} sources)
+## Search Progress
+- **Iteration**: {iteration}/{max_iterations}
+- **Total evidence collected**: {total_count} sources
+- **Evidence shown below**: {len(evidence)} diverse sources (selected for relevance)
+
+## Available Evidence
 
 {evidence_text}
 
 ## Your Task
 
-Evaluate this evidence and determine if it's sufficient to recommend drug repurposing candidates.
-Respond with a JSON object matching the JudgeAssessment schema.
+Score this evidence for drug repurposing potential. Provide ONLY scores and extracted data.
+DO NOT decide "synthesize" vs "continue" - that decision is made by the system.
+
+## REMINDER: Original Question (stay focused)
+{question}
 """
 
 

tests/e2e/conftest.py

@@ -39,7 +39,7 @@ def mock_judge_handler():
     """Return a mock judge that always says 'synthesize'."""
     mock = MagicMock()
 
-    async def mock_assess(question, evidence):
+    async def mock_assess(question, evidence, iteration=1, max_iterations=10):
         return JudgeAssessment(
             sufficient=True,
             confidence=0.9,

tests/integration/test_simple_mode_synthesis.py

@@ -0,0 +1,147 @@
+from unittest.mock import AsyncMock
+
+import pytest
+
+from src.orchestrators.simple import Orchestrator
+from src.utils.models import (
+    AssessmentDetails,
+    Citation,
+    Evidence,
+    JudgeAssessment,
+    OrchestratorConfig,
+    SearchResult,
+)
+
+
+def make_evidence(title: str) -> Evidence:
+    return Evidence(
+        content="content",
+        citation=Citation(title=title, url="http://test.com", date="2025", source="pubmed"),
+    )
+
+
+@pytest.mark.integration
+@pytest.mark.asyncio
+async def test_simple_mode_synthesizes_before_max_iterations():
+    """Verify simple mode produces useful output with mocked judge."""
+    # Mock search to return evidence
+    mock_search = AsyncMock()
+    mock_search.execute.return_value = SearchResult(
+        query="test query",
+        evidence=[make_evidence(f"Paper {i}") for i in range(5)],
+        errors=[],
+        sources_searched=["pubmed"],
+        total_found=5,
+    )
+
+    # Mock judge to return GOOD scores eventually
+    # We can use MockJudgeHandler or a pure mock. Let's use a pure mock to control scores precisely.
+    mock_judge = AsyncMock()
+
+    # Iteration 1: Low scores
+    assess_1 = JudgeAssessment(
+        details=AssessmentDetails(
+            mechanism_score=2,
+            mechanism_reasoning="reasoning is sufficient for valid model",
+            clinical_evidence_score=2,
+            clinical_reasoning="reasoning is sufficient for valid model",
+            drug_candidates=[],
+            key_findings=[],
+        ),
+        sufficient=False,
+        confidence=0.5,
+        recommendation="continue",
+        next_search_queries=["q2"],
+        reasoning="need more evidence to support conclusions about this topic",
+    )
+
+    # Iteration 2: High scores (should trigger synthesis)
+    assess_2 = JudgeAssessment(
+        details=AssessmentDetails(
+            mechanism_score=8,
+            mechanism_reasoning="reasoning is sufficient for valid model",
+            clinical_evidence_score=7,
+            clinical_reasoning="reasoning is sufficient for valid model",
+            drug_candidates=["MagicDrug"],
+            key_findings=["It works"],
+        ),
+        sufficient=False,  # Judge is conservative
+        confidence=0.9,
+        recommendation="continue",  # Judge still says continue (simulating bias)
+        next_search_queries=[],
+        reasoning="good scores but maybe more evidence needed technically",
+    )
+
+    mock_judge.assess.side_effect = [assess_1, assess_2]
+
+    orchestrator = Orchestrator(
+        search_handler=mock_search,
+        judge_handler=mock_judge,
+        config=OrchestratorConfig(max_iterations=5),
+    )
+
+    events = []
+    async for event in orchestrator.run("test query"):
+        events.append(event)
+        if event.type == "complete":
+            break
+
+    # Must have synthesis with drug candidates
+    complete_events = [e for e in events if e.type == "complete"]
+    assert len(complete_events) == 1
+    complete_event = complete_events[0]
+
+    assert "MagicDrug" in complete_event.message
+    assert "Drug Candidates" in complete_event.message
+    assert complete_event.data.get("synthesis_reason") == "high_scores_with_candidates"
+    assert complete_event.iteration == 2  # Should stop at it 2
+
+
+@pytest.mark.integration
+@pytest.mark.asyncio
+async def test_partial_synthesis_generation():
+    """Verify partial synthesis includes drug candidates even if max iterations reached."""
+    mock_search = AsyncMock()
+    mock_search.execute.return_value = SearchResult(
+        query="test", evidence=[], errors=[], sources_searched=["pubmed"], total_found=0
+    )
+
+    mock_judge = AsyncMock()
+    # Always return low scores but WITH candidates
+    # Scores 3+3 = 6 < 8 (late threshold), so it should NOT synthesize early
+    mock_judge.assess.return_value = JudgeAssessment(
+        details=AssessmentDetails(
+            mechanism_score=3,
+            mechanism_reasoning="reasoning is sufficient for valid model",
+            clinical_evidence_score=3,
+            clinical_reasoning="reasoning is sufficient for valid model",
+            drug_candidates=["PartialDrug"],
+            key_findings=["Partial finding"],
+        ),
+        sufficient=False,
+        confidence=0.5,
+        recommendation="continue",
+        next_search_queries=[],
+        reasoning="keep going to find more evidence about this topic please",
+    )
+
+    orchestrator = Orchestrator(
+        search_handler=mock_search,
+        judge_handler=mock_judge,
+        config=OrchestratorConfig(max_iterations=2),
+    )
+
+    events = []
+    async for event in orchestrator.run("test"):
+        events.append(event)
+
+    complete_events = [e for e in events if e.type == "complete"]
+    assert (
+        len(complete_events) == 1
+    ), f"Expected exactly one complete event, got {len(complete_events)}"
+    complete_event = complete_events[0]
+    assert complete_event.data.get("max_reached") is True
+
+    # The output message should contain the drug candidate from the last assessment
+    assert "PartialDrug" in complete_event.message
+    assert "Maximum iterations reached" in complete_event.message

tests/unit/orchestrators/test_termination.py

@@ -0,0 +1,104 @@
+from typing import Literal
+from unittest.mock import MagicMock
+
+import pytest
+
+from src.orchestrators.simple import Orchestrator
+from src.utils.models import AssessmentDetails, JudgeAssessment
+
+
+def make_assessment(
+    mechanism: int,
+    clinical: int,
+    drug_candidates: list[str],
+    sufficient: bool = False,
+    recommendation: Literal["continue", "synthesize"] = "continue",
+    confidence: float = 0.8,
+) -> JudgeAssessment:
+    return JudgeAssessment(
+        details=AssessmentDetails(
+            mechanism_score=mechanism,
+            mechanism_reasoning="reasoning is sufficient for testing purposes",
+            clinical_evidence_score=clinical,
+            clinical_reasoning="reasoning is sufficient for testing purposes",
+            drug_candidates=drug_candidates,
+            key_findings=["finding"],
+        ),
+        sufficient=sufficient,
+        confidence=confidence,
+        recommendation=recommendation,
+        next_search_queries=[],
+        reasoning="reasoning is sufficient for testing purposes",
+    )
+
+
+@pytest.fixture
+def orchestrator():
+    search = MagicMock()
+    judge = MagicMock()
+    return Orchestrator(search, judge)
+
+
+@pytest.mark.unit
+def test_should_synthesize_high_scores(orchestrator):
+    """High scores with drug candidates triggers synthesis."""
+    assessment = make_assessment(mechanism=7, clinical=6, drug_candidates=["Metformin"])
+
+    # Access the private method via name mangling or just call it if it was public.
+    # Since I made it private _should_synthesize, I access it directly.
+    should_synth, reason = orchestrator._should_synthesize(
+        assessment, iteration=3, max_iterations=10, evidence_count=50
+    )
+
+    assert should_synth is True
+    assert reason == "high_scores_with_candidates"
+
+
+@pytest.mark.unit
+def test_should_synthesize_late_iteration(orchestrator):
+    """Late iteration with acceptable scores triggers synthesis."""
+    assessment = make_assessment(mechanism=5, clinical=4, drug_candidates=[])
+    should_synth, reason = orchestrator._should_synthesize(
+        assessment, iteration=9, max_iterations=10, evidence_count=80
+    )
+
+    assert should_synth is True
+    assert reason in ["late_iteration_acceptable", "emergency_synthesis"]
+
+
+@pytest.mark.unit
+def test_should_not_synthesize_early_low_scores(orchestrator):
+    """Early iteration with low scores continues searching."""
+    assessment = make_assessment(mechanism=3, clinical=2, drug_candidates=[])
+    should_synth, reason = orchestrator._should_synthesize(
+        assessment, iteration=2, max_iterations=10, evidence_count=20
+    )
+
+    assert should_synth is False
+    assert reason == "continue_searching"
+
+
+@pytest.mark.unit
+def test_judge_approved_overrides_all(orchestrator):
+    """If judge explicitly says synthesize with good scores, do it."""
+    assessment = make_assessment(
+        mechanism=6, clinical=5, drug_candidates=[], sufficient=True, recommendation="synthesize"
+    )
+    should_synth, reason = orchestrator._should_synthesize(
+        assessment, iteration=2, max_iterations=10, evidence_count=20
+    )
+
+    assert should_synth is True
+    assert reason == "judge_approved"
+
+
+@pytest.mark.unit
+def test_max_evidence_threshold(orchestrator):
+    """Force synthesis if we have tons of evidence."""
+    assessment = make_assessment(mechanism=2, clinical=2, drug_candidates=[])
+    should_synth, reason = orchestrator._should_synthesize(
+        assessment, iteration=5, max_iterations=10, evidence_count=150
+    )
+
+    assert should_synth is True
+    assert reason == "max_evidence_reached"

tests/unit/prompts/test_judge_prompt.py

@@ -0,0 +1,61 @@
+from unittest.mock import patch
+
+import pytest
+
+from src.prompts.judge import format_user_prompt, select_evidence_for_judge
+from src.utils.models import Citation, Evidence
+
+
+def make_evidence(title: str, content: str = "content") -> Evidence:
+    return Evidence(
+        content=content,
+        citation=Citation(title=title, url="http://test.com", date="2025", source="pubmed"),
+    )
+
+
+@pytest.mark.unit
+@pytest.mark.asyncio
+async def test_evidence_selection_diverse():
+    """Verify evidence selection includes early and recent items (fallback logic)."""
+    # Create enough evidence to trigger selection
+    evidence = [make_evidence(f"Paper {i}") for i in range(100)]
+
+    # Mock select_diverse_evidence to raise ImportError to trigger fallback logic
+    with patch("src.utils.text_utils.select_diverse_evidence", side_effect=ImportError):
+        selected = await select_evidence_for_judge(evidence, "test query", max_items=30)
+
+    assert len(selected) == 30
+
+    # Should include some early evidence (lost-in-the-middle mitigation)
+    titles = [e.citation.title for e in selected]
+
+    # Check for start (Paper 0..9) - using set intersection for clarity
+    early_papers = {f"Paper {i}" for i in range(10)}
+    has_early = any(title in early_papers for title in titles)
+    # Check for end (Paper 90..99)
+    late_papers = {f"Paper {i}" for i in range(90, 100)}
+    has_late = any(title in late_papers for title in titles)
+
+    assert has_early, "Should include early evidence"
+    assert has_late, "Should include recent evidence"
+
+
+@pytest.mark.unit
+def test_prompt_includes_question_at_edges():
+    """Verify lost-in-the-middle mitigation in prompt formatting."""
+    evidence = [make_evidence("Test Paper")]
+    question = "CRITICAL RESEARCH QUESTION"
+
+    prompt = format_user_prompt(question, evidence, iteration=5, max_iterations=10)
+
+    # Question should appear at START and END of prompt
+    lines = prompt.split("\n")
+
+    # Check start (first few lines)
+    start_content = "\n".join(lines[:10])
+    assert question in start_content
+
+    # Check end (last few lines)
+    end_content = "\n".join(lines[-10:])
+    assert question in end_content
+    assert "REMINDER: Original Question" in end_content