docs: Add P2 bug report for 7B model producing garbage output

This commit introduces a new documentation file detailing a P2 bug where the Qwen2.5-7B-Instruct model generates incoherent streaming output, displaying random tokens instead of meaningful responses. The report includes symptoms, reproduction steps, root cause analysis, impact assessment, potential solutions, and a recommended action plan.

Key findings indicate that the 7B model lacks the reasoning capacity for complex multi-agent prompts, necessitating a review of model selection and architecture for the Free Tier.

Files added:
- P2_7B_MODEL_GARBAGE_OUTPUT.md

P2_7B_MODEL_GARBAGE_OUTPUT.md

@@ -0,0 +1,224 @@
+# P2 Bug: 7B Model Produces Garbage Streaming Output
+
+**Date**: 2025-12-02
+**Status**: OPEN - Investigating
+**Severity**: P2 (Major - Degrades User Experience)
+**Component**: Free Tier / HuggingFace + Multi-Agent Orchestration
+
+---
+
+## Symptoms
+
+When running a research query on Free Tier (Qwen2.5-7B-Instruct), the streaming output shows **garbage tokens** instead of coherent agent reasoning:
+
+```
+📡 **STREAMING**: yarg
+📡 **STREAMING**: PostalCodes
+📡 **STREAMING**: PostalCodes
+📡 **STREAMING**: FunctionFlags
+📡 **STREAMING**: search_pubmed
+📡 **STREAMING**: search_clinical_trials
+📡 **STREAMING**: system
+📡 **STREAMING**: Transferred to searcher, adopt the persona immediately.
+```
+
+The model outputs random tokens like "yarg", "PostalCodes", "FunctionFlags" instead of actual research reasoning.
+
+---
+
+## Reproduction Steps
+
+1. Go to HuggingFace Spaces: https://huggingface.co/spaces/vcms/deepboner
+2. Leave API key empty (Free Tier)
+3. Click any example query or type a question
+4. Click submit
+5. Observe streaming output - garbage tokens appear
+
+**Expected**: Coherent agent reasoning like "Searching PubMed for female libido treatments..."
+**Actual**: Random tokens like "yarg", "PostalCodes"
+
+---
+
+## Root Cause Analysis
+
+### Primary Cause: 7B Model Too Small for Multi-Agent Prompts
+
+The Qwen2.5-7B-Instruct model has **insufficient reasoning capacity** for the complex multi-agent framework. The system requires the model to:
+
+1. **Adopt agent personas** with specialized instructions
+2. **Follow structured workflows** (Search → Judge → Hypothesis → Report)
+3. **Make tool calls** (search_pubmed, search_clinical_trials, etc.)
+4. **Generate JSON-formatted progress ledgers** for workflow control
+5. **Understand manager instructions** and delegate appropriately
+
+A 7B parameter model simply does not have the reasoning depth to handle this. Larger models (70B+) were originally intended, but those are routed to unreliable third-party providers (see `HF_FREE_TIER_ANALYSIS.md`).
+
+### Technical Flow (Where Garbage Appears)
+
+```
+User Query
+    ↓
+AdvancedOrchestrator.run() [advanced.py:247]
+    ↓
+workflow.run_stream(task) [builds Magentic workflow]
+    ↓
+MagenticAgentDeltaEvent emitted with event.text
+    ↓
+Yields AgentEvent(type="streaming", message=event.text) [advanced.py:314-319]
+    ↓
+Gradio displays: "📡 **STREAMING**: {garbage}"
+```
+
+The garbage tokens are **raw model output**. The 7B model is:
+- Not following the system prompt
+- Outputting partial/incomplete token sequences
+- Possibly attempting tool calls but formatting incorrectly
+- Hallucinating random words
+
+### Evidence from Microsoft Reference Framework
+
+The Microsoft Agent Framework's `_magentic.py` (lines 1717-1741) shows how agent invocation works:
+
+```python
+async for update in agent.run_stream(messages=self._chat_history):
+    updates.append(update)
+    await self._emit_agent_delta_event(ctx, update)
+```
+
+The framework passes through whatever the underlying chat client produces. If the model produces garbage, the framework streams it directly.
+
+### Why Click Example vs Submit Shows Different Initial State
+
+Both code paths go through the same `research_agent()` function in `app.py`. The difference:
+
+- **Example click**: Immediately submits query, so you see garbage quickly
+- **Submit button click**: Shows "Starting research (Advanced mode)" banner first, then garbage
+
+Both ultimately produce the same garbage output from the 7B model.
+
+---
+
+## Impact Assessment
+
+| Aspect | Impact |
+|--------|--------|
+| Free Tier Users | Cannot get usable research results |
+| Demo Quality | Appears broken/unprofessional |
+| Trust | Users may think the entire system is broken |
+| Differentiation | Undermines "free tier works!" messaging |
+
+---
+
+## Potential Solutions
+
+### Option 1: Switch to Better Small Model (Recommended - Quick Fix)
+
+Find a small model that better handles complex instructions. Candidates:
+
+| Model | Size | Tool Calling | Instruction Following |
+|-------|------|--------------|----------------------|
+| `mistralai/Mistral-7B-Instruct-v0.3` | 7B | Yes | Better |
+| `microsoft/Phi-3-mini-4k-instruct` | 3.8B | Limited | Good |
+| `google/gemma-2-9b-it` | 9B | Yes | Good |
+| `Qwen/Qwen2.5-14B-Instruct` | 14B | Yes | Better |
+
+**Risk**: 14B model might still be routed to third-party providers. Need to test each.
+
+### Option 2: Simplify Free Tier Architecture
+
+Create a **simpler single-agent mode** for Free Tier:
+- Remove multi-agent coordination (Manager, multiple ChatAgents)
+- Use a single direct query → search → synthesize flow
+- Reduce prompt complexity significantly
+
+**Pros**: More reliable with smaller models
+**Cons**: Loses sophisticated multi-agent research capability
+
+### Option 3: Output Filtering/Validation
+
+Add validation layer to detect and filter garbage output:
+
+```python
+def is_valid_streaming_token(text: str) -> bool:
+    """Check if streaming token appears valid."""
+    # Garbage patterns we've seen
+    garbage_patterns = ["yarg", "PostalCodes", "FunctionFlags"]
+    if any(g in text for g in garbage_patterns):
+        return False
+    # Check for minimum coherence (has spaces, reasonable length)
+    return len(text) > 0 and text.strip()
+```
+
+**Pros**: Band-aid fix, quick to implement
+**Cons**: Doesn't fix root cause, will miss new garbage patterns
+
+### Option 4: Graceful Degradation
+
+Detect when model output is incoherent and fall back to:
+- Returning an error message
+- Suggesting user provide an API key
+- Using a cached/templated response
+
+### Option 5: Prompt Engineering for 7B Models
+
+Significantly simplify the agent prompts for 7B compatibility:
+- Shorter system prompts
+- More explicit step-by-step instructions
+- Remove abstract concepts
+- Use few-shot examples
+
+---
+
+## Recommended Action Plan
+
+### Phase 1: Quick Fix (P2)
+1. Test `mistralai/Mistral-7B-Instruct-v0.3` or `Qwen/Qwen2.5-14B-Instruct`
+2. Verify they stay on HuggingFace native infrastructure (no third-party routing)
+3. Evaluate output quality on sample queries
+
+### Phase 2: Architecture Review (P3)
+1. Consider simplified single-agent mode for Free Tier
+2. Design graceful degradation when model output is invalid
+3. Add output validation layer
+
+### Phase 3: Long-term (P4)
+1. Consider hybrid approach: simple mode for free tier, advanced for paid
+2. Explore fine-tuning a small model specifically for research agent tasks
+
+---
+
+## Files Involved
+
+| File | Relevance |
+|------|-----------|
+| `src/orchestrators/advanced.py` | Main orchestrator, streaming event handling |
+| `src/clients/huggingface.py` | HuggingFace chat client adapter |
+| `src/agents/magentic_agents.py` | Agent definitions and prompts |
+| `src/app.py` | Gradio UI, event display |
+| `src/utils/config.py` | Model configuration |
+
+---
+
+## Relation to Previous Bugs
+
+- **P0 Repr Bug (RESOLVED)**: Fixed in PR #117 - Was about `<generator object>` appearing due to async generator mishandling
+- **P1 HuggingFace Novita Error (RESOLVED)**: Fixed in PR #118 - Was about 72B models being routed to failing third-party providers
+
+This P2 bug is **downstream** of the P1 fix - we fixed the 500 errors by switching to 7B, but now the 7B model doesn't produce quality output.
+
+---
+
+## Questions to Investigate
+
+1. What models in the 7-20B range stay on HuggingFace native infrastructure?
+2. Can we detect third-party routing before making the full request?
+3. Is the chat template correct for Qwen2.5-7B? (Some models need specific formatting)
+4. Are there HuggingFace serverless models specifically optimized for tool calling?
+
+---
+
+## References
+
+- `HF_FREE_TIER_ANALYSIS.md` - Analysis of HuggingFace provider routing
+- `CLAUDE.md` - Critical HuggingFace Free Tier section
+- Microsoft Agent Framework `_magentic.py` - Reference implementation

P3_REMOVE_ANTHROPIC_PARTIAL_WIRING.md

@@ -0,0 +1,160 @@
+# P3 Tech Debt: Remove Anthropic Partial Wiring
+
+**Date**: 2025-12-03
+**Status**: OPEN
+**Severity**: P3 (Tech Debt / Simplification)
+**Component**: Architecture / Provider Integration
+
+---
+
+## Summary
+
+Remove all Anthropic-related code, configuration, and references from the codebase. Anthropic is partially wired but **not fully threaded through the architecture**, creating confusion and half-implemented code paths.
+
+---
+
+## Rationale
+
+### 1. Anthropic Does NOT Provide Embeddings
+
+Our architecture requires embeddings for:
+- RAG (LlamaIndex/ChromaDB)
+- Evidence deduplication
+- Semantic search
+
+Anthropic only provides chat completion, not embeddings. This means even with a working Anthropic chat client, users would need a **second provider** for embeddings, breaking the unified experience.
+
+### 2. Partial Implementation Creates Confusion
+
+Current state:
+- `settings.anthropic_api_key` exists ✅
+- `settings.has_anthropic_key` property exists ✅
+- `settings.anthropic_model` configured ✅
+- `AnthropicChatClient` for agent_framework **DOES NOT EXIST** ❌
+- Code raises `NotImplementedError` when Anthropic detected ❌
+
+This half-state causes:
+- User confusion ("Why doesn't my Anthropic key work?")
+- Developer confusion ("Is Anthropic supported or not?")
+- Dead code paths that need maintenance
+
+### 3. Unified Architecture Principle
+
+**Principle**: Only support providers that work **end-to-end** through the entire stack:
+
+```
+Provider Requirements:
+├── Chat Completion (for agents)     ✅ Required
+├── Function/Tool Calling            ✅ Required
+├── Embeddings (for RAG)             ✅ Required
+└── Streaming                        ✅ Required
+```
+
+| Provider | Chat | Tools | Embeddings | Streaming | Status |
+|----------|------|-------|------------|-----------|--------|
+| OpenAI | ✅ | ✅ | ✅ | ✅ | **KEEP** |
+| HuggingFace | ✅ | ✅ | ✅ (local) | ✅ | **KEEP** |
+| Gemini | ✅ | ✅ | ✅ | ✅ | Future (Phase 4) |
+| Anthropic | ✅ | ✅ | ❌ | ✅ | **REMOVE** |
+
+---
+
+## Files to Clean Up
+
+### Configuration
+- [ ] `src/utils/config.py` - Remove `anthropic_api_key`, `anthropic_model`, `has_anthropic_key`
+
+### Client Factory
+- [ ] `src/clients/factory.py` - Remove Anthropic detection and `NotImplementedError`
+
+### Legacy Code (pydantic-ai based)
+- [ ] `src/utils/llm_factory.py` - Remove `AnthropicModel`, `AnthropicProvider` imports and handling
+- [ ] `src/agent_factory/judges.py` - Remove Anthropic model selection
+
+### App/UI
+- [ ] `src/app.py` - Remove `has_anthropic_key` checks and "Anthropic from env" backend info
+
+### Documentation
+- [ ] `CLAUDE.md` - Update LLM provider list
+- [ ] `AGENTS.md` - Update LLM provider list
+- [ ] `GEMINI.md` - Update LLM provider list
+
+### Tests
+- [ ] `tests/unit/clients/test_chat_client_factory.py` - Remove Anthropic test cases
+- [ ] `tests/unit/utils/test_config.py` - Remove Anthropic config tests
+
+---
+
+## Code Snippets to Remove
+
+### `src/utils/config.py`
+```python
+# REMOVE these lines:
+anthropic_api_key: str | None = Field(default=None, description="Anthropic API key")
+anthropic_model: str = Field(
+    default="claude-sonnet-4-5-20250929", description="Anthropic model"
+)
+
+@property
+def has_anthropic_key(self) -> bool:
+    """Check if Anthropic API key is available."""
+    return bool(self.anthropic_api_key)
+```
+
+### `src/clients/factory.py`
+```python
+# REMOVE these lines:
+if api_key.startswith("sk-ant-"):
+    normalized = "anthropic"
+
+if normalized == "anthropic":
+    raise NotImplementedError(
+        "Anthropic client not yet implemented. "
+        "Use OpenAI key (sk-...) or leave empty for free HuggingFace tier."
+    )
+```
+
+### `src/app.py`
+```python
+# REMOVE these lines:
+elif settings.has_anthropic_key:
+    backend_info = "Paid API (Anthropic from env)"
+
+has_anthropic = settings.has_anthropic_key
+has_paid_key = has_openai or has_anthropic or bool(user_api_key)
+# Change to:
+has_paid_key = has_openai or bool(user_api_key)
+```
+
+---
+
+## Migration Notes
+
+### For Users with Anthropic Keys
+
+If users have `ANTHROPIC_API_KEY` set in their environment:
+1. It will be **silently ignored** (not an error)
+2. System falls through to HuggingFace free tier
+3. Users should use `OPENAI_API_KEY` instead for paid tier
+
+### Future Consideration
+
+If Anthropic adds embeddings API in the future, we can re-add support. But until then, partial support creates more confusion than value.
+
+---
+
+## Definition of Done
+
+- [ ] All Anthropic references removed from `src/`
+- [ ] All Anthropic tests removed or updated
+- [ ] Documentation updated to reflect supported providers: OpenAI, HuggingFace, (future: Gemini)
+- [ ] `make check` passes (lint, typecheck, tests)
+- [ ] PR reviewed and merged
+
+---
+
+## Related Documents
+
+- `P2_7B_MODEL_GARBAGE_OUTPUT.md` - Current free tier model quality issues
+- `HF_FREE_TIER_ANALYSIS.md` - HuggingFace provider routing analysis
+- `CLAUDE.md` - Agent context with provider documentation