File size: 2,972 Bytes
4b9f54a 915b009 4b9f54a 915b009 4b9f54a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
# P2 Architectural: BYOK Gaps in Non-Critical Paths
**Date**: 2025-12-03
**Status**: ✅ RESOLVED
**Severity**: P2 (Architectural Debt)
**Component**: LLM Routing / BYOK Support
**Resolution**: Fixed end-to-end BYOK support in this PR
---
## Summary
Two code paths do NOT support BYOK (Bring Your Own Key) from Gradio:
1. **HierarchicalOrchestrator** - Doesn't receive `api_key` parameter
2. **get_model() (PydanticAI)** - Only checks env vars, no BYOK
These are **latent bugs** - they don't affect the main user flow currently.
---
## Bug 1: HierarchicalOrchestrator Missing api_key
**Location**: `src/orchestrators/factory.py:61-64`
```python
if effective_mode == "hierarchical":
from src.orchestrators.hierarchical import HierarchicalOrchestrator
return HierarchicalOrchestrator(config=effective_config, domain=domain)
# BUG: api_key is NOT passed to HierarchicalOrchestrator
```
**Impact**: If hierarchical mode were exposed in UI, BYOK would not work.
**Current State**: Hierarchical mode is NOT exposed in Gradio UI, so this is latent.
**Fix**: Pass `api_key` to HierarchicalOrchestrator when instantiating.
---
## Bug 2: get_model() Doesn't Support BYOK
**Location**: `src/agent_factory/judges.py:62-91` (function `get_model()`)
```python
def get_model() -> Any:
# Priority 1: OpenAI
if settings.has_openai_key: # Only checks ENV VAR
...
# Priority 2: Anthropic
if settings.has_anthropic_key: # Only checks ENV VAR
...
# Priority 3: HuggingFace
if settings.has_huggingface_key: # Only checks ENV VAR
...
```
**Impact**: PydanticAI-based components (judges, statistical analyzer) cannot use BYOK keys.
**Current State**: The main Advanced mode flow uses `get_chat_client()` (Microsoft Agent Framework), NOT `get_model()`. So this is latent.
**Fix**: Either:
1. Add `api_key` parameter to `get_model()`
2. Or deprecate `get_model()` in favor of `get_chat_client()` everywhere
---
## Architecture Notes
The codebase has **TWO separate LLM routing systems**:
| System | Function | BYOK Support | Used By |
|--------|----------|--------------|---------|
| Microsoft Agent Framework | `get_chat_client()` | **YES** (key prefix detection) | Advanced mode (main flow) |
| PydanticAI | `get_model()` | **NO** (env vars only) | Judges, statistical analyzer |
This dual-system architecture creates confusion and maintenance burden.
---
## Recommendation
**Short-term**: Leave as-is (latent, not blocking)
**Long-term**: Unify on `get_chat_client()` and deprecate `get_model()` (see P3_REMOVE_ANTHROPIC_PARTIAL_WIRING.md for related cleanup)
---
## Test Results
- All 310 unit tests pass
- Main user flow (Gradio → Advanced) works with BYOK
---
## Related Documents
- `P3_REMOVE_ANTHROPIC_PARTIAL_WIRING.md` - Related architecture cleanup
- `src/clients/factory.py` - BYOK-capable factory (correct implementation)
- `src/agent_factory/judges.py` - Non-BYOK factory (needs fix)
|