| # META_PLAN: DeepBoner Stabilization Roadmap | |
| **Created**: 2025-12-03 | |
| **Updated**: 2025-12-04 | |
| **Status**: ✅ COMPLETE | |
| **Purpose**: Single source of truth for what to do next before adding features | |
| --- | |
| ## Executive Summary | |
| **Codebase Health**: PRODUCTION-READY | |
| - 302 tests passing | |
| - No type errors (mypy clean) | |
| - No linting issues (ruff clean) | |
| - All P3 tech debt resolved | |
| **Key Finding**: Architecture is sound. All specs implemented. Tech debt cleaned. | |
| **Status**: Steps 1-3 COMPLETE. Ready for feature development or documentation polish. | |
| --- | |
| ## Current State Assessment | |
| ### Documentation Status | |
| | Document | Status | Action | | |
| |----------|--------|--------| | |
| | `docs/STATUS_LLAMAINDEX_INTEGRATION.md` | DONE | Keep as-is | | |
| | `docs/specs/archive/SPEC_13_EVIDENCE_DEDUPLICATION.md` | ✅ IMPLEMENTED | Archived | | |
| | `docs/specs/archive/SPEC_14_CLINICALTRIALS_OUTCOMES.md` | ✅ IMPLEMENTED | Archived | | |
| | `docs/future-roadmap/TOOL_ANALYSIS_CRITICAL.md` | ANALYSIS DONE | Reference for future | | |
| | `docs/ARCHITECTURE.md` | PARTIAL | Expand with diagrams | | |
| | `docs/architecture/system_registry.md` | DONE | Canonical SSOT for wiring | | |
| ### Architecture Status | |
| | Component | Status | Notes | | |
| |-----------|--------|-------| | |
| | `src/orchestrators/` | COMPLETE | Factory pattern, protocols | | |
| | `src/clients/` | COMPLETE | OpenAI/HuggingFace working, Provider Registry pattern | | |
| | `src/tools/` | COMPLETE | Deduplication + outcomes extraction done | | |
| | `src/agents/` | FUNCTIONAL | All agents wired, some experimental | | |
| | `src/services/` | COMPLETE | Embeddings, RAG, memory all working | | |
| ### Open Issues | |
| | Issue | Priority | Effort | | |
| |-------|----------|--------| | |
| | ~~Evidence deduplication (SPEC_13)~~ | ~~HIGH~~ | ✅ DONE | | |
| | ~~ClinicalTrials outcomes (SPEC_14)~~ | ~~HIGH~~ | ✅ DONE | | |
| | ~~Remove Anthropic wiring (P3)~~ | ~~P3~~ | ✅ DONE (PR #130) | | |
| | ~~Remove Modal wiring (P3)~~ | ~~P3~~ | ✅ DONE (PR #130) | | |
| | Expand ARCHITECTURE.md | LOW | 2 hours | | |
| | P3 Progress Bar positioning | P3 | 30 min | | |
| --- | |
| ## The Next 5 Steps | |
| ### ~~Step 1: Implement SPEC_13 - Evidence Deduplication~~ ✅ COMPLETE | |
| **Priority**: ~~HIGH~~ DONE | **Effort**: ~~3-4 hours~~ | **Impact**: 30-50% token savings | |
| ✅ **COMPLETED** - Deduplication now removes duplicate papers from PubMed/Europe PMC/OpenAlex. | |
| **Files modified**: | |
| - `src/tools/search_handler.py` - Added `extract_paper_id()` and `deduplicate_evidence()` | |
| - `src/tools/openalex.py` - Extracts PMID from `work.ids.pmid` | |
| - `tests/unit/tools/test_search_handler.py` - 22 dedup tests | |
| - `tests/integration/test_search_deduplication.py` - Integration test | |
| **Spec**: `docs/specs/archive/SPEC_13_EVIDENCE_DEDUPLICATION.md` (Status: Implemented) | |
| --- | |
| ### ~~Step 2: Implement SPEC_14 - ClinicalTrials Outcomes~~ ✅ COMPLETE | |
| **Priority**: ~~HIGH~~ DONE | **Effort**: ~~2-3 hours~~ | **Impact**: Critical efficacy data | |
| ✅ **COMPLETED** - ClinicalTrials now extracts outcome measures and results status. | |
| **Files modified**: | |
| - `src/tools/clinicaltrials.py` - Added `OutcomesModule`, `HasResults` fields, `_extract_primary_outcome()` | |
| - `tests/unit/tools/test_clinicaltrials.py` - 4 outcome tests + 2 integration tests | |
| **Spec**: `docs/specs/archive/SPEC_14_CLINICALTRIALS_OUTCOMES.md` (Status: Implemented) | |
| --- | |
| ### ~~Step 3: Remove Anthropic + Modal Tech Debt~~ ✅ COMPLETE | |
| **Priority**: ~~P3~~ DONE | **Effort**: ~~1 hour~~ | **Impact**: Code clarity | |
| ✅ **COMPLETED** - Both Anthropic and Modal partial integrations removed in PR #130. | |
| **Files removed/modified**: | |
| - `src/utils/llm_factory.py` - DELETED | |
| - `src/tools/code_execution.py` - DELETED | |
| - `src/services/statistical_analyzer.py` - DELETED | |
| - `src/agents/analysis_agent.py`, `code_executor_agent.py` - DELETED | |
| - All config, factory, and agent files cleaned | |
| **Docs archived**: `docs/bugs/archive/P3_REMOVE_ANTHROPIC_PARTIAL_WIRING.md`, `P3_MODAL_INTEGRATION_REMOVAL.md` | |
| --- | |
| ### Step 4: Documentation Consolidation | |
| **Priority**: MEDIUM | **Effort**: 2 hours | **Impact**: Developer clarity | |
| Create single canonical architecture doc with: | |
| - System flow diagram | |
| - Component interaction map | |
| - Error handling patterns | |
| - Deployment topology | |
| **Output**: Expanded `docs/ARCHITECTURE.md` | |
| --- | |
| ### Step 5: Create Implementation Status Matrix | |
| **Priority**: LOW | **Effort**: 1 hour | **Impact**: Project tracking | |
| Update `docs/index.md` or create `docs/IMPLEMENTATION_STATUS.md` with: | |
| - Phase completion tracking (14 phases) | |
| - Post-hackathon roadmap status | |
| - Clear DONE vs TODO markers | |
| --- | |
| ## What NOT To Do (Yet) | |
| 1. **Add new features** - Stabilize first | |
| 2. **Add new LLM providers** - OpenAI/HuggingFace cover all use cases | |
| 3. **Build Neo4j knowledge graph** - Overkill for current needs | |
| 4. **Implement full-text retrieval** - Phase 15+ (after stabilization) | |
| 5. **Add MeSH term expansion** - Phase 15+ (optimization) | |
| --- | |
| ## Documentation Sprawl Analysis | |
| **Total docs**: 91 markdown files in `docs/` | |
| **Organization**: | |
| ```text | |
| docs/ | |
| ├── architecture/ # Canonical architecture docs (4 files) | |
| ├── brainstorming/ # Ideas, not commitments (6 files) | |
| ├── bugs/ # Active bugs + archive (25+ files) | |
| ├── decisions/ # ADRs from Nov 2025 (2 files) | |
| ├── development/ # Dev guides (1 file) | |
| ├── future-roadmap/ # Deferred work (5 files) | |
| ├── guides/ # User guides (1 file) | |
| ├── implementation/ # Phase docs 1-14 (15 files) | |
| ├── specs/ # Feature specs (4 files) | |
| ├── ARCHITECTURE.md # High-level overview | |
| └── index.md # Entry point | |
| ``` | |
| **Recommendation**: Structure is fine. Both SPEC_13 and SPEC_14 are now implemented. | |
| --- | |
| ## Success Criteria | |
| After completing Steps 1-5: | |
| - [x] Evidence deduplication reduces duplicate papers by 80%+ ✅ | |
| - [x] ClinicalTrials shows outcome measures and results status ✅ | |
| - [x] No Anthropic references in codebase ✅ (PR #130) | |
| - [x] No Modal references in codebase ✅ (PR #130) | |
| - [ ] ARCHITECTURE.md has flow diagrams (optional) | |
| - [ ] All 14 implementation phases marked DONE/TODO (optional) | |
| --- | |
| ## Decision Log | |
| | Date | Decision | Rationale | | |
| |------|----------|-----------| | |
| | 2025-12-03 | Implement specs before doc cleanup | Specs are ready, high impact | | |
| | 2025-12-03 | Remove Anthropic over adding Gemini | Tech debt cleanup > new features | | |
| | 2025-12-03 | Defer full-text retrieval | Stabilize core first | | |
| | 2025-12-03 | Mark SPEC_13 complete | All acceptance criteria verified, PR #122 | | |
| | 2025-12-03 | Mark SPEC_14 complete | All acceptance criteria verified (was already implemented) | | |
| --- | |
| ## References | |
| - `docs/architecture/system_registry.md` - Decorator/marker/tool wiring SSOT | |
| - `docs/bugs/ACTIVE_BUGS.md` - Current bug tracking | |
| - `CLAUDE.md` - Development commands and patterns | |