docs/ROADMAP.md

# Ares engineering roadmap and review gates

## Corrections to the initial concept

- **128K vocabulary is not an early milestone.** It makes embedding/output matrices very large. Validate 8–16K first; grow only after corpus-language analysis.
- **8192 context is architecture-ready but not the first training length.** Attention is quadratic. Train 512/1024 first, then lengthen through a measured curriculum and RoPE validation.
- **Petabyte SQLite is unsuitable.** SQLite is for local metadata/profiles. Large corpora need immutable shards, an index, and separate metadata services.
- **RLHF requires human preference data and governance.** Start with SFT/evaluation; use DPO/RLHF only with consented data, reward evaluation, and rollback.
- Capability never justifies unrestricted execution: tool use is reviewed, sandboxed, and least-privilege.

## Phases
1. **Foundation (included):** local BPE, reference decoder transformer, causal loss/AdamW, corpus stream, checkpoint, SQLite memory, static UI.
2. **Data/evaluation:** dataset manifest with licence/provenance/hash, dedupe, PII filtering, held-out perplexity and safety/task evals. No automatic downloads.
3. **Scale:** mixed precision, gradient accumulation/checkpointing, distributed plan, token budget and hardware estimate. Compare 20M → 100M before proposing 1B.
4. **Post-training:** curated SFT and adversarial tests; preference governance before DPO/RLHF.
5. **RAG/tools:** retrieval from approved docs with citations. Xiphos only drafts plans; user approves each action.
6. **Deployment:** Static Space UI plus separately approved inference. Browser inference is optional and constrained by device RAM/download.

## Xiphos plan contract
Every plan states purpose, scope, files/data, licences, risks, tests, rollback, resource estimate, and proposed commands. It is never executable until explicit user approval, then runs only in a sandbox allowlist.