Ares engineering roadmap and review gates
Corrections to the initial concept
- 128K vocabulary is not an early milestone. It makes embedding/output matrices very large. Validate 8–16K first; grow only after corpus-language analysis.
- 8192 context is architecture-ready but not the first training length. Attention is quadratic. Train 512/1024 first, then lengthen through a measured curriculum and RoPE validation.
- Petabyte SQLite is unsuitable. SQLite is for local metadata/profiles. Large corpora need immutable shards, an index, and separate metadata services.
- RLHF requires human preference data and governance. Start with SFT/evaluation; use DPO/RLHF only with consented data, reward evaluation, and rollback.
- Capability never justifies unrestricted execution: tool use is reviewed, sandboxed, and least-privilege.
Phases
- Foundation (included): local BPE, reference decoder transformer, causal loss/AdamW, corpus stream, checkpoint, SQLite memory, static UI.
- Data/evaluation: dataset manifest with licence/provenance/hash, dedupe, PII filtering, held-out perplexity and safety/task evals. No automatic downloads.
- Scale: mixed precision, gradient accumulation/checkpointing, distributed plan, token budget and hardware estimate. Compare 20M → 100M before proposing 1B.
- Post-training: curated SFT and adversarial tests; preference governance before DPO/RLHF.
- RAG/tools: retrieval from approved docs with citations. Xiphos only drafts plans; user approves each action.
- Deployment: Static Space UI plus separately approved inference. Browser inference is optional and constrained by device RAM/download.
Xiphos plan contract
Every plan states purpose, scope, files/data, licences, risks, tests, rollback, resource estimate, and proposed commands. It is never executable until explicit user approval, then runs only in a sandbox allowlist.