ROADMAP.md

Ares Roadmap and Reality Checks

Constraint checks

1. Static Hugging Face Space

   • Good for: UI, documentation, browser-only demos, downloadable checkpoints, localStorage/IndexedDB memory.
   • Not good for: server-side Python, GPU training, persistent server databases, private compute, live web search.

2. No external AI APIs for core intelligence

   • Good principle. Ares training and inference code here is from-scratch.
   • Consequence: no OpenAI/Claude/Gemini/etc. behind the chatbot. Capabilities come only from Ares weights, RAG data, and deterministic tools.

3. Billion-parameter training from scratch

   • Possible in principle.
   • Not realistic on free static hosting. Requires many billions to trillions of tokens, multi-GPU training, checkpoint sharding, monitoring, and data governance.

4. Petabyte-scale database

   • A single SQLite database is not a petabyte system.
   • Practical path: SQLite/FTS5 now, then sharded SQLite or DuckDB/Parquet, then distributed object storage + vector index if compute becomes available.

5. Actual intelligence vs. prediction

   • A Transformer learns by next-token prediction during pretraining.
   • More agentic behavior comes from post-training, retrieval, planning loops, tool use, evaluation, self-critique, and environment feedback. Do not claim consciousness or full AGI.

Development phases

Phase 0 — Static Space and honest prototype

• Static UI.
• Local memory/RAG in browser.
• Architecture dashboard.
• Training instructions.

Phase 1 — Tokenizer and data pipeline

• Train BPE tokenizer.
• Dataset cleaning and deduplication.
• Packing into fixed-length token blocks.
• Data mixture manifests.

Phase 2 — Tiny Transformer smoke tests

• 1M–10M parameters.
• CPU/GPU smoke training.
• Loss decreases on sample data.
• Generate text from checkpoint.

Phase 3 — Small useful model

• 30M–150M parameters.
• More curated text/code/chat data.
• SFT formatting.
• Basic eval suite.

Phase 4 — RAG and long-term memory

• SQLite/FTS5 store.
• Hashed-vector bootstrap retrieval.
• Later replace with Ares-trained embedding head.
• Evidence-grounded answer templates.

Phase 5 — Planning and tools

• Planner proposes steps.
• Sandbox/code actuator runs controlled commands.
• Web search side-goal only after base model exists.
• Static Space can show UI; tools need a backend elsewhere.

Phase 6 — Scaling toward 1B

• 1B target config.
• Multi-GPU FSDP/DeepSpeed or similar.
• Checkpoint sharding.
• Mixed precision.
• Activation checkpointing.
• Larger context and RoPE scaling.

Phase 7 — SFT and preference alignment

• Instruction/chat/code data.
• Supervised fine-tuning.
• Preference data collection.
• DPO/RLHF-style optimization.

Immediate next success criterion

Ares v0.1 is successful when:

• The Static Space loads.
• The browser prototype can store and retrieve local memory.
• The tokenizer trains on sample text.
• The tiny model trains for 100 steps without crashing in a Python GPU/CPU environment.
• Loss decreases on the sample corpus.