| --- |
| title: Ares Lab |
| emoji: ⚔️ |
| colorFrom: red |
| colorTo: yellow |
| sdk: static |
| pinned: false |
| --- |
| |
| # Ares — from-scratch transformer research project |
|
|
| Ares is a two-role system: **Ares** answers and assists; **Xiphos** drafts *reviewable* capability plans. Xiphos never executes code, modifies tools, browses, or trains anything without an explicit user approval recorded in a plan. |
|
|
| ## What this first increment delivers |
|
|
| * A dependency-light, trained byte-level BPE tokenizer (special tokens and chat sections). |
| * A decoder-only PyTorch transformer: tied token embedding/unembedding, RoPE, causal grouped-query attention, KV cache, pre-norm RMSNorm, SwiGLU, and AdamW training. |
| * Streaming text pipeline, checkpoint/resume, SQLite profile/memory store, and an approval-gated Xiphos plan schema. |
| * A dependency-free **static** Hugging Face Spaces-compatible UI. It is an interface and local memory/planning demonstration; static hosting cannot run a multi-million parameter Python/PyTorch model server-side. |
|
|
| It deliberately starts as a small, testable model. Scaling configuration is a controlled experiment, not a claim of capability. |
|
|
| ## Honest deployment constraints |
|
|
| Training a credible 1B-parameter foundation model from scratch needs a very large licensed/clean corpus, distributed GPU infrastructure, lengthy runs, and substantial cost. A free static HF Space cannot train or serve it. Static Pages can host this UI and model artifacts, but **inference requires either browser-compatible quantized weights with enough client RAM/download, or a separately operated inference service**. Neither is secretly substituted with an external model/API here. |
|
|
| Do not scrape or train on data without checking licenses, terms, privacy, and provenance. Never store secrets in this repository. |
|
|
| ## Quick start (Python 3.10+) |
|
|
| ```bash |
| python -m venv .venv && source .venv/bin/activate |
| pip install -r requirements.txt |
| # Place UTF-8, license-reviewed .txt files below corpus/. |
| python -m ares.tokenizer train --input corpus --out artifacts/tokenizer.json --vocab-size 16000 |
| python -m ares.train --tokenizer artifacts/tokenizer.json --data corpus --out runs/tiny |
| python -m ares.cli --checkpoint runs/tiny/latest.pt --tokenizer artifacts/tokenizer.json |
| ``` |
|
|
| `python -m unittest discover -s tests` runs architecture smoke tests. |
|
|
| ## Hugging Face static setup (jacmor64) |
|
|
| 1. Create a **Static** Space named `Ares` under `jacmor64` (no Docker/Gradio SDK). |
| 2. Upload the contents of `apps/static/` to its repository root (including `index.html`). |
| 3. Host model checkpoints/datasets separately only after their licences, safety review, and sizes are approved. Do not put giant checkpoints in a static UI repository. |
|
|
| Read `docs/ROADMAP.md` before moving beyond this foundation. |
|
|
| ## Local two-model runtime (Phase 1) |
|
|
| A Static Space cannot host this Python process. Run it on a computer or separately approved service after training: |
|
|
| ```bash |
| python -m ares.init_models --tokenizer artifacts/tokenizer.json --out models |
| python -m ares.server --ares models/ares.pt --xiphos models/xiphos.pt --tokenizer artifacts/tokenizer.json |
| ``` |
|
|
| `init_models` produces two **random, untrained** checkpoints purely to validate wiring. The server explicitly refuses to present them as intelligent. Train Ares and Xiphos separately, evaluate each checkpoint, then mark only reviewed checkpoints as `training_complete: true`. `POST /chat` serves Ares; `POST /xiphos/plan` produces a proposal envelope and **never executes it**. |
|
|
| ## Resumable training and role-specific SFT |
|
|
| Training emits `metrics.jsonl` (loss, learning rate, gradient norm) and checkpointed model + optimizer state. Resume an interrupted run with the same architecture arguments: |
|
|
| ```bash |
| python -m ares.train --resume --role ares --tokenizer artifacts/tokenizer-512.json --data corpus --out runs/ares-general-20m --steps 5000 --batch-size 4 --seq-len 256 --dim 384 --layers 12 --heads 6 --kv-heads 2 |
| ``` |
|
|
| Prepare only reviewed, locally supplied SFT examples: |
|
|
| ```bash |
| python -m ares.sft --role ares --input data/sft/ares_examples.jsonl --out corpus_sft/ares.txt |
| python -m ares.sft --role xiphos --input data/sft/xiphos_examples.jsonl --out corpus_sft/xiphos.txt |
| ``` |
|
|
