README.md

metadata

title: Ares Lab
emoji: ⚔️
colorFrom: red
colorTo: yellow
sdk: static
pinned: false

Ares — from-scratch transformer research project

Ares is a two-role system: Ares answers and assists; Xiphos drafts reviewable capability plans. Xiphos never executes code, modifies tools, browses, or trains anything without an explicit user approval recorded in a plan.

What this first increment delivers

• A dependency-light, trained byte-level BPE tokenizer (special tokens and chat sections).
• A decoder-only PyTorch transformer: tied token embedding/unembedding, RoPE, causal grouped-query attention, KV cache, pre-norm RMSNorm, SwiGLU, and AdamW training.
• Streaming text pipeline, checkpoint/resume, SQLite profile/memory store, and an approval-gated Xiphos plan schema.
• A dependency-free static Hugging Face Spaces-compatible UI. It is an interface and local memory/planning demonstration; static hosting cannot run a multi-million parameter Python/PyTorch model server-side.

It deliberately starts as a small, testable model. Scaling configuration is a controlled experiment, not a claim of capability.

Honest deployment constraints

Training a credible 1B-parameter foundation model from scratch needs a very large licensed/clean corpus, distributed GPU infrastructure, lengthy runs, and substantial cost. A free static HF Space cannot train or serve it. Static Pages can host this UI and model artifacts, but inference requires either browser-compatible quantized weights with enough client RAM/download, or a separately operated inference service. Neither is secretly substituted with an external model/API here.

Do not scrape or train on data without checking licenses, terms, privacy, and provenance. Never store secrets in this repository.

Quick start (Python 3.10+)

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Place UTF-8, license-reviewed .txt files below corpus/.
python -m ares.tokenizer train --input corpus --out artifacts/tokenizer.json --vocab-size 16000
python -m ares.train --tokenizer artifacts/tokenizer.json --data corpus --out runs/tiny
python -m ares.cli --checkpoint runs/tiny/latest.pt --tokenizer artifacts/tokenizer.json

python -m unittest discover -s tests runs architecture smoke tests.

Hugging Face static setup (jacmor64)

1. Create a Static Space named Ares under jacmor64 (no Docker/Gradio SDK).
2. Upload the contents of apps/static/ to its repository root (including index.html).
3. Host model checkpoints/datasets separately only after their licences, safety review, and sizes are approved. Do not put giant checkpoints in a static UI repository.

Read docs/ROADMAP.md before moving beyond this foundation.

Local two-model runtime (Phase 1)

A Static Space cannot host this Python process. Run it on a computer or separately approved service after training:

python -m ares.init_models --tokenizer artifacts/tokenizer.json --out models
python -m ares.server --ares models/ares.pt --xiphos models/xiphos.pt --tokenizer artifacts/tokenizer.json

init_models produces two random, untrained checkpoints purely to validate wiring. The server explicitly refuses to present them as intelligent. Train Ares and Xiphos separately, evaluate each checkpoint, then mark only reviewed checkpoints as training_complete: true. POST /chat serves Ares; POST /xiphos/plan produces a proposal envelope and never executes it.

Resumable training and role-specific SFT

Training emits metrics.jsonl (loss, learning rate, gradient norm) and checkpointed model + optimizer state. Resume an interrupted run with the same architecture arguments:

python -m ares.train --resume --role ares --tokenizer artifacts/tokenizer-512.json --data corpus --out runs/ares-general-20m --steps 5000 --batch-size 4 --seq-len 256 --dim 384 --layers 12 --heads 6 --kv-heads 2

Prepare only reviewed, locally supplied SFT examples:

python -m ares.sft --role ares --input data/sft/ares_examples.jsonl --out corpus_sft/ares.txt
python -m ares.sft --role xiphos --input data/sft/xiphos_examples.jsonl --out corpus_sft/xiphos.txt