File size: 4,200 Bytes
4450dd8 8efdc48 4f53876 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
title: Ares Lab
emoji: ⚔️
colorFrom: red
colorTo: yellow
sdk: static
pinned: false
---
# Ares — from-scratch transformer research project
Ares is a two-role system: **Ares** answers and assists; **Xiphos** drafts *reviewable* capability plans. Xiphos never executes code, modifies tools, browses, or trains anything without an explicit user approval recorded in a plan.
## What this first increment delivers
* A dependency-light, trained byte-level BPE tokenizer (special tokens and chat sections).
* A decoder-only PyTorch transformer: tied token embedding/unembedding, RoPE, causal grouped-query attention, KV cache, pre-norm RMSNorm, SwiGLU, and AdamW training.
* Streaming text pipeline, checkpoint/resume, SQLite profile/memory store, and an approval-gated Xiphos plan schema.
* A dependency-free **static** Hugging Face Spaces-compatible UI. It is an interface and local memory/planning demonstration; static hosting cannot run a multi-million parameter Python/PyTorch model server-side.
It deliberately starts as a small, testable model. Scaling configuration is a controlled experiment, not a claim of capability.
## Honest deployment constraints
Training a credible 1B-parameter foundation model from scratch needs a very large licensed/clean corpus, distributed GPU infrastructure, lengthy runs, and substantial cost. A free static HF Space cannot train or serve it. Static Pages can host this UI and model artifacts, but **inference requires either browser-compatible quantized weights with enough client RAM/download, or a separately operated inference service**. Neither is secretly substituted with an external model/API here.
Do not scrape or train on data without checking licenses, terms, privacy, and provenance. Never store secrets in this repository.
## Quick start (Python 3.10+)
```bash
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Place UTF-8, license-reviewed .txt files below corpus/.
python -m ares.tokenizer train --input corpus --out artifacts/tokenizer.json --vocab-size 16000
python -m ares.train --tokenizer artifacts/tokenizer.json --data corpus --out runs/tiny
python -m ares.cli --checkpoint runs/tiny/latest.pt --tokenizer artifacts/tokenizer.json
```
`python -m unittest discover -s tests` runs architecture smoke tests.
## Hugging Face static setup (jacmor64)
1. Create a **Static** Space named `Ares` under `jacmor64` (no Docker/Gradio SDK).
2. Upload the contents of `apps/static/` to its repository root (including `index.html`).
3. Host model checkpoints/datasets separately only after their licences, safety review, and sizes are approved. Do not put giant checkpoints in a static UI repository.
Read `docs/ROADMAP.md` before moving beyond this foundation.
## Local two-model runtime (Phase 1)
A Static Space cannot host this Python process. Run it on a computer or separately approved service after training:
```bash
python -m ares.init_models --tokenizer artifacts/tokenizer.json --out models
python -m ares.server --ares models/ares.pt --xiphos models/xiphos.pt --tokenizer artifacts/tokenizer.json
```
`init_models` produces two **random, untrained** checkpoints purely to validate wiring. The server explicitly refuses to present them as intelligent. Train Ares and Xiphos separately, evaluate each checkpoint, then mark only reviewed checkpoints as `training_complete: true`. `POST /chat` serves Ares; `POST /xiphos/plan` produces a proposal envelope and **never executes it**.
## Resumable training and role-specific SFT
Training emits `metrics.jsonl` (loss, learning rate, gradient norm) and checkpointed model + optimizer state. Resume an interrupted run with the same architecture arguments:
```bash
python -m ares.train --resume --role ares --tokenizer artifacts/tokenizer-512.json --data corpus --out runs/ares-general-20m --steps 5000 --batch-size 4 --seq-len 256 --dim 384 --layers 12 --heads 6 --kv-heads 2
```
Prepare only reviewed, locally supplied SFT examples:
```bash
python -m ares.sft --role ares --input data/sft/ares_examples.jsonl --out corpus_sft/ares.txt
python -m ares.sft --role xiphos --input data/sft/xiphos_examples.jsonl --out corpus_sft/xiphos.txt
```
|