---
title: Ares Lab
emoji: ⚔️
colorFrom: red
colorTo: yellow
sdk: static
pinned: false
---

# Ares — from-scratch transformer research project

Ares is a two-role system: **Ares** answers and assists; **Xiphos** drafts *reviewable* capability plans. Xiphos never executes code, modifies tools, browses, or trains anything without an explicit user approval recorded in a plan.

## What this first increment delivers

* A dependency-light, trained byte-level BPE tokenizer (special tokens and chat sections).
* A decoder-only PyTorch transformer: tied token embedding/unembedding, RoPE, causal grouped-query attention, KV cache, pre-norm RMSNorm, SwiGLU, and AdamW training.
* Streaming text pipeline, checkpoint/resume, SQLite profile/memory store, and an approval-gated Xiphos plan schema.
* A dependency-free **static** Hugging Face Spaces-compatible UI. It is an interface and local memory/planning demonstration; static hosting cannot run a multi-million parameter Python/PyTorch model server-side.

It deliberately starts as a small, testable model. Scaling configuration is a controlled experiment, not a claim of capability.

## Honest deployment constraints

Training a credible 1B-parameter foundation model from scratch needs a very large licensed/clean corpus, distributed GPU infrastructure, lengthy runs, and substantial cost. A free static HF Space cannot train or serve it. Static Pages can host this UI and model artifacts, but **inference requires either browser-compatible quantized weights with enough client RAM/download, or a separately operated inference service**. Neither is secretly substituted with an external model/API here.

Do not scrape or train on data without checking licenses, terms, privacy, and provenance. Never store secrets in this repository.

## Quick start (Python 3.10+)

```bash
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Place UTF-8, license-reviewed .txt files below corpus/.
python -m ares.tokenizer train --input corpus --out artifacts/tokenizer.json --vocab-size 16000
python -m ares.train --tokenizer artifacts/tokenizer.json --data corpus --out runs/tiny
python -m ares.cli --checkpoint runs/tiny/latest.pt --tokenizer artifacts/tokenizer.json
```

`python -m unittest discover -s tests` runs architecture smoke tests.

## Hugging Face static setup (jacmor64)

1. Create a **Static** Space named `Ares` under `jacmor64` (no Docker/Gradio SDK).
2. Upload the contents of `apps/static/` to its repository root (including `index.html`).
3. Host model checkpoints/datasets separately only after their licences, safety review, and sizes are approved. Do not put giant checkpoints in a static UI repository.

Read `docs/ROADMAP.md` before moving beyond this foundation.

## Local two-model runtime (Phase 1)

A Static Space cannot host this Python process. Run it on a computer or separately approved service after training:

```bash
python -m ares.init_models --tokenizer artifacts/tokenizer.json --out models
python -m ares.server --ares models/ares.pt --xiphos models/xiphos.pt --tokenizer artifacts/tokenizer.json
```

`init_models` produces two **random, untrained** checkpoints purely to validate wiring. The server explicitly refuses to present them as intelligent. Train Ares and Xiphos separately, evaluate each checkpoint, then mark only reviewed checkpoints as `training_complete: true`. `POST /chat` serves Ares; `POST /xiphos/plan` produces a proposal envelope and **never executes it**.

## Resumable training and role-specific SFT

Training emits `metrics.jsonl` (loss, learning rate, gradient norm) and checkpointed model + optimizer state. Resume an interrupted run with the same architecture arguments:

```bash
python -m ares.train --resume --role ares --tokenizer artifacts/tokenizer-512.json --data corpus --out runs/ares-general-20m --steps 5000 --batch-size 4 --seq-len 256 --dim 384 --layers 12 --heads 6 --kv-heads 2
```

Prepare only reviewed, locally supplied SFT examples:

```bash
python -m ares.sft --role ares --input data/sft/ares_examples.jsonl --out corpus_sft/ares.txt
python -m ares.sft --role xiphos --input data/sft/xiphos_examples.jsonl --out corpus_sft/xiphos.txt
```