README.md

---
title: ASTERIZER
emoji: 🚀
colorFrom: gray
colorTo: gray
sdk: static
pinned: false
---
# ASTERIZER

**Building AI from the model to the device.** In-house LLMs trained from scratch (100% owned IP), a multilingual tokenizer, a memory layer and on-device inference — no third-party weights.

**Models (trained from scratch)**
- [LUNA-100M](https://huggingface.co/ASTERIZER/LUNA-100M) — 4.5B-token pretrain + RAG/MCP SFT
- [LUNA-300M](https://huggingface.co/ASTERIZER/LUNA-300M) — 4.5B-token pretrain (2.7x scale-up)
- [Ezaris-1B](https://huggingface.co/ASTERIZER/Ezaris-Instruct) — 27.2B-token pretrain + 32B-token CPT + SFT/Instruct (11x scale-up)

**Scaling — LUNA-100M → LUNA-300M → Ezaris-1B** *(latest base checkpoints, held-out multilingual eval, RTX 4060 Ti, higher is better)*

| metric | LUNA-100M | LUNA-300M | Ezaris-1B |
|---|---|---|---|
| Parameters | 109M | 303M | 1.21B |
| Next-token acc (top-1) | 32.5% | 32.7% | **40.5%** |
| Next-token acc (top-5) | 56.0% | 57.6% | **60.3%** |
| Throughput | 40.4K tok/s | 14.6K tok/s | 6.8K tok/s |

*LAMBADA-style exact-match is not cross-tokenizer comparable (the 128K tokenizer merges word+punctuation into single tokens), so it is omitted here; next-token columns are the consistent family comparison.

**vs top open same-tier models** *(same held-out corpus, each model with its own tokenizer, RTX 4060 Ti, bf16)*

| model | train tokens | params | next-tok top-1 | bits/byte (lower=better) | LAMBADA | tok/s |
|---|---|---|---|---|---|---|
| **Ezaris-1B (ours)** | 27B + 32B | 1.21B | 40.5% | 0.98 | 12.9% | 6.8K |
| SmolLM2-1.7B | 11T | 1.71B | 64.2% | 0.70 | 35.1% | 8.9K |
| Qwen2.5-1.5B | ~18T | 1.54B | 58.7% | 0.69 | 25.7% | 6.1K |
| TinyLlama-1.1B | 3T | 1.10B | 59.3% | 0.78 | 35.1% | 4.7K |

Honest notes:
- Ezaris-1B is a 27.2B pretrain + 32B CPT base (~1/100th of the reference models' training tokens). The quality gap (bits/byte 0.98 vs 0.69–0.78) is real and is the focus of our ongoing CPT/SFT program.
- Per-position next-token favors fine tokenizers (TinyLlama's 32K = many easy positions; Ezaris's 128K = fewer, harder positions with 4x the vocab classes). bits/byte is the tokenizer-independent metric.
- Where Ezaris wins by design: ~2x fewer tokens per text, ~2x faster end-to-end per text, native Indic coverage, and code next-token accuracy 66% (beats TinyLlama's 60%).

**Tokenizer — multilingual BPE (tokens per 1,000 bytes; lower = better)**

| tokenizer | Indic avg* | Hindi | English | Code | ALL |
|---|---|---|---|---|---|
| **Ezaris 128K** | **97** | 127 | 219 | 276 | **178** |
| OpenAI o200k (GPT-4o) | 137 | 122 | **200** | **222** | **176** |
| Sarvam-1 | 110 | **101** | 241 | 323 | 193 |
| Ezaris 64K | 111 | 147 | 235 | 305 | 193 |
| Ezaris 32K | 130 | 174 | 260 | 338 | 217 |
| GPT-NeoX 50K (baseline) | 574 | 386 | 202 | 268 | 319 |
| Llama-3 128K | 592 | 199 | 202 | 218 | 301 |
| TinyLlama 32K | 756 | 416 | 235 | 294 | 358 |
| SmolLM2 50K | 874 | 418 | 211 | 262 | 387 |

*Indic avg = Kannada/Telugu/Tamil mean.

**Same text, tokens needed vs Ezaris-128K (lower = better):** GPT-NeoX **~6x on Indic, 1.8x overall** · Llama-3 **~6x on Indic, 1.7x overall** · SmolLM2 2.2x overall · TinyLlama 2.0x overall · Sarvam-1 1.14x on Indic · OpenAI o200k **0.99x overall (on par with GPT-4o's tokenizer)**.

Honest trade-offs: Ezaris deliberately spends its 128K vocab budget on South-Indian scripts (that is the product bet) — English ~9% and code ~24% denser under o200k, and Hindi lags Sarvam-1 ~25% (next optimization target).

**Programs**
- [LUNA-100M](https://huggingface.co/collections/ASTERIZER/luna-100m-program-6a9115383c18e52460bf67c9) · [LUNA-300M](https://huggingface.co/collections/ASTERIZER/luna-300m-program-6a91195ab75671026f83ac94) · [Ezaris](https://huggingface.co/collections/ASTERIZER/luna-1b-program-6a9134d895b9571643e9b1cb)
- Tokenizers: [LUNA (100M/300M)](https://huggingface.co/ASTERIZER/LUNA-Tokenizer) · [Ezaris (128K/64K/32K)](https://huggingface.co/datasets/ASTERIZER/Ezaris-Tokenizer)