Hugging Face's logo

Spaces:
ASTERIZER
/
README
Running

App Files Community
README / README.md
REZIRETSA's picture
REZIRETSA
Upload README.md with huggingface_hub
35787b1 verified
preview code
|
Raw
History Blame Contribute Delete
4.04 kB
metadata 
title: ASTERIZER
emoji: 🚀
colorFrom: gray
colorTo: gray
sdk: static
pinned: false

ASTERIZER

Building AI from the model to the device. In-house LLMs trained from scratch (100% owned IP), a multilingual tokenizer, a memory layer and on-device inference — no third-party weights.

Models (trained from scratch)

  • LUNA-100M — 4.5B-token pretrain + RAG/MCP SFT
  • LUNA-300M — 4.5B-token pretrain (2.7x scale-up)
  • Ezaris-1B — 27.2B-token pretrain + 32B-token CPT + SFT/Instruct (11x scale-up)

Scaling — LUNA-100M → LUNA-300M → Ezaris-1B (latest base checkpoints, held-out multilingual eval, RTX 4060 Ti, higher is better)

metric LUNA-100M LUNA-300M Ezaris-1B
Parameters 109M 303M 1.21B
Next-token acc (top-1) 32.5% 32.7% 40.5%
Next-token acc (top-5) 56.0% 57.6% 60.3%
Throughput 40.4K tok/s 14.6K tok/s 6.8K tok/s

*LAMBADA-style exact-match is not cross-tokenizer comparable (the 128K tokenizer merges word+punctuation into single tokens), so it is omitted here; next-token columns are the consistent family comparison.

vs top open same-tier models (same held-out corpus, each model with its own tokenizer, RTX 4060 Ti, bf16)

model train tokens params next-tok top-1 bits/byte (lower=better) LAMBADA tok/s
Ezaris-1B (ours) 27B + 32B 1.21B 40.5% 0.98 12.9% 6.8K
SmolLM2-1.7B 11T 1.71B 64.2% 0.70 35.1% 8.9K
Qwen2.5-1.5B ~18T 1.54B 58.7% 0.69 25.7% 6.1K
TinyLlama-1.1B 3T 1.10B 59.3% 0.78 35.1% 4.7K

Honest notes:

  • Ezaris-1B is a 27.2B pretrain + 32B CPT base (~1/100th of the reference models' training tokens). The quality gap (bits/byte 0.98 vs 0.69–0.78) is real and is the focus of our ongoing CPT/SFT program.
  • Per-position next-token favors fine tokenizers (TinyLlama's 32K = many easy positions; Ezaris's 128K = fewer, harder positions with 4x the vocab classes). bits/byte is the tokenizer-independent metric.
  • Where Ezaris wins by design: ~2x fewer tokens per text, ~2x faster end-to-end per text, native Indic coverage, and code next-token accuracy 66% (beats TinyLlama's 60%).

Tokenizer — multilingual BPE (tokens per 1,000 bytes; lower = better)

tokenizer Indic avg* Hindi English Code ALL
Ezaris 128K 97 127 219 276 178
OpenAI o200k (GPT-4o) 137 122 200 222 176
Sarvam-1 110 101 241 323 193
Ezaris 64K 111 147 235 305 193
Ezaris 32K 130 174 260 338 217
GPT-NeoX 50K (baseline) 574 386 202 268 319
Llama-3 128K 592 199 202 218 301
TinyLlama 32K 756 416 235 294 358
SmolLM2 50K 874 418 211 262 387

*Indic avg = Kannada/Telugu/Tamil mean.

Same text, tokens needed vs Ezaris-128K (lower = better): GPT-NeoX ~6x on Indic, 1.8x overall · Llama-3 ~6x on Indic, 1.7x overall · SmolLM2 2.2x overall · TinyLlama 2.0x overall · Sarvam-1 1.14x on Indic · OpenAI o200k 0.99x overall (on par with GPT-4o's tokenizer).

Honest trade-offs: Ezaris deliberately spends its 128K vocab budget on South-Indian scripts (that is the product bet) — English ~9% and code ~24% denser under o200k, and Hindi lags Sarvam-1 ~25% (next optimization target).

Programs