README.md

---
title: README
emoji: 🚀
colorFrom: blue
colorTo: purple
sdk: static
pinned: false
license: mit
short_description: Aria Compute
---

# Aria Compute

> [huggingface.co/ariacompute](https://huggingface.co/ariacompute) · [ariacompute.com](https://ariacompute.com) · [github.com/ariacompute](https://github.com/ariacompute) · Shanghai, China

**On-device AI, cloud-cooperative.** Aria Compute builds an end-to-end quantized model and inference engine stack that brings production-grade LLMs, VLMs, and VLA models to phones, robots, wearables, and embedded devices — all running purely on-device CPU, with no network dependency.

## Who We Are

Aria Compute is an AI infrastructure company founded in June 2026, focused on **on-device multimodal foundation models** and **hybrid inference infrastructure**. Our core offering is the **Aria Compute platform**: a unified toolchain for voice, vision, text, and embodied intelligence (VLA) models, built from the ground up for mobile and edge hardware.

## What We Ship

### Aria Engine

A from-scratch, pure-Rust inference runtime optimized for our quantized model bundles. Features include:
- **CPU-only execution** — No GPU or cloud required
- **Blocked Hadamard domain multiplication (HDM)** — Applies the same rotation tiles used during quantization to activations at runtime, enabling exact dequantization-free matrix multiplication
- **mmap-based weight loading** — Direct memory-mapped IO with zero-copy weight access, minimizing startup latency and RAM overhead
- **Quantized KV cache** — 4-bit per-entry cache reduces context memory ~4× vs FP16, keeping multi-turn conversation viable on low-RAM devices
- **Streaming token generation** — Low-latency output suitable for real-time chat and tool-calling loops

### Aria Quant Bundles

We quantize, package, and distribute production-ready bundles of popular open-source models. Every bundle is produced by our Python quantization toolkit using a common recipe stack:

| Recipe | Strategy | Best For |
|--------|----------|----------|
| **q4** (`--bits 4`) | Uniform 4-bit, per-group codebooks (group_size=32) | Default baseline — smallest bundle |
| **q8** (`--bits 8`) | Uniform 8-bit, per-group codebooks (group_size=32) | Near-lossless — maximum fidelity |
| **q326_channel** (`--bits 3.26 --codebook-share channel`) | Mixed precision (sensitive layers 4-bit, others ~3-bit), **per-channel codebooks** | Recommended generation-quality recipe — optimal quality-size trade-off |

All recipes share the same pipeline: **Hadamard rotation preprocessing** followed by **Lloyd-Max codebook quantization**. The Hadamard transform rotates weight matrices to decorrelate columns, making them amenable to low-bit codebook compression with minimal reconstruction error. No calibration data is required — the method is data-free and deterministic given a random seed.

Each bundle ships as a lightweight directory: `weight.bin` (codebook-indices + per-group/per-channel codebooks) + `config.json` + tokenizer files. No GGUF, no safetensors — just Aria-native format.

### Model Portfolio

We provide quantized bundles across **9 model families** from the open-source community, each with q4 / q8 / q326_channel variants:

| Family | Models | Domain | Key Partners |
|--------|--------|--------|-------------|
| **Qwen3** | 0.6B, 1.7B, 3.5-0.8B, 3.5-2B | Text generation, chat, tool-calling | Alibaba Cloud |
| **Gemma** | 2-2B/9B, 3-270M/1B, 3n-E2B/E4B, 4-E2B/E4B | Text generation, chat, tool-calling | Google |
| **LFM** | LFM2 350M–8B-A1B, LFM2.5 350M–2.6B, VL 450M/1.6B | Text + vision, Liquid Neural Networks | Liquid AI |
| **Nanbeige** | 4.2-3B | Chinese LLM | Nanbeige |
| **Bonsai** | 1.7B, 27B | General text generation | Prism |
| **Inkling** | Small | General text generation | Thinking Machines |
| **OpenVLA** | 7B | Vision-Language-Action (robotics) | Stanford / OpenVLA |
| **OpenPI** | Pi0-3B, Pi0.5-3B | Vision-Language-Action (robotics) | Physical Intelligence / LeRobot |
| **LingBot** | VLA-v2-6B | Vision-Language-Action (robotics) | Robbyant |

### Bench Evaluation

We maintain an automated cross-engine benchmark suite at [`bench/`](https://github.com/ariacompute/bench) that compares Aria-quantized models head-to-head against the same base models running in their officially recommended engines (llama.cpp GGUF, HuggingFace Transformers). Metrics span:

- **Capability**: MMLU, GSM8K, C-Eval, HumanEval, Needle-in-Haystack, IFEval
- **Performance**: TTFT, tokens/sec, peak memory, model size, power draw
- **Output quality**: Token overlap, exact prefix fraction, logprob delta vs FP16

## Design Philosophy

- **Local-first.** All inference happens on-device. No data leaves the device. No network calls during inference.
- **Calibration-free quantization.** Hadamard + Lloyd-Max requires no calibration dataset, no fine-tuning, no distillation — just the weight matrix.
- **One bundle, any CPU.** ARM (Android, iOS, Linux) and x86_64 are first-class targets. Rust portability ensures the same bundle runs everywhere.
- **Progressive quality.** Choose the recipe that fits your device: q4 for smallest footprint, q8 for near-lossless, q326_channel for the sweet spot in between.

## Getting Started

- **HuggingFace:** [huggingface.co/ariacompute](https://huggingface.co/ariacompute) — Browse and download quantized model bundles
- **Website:** [ariacompute.com](https://ariacompute.com) — Dashboard and documentation
- **GitHub:** [github.com/ariacompute](https://github.com/ariacompute) — Engine, model quantization toolkit, and benchmark suite
- **Dashboard:** [ariacompute.com/dashboard/models](https://ariacompute.com/dashboard/models) — Authenticated model download

## License

All Aria Compute quantized bundles inherit the license of their base model. See individual model cards for details. The Aria Engine and quantization toolkit are separately licensed.