--- title: README emoji: ๐Ÿš€ colorFrom: blue colorTo: purple sdk: static pinned: false license: mit short_description: Aria Compute --- # Aria Compute > [huggingface.co/ariacompute](https://huggingface.co/ariacompute) ยท [ariacompute.com](https://ariacompute.com) ยท [github.com/ariacompute](https://github.com/ariacompute) ยท Shanghai, China **On-device AI, cloud-cooperative.** Aria Compute builds an end-to-end quantized model and inference engine stack that brings production-grade LLMs, VLMs, and VLA models to phones, robots, wearables, and embedded devices โ€” all running purely on-device CPU, with no network dependency. ## Who We Are Aria Compute is an AI infrastructure company founded in June 2026, focused on **on-device multimodal foundation models** and **hybrid inference infrastructure**. Our core offering is the **Aria Compute platform**: a unified toolchain for voice, vision, text, and embodied intelligence (VLA) models, built from the ground up for mobile and edge hardware. ## What We Ship ### Aria Engine A from-scratch, pure-Rust inference runtime optimized for our quantized model bundles. Features include: - **CPU-only execution** โ€” No GPU or cloud required - **Blocked Hadamard domain multiplication (HDM)** โ€” Applies the same rotation tiles used during quantization to activations at runtime, enabling exact dequantization-free matrix multiplication - **mmap-based weight loading** โ€” Direct memory-mapped IO with zero-copy weight access, minimizing startup latency and RAM overhead - **Quantized KV cache** โ€” 4-bit per-entry cache reduces context memory ~4ร— vs FP16, keeping multi-turn conversation viable on low-RAM devices - **Streaming token generation** โ€” Low-latency output suitable for real-time chat and tool-calling loops ### Aria Quant Bundles We quantize, package, and distribute production-ready bundles of popular open-source models. Every bundle is produced by our Python quantization toolkit using a common recipe stack: | Recipe | Strategy | Best For | |--------|----------|----------| | **q4** (`--bits 4`) | Uniform 4-bit, per-group codebooks (group_size=32) | Default baseline โ€” smallest bundle | | **q8** (`--bits 8`) | Uniform 8-bit, per-group codebooks (group_size=32) | Near-lossless โ€” maximum fidelity | | **q326_channel** (`--bits 3.26 --codebook-share channel`) | Mixed precision (sensitive layers 4-bit, others ~3-bit), **per-channel codebooks** | Recommended generation-quality recipe โ€” optimal quality-size trade-off | All recipes share the same pipeline: **Hadamard rotation preprocessing** followed by **Lloyd-Max codebook quantization**. The Hadamard transform rotates weight matrices to decorrelate columns, making them amenable to low-bit codebook compression with minimal reconstruction error. No calibration data is required โ€” the method is data-free and deterministic given a random seed. Each bundle ships as a lightweight directory: `weight.bin` (codebook-indices + per-group/per-channel codebooks) + `config.json` + tokenizer files. No GGUF, no safetensors โ€” just Aria-native format. ### Model Portfolio We provide quantized bundles across **9 model families** from the open-source community, each with q4 / q8 / q326_channel variants: | Family | Models | Domain | Key Partners | |--------|--------|--------|-------------| | **Qwen3** | 0.6B, 1.7B, 3.5-0.8B, 3.5-2B | Text generation, chat, tool-calling | Alibaba Cloud | | **Gemma** | 2-2B/9B, 3-270M/1B, 3n-E2B/E4B, 4-E2B/E4B | Text generation, chat, tool-calling | Google | | **LFM** | LFM2 350Mโ€“8B-A1B, LFM2.5 350Mโ€“2.6B, VL 450M/1.6B | Text + vision, Liquid Neural Networks | Liquid AI | | **Nanbeige** | 4.2-3B | Chinese LLM | Nanbeige | | **Bonsai** | 1.7B, 27B | General text generation | Prism | | **Inkling** | Small | General text generation | Thinking Machines | | **OpenVLA** | 7B | Vision-Language-Action (robotics) | Stanford / OpenVLA | | **OpenPI** | Pi0-3B, Pi0.5-3B | Vision-Language-Action (robotics) | Physical Intelligence / LeRobot | | **LingBot** | VLA-v2-6B | Vision-Language-Action (robotics) | Robbyant | ### Bench Evaluation We maintain an automated cross-engine benchmark suite at [`bench/`](https://github.com/ariacompute/bench) that compares Aria-quantized models head-to-head against the same base models running in their officially recommended engines (llama.cpp GGUF, HuggingFace Transformers). Metrics span: - **Capability**: MMLU, GSM8K, C-Eval, HumanEval, Needle-in-Haystack, IFEval - **Performance**: TTFT, tokens/sec, peak memory, model size, power draw - **Output quality**: Token overlap, exact prefix fraction, logprob delta vs FP16 ## Design Philosophy - **Local-first.** All inference happens on-device. No data leaves the device. No network calls during inference. - **Calibration-free quantization.** Hadamard + Lloyd-Max requires no calibration dataset, no fine-tuning, no distillation โ€” just the weight matrix. - **One bundle, any CPU.** ARM (Android, iOS, Linux) and x86_64 are first-class targets. Rust portability ensures the same bundle runs everywhere. - **Progressive quality.** Choose the recipe that fits your device: q4 for smallest footprint, q8 for near-lossless, q326_channel for the sweet spot in between. ## Getting Started - **HuggingFace:** [huggingface.co/ariacompute](https://huggingface.co/ariacompute) โ€” Browse and download quantized model bundles - **Website:** [ariacompute.com](https://ariacompute.com) โ€” Dashboard and documentation - **GitHub:** [github.com/ariacompute](https://github.com/ariacompute) โ€” Engine, model quantization toolkit, and benchmark suite - **Dashboard:** [ariacompute.com/dashboard/models](https://ariacompute.com/dashboard/models) โ€” Authenticated model download ## License All Aria Compute quantized bundles inherit the license of their base model. See individual model cards for details. The Aria Engine and quantization toolkit are separately licensed.