---
license: apache-2.0
language:
- en
library_name: safetensors
pipeline_tag: text-classification
tags:
- artoru
- artoru-edge
- artoru-desk
- edge-ai
- intent-routing
- agentic
- on-device
- local-inference
- safetensors
- onnx-runtime
- int8
- quantization
---
# Artoru # EDGE 65M
**First preview. The local agentic brand starts here.**
Not another cloud chat clone. **Artoru** is the local-first, multi-agent line you run on your own machine — and **EDGE 65M** is the opening chapter: the tiny routing engine inside **Artoru Desk**, shipped as a public first preview with LoRAs and more already on the road.
The arena already has its myths. Artoru is our line in that sky — built to live on *your* hardware.
---
| | |
| --- | --- |
| **Brand** | **Artoru** |
| **Model** | **Artoru EDGE 65M** (first EDGE preview) |
| **Product** | **Artoru Desk** — local-first multi-agent workspace |
| **Routing win** | **~98.66%** product-scored · **~99.16%** handled (**EDGE + Chief**) |
| **Speed** | ORT CUDA FP32 **~134,462 tok/s** (batch 32, RTX 5090) |
| **Params** | ~**65.39M** |
---
## Why this card exists
Most model cards apologize for being small. This one doesn’t.
**EDGE 65M** is not trying to write your novel, debate philosophy, or impersonate a search engine. It has one job and it does that job like a blade: **read the turn, name the intent, route the desk** — on-device, at reflex speed, without asking the cloud for permission to think.
If Desk is the stage, EDGE is the stage manager with a stopwatch.
---
## The win
Cloud models talk. **EDGE routes.**
**EDGE + Chief** on the product eval (n=372):
- **~98.66%** product-scored accuracy
- **~99.16%** handled-rate
That’s the number. Own it.
**Throughput:** ORT CUDA FP32 batch 32 ≈ **134k tok/s** (~0.11 ms/sample, ~8.9k samples/s) on short routing utterances. Batched PyTorch FP16 ≈ **84k tok/s**.
Tiny model. Massive local routing power.
No asterisk that says “but only if the API is up.” No waiting room. No meter ticking while you decide whether to draft an email or ask the Chief.
---
## The story: from Micro to EDGE
Before Artoru had a name people could shout, there was a quieter public router — **AMS Micro** on `vvarghese/ams-micro-70m`. Honest work. On-device. Already thinking in intents.
**Artoru EDGE 65M** is that lineage stepping into the light with a brand, a desk, and a scoreboard:
| | Prior public Micro | **Artoru EDGE 65M** |
| --- | --- | --- |
| **Identity** | Earlier published on-device router (`vvarghese/ams-micro-70m`) | **Today’s Artoru preview** — Desk’s local routing engine |
| **Product routing** | Earlier public baseline | **~98.66% / ~99.16%** handled with **EDGE + Chief** |
| **Speed** | Prior CPU ORT full-test ~9.2 s (older publish) | **~134k tok/s** ORT CUDA FP32 b32 |
| **This pack** | — | FP32 + int8 + Q4 |
Same obsession — understand first, spend heavy compute second. New costume. Sharper teeth. Product attached.
---
## Why local still wins
Because your desk is not a demo.
Because “upload everything to someone else’s GPU” is a lifestyle choice, not a law of physics.
Because routing is a *reflex*, and reflexes that cross the ocean feel like lag even when the chat model is brilliant.
Local means:
- **Privacy by architecture** — intent classification happens where the keystrokes already are
- **Latency you can feel** — ~134k tok/s class throughput on the FP32 ORT path (batch 32, RTX 5090)
- **A desk that still works when the wifi sulks**
- **Agents that ask permission** instead of shipping your draft into the void
Artoru is not anti-cloud. Artoru is **pro-sovereignty**. Cloud models are invited guests. EDGE is the bouncer who knows your name.
---
## Artoru Desk — the product frame
**Artoru Desk** is the product: a **local-first multi-agent workspace** — agents, tools, approvals, panic, memory, and a full surface that lives on your hardware.
**EDGE 65M** is the fast intent/routing model in that stack.
**Chief** is the handoff layer that rides with EDGE on hard turns — one piece of Desk, not the whole story.
```text
[ You type ] → [ EDGE 65M names the intent ] → [ Desk routes ]
└─ low confidence / hard intent → [ Chief ]
```
Desk > any single model.
EDGE is how Desk decides *what to do next* without round-tripping the cloud for every reflex. Chief is how Desk stays honest when the turn is sharper than a tiny router should pretend.
Chat is never empty theater. Someone is always addressed — usually the Chief — and EDGE is the quiet nerve that decides whether the turn is a greeting, a task assign, an escalate, a skill suggest, or “ask the human.”
---
## How EDGE + Chief feels
Imagine opening Desk.
You say something ordinary. EDGE fires. The room already knows whether this is smalltalk, a meeting, a fetch, or a handoff.
You say something that matters — escalate, assign, a Chief question. Confidence dips or the intent lands in the handoff set. **Chief** steps in. Visible. Ownable. Not a silent model swap behind a spinner.
That duet is the product eval story:
| Metric | Value |
| --- | --- |
| Product-scored accuracy | **~98.66%** |
| Handled-rate (**EDGE + Chief**) | **~99.16%** |
| Eval size | n=372 |
EDGE alone is the sprinter.
EDGE + Chief is the relay team that finishes the race.
If routing were only accuracy cosplay, we’d ship a bigger model and call it a day. This pack is about *feel*: instant local triage, then a named handoff when the desk needs a brain with more room.
---
## What’s in the pack (and why the drama)
Four files. Four moods. Same EDGE soul.
| File | Precision | Use when |
| --- | --- | --- |
| `artoru-edge-65m.safetensors` | **FP32** | Flagship PyTorch load + vocab metadata |
| `artoru-edge-65m.onnx` | **FP32** | ORT serve / GPU peak throughput (self-contained) |
| `artoru-edge-65m-int8.onnx` | **INT8** | Smaller CPU-oriented ORT (~63 MB) |
| `artoru-edge-65m-q4.safetensors` | **INT4** | Smallest download (~33 MB); dequant then load |
**FP32** is the headline — quality plus the **~134k tok/s** ORT CUDA story.
**Int8** is the everyday carry for lean CPU desks.
**Q4** is the suitcase that still somehow holds the whole outfit: a custom RTN int4 pack — dequant before `load_state_dict`. After dequant, compute is FP32-class.
Download the flagship if you want the poster on the wall.
Download int8 if your fans already hate you.
Download Q4 if you believe small can still be serious.
---
## Architecture
~**65.39M** params · vocab **9225** · up to **2048** positions · **768-d** · **8** heads · **8** layers · intents + skills + slots + risk.
Built for agentic routing inside Desk: register, assign/resume, browse/search/fetch, skills, memory/meeting, calculate/time/weather.
Not a toy transformer cosplaying as a chat model. A routing encoder with a job description carved into the heads.
---
## Load — FP32 Safetensors
```python
import json
from pathlib import Path
import torch
from safetensors import safe_open
from safetensors.torch import load_file
from scripts.train_canonical import build_canonical_encoder
from ams.tokenize import tokenize_legacy
path = Path("Model/artoru-edge-65m-hf/artoru-edge-65m.safetensors")
tensors = load_file(str(path))
with safe_open(str(path), framework="pt") as f:
meta = f.metadata() or {}
vocab = json.loads(meta["vocab_json"])
canonical = json.loads(meta["canonical_json"])
skills = json.loads(meta["skills_json"])
model = build_canonical_encoder(len(vocab), len(canonical), len(skills), dropout=0.0)
model.load_state_dict(tensors, strict=False)
model.eval()
def encode(text: str, pack: int = 512):
content = tokenize_legacy(text)
room = pack - 1
if len(content) > room:
content = content[-room:]
ids = [vocab[""]] + [vocab.get(tok, 1) for tok in content]
return torch.tensor([ids], dtype=torch.long)
```
---
## Load — FP32 ONNX
```python
import onnxruntime as ort
sess = ort.InferenceSession(
"Model/artoru-edge-65m-hf/artoru-edge-65m.onnx",
providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
```
---
## Load — INT8 ONNX
```python
import onnxruntime as ort
sess = ort.InferenceSession(
"Model/artoru-edge-65m-hf/artoru-edge-65m-int8.onnx",
providers=["CPUExecutionProvider"],
)
```
Pad/truncate to the graph width when needed (bench used seq≈42).
---
## Load — Q4
Unpack `*.q4` + `*.scale` from safetensors metadata (`layouts_json`), rebuild FP32 tensors, then `load_state_dict` into the EDGE encoder. After dequant, compute is FP32-class.
---
## Roadmap swagger
This is the **1st public preview** of Artoru EDGE on the path to Desk.
**Coming next — proudly:**
- LoRAs that specialize the router without replacing the soul
- More EDGE drops as the desk surface deepens
- Richer agent lanes, sharper handoffs, louder product feel
- The boring-but-holy work: eval harnesses, calibration, thresholds you can actually trust
The brand ladder is just starting. Watch Artoru climb.
We are not shipping “a 65M curiosity.” We are shipping the **first named nerve** of a local multi-agent brand — and inviting you to load it, break it, score it, and tell us where it flinches.
---
## Naming
| Name | What it is |
| --- | --- |
| **Artoru** | The brand line — local / agentic |
| **Artoru EDGE 65M** | This model (first EDGE preview weights) |
| **Chief** | Handoff layer with EDGE |
| **Artoru Desk** | Local-first multi-agent product |
---
## Intended use (short and honest)
**Yes:** on-device intent / routing for agentic desks; triage before expensive generation; local-first pipelines that want a fast first decision.
Use EDGE to decide. Use your generative stack to speak. Use Chief when the turn deserves a longer brain.
---
**Artoru EDGE 65M — first preview. The ladder begins.**