| --- |
| license: cc-by-nc-sa-4.0 |
| base_model: Qwen/Qwen3-14B-Base |
| tags: |
| - aria |
| - mnemonicai |
| - qwen3 |
| - text-generation |
| library_name: transformers |
| pipeline_tag: text-generation |
| language: |
| - en |
| --- |
| |
| # Aria V7.1 |
|
|
| Aria is a 14B assistant built by **Sergio Williams** on **Qwen3-14B-Base**, developed as the core model for MnemonicAI. This repo's `main` holds **V7.1**, the current release. (V7.0 weights remain available in this repo's commit history.) |
|
|
| ## Benchmarks |
| 100 held-out GSM8K-style math + 10 identity probes, identical raw plain-ChatML serving for all three (Intel Arc A770 / ollama): |
|
|
| | Model | Math | Identity | Wall time | |
| |---|---|---|---| |
| | v5 (previous live) | 46/100 | 0/10 | 695s | |
| | V7.0 | 69/100 | 8/10 | 837s | |
| | **V7.1** | **71/100** | **8/10** | **514s** | |
|
|
| Honest read: +2 math over V7.0 is within noise on a 100-question set. The meaningful gains are the **38% latency reduction** and a fully clean, verified training corpus. Against the previous live v5: **+25 math points** and **0→8 identity**. |
|
|
| ## Training |
| - Unsloth QLoRA on Qwen3-14B-Base, ~2.55 epochs, seq 2048, LoRA r32/α32 |
| - Target modules include **`lm_head` + `embed_tokens`** (see below), plain ChatML, response-only loss |
| - **17,522 verified examples**, deduplicated: math 11,183 (answer-checked vs gold) · code 3,309 (**execution-verified** — every solution ran its own assert tests) · deep/ultra reasoning 1,206 · chat 937 · balancing 379 · multi-turn 250 · identity 205 · max-thinking 53 |
| - Teachers: kimi-k2.5/2.6/2.7-code, Claude Opus & Sonnet, Fable 5, deepseek-v4-pro |
| - Dedup removed 501 duplicate prompts and 768 duplicate code solutions |
|
|
| ## The stop-token fix |
| Earlier versions rambled and never terminated. Two causes: |
| 1. Serving with an injected empty `<think>\n\n</think>` block (`enable_thinking=false`) triggers non-stop generation. |
| 2. More fundamentally: **Qwen3-Base never emits `<|im_end|>` during pretraining**, and attention/MLP-only LoRA can never surface that token — the output head must be trainable. |
| |
| V7.1 fixes this structurally by putting `lm_head`+`embed_tokens` in the LoRA and leaving terminators unmasked in the loss. Result: a **5/5 clean-stop gate** — every probe emits `<|im_end|>` and terminates. |
| |
| ## Serving requirements (important) |
| - Use the **plain Qwen3 ChatML template**. Never inject an empty `<think>` block / `enable_thinking=false`. |
| - Stop tokens: `<|im_end|>` and `<|endoftext|>`. |
| |
| ``` |
| <|im_start|>system |
| You are Aria, a helpful AI assistant created by Sergio Williams.<|im_end|> |
| <|im_start|>user |
| {message}<|im_end|> |
| <|im_start|>assistant |
| ``` |
| |
| ## Known quirks |
| - A rare stray non-English token can appear immediately before the stop token (cosmetic; trim app-side). |
| |
| Base model licence: Qwen3-14B-Base, Apache-2.0. |
