README.md

---
base_model: unsloth/gemma-4-12b-it
library_name: transformers
license: gemma
language:
  - en
tags:
  - gemma4
  - gemma4_unified
  - multimodal
  - tool-use
  - personal-assistant
  - gguf
  - qlora
  - unsloth
pipeline_tag: image-text-to-text
---

# Aria V8

**Aria** is a personal assistant built on **`google/gemma-4-12b-it`**, tuned for **tool calling**,
**memory-aware behaviour**, and a stable identity — while keeping the base model's math and vision intact.

This repo ships the **merged, standalone model**. No adapter to apply, no base model to fetch
separately: `from_pretrained("SurgeFF/AriaV8")` and you have Aria. **GGUF quants** are in
[`gguf/`](./gguf) for llama.cpp / Ollama / LM Studio, including a multimodal projector so vision
and audio survive quantization.

Trained on a single RTX 3090 (24 GB). Every teacher used to generate the training data was an
open-weight model, so the corpus is legally clean.

---

## Evaluation

Five-capability harness on a fixed held-out set (`math-n=100` sampled from GSM8K **test** with
seed 42; 10 tool cases; 10 identity cases; 20 memory-behaviour cases; a vision smoke test).

| Capability | Stock `gemma-4-12b-it` | **Aria V8 (merged)** |
|---|---|---|
| Math (GSM8K, 100 held-out) | 81 / 100 | **92 / 100** |
| Tool calling | 8 / 10 | **10 / 10** |
| Identity consistency *(with system prompt)* | 9 / 10 | **8 / 10** |
| Memory behaviour | see note | **18 / 20** |
| Multimodal (vision) | pass | **pass** |

**Read these numbers carefully.** The pre-merge adapter scored math 89 / tools 10 / identity 9 /
memory 17. The merged model scores 92 / 10 / 8 / 18. Those differences (+3 math, −1 identity,
+1 memory) are **run-to-run noise, not improvement** — on a 100-item math suite a ±3 swing is well
inside variance, and ±1 on a 10- or 20-item suite is a coin flip. The merge is *faithful*, which is
all it should be. Treat Aria V8 as **~89–92 on math**, not "92".

> **On the memory baseline.** The stock baseline was scored on an earlier 5-case memory suite
> (it got 1/5). That suite was later expanded to 20 cases and the baseline was never re-run on it,
> so there is no honest like-for-like memory delta to quote. The 18/20 is real and measured; the
> improvement over stock is not precisely claimable.

Math *improving* during a fine-tune is unusual and was the main thing being watched: an earlier
generation (V7.1) collapsed from 65 to 24 on math after training on a corpus containing ~43K rows
of performative self-correction. That corpus was discarded and the failure mode is now screened for.

## Usage — transformers

> **Requires `transformers` 5.15.0.dev0 (from source).** Stock `transformers` <= 5.5.0 cannot load
> `gemma4_unified` at all.

```python
import torch
import transformers.integrations.heterogeneity.configuration_utils as het

# gemma4_unified has a HETEROGENEOUS per-layer config; reading a global attr that
# varies per layer raises AmbiguousGlobalPerLayerAttributeError. Install this shim
# BEFORE loading or most loaders will fail.
_HCM, _Err = het.HeterogeneousConfigMixin, het.AmbiguousGlobalPerLayerAttributeError
_orig = _HCM.__getattribute__
def _permissive(self, key):
    try:
        return _orig(self, key)
    except _Err:
        self.__dict__["allow_global_per_layer_attribute_access"] = True
        return _orig(self, key)
_HCM.__getattribute__ = _permissive

from transformers import AutoProcessor
from transformers.models.gemma4_unified.modeling_gemma4_unified import (
    Gemma4UnifiedForConditionalGeneration as ModelCls)

model = ModelCls.from_pretrained("SurgeFF/AriaV8", dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained("SurgeFF/AriaV8")
```

Suggested system prompt: `You are Aria, created by Sergio Williams.`

Tool calls are emitted in the convention the model was trained on:

```
<tool_call>{"name": "recall", "arguments": {"query": "..."}}</tool_call>
```

## Usage — GGUF / llama.cpp

| File | Quant | Size | Notes |
|---|---|---:|---|
| [`gguf/AriaV8-Q4_0.gguf`](./gguf/AriaV8-Q4_0.gguf) | Q4_0 | 7.5 GB | smallest; legacy format, widest compatibility |
| [`gguf/AriaV8-Q4_K_M.gguf`](./gguf/AriaV8-Q4_K_M.gguf) | Q4_K_M | 7.9 GB | **recommended default** — best size/quality trade |
| [`gguf/AriaV8-Q5_K_M.gguf`](./gguf/AriaV8-Q5_K_M.gguf) | Q5_K_M | 9.2 GB | noticeably closer to full precision |
| [`gguf/AriaV8-Q6_K.gguf`](./gguf/AriaV8-Q6_K.gguf) | Q6_K | 10.6 GB | near-lossless |
| [`gguf/AriaV8-Q8_0.gguf`](./gguf/AriaV8-Q8_0.gguf) | Q8_0 | 13.7 GB | effectively lossless |
| [`gguf/AriaV8-mmproj-F16.gguf`](./gguf/AriaV8-mmproj-F16.gguf) | F16 | 122 MB | **vision + audio projector** — pair with any quant above for multimodal |

An F16 GGUF is deliberately **not** shipped: it would be the same precision as the safetensors
already at the repo root (~26 GB each), doubling the repo for no benefit. Regenerate it with
`convert_hf_to_gguf.py` if you need one for further quantization.

```bash
# text
llama-cli -m AriaV8-Q4_K_M.gguf -p "What is 17*23?"

# with vision/audio (pair the projector with any quant)
llama-mtmd-cli -m AriaV8-Q4_K_M.gguf --mmproj AriaV8-mmproj-F16.gguf --image photo.jpg -p "Describe this."
```

> The GGUF quants have **not** been re-scored on the eval harness. Quantization costs quality,
> especially at Q4. The table above reports the bf16 model's numbers; assume the quants are somewhat
> lower and test for your use case.

## Repo layout

```
.                      merged standalone model (bf16 safetensors, 7 shards, ~26 GB)
adapter/               the original Stage-A LoRA, if you'd rather stack it on the base yourself
gguf/                  quantized GGUFs + multimodal projector
```

## Training

Aria V8 is the **Stage A** adapter of a staged DMT ([arXiv:2310.05492](https://arxiv.org/abs/2310.05492))
fine-tune, merged into the base. Stage A injects capability; a later integration stage and two
follow-up repair experiments were all trained, evaluated, and **deliberately not promoted**
(see [Things that did not work](#things-that-did-not-work)).

**Architecture note:** `gemma-4` is `gemma4_unified` — an **encoder-free multimodal** model where
vision, audio and text share the same weights. There is no separate vision tower to freeze, so a
text-only fine-tune degrades vision. The mix therefore carries a deliberate **multimodal floor**.

### Data mix (Stage A)

| Source | Rows | Purpose |
|---|---:|---|
| `tools.jsonl` | 4,000 | tool-calling behaviour |
| `memory.jsonl` | 1,915 | memory-aware behaviour (when to save/recall/update/refuse) |
| multimodal floor | ~3,000 | image-text pairs, **28.5%** of the mix — prevents vision degradation |
| replay pool | ~1,200 | general-capability retention |
| **Total** | **10,198 train / 315 eval** | |

Identity and curiosity corpora were **deliberately excluded** from Stage A so persona training could
not contaminate capability learning.

**Teachers** (all open-weight): `nemotron-3-ultra`, `kimi-k2.7-code`, `deepseek-v4-flash`,
`gemma4:31b`, `deepseek-v4-pro`. Multimodal pairs from permissive subsets of
[`HuggingFaceM4/the_cauldron`](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron)
(vqav2, ai2d, cocoqa).

### LoRA / hyperparameters

```python
r = 32, lora_alpha = 32, lora_dropout = 0.0, bias = "none"
target_modules = ["q_proj","k_proj","v_proj","o_proj",
                  "gate_proj","up_proj","down_proj",
                  "lm_head","embed_tokens"]
finetune_vision_layers = True          # encoder-free: shared weights must stay trainable
max_seq_length = 4096

epochs 2 - batch 2 x grad-accum 8 (effective 16) - lr 1e-4 cosine - warmup_steps 40
optim adamw_8bit - weight_decay 0.01 - eval every 250 steps - save_total_limit 4
1,276 steps total
```

Including **`lm_head` + `embed_tokens`** was the decisive ingredient — without them the model cannot
adjust output-token behaviour, which is exactly where earlier EOS/formatting damage lived.

### Merge details

- Merged in **bf16 on CPU**, *not* 4-bit. Merging a QLoRA into a quantized base would bake
  quantization error permanently into the released weights.
- The base ties `lm_head` to `embed_tokens`, and **both were LoRA targets**. Saving without
  intervention would have collapsed them into one tensor and discarded half the trained output
  behaviour. `lm_head` is therefore **untied and cloned** before saving, and
  `tie_word_embeddings` is `false` in the released config.
- `transformers`' `save_pretrained` normalizes the heterogeneous config and drops
  `text_config.global_head_dim` and `num_global_key_value_heads`. Both are restored in the released
  `config.json` — without them `convert_hf_to_gguf.py` fails with `KeyError: 'global_head_dim'`.
- Because the released checkpoint has an untied `lm_head`, llama.cpp's gemma4 **mmproj** exporter
  needs a one-line skip for `lm_head.*` (tied upstream checkpoints never contain that tensor).
  This affects re-generating the projector yourself, not using the shipped one.

## Things that did not work

Recorded because negative results are the useful part. All three were fully trained and evaluated on
the same suite, and all three were **declined** under a promotion rule fixed *before* the numbers
were seen (promote only if the target capability improves **and** nothing else regresses).

| Experiment | Result | Decision |
|---|---|---|
| **Stage B** (integration: all four corpora + replay) | memory 3/5 to 4/5 *(older 5-case suite)*, but **math 89 to 84** | rejected — traded real math for a marginal memory gain |
| **Memory top-up** (1 epoch, memory-weighted, resumed from Stage A) | memory 17 to 18 (+1 case, noise), **math 89 to 85**, identity 9 to 8 | rejected |
| **Math DPO** (84 preference pairs mined from GSM8K *train*) | **identical on every capability** | rejected — null result |

The DPO run is the instructive one: it trained *correctly* (held-out preference accuracy 0.875,
reward margins +0.72 — it genuinely learned to rank the right answer above its own wrong one), but
that never transferred to producing better answers at decode time. Likely causes: only 10 optimizer
steps from 84 pairs, and "chosen" being terse human GSM8K gold rationale — stylistically far from the
model's own verbose reasoning, so it may have learned *"prefer terse gold-style text"* rather than
*"reason correctly."* A mid-run partial read of that eval showed 91%, and the final landed on exactly
89 — a clean demonstration that small deltas on a 100-item suite are noise.

**If math is pushed again, the better method is rejection-sampling SFT (STaR):** sample k=4–8
solutions per *train* problem, keep only those reaching the correct answer, and fine-tune on the
model's own correct reasoning. On-policy, no style mismatch.

## Limitations

- **Not a general-purpose assistant release.** Tuned for one person's fleet, tools, and conventions.
- **Identity REQUIRES the system prompt.** `identity.jsonl` was deliberately excluded from Stage A so
  persona could not contaminate capability learning; it was meant to land in the Stage B integration
  pass, which failed its eval gate and was declined. So this checkpoint has **no trained identity**.
  Given `You are Aria, created by Sergio Williams.` it answers correctly and consistently. With **no**
  system prompt it falls back to the base model and says it is *Gemma 4, developed by Google DeepMind*.
  Always set the system prompt. (The published identity score of 8-10/10 was measured *with* that
  system prompt — it does not measure unprompted identity, which was an untested blind spot.)
- **Memory behaviour is not a memory system.** The model is trained to *behave* correctly around
  memory (when to save, when to admit it doesn't know, when to reconcile a contradiction). It has no
  memory of its own — you must supply the tools and the store.
- **Tool schema is specific** to the five tools it was trained on (`remember`, `recall`, `exec`,
  `web_search`, `send_message`). Generalisation to arbitrary schemas is untested.
- **Vision is verified, not optimised.** The multimodal floor exists to prevent regression; the eval
  is a smoke test, not a VQA benchmark. Don't assume VLM-grade performance.
- **Math is ~89–92 on GSM8K-style problems.** Not evaluated on MATH, competition problems, or long
  symbolic derivation.
- **GGUF quants are unscored** — see the note above.

## License

Derived from `google/gemma-4-12b-it` and therefore governed by the
**[Gemma Terms of Use](https://ai.google.dev/gemma/terms)**. You must comply with the Gemma license
and the Gemma Prohibited Use Policy. Training data was generated exclusively with open-weight
teacher models.

## Citation

```bibtex
@misc{aria-v8,
  title  = {Aria V8: a tool-using, memory-aware assistant merged from Gemma-4-12B},
  author = {Williams, Sergio},
  year   = {2026},
  url    = {https://huggingface.co/SurgeFF/AriaV8}
}
```