---
license: cc-by-nc-4.0
language: [ar, en, de, fr, es, it]
tags: [multilingual, research, ablation, arkadiko, laser2, moda, sophia]
library_name: pytorch
---

# Arkadiko V5 Experiments — 150M Architecture Validation

Research artifacts from the 150M production prep sprint (2026-04-12).
Single RTX PRO 4000 Blackwell (24GB), /bin/zsh compute.

## Contents

| Directory | What | Key result |
|---|---|---|
| tokenizer/ | 60K SentencePiece BPE (6-lang) | ar 4.97, en 5.72 chars/tok |
| aeq_v10_adamw/ | 150M calibration (MoDA + LASER2 cross-attn) | val 2.81, 37.8K tok/s |
| aeq_v10b_sophia/ | Sophia-G matched-step head-to-head | val 2.57 but SFT WORSE |
| sft_adamw/ | AdamW SFT (22K Aya examples) | delta -0.53 overall |
| sft_sophia/ | Sophia SFT comparison | delta -0.37 (REJECTED) |

## Architecture (locked for production)
14L x 640d x GQA 5:1 x MoDA(block_size=2), per-layer cross-attention to
frozen LASER2 BiLSTM. 173.5M params. AdamW optimizer.

## Sophia verdict (ADR-192): REJECTED
Pre-training val loss 8.6% better, but SFT transfer 30% WORSE. Russian
regressed. Generations degenerate. Third confirmation that pre-training
loss != downstream quality (L-296).

## License
CC BY-NC 4.0