---
license: apache-2.0
language: [ja]
pipeline_tag: text-generation
library_name: gguf
tags: [japanese, instruction-tuning, little-language-model, tiny-language-model, edge-ai, embedded-ai, ex-word, gguf, llama-cpp, lm-studio, custom-code]
---

# EXLLM-0.005B-Instruct

EXLLM-0.005B-Instruct is a **0.005377824B-parameter** Japanese instruction model for local inference on low-memory embedded hardware. Physical-device inference is verified on the CASIO EX-word XD-B4800 (DATAPLUS 6) and XD-N6500 (DATAPLUS 7). A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained **0.005441184B-parameter** Llama-compatible GGUF companion for LM Studio and llama.cpp.

Here, LLM stands for Little Language Model.

The embedded checkpoint uses a custom decoder-only Transformer and a hybrid character/UTF-8-byte tokenizer; it is not directly compatible with standard `transformers` auto classes or GGUF runtimes. Use `EXLLM-0.005B-LMStudio-F16.gguf` for LM Studio or llama.cpp. The GGUF is a companion model trained on the EXLLM project data, not a numerical conversion of the embedded checkpoint.

日本語版の説明は[こちら](#日本語)です。

## Highlights

- **0.005377824B parameters**
- **0.000868M-token vocabulary** with UTF-8 byte fallback
- **0.000128M-token context length**
- **0.021526737 GB** fp32 checkpoint
- **0.005443105 GB** int8 and EXQ12 deployment artifacts
- Reference inference at **239.082 token/s** on an RTX 3060
- Integer inference at **0.52–0.55 token/s** on a 0.024184 GHz SH-4A-class CPU
- Apache-2.0 weights and reference code
- Ready-to-run **0.010915168 GB** F16 GGUF companion for LM Studio

## Model Details

| Field | Value |
|---|---|
| Model type | Instruction-tuned decoder-only Transformer |
| Parameters | 0.005377824B |
| Language | Japanese |
| Context length | 0.000128M tokens |
| Vocabulary | 0.000868M tokens |
| Layers | 6 |
| Hidden size | 288 |
| Attention heads | 9 |
| Head dimension | 32 |
| FFN hidden size | 896 |
| Activation | ReLU |
| Normalization | Pre-norm RMSNorm, epsilon 1e-5 |
| Position encoding | Learned absolute position embedding |
| Output head | Weight-tied token embedding |
| Tokenizer | Frequent Japanese characters + UTF-8 byte fallback, NFC normalization |
| License | Apache-2.0 |

The parameter count reports unique trainable parameters. The tied language-model head and token embedding refer to the same parameter tensor.

## Training

EXLLM was trained only on project-generated Japanese instruction/response data. It does not use an external pretrained checkpoint or imported public text corpus. The EXLLM project originates from random initialization. The 0.005B release was expanded from an earlier in-project EXLLM checkpoint by transplanting dimension-compatible parameters, followed by staged training and corrective fine-tuning. No external pretrained checkpoint was used.

| Item | Value |
|---|---|
| Published staged data | 0.086408M JSONL records |
| External pretrained base | None |
| Optimizer | AdamW |
| Base learning rate | 5e-4 |
| Adam betas | 0.9, 0.95 |
| Base weight decay | 0.03 |
| Gradient clipping | 1.0 |
| LR schedule | Linear warm-up + cosine decay |
| Checkpoint global step | 1,810 |
| Final corrective stage | 100 steps at 6e-6 |
| Final corrective loss | 0.03875 → 0.02420 |

The published staged files include overlapping base, robustness, recovery, and corrective sets; 0.086408M is a file-record total, not a deduplicated example count. Exact end-to-end training time and training-device telemetry were not retained in the release checkpoint and are therefore not reported.

Data scope and weight lineage are documented in [`DATA_PROVENANCE.md`](DATA_PROVENANCE.md).
The machine-readable release metadata is available in [`training/release-5m.json`](training/release-5m.json).
The direct v1.0 parent and exact 5M stage parameters and hashes are published in [`training/release-5m-stages.json`](training/release-5m-stages.json). The source repository provides all staged datasets and `tools/replay_5m.py`.

## Inference Performance

All results use greedy decoding. TTFT means time to first generated token. Decode throughput excludes the first generated token.

| Runtime | Compute | Model format | Memory | Load | TTFT | Decode throughput |
|---|---|---|---:|---:|---:|---:|
| PyTorch 2.11.0+cu130 | RTX 3060, 12 GB VRAM | fp32 | 0.0307584 GB peak allocated | 0.4645 s | 0.003990 s | 239.082 token/s |
| EX-word integer runtime | SH-4A-class, 1C/1T, 0.024184 GHz | int8 weights / Q12 activations | 0.00786432 GB verified contiguous application heap | device storage | 23.711 s | 0.52–0.55 token/s |

### RTX 3060 methodology

- Driver 580.178.04; Python 3.12.13; CUDA 13.0.
- 40 warm-up inferences followed by 120 measured inferences.
- Median TTFT: 0.003990 s; observed range: 0.003913–0.004072 s.
- Median total generation time: 0.079589 s.
- Tukey 1.5 IQR filtering removed 0/120 observations for TTFT, total time, and decode throughput.
- The first cold warm-up inference recorded 0.193056 s TTFT.
- Raw per-run results are published in [`benchmarks/rtx3060-cuda-robust-20260925.json`](benchmarks/rtx3060-cuda-robust-20260925.json).

### EX-word methodology

- Five-run baseline: CASIO EX-word XD-B4800 / DATAPLUS 6.
- SH7305-family SH-4A, 1 core / 1 thread; measured instruction clock 0.024184 GHz.
- Five prompts, Thinking disabled, model stored in internal storage.
- Application UI rendering and UTF-8 display handling are included.
- Median TTFT: 23.711 s; observed range: 12.705–28.440 s.
- Median total generation time: 54.145 s.

A matched v1.2.0 deployment check used the same `model.q12` and Thinking OFF setting on both devices. Two `こんにちは` runs per device produced the same 24-token answer at 0.55 token/s. Median TTFT / total time was 12.702 s / 54.138 s on XD-B4800 and 12.707 s / 54.164 s on XD-N6500. Four matched fixed prompts differed by at most 11 ms in TTFT and 32 ms in total time, with no consistent winner. The recovered logs therefore validate deployment compatibility but do not establish a meaningful performance difference between these generations.

The two runtimes use the same architecture and training lineage but different numerical representations. The results are deployment measurements, not a direct hardware-compute comparison.

## Evaluation

The release was evaluated with a project-specific regression suite after dequantizing the EXLLM8 artifact into the reference runtime.

| Suite | Result |
|---|---:|
| Semantic behavior | 35 / 35 |
| Deterministic calculator routing | 300 / 300 |
| Fuzz prompts | 200 / 200 |
| Sampling checks | 48 / 48 |
| Release gate | PASS |

These are internal regression tests for the intended application scope. They are not standard language-model benchmarks and should not be interpreted as evidence of broad knowledge, general reasoning, or benchmark comparability with larger models.

## Usage

### LM Studio / llama.cpp

In LM Studio, search for `ToTo-40417/EXLLM`, download `EXLLM-0.005B-LMStudio-F16.gguf`, and start a chat. The required single-turn chat template is embedded in the GGUF. Set the loaded context length to **512 tokens** and use greedy decoding or a low temperature; the model is intentionally tiny and is intended for short Japanese prompts. Training sequences were limited to 128 tokens; the larger runtime window reserves space for LM Studio's chat wrapper and short history.

With llama.cpp:

```bash
llama-cli -m EXLLM-0.005B-LMStudio-F16.gguf \
  -p "RAMとは何ですか？" -n 48 --temp 0 --ctx-size 512 --single-turn
```

The GGUF companion uses a Llama-compatible decoder, a 1,024-piece SentencePiece tokenizer with byte fallback, 6 layers, hidden size 288, intermediate size 608, 9 attention/KV heads, 128-token training sequences, and a 512-token runtime context. It was trained separately for five epochs (3,325 optimizer steps) on 85,005 project-data records; best validation loss was 0.0390303. Exact metadata is in [`lmstudio/training_manifest.json`](lmstudio/training_manifest.json).

### Original EXLLM reference runtime

```bash
git clone https://huggingface.co/ToTo-40417/EXLLM
cd EXLLM
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
python chat.py こんにちは
```

```python
from src.loader import load_release_model
from src.runtime import answer

model, tokenizer = load_release_model()
print(answer(model, tokenizer, "RAMとは何ですか？", temperature=0.0))
```

### Generation Examples

| Prompt | Greedy output |
|---|---|
| `こんにちは` | `こんにちは。ことばの意味や簡単な質問ならどうぞ。` |
| `あなたは何というモデルですか？` | `私はEXLLMです。` |
| `RAMとは何ですか？` | `RAMは、処理中のデータやプログラムを一時的に保持する記憶装置です。` |
| `オフラインとは何ですか？` | `ウェブ検索はできません。` |

## Release Artifacts

| Artifact | Size | SHA-256 | Purpose |
|---|---:|---|---|
| `weights/EXLLM-v1.1-5m-release3.pt` | 21,526,737 bytes (20.53 MiB; 0.021526737 GB) | `48ad00c6640689595a253fc949cad83f3000e14cccdf69275fcf245e85be3ceb` | fp32 PyTorch checkpoint |
| `weights/EXLLM-v1.1-5m-int8.bin` | 5,443,105 bytes (5.19 MiB; 0.005443105 GB) | `3b2bf2d9f103cba714bc98976709c5ad34860c1ff55fed2fce91fc95afb5b34a` | EXLLM8 interchange artifact |
| `weights/model.q12` | 5,443,105 bytes (5.19 MiB; 0.005443105 GB) | `d64037fde791e5c0e48101bc1a8ab366a3287f1f36b4464879e36495ea7e5a53` | EX-word EXQ12 artifact |
| `weights/EXLLM-v1.0.0.safetensors` | 4,905,568 bytes (4.68 MiB; 0.004905568 GB) | `caad6a93aa2a8f6f2b525c4671dc8d2f8e4328d262a093172a55c672cd4e24c9` | Direct project-owned parent of the 5M expansion |
| `EXLLM-0.005B-LMStudio-F16.gguf` | 10,915,168 bytes (10.41 MiB; 0.010915168 GB) | `0e3c519d4a044fac45be84dc607eb8a43de4a7c60083983bf246c5b9c8a43472` | Separately trained Llama-compatible F16 companion for LM Studio / llama.cpp |

EXLLM8 and EXQ12 have the same file size but are not interchangeable formats. Rebuild and verify `model.q12` with `python tools/export_exq12.py`; see [`EXQ12_FORMAT.md`](EXQ12_FORMAT.md).

## Intended Use

- Ultra-small language-model research
- Embedded and low-memory inference experiments
- Short Japanese UI responses and narrow-domain instruction handling
- Integer-runtime and quantization experiments
- Educational analysis of language models below 0.01B parameters

## Limitations

- The model is not a general-purpose assistant.
- The original embedded checkpoint and the GGUF training sequences use 0.000128M tokens. The GGUF advertises a 0.000512M-token runtime window for LM Studio framing and short history; quality beyond the trained 128-token range is not guaranteed.
- Knowledge coverage is restricted to the project-generated training scope.
- Unseen concepts, long instructions, translation, and multi-step reasoning are unreliable.
- No RLHF, DPO, tool use, retrieval, web access, or current-information source is included.
- The deterministic calculator is runtime routing, not learned arithmetic capability.
- The model must not be used as an authority for medical, legal, financial, safety-critical, or other high-stakes decisions.
- Standard Hugging Face Inference Providers cannot serve the original embedded checkpoint without custom architecture support. The bundled GGUF companion is intended for local LM Studio / llama.cpp execution.

## Links

- Source and training code: [`ToTo-40417/exllm`](https://github.com/ToTo-40417/exllm)
- EX-word runtime: [`ToTo-40417/exllm-exword`](https://github.com/ToTo-40417/exllm-exword)
- EXQ12 validator: [`ToTo-40417/exllm-model-check`](https://github.com/ToTo-40417/exllm-model-check)
- Hardware probe: [`ToTo-40417/exword-hardware-dump`](https://github.com/ToTo-40417/exword-hardware-dump)
- RAM qualification tool: [`ToTo-40417/exword-ram-scanner`](https://github.com/ToTo-40417/exword-ram-scanner)
- Gnuboy save-transfer utility: [`ToTo-40417/exword-gnuboy-save-importer`](https://github.com/ToTo-40417/exword-gnuboy-save-importer)
- Project article (Japanese): [高校生用の電子辞書でLLMを動かしてみた ―0.024GHz/0.016GB](https://note.com/joyful_beetle869/n/nbd1e26679b78)

## 日本語

EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、**0.005377824Bパラメータ**の日本語instruction modelです。CASIO EX-word XD-B4800（DATAPLUS 6）とXD-N6500（DATAPLUS 7）で実機推論を確認しています。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの**0.005441184Bパラメータ**GGUF companionも収録しています。

LLMとは、Little Language Modelの略です。

電子辞書用checkpointは独自のdecoder-only Transformerとtokenizerを使用するため、GGUFに直接変換したものではありません。LM Studioでは`ToTo-40417/EXLLM`を検索し、`EXLLM-0.005B-LMStudio-F16.gguf`を選択してください。チャット用templateはGGUFに内蔵済みで、LM Studioのロード時contextは512 tokensに設定します。学習時の系列長は128 tokensであり、追加領域はチャット制御情報と短い履歴のための余白です。このGGUFはプロジェクトデータで別途学習したPC用companionであり、電子辞書版と同一重みではありません。

RTX 3060でのfp32参照実行は、40回のwarm-up後に120回測定し、TTFT中央値0.003990秒、生成速度239.082 token/sでした。EX-word整数runtimeでは、0.024184 GHzのSH-4A-class CPU上でTTFT中央値23.711秒、生成速度0.52–0.55 token/sでした。

用途は、0.01B未満の言語モデル、組込み推論、量子化、整数runtimeの研究です。汎用的な質問応答、長文処理、専門判断、最新情報の取得を目的としたモデルではありません。

コード、重み、付属データはApache License 2.0で公開します。

## Citation

```bibtex
@software{toto_exllm_2026,
  author  = {ToTo},
  title   = {EXLLM-0.005B-Instruct},
  year    = {2026},
  url     = {https://huggingface.co/ToTo-40417/EXLLM}
}
```