Kiel-mini-26M-v5

⚠️ Status: research artifact / early experiment, not a usable language model today. The tokenizer used at training time was not preserved in this repository, so the checkpoint cannot be loaded end-to-end with HF AutoModel / AutoTokenizer. Output quality on standard Indonesian benchmarks is far below the org's working models (Kiel-Pro-0.5B-v3, AksaraLLM-Qwen-1.5B-v5-public).

What this is

A 39M-parameter decoder-only transformer trained from scratch as part of the early AksaraLLM experiments. Architecture (inferred from weight shapes):

Property	Value
Parameters	38.7M
Layers	6
Heads	6
Hidden size	384
FFN size (SwiGLU)	1536
Vocabulary	32000
Context length	256
RMSNorm + RoPE + SwiGLU	yes

Measured baseline (Devin audit, CPU eval)

We loaded this checkpoint with aksarallm.model.aksaraLLMModel, tested several candidate tokenizers (AksaraLLM/aksara-tokenizer-v1/v2/v3, Llama-2 SentencePiece, GPT-2 BPE), and ran perplexity on 50 short Indonesian Wikipedia-style sentences plus 5 free-form prompts. Best-case results:

Perplexity: BROKEN (tokenizer mismatch — see Limitations) — meaning the model is not modelling Indonesian distribution.
English-stopword ratio in ID-prompted output: 7.7%
Indonesian-stopword ratio in ID-prompted output: 0.0%

Sample output for prompt "Indonesia adalah negara":

Indonesia adalah negara2okics lic success4!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Why the original "Skor 10/11 (Grade S)" claim is misleading

The score in earlier versions of this README is from a custom 11-question in-house scorecard graded on a tiny SFT set, not from a standard language modelling metric. By any standard LM evaluation (perplexity, response coherence on out-of-distribution prompts, identity calibration), this model does not function.

Limitations

Tokenizer not preserved. Without it, downstream usage is impossible.
No HF-compatible config.json in the original training pipeline; the aksarallm package is required for loading.
Vocab size 32000 — does not match any published AksaraLLM tokenizer (32 768, 64 000) or common open tokenizers (Llama-2 = 32 000).
Trained on a small mixed corpus (Wikipedia / SFT pairs), insufficient for general Indonesian generation at this scale.

What to use instead

If you want a small Indonesian LM in the AksaraLLM family, use:

AksaraLLM/Kiel-Pro-0.5B-v3 — 494M Qwen2-based, perplexity ≈ 15 on the same eval set.
AksaraLLM/AksaraLLM-Qwen-1.5B-v5-public — 1.78B Qwen2-based, perplexity ≈ 8.4.

Citation

@misc{aksarallm-kiel-mini-26m-v5,
  author       = {Cahyok Putra and AksaraLLM Community},
  title        = {Kiel-mini-26M-v5: early-experiment Indonesian transformer (39M params)},
  year         = 2025,
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/AksaraLLM/Kiel-Mini-26M-v5}},
}

License

Apache 2.0

Downloads last month: -; Downloads are not tracked for this model. How to track