Haru

Haru is a compact Korean story continuation model built with the custom CFRD causal architecture. It has 6,793,363 parameters and supports recurrent inference depths 2, 4, and 6.

• Source code
• Interactive demo

Usage

Review the included Python files before enabling remote custom code.

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "gaon12/haru"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, trust_remote_code=True)

inputs = tokenizer("작은 마을에 아침이 찾아왔어요.", return_tensors="pt")
output = model.generate(
    **inputs,
    max_new_tokens=120,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    top_k=40,
    repetition_penalty=1.08,
    use_cache=False,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Model details

• Parameters: 6,793,363
• Context length: 512
• Recurrent depths supervised during training: (2, 4, 6)
• Exported checkpoint step: 6104
• Training tokens seen: 800,063,488

Evaluation

Recurrent depth	Validation loss	Perplexity
2	2.37096	10.708
4	2.06052	7.850
6	2.00630	7.436

Training data attribution

Tiny-Ko-Stories by psymon, licensed under CC BY 4.0. The dataset is not redistributed with this model.

Limitations

• Haru is a continuation model, not an instruction-following assistant.
• Longer generations can repeat ideas or drift between entities.
• The model is not suitable for factual or safety-critical use.
• There is no inference cache yet, so generation recomputes the active context.

License

Haru model weights and included code are released under the MIT License. The training dataset remains under its separate CC BY 4.0 license.