Model Card for Kiseki-0.3B

A experiment to make NanoSakura multilingual.

Model Details

Model Description

• Developed by: telecomadm1145
• Model type: Transformer-Mamba2 Seq2Seq
• Language(s) (NLP): Japanese (ja), Chinese (zh), English (en)
• License: MIT
• Parameters: 0.3B

How to Get Started with the Model

<|2zh|>/<|2ja|>/<|2en|>{source text}<eos>

import torch
from transformers import AutoModelForSeq2SeqLM, PreTrainedTokenizerFast

repo_id = "telecomadm1145/Kiseki-0.3B"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = PreTrainedTokenizerFast.from_pretrained(repo_id)
model = AutoModelForSeq2SeqLM.from_pretrained(
    repo_id, 
    trust_remote_code=True, 
    dtype=torch.float32 
)
model.to(device)

text = "おはようございます。"
input_ids = tokenizer.encode("<|2zh|>" + text + "<eos>")
input_tensor = torch.tensor([input_ids]).to(device)

output_ids = model.generate(
    input_tensor, 
    max_new_tokens=256,
)
result = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(f"Translation: {result}")

Metrics

Table 1: Multi-Model Comparison (ja -> zh)

Metric	opus-mt-ja-zh (~73M)	NanoSakura-2-0.2B	NanoSakura-2.2-0.2B	NanoSakura-2.3-0.2B	NanoSakura-0.3B	nllb-200-1.3B	Qwen3-0.6B (fp16)	Qwen3-0.6B (fp16, thinking)	Qwen3-1.7B (fp16)	Qwen3-1.7B (fp16, thinking)	Kiseki-0.3B
FLORES-200 BLEU	25.67	23.27	26.55	28.67	22.36	20.87	12.58	21.13	27.12	27.56	22.51
FLORES-200 spBLEU	-	-	20.08	22.02	16.13	-	-	-	-	-	16.09
FLORES-200 COMET	0.8371	0.8380	0.8494	0.8563	0.8307	0.7805	0.8020	0.8220	0.8561	0.8571	0.8272
shard_00134 BLEU	8.07	58.13	57.55	49.32	58.71	5.73	6.89	14.57	23.37	24.60	53.29
shard_00134 spBLEU	-	-	54.12	45.10	55.57	-	-	-	-	-	49.75
shard_00134 COMET	0.4493	0.8615	0.8608	0.8558	0.8654	0.5181	0.6930	0.7414	0.8158	0.8182	-
shard_00134 COMET [Note 1]	-	-	0.9016	0.8899	0.9044	-	-	-	-	-	0.8910

[Note 1] Methodology Variance in SHARD-00134 COMET Evaluation: Evaluated on SHARD-00134 at the sentence level (the 200 raw items were split into 3,197 individual sentences). Sentence-level evaluation reduces length penalties and aligns semantic embeddings more tightly, resulting in a higher absolute score.

---

Table 2: Complete Benchmark Results for Kiseki-0.3B (All Directions)

Below is the comprehensive performance report across all 8 tested language pairs/datasets:

Dataset	Direction	SacreBLEU	spBLEU	COMET (Ref)	COMET-QE (Ref-less)
FLORES-200_JA-ZH	ja -> zh	22.51	16.09	0.8272	0.7894
FLORES-200_ZH-JA	zh -> ja	14.99	13.49	0.8479	0.7738
FLORES-200_JA-EN	ja -> en	14.35	15.30	0.7620	0.7564
FLORES-200_EN-JA	en -> ja	21.73	17.61	0.8179	0.7868
FLORES-200_ZH-EN	zh -> en	11.01	11.91	0.6655	0.6467
FLORES-200_EN-ZH	en -> zh	21.28	14.21	0.7017	0.6733
SHARD-00134_JA-ZH	ja -> zh	53.29	49.75	0.8910	0.6873
SHARD-00134_ZH-JA	zh -> ja	32.85	30.18	0.9070	0.7270