max-0.1b v0

Newer version: quantumaikr/max-0.1b-v1 continues this model to 10.0B training tokens (better validation loss and public Korean task scores). / ํ›„์† ๋ชจ๋ธ: max-0.1b-v1(ํ•™์Šต 100์–ต ํ† ํฐ).

A small Korean-first base language model (109.5M parameters) for industrial small functions such as document classification, request routing, field extraction and form filling. This is an early base model (not instruction-tuned). It is intended as a starting point for task fine-tuning and for research, not for direct end-user use.

ํ•œ๊ตญ์–ด ์ค‘์‹ฌ์˜ ์†Œํ˜• ๊ธฐ์ดˆ ์–ธ์–ด๋ชจ๋ธ(1.095์–ต ๋งค๊ฐœ๋ณ€์ˆ˜)์ž…๋‹ˆ๋‹ค. ๋ฌธ์„œ ๋ถ„๋ฅ˜, ์š”์ฒญ ๋ถ„๊ธฐ, ํ•ญ๋ชฉ ์ถ”์ถœ, ์–‘์‹ ์ž‘์„ฑ ๊ฐ™์€ ์‚ฐ์—…์šฉ ์ž‘์€ ๊ธฐ๋Šฅ์˜ ์ถœ๋ฐœ์ ์œผ๋กœ ์“ฐ๊ธฐ ์œ„ํ•œ ์ดˆ๊ธฐ ๊ธฐ์ดˆ ๋ชจ๋ธ์ด๋ฉฐ, ์ง€์‹œ ๋”ฐ๋ฅด๊ธฐ ๋ฏธ์„ธ์กฐ์ • ์ „ ๋‹จ๊ณ„์ž…๋‹ˆ๋‹ค.

Files

File Use
model.safetensors, config.json, tokenizer.json Transformers (Qwen3ForCausalLM-compatible layout)
gguf/max01b-v0-f16.gguf (221 MB) llama.cpp, full precision
gguf/max01b-v0-q8_0.gguf (118 MB) llama.cpp, 8-bit
gguf/max01b-v0-q4_k_m.gguf (73 MB) llama.cpp, 4-bit, runs on ordinary CPUs

Context length: 1,024 tokens. Vocabulary: 32,000 (Korean-optimized, digits split). Input text should be Unicode NFC-normalized.

from transformers import AutoModelForCausalLM, PreTrainedTokenizerFast
tok = PreTrainedTokenizerFast.from_pretrained("quantumaikr/max-0.1b-v0")
model = AutoModelForCausalLM.from_pretrained("quantumaikr/max-0.1b-v0")
llama-cli -m max01b-v0-q4_k_m.gguf -p "์˜ค๋Š˜ ํšŒ์˜์—์„œ ๋…ผ์˜๋œ ์•ˆ๊ฑด์€" -n 64

Evaluation (base model, held-out sets)

Validation (bits per byte, lower is better): Korean web 0.90 ยท Korean documents 0.83 ยท English 1.03.

Task (0-shot, n=300) Accuracy Chance
ARC-Easy 0.43 0.25
PIQA 0.60 0.50
HellaSwag 0.29 0.25
KoBEST COPA 0.56 0.50
KoBEST HellaSwag 0.32 (norm 0.41) 0.25
KoBEST BoolQ 0.49 0.50

Few-shot industrial tasks (n=40 each, before fine-tuning): document-type classification 0.63 (calibrated), complaint classification 0.45, request routing 0.28, JSON output parse rate 0.48โ€“0.58. Field extraction and number-faithful summarization are not usable without fine-tuning.

Limitations

  • Trained on 2.8B tokens only: factual knowledge is weak and the model repeats itself.
  • It may invent numbers and facts. Do not use its output without verification.
  • Base model: it continues text; it does not follow instructions reliably.
  • Personal identifiers in the training text were masked, but small models can still reproduce fragments of web text.

Training data and attribution

All training data is publicly released under the Open Data Commons Attribution License (ODC-By 1.0):

  • FineWeb-2 (Korean subset) โ€” Hugging Face, ODC-By 1.0
  • FinePDFs (Korean subset) โ€” Hugging Face, ODC-By 1.0
  • FineWeb-Edu โ€” Hugging Face, ODC-By 1.0
  • FineMath โ€” Hugging Face, ODC-By 1.0

These datasets are derived from Common Crawl and are also subject to the Common Crawl Terms of Use.

License

Apache License 2.0.

Downloads last month
173
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support