Instructions to use properly59/Jumini-Ko-1.2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use properly59/Jumini-Ko-1.2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="properly59/Jumini-Ko-1.2B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("properly59/Jumini-Ko-1.2B")
model = AutoModelForCausalLM.from_pretrained("properly59/Jumini-Ko-1.2B")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use properly59/Jumini-Ko-1.2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "properly59/Jumini-Ko-1.2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "properly59/Jumini-Ko-1.2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/properly59/Jumini-Ko-1.2B

SGLang

How to use properly59/Jumini-Ko-1.2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "properly59/Jumini-Ko-1.2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "properly59/Jumini-Ko-1.2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "properly59/Jumini-Ko-1.2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "properly59/Jumini-Ko-1.2B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use properly59/Jumini-Ko-1.2B with Docker Model Runner:
```
docker model run hf.co/properly59/Jumini-Ko-1.2B
```

Jumini-Ko-1.2B

This repository contains a from-scratch Korean decoder-only LLM checkpoint prepared for HW2.

Architecture: RoPE, RMSNorm, SwiGLU, GQA
Parameters: 1,260,505,088
Export format: Transformers-compatible LlamaForCausalLM key layout
Stage: anneal
Repo target: properly59/Jumini-Ko-1.2B
Source checkpoint: step_00006200/model.pt
Export dtype: float16

Training Summary

The model code, tokenizer pipeline, data filters, training loop, checkpointing, evaluation scripts, and export scripts were implemented in this project rather than copied from an existing LLM training repository.

The current export candidate is the V3A retention step-6200 checkpoint. A later low-LR response-only SFT pilot improved SFT validation loss but was not promoted because it did not improve generation or mini-MCQ and degraded all source-bucket losses.

Additional post-export probes were also rejected. A V3B TinyStories low-ratio probe improved its own validation split from step 6200 to step 6300, but it did not beat step 6200 on a same-condition source-bucket comparison and generation remained repetitive/malformed. The exported artifact therefore remains the V3A step-6200 base checkpoint.

License and Data Provenance

The released model artifacts are provided under Apache-2.0. The final promoted training sources were selected from datasets with explicit permissive licenses:

Source	Role	License
`lcw99/wikipedia-korean-20240501`	Korean wiki pretraining/replay	Apache-2.0
`eliceai/korean-webtext-edu`	Korean educational web pretraining/replay	MIT
`maywell/korean_textbooks`	textbook/factual synthetic Korean data	Apache-2.0
`heegyu/open-korean-instructions`	capped phase-2 instruction/chat anneal	MIT
`IkJun1/korean-qa-dataset`	capped phase-2 Korean QA anneal	MIT

Benchmark and diagnostic datasets were kept evaluation-only and were not promoted into the final training mixture.

Local Validation

Train-time held-out validation on the V3A retention split improved monotonically through step 6200:

Step	loss	perplexity
5700	4.3200	75.19
5800	4.3079	74.29
5900	4.3036	73.96
6000	4.2999	73.69
6100	4.2970	73.48
6200	4.2950	73.33

Source-bucket validation at step 6200. Web/wiki rows are from the stage-gate run with steps=50:

Source	loss	perplexity
`eliceai/korean-webtext-edu`	6.2910	539.71
`lcw99/wikipedia-korean-20240501`	4.9192	136.90
`maywell/korean_textbooks:claude_evol`	2.7999	16.44
`maywell/korean_textbooks:helpsteer`	2.7718	15.99
`maywell/korean_textbooks:ko_wikidata`	2.8092	16.60
`maywell/korean_textbooks:tiny-textbooks`	2.7239	15.24

Diagnostic results:

Task	Result
Korean mini-MCQ	10/20
KMMLU-HARD subset	36/160
Ko-WinoGrande subset	104/200
Ko-PIQA subset	95/200
KoBALT subset	20/200
HAE_RAE_BENCH_2.0 subset	14/60
BELEBELE-KOR subset	47/200
GSM8K-Ko subset	0/100
HRM8K subset	1/60

Rejected Follow-Up Probes

Probe	Decision
Low-LR response-only SFT to step 6400	Rejected: SFT validation improved, but generation did not improve and all source-bucket losses worsened.
Average of step 6100/6150/6200	Rejected: no mini-MCQ/generation gain and source buckets were slightly worse than step 6200.
Mixed retention/SFT step 6250	Rejected: retention validation improved slightly, but generation stayed repetitive and source buckets were mostly slightly worse.
Low-LR V3A retention extension to step 6300	Rejected: validation worsened versus step 6200.
V3B TinyStories low-ratio step 6300	Rejected: V3B validation improved, but same-condition source-bucket losses were flat/slightly worse and generation did not improve.

Export Checks

Native-vs-HF logits equivalence: ok=true, max absolute diff 0.015625, argmax mismatch rate 0.0.
Local Transformers smoke: AutoConfig, AutoTokenizer, and AutoModelForCausalLM load without custom code as LlamaForCausalLM.
Local parameter count after HF load: 1,260,505,088.
Local submission secret scan: passed on text artifacts; no hf_... token pattern or generic long secret assignment was found.

Intended Use

This checkpoint is intended for course evaluation, reproducibility review, and further controlled research on Korean language-model training from scratch. It is not instruction aligned and should not be treated as a production assistant.

Limitations

Free-form generation is still weak and often repetitive or malformed.
Math, reasoning, and safety behavior are not reliable.
The model can produce incorrect, biased, or unsafe text.
Evaluation numbers are local diagnostics unless explicitly marked as public benchmark results.

Downloads last month: 13

Safetensors

Model size

2B params

Tensor type

F16