Instructions to use izlley2/LLM0to1-10b-step83000 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use izlley2/LLM0to1-10b-step83000 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="izlley2/LLM0to1-10b-step83000")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("izlley2/LLM0to1-10b-step83000")
model = AutoModelForCausalLM.from_pretrained("izlley2/LLM0to1-10b-step83000", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use izlley2/LLM0to1-10b-step83000 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "izlley2/LLM0to1-10b-step83000"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "izlley2/LLM0to1-10b-step83000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/izlley2/LLM0to1-10b-step83000

SGLang

How to use izlley2/LLM0to1-10b-step83000 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "izlley2/LLM0to1-10b-step83000" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "izlley2/LLM0to1-10b-step83000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "izlley2/LLM0to1-10b-step83000" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "izlley2/LLM0to1-10b-step83000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use izlley2/LLM0to1-10b-step83000 with Docker Model Runner:
```
docker model run hf.co/izlley2/LLM0to1-10b-step83000
```

LLM0to1-10b — step 83000

바닥부터 학습한 ~10B 한/영 이중언어 LLM의 사전학습 중간 체크포인트입니다.

⚠️ base 모델 — instruction/chat 튜닝이 되어 있지 않습니다. ⚠️ LR 감쇠(anneal) 이전 체크포인트로 생성 품질이 낮습니다. 연구 목적 아카이브입니다.

스펙

항목	값
아키텍처	`SmolLM3ForCausalLM`
파라미터	약 10B (dense)
hidden / layers	4096 / 44
heads	32 Q / 8 KV (GQA), head_dim 128
intermediate	13312
vocab	160,000 (자체 학습 토크나이저)
컨텍스트	4096
dtype	bfloat16
기타	qk_norm, z-loss, partial RoPE(4레이어마다 생략)

학습

프레임워크: nanotron
옵티마이저: 하이브리드 Muon(2D 행렬) + AdamW(임베딩/norm)
LR: WSD 스케줄, stable 2e-4 (이 체크포인트는 stable 구간)
global batch: 2.1M 토큰 (512 × 4096)
학습량: 약 175B 토큰

사용법

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
m = "izlley2/LLM0to1-10b-step83000"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="auto")
ids = tok("대한민국의 수도는", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=32)[0], skip_special_tokens=True))

이어서 사전학습하려면 원본 nanotron 체크포인트(옵티마이저 상태 포함)를 사용하세요: izlley2/LLM0to1-10b-step83000-nanotron

주의

안전성 정렬(alignment)이 되어 있지 않습니다.
학습 데이터에 이용약관이 있는 국내 코퍼스가 포함되어 있어 연구 목적으로 제한합니다. 상업적 이용 전 확인이 필요합니다.

Downloads last month: 100

Safetensors

Model size

10B params

Tensor type

BF16