Instructions to use aaaaaaaieuyd/Small-LLM1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aaaaaaaieuyd/Small-LLM1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aaaaaaaieuyd/Small-LLM1")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aaaaaaaieuyd/Small-LLM1")
model = AutoModelForCausalLM.from_pretrained("aaaaaaaieuyd/Small-LLM1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aaaaaaaieuyd/Small-LLM1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aaaaaaaieuyd/Small-LLM1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaaaaaaieuyd/Small-LLM1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aaaaaaaieuyd/Small-LLM1

SGLang

How to use aaaaaaaieuyd/Small-LLM1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aaaaaaaieuyd/Small-LLM1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaaaaaaieuyd/Small-LLM1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aaaaaaaieuyd/Small-LLM1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aaaaaaaieuyd/Small-LLM1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aaaaaaaieuyd/Small-LLM1 with Docker Model Runner:
```
docker model run hf.co/aaaaaaaieuyd/Small-LLM1
```

Small-LLM1 / README.md

aaaaaaaieuyd

Upload 7 files

16dce9d verified 7 days ago

preview code

Raw History Blame Contribute Delete

2.51 kB

metadata

license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation
datasets:
  - roneneldan/TinyStories
tags:
  - tiny
  - toy-model
  - from-scratch
  - llama

Small LLM1

재미로 만든 약 5M(4.98M) 파라미터짜리 초소형 언어 모델입니다.

⚠️ 재미로 만든 모델이라 성능은 낮습니다. 실제 용도로 쓰기에는 부족하고, 짧은 영어 동화 문장을 이어 쓰는 정도만 할 수 있습니다.

⚠️ 파인튜닝(instruction tuning / chat tuning)은 하지 않았습니다. 사전학습만 한 base 모델이라 질문에 답하거나 대화하지 못하고, 주어진 문장을 이어서 쓰기만 합니다.

모델 정보

항목	값
구조	Llama 스타일 decoder-only Transformer (처음부터 학습)
파라미터	4.98M
레이어 / hidden / heads	6 / 256 / 8
FFN 크기	512
어휘 크기	4,096 (직접 학습한 byte-level BPE)
컨텍스트 길이	256 토큰
학습 데이터	TinyStories 앞부분 약 40만 개 이야기 (약 92M 토큰)
학습	6,000 step × 64 × 256 토큰, RTX 5070 1장에서 약 4분
검증 loss	1.77 (perplexity 약 5.9, TinyStories 검증 구간)

사용법

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("aaaaaaaieuyd/Small-LLM1")
tok = AutoTokenizer.from_pretrained("aaaaaaaieuyd/Small-LLM1")

x = tok("Once upon a time", return_tensors="pt")
out = model.generate(**x, max_new_tokens=80, do_sample=True, temperature=0.8, top_k=40)
print(tok.decode(out[0], skip_special_tokens=True))

출력 예시:

Once upon a time, there was a little girl named Lily. She had a little tooth that was very important to her. The tooth was her special tooth, and she could always count on it to show it to her friends.

한계

영어 동화체 문장만 학습해서 한국어나 다른 주제는 거의 못 다룹니다.
사실 정보, 추론, 코드 같은 건 기대하면 안 됩니다.
앞뒤가 안 맞는 이야기를 자주 만듭니다.
파인튜닝을 하지 않아서 지시를 따르지 못합니다.

English: Small LLM1 is a ~5M-parameter toy language model trained from scratch on TinyStories, just for fun. Performance is low, and it is a base model with no fine-tuning (no instruction or chat tuning). It can only continue short, simple English stories.