How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "craa/exceptions" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "craa/exceptions",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
exceptions
GPT-2 language models trained from scratch for the exceptions project (Leong and Linzen, Language Models Can Learn Exceptions to Syntactic Rules, and follow-up experiments). Code: github.com/craaaa/exceptions.
All models are GPT-2 small (12 layers, 12 heads, 768 hidden). Each folder holds the final model of one training run
(best checkpoint by validation loss), with its tokenizer and Trainer logs (trainer_state.json, all_results.json).
Contents
| Folder | Runs | Naming | Seeds | Vocab |
|---|---|---|---|---|
exp1b/ |
5 | 100M_<seed> |
1001, 2128, 3591, 5039, 40817 | 50,257 (GPT-2) |
exp2a/ |
60 | <cond>_to_<verb>_frequency_<seed> — cond ∈ {cost, last, resemble}, verb ∈ {carry, drop, hit, push} |
1001, 2128, 3591, 5039, 40817 | 50,257 (GPT-2) |
exp3/ |
170 | {high,low}_<n>_<seed> — n ∈ {0, 1, 2, 4, 8, 16, 32, 64, 10, 100, 128, 256, 500, 512, 1000, 1500, 2000} |
495, 634, 1208, 6910, 8397 | 52,000 (custom) |
Training corpora are in the dataset repo craa/exceptions;
the exp2a folder names correspond to its frequency_<cond>_to_<verb> configs.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo, run = "craa/exceptions", "exp3/high_0_1208"
model = AutoModelForCausalLM.from_pretrained(repo, subfolder=run)
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=run)
Intermediate checkpoints are not published here.
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "craa/exceptions" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "craa/exceptions", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'