Instructions to use crumb/horizon-pythia-ft-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use crumb/horizon-pythia-ft-1b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="crumb/horizon-pythia-ft-1b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("crumb/horizon-pythia-ft-1b")
model = AutoModelForCausalLM.from_pretrained("crumb/horizon-pythia-ft-1b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use crumb/horizon-pythia-ft-1b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "crumb/horizon-pythia-ft-1b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crumb/horizon-pythia-ft-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/crumb/horizon-pythia-ft-1b

SGLang

How to use crumb/horizon-pythia-ft-1b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "crumb/horizon-pythia-ft-1b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crumb/horizon-pythia-ft-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "crumb/horizon-pythia-ft-1b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crumb/horizon-pythia-ft-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use crumb/horizon-pythia-ft-1b with Docker Model Runner:
```
docker model run hf.co/crumb/horizon-pythia-ft-1b
```

See the blog post!

pythia-1b "reweighted" (very short continued pretraining) with a mix of data from the crumbly Horizon dataset found via crumb's "Paying 'Attention' to your Dataset" method on pythia-70m -> pythia-160m -> and pythia-410m at different stages.

hparams:

LR: 1e-5
SCHEDULE: cosine with 20% warmup from 0, cooldown to 0
BS: 64
CTX: 2048
everything else you can think of is set to it's default value in the huggingface trainer

validation loss (loss on data not used in training, lower is better)

model	arxiv	github	books	wiki	webtext
horizon-pythia-ft-1b	2.13	1.30	2.00	2.22	2.71
pythia-1b	2.21	1.30	2.02	2.29	2.72

optimized mixture after 12 training runs, starting with 100 samples each:

subset	documents
arxiv	608
github	226
books	613
wiki	1438
webtext	8516

benchmarks that i can actually run in reasonable time:

model	arc	truthfulqa	winogrande
horizon-pythia-ft-1b	28.24	41.13	53.75
pythia-1b deduped*	29.1	38.94	53.59
the actual pythia model isnt on the leaderboard and im really tired and dont want to open the eval script again and wait

Downloads last month: 7