Text Generation
Transformers
PyTorch
Safetensors
English
gpt_neox
causal-lm
pythia
text-generation-inference
Instructions to use EleutherAI/pythia-12b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use EleutherAI/pythia-12b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="EleutherAI/pythia-12b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("EleutherAI/pythia-12b") model = AutoModelForCausalLM.from_pretrained("EleutherAI/pythia-12b") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use EleutherAI/pythia-12b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "EleutherAI/pythia-12b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "EleutherAI/pythia-12b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/EleutherAI/pythia-12b
- SGLang
How to use EleutherAI/pythia-12b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "EleutherAI/pythia-12b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "EleutherAI/pythia-12b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "EleutherAI/pythia-12b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "EleutherAI/pythia-12b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use EleutherAI/pythia-12b with Docker Model Runner:
docker model run hf.co/EleutherAI/pythia-12b
Commit History
fix batch size and add paper 35c9d7f
fix checkpoint count and shorten intro section 3fef353
wording fix 1f16131
updated model card 555c484
add updated eval plots 539109a
Upload pytorch_model-00002-of-00003.bin 818d2d2
Upload pytorch_model-00003-of-00003.bin 4bfdd5b
Upload pytorch_model-00001-of-00003.bin 5a9ce6b
Upload config.json 8c00317
Upload tokenizer_config.json 014f3e8
Upload special_tokens_map.json 088405f
Upload tokenizer.json d6a1729
Upload pytorch_model.bin.index.json ab099b3
Upload tokenizer.json 9a24c5a
Upload special_tokens_map.json 20b2cee
Upload pytorch_model-00003-of-00003.bin 6d11710
Upload pytorch_model-00002-of-00003.bin 9df309a
Upload pytorch_model-00001-of-00003.bin 50079e1
Upload tokenizer_config.json 6ebee27
Upload pytorch_model.bin.index.json 2551490
Upload config.json f3f986a
Upload pytorch_model.bin.index.json 3c22ede
Upload pytorch_model-00001-of-00003.bin 02da881
Upload pytorch_model-00003-of-00003.bin 8419c53
Upload pytorch_model-00002-of-00003.bin 53bc2e0
Upload tokenizer_config.json 11fede8
Upload config.json 0929857
Upload special_tokens_map.json 16de778
Upload tokenizer.json e4c399e
initial commit 878bced
Hailey Schoelkopf commited on