Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloomz with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloomz with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloomz")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz") model = AutoModelForCausalLM.from_pretrained("bigscience/bloomz", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloomz with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloomz" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloomz", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloomz
- SGLang
How to use bigscience/bloomz with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloomz" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloomz", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloomz" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloomz", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloomz with Docker Model Runner:
docker model run hf.co/bigscience/bloomz
Commit History
-
Update README.md
c3f7ca1 -
Add prompts
d85a6ee -
Add bibtex
3e2f262 -
Add paper
81497b2
-
Adding `safetensors` variant of this model (#1)
- Adding `safetensors` variant of this model (f0e3534bb48531cdde5a8d9b71ebe420572df397)281d7ec -
Update README.md
50b60e0 -
Update README.md
65a35bb -
Update README.md
d253e3e -
Update README.md
64844fb -
Update README.md
26b28df -
Update README.md
8d8a93f -
Update README.md
4465a27 -
Update model family table
a7c87ea -
Update README.md
65566d0 -
Update README.md
dab3029 -
Update README.md
b21a48a -
Update README.md
81d56cf -
Update README.md
928e9ab -
Update README.md
f6e005a -
Update README.md
05d6b62 -
Update README.md
ad0c46c -
Update README.md
bc34f28 -
Update README.md
8b95958 -
Update README.md
f0d4457 -
Update README.md
8c53520 -
Update README.md
4cb792b -
Updates
44f3fe5 -
Update README.md
fd4a9a8
-
Update README.md
4d349c9 -
Update README.md
259952a -
Update README.md
29e57a5 -
Update README.md
f8d4850 -
Update README.md
016ab74 -
Update README.md
943694b -
Update README.md
da2f37f -
Update README.md
028d295 -
Update README.md
99b17ec -
Update README.md
27464b7 -
Update README.md
a75cdc0
-
Merge branch 'main' of https://huggingface.co/bigscience/bloomz
3432304 -
Add res
37a1107
-
Create README.md
59290ca
-
Logs
480af3c -
Organize val files
baaf0c1
-
Update
cb6d47c
-
Upload files
f2124a1 -
initial commit
ed9014d