Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
-
Updates the model card in light of more developed format.
Updates the model card in light of the specifications HF has been setting out based on our exhaustive survey of model cards. This requires adding a little section detailing the model card, being more mindful of the use of <detail> expansions, and making key terms/sections more clear. Since the markdown seems to render only a slight difference between # Section headers and ## Subsection headers, I have added a line -- to better demarcate them.e7e28b1
-
Update README.md
Younes BelkadaUpdate global step826ae56
-
Merge branch 'main' of https://huggingface.co/bigscience/bloom into main
c2f6b0a -
new data
18894c7 -
new data
2690f53
-
Update README.md
215bee9
-
Edit intermediary-checkpoint warning
Teven Le Scao55d00a5
-
Update README.md
Younes Belkadae8fa22e -
Update README.md
Younes Belkadacf39b50 -
Update README.md
Younes Belkada39b727a -
Update README.md
Younes Belkada945bfb3 -
Update README.md
Younes Belkada4320d65
-
Update README.md
Younes Belkada3351c7c -
Update README.md
Younes Belkada6dbe969 -
new data
5e650f0
-
new data
bea8142 -
new data
e0e6b58
-
Update config.json
Younes Belkada1d7d7cc
-
Add correct pipeline tag (#8)
ybelkada- Add correct pipeline tag (9d9c82605d1e3c66d79f1635336eb4f60e77b394)0bc3d2e -
Upload tokenizer.json
Younes Belkada9744a05 -
Upload special_tokens_map.json
Younes Belkada55dd66c -
Upload tokenizer_config.json
Younes Belkada8f193aa
-
Update config.json
Younes Belkadad6e3260
-
Different wording for "derivative" models (#5)
- Different wording for "derivative" models (ccfb08dafdd8a94a202e8c0d613ad4d760fa5f3c) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>bf2a471 -
Open-source -> Open-science in citation
8ffc30e
-
Update date.
908fd7c -
Training cost estimate (#4)
- Training cost estimate (57285788330586941122ac23f9eee9a675de942f) Co-authored-by: Teven Le Scao <teven@users.noreply.huggingface.co>359f0e6
-
Moves Chris up in the author list
Due to his heroic efforts getting this out over the past few days.9b8168a -
Changing name from "Open-source" to "Open-science"
RAIL license is open-source-like; perhaps open-science is a better term. Discussion here: https://huggingface.slack.com/archives/C020B9V3L1K/p16534101950014292701b5e -
Updating the loss and ppl values for today.
114e931 -
Adds link to the tensorboard, yay! (#3)
- Adds link to the tensorboard, yay! (4f46d79b9818fc7b4bc5746b0c203b00d4aa7dfa) Co-authored-by: Margaret Mitchell <meg@users.noreply.huggingface.co>51eb23c -
Updates the BLOOM model card to sync with the updated https://github.com/bigscience-workshop/model_card (#2)
- Updates the BLOOM model card to sync with the updated https://github.com/bigscience-workshop/model_card (bec8ba3575dbd338dea1dc36044e8d54aba46c50) Co-authored-by: Margaret Mitchell <meg@users.noreply.huggingface.co>472cdd0 -
Update README.md
e9247ea
-
Update README.md
Younes Belkadaa655f9e -
Update Model Card
Younes Belkada536a915
-
new data
510180a -
new data
e1cf111
-
initial commit
Younes Belkada0b49f22