Instructions to use microsoft/bloom-deepspeed-inference-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/bloom-deepspeed-inference-fp16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="microsoft/bloom-deepspeed-inference-fp16")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("microsoft/bloom-deepspeed-inference-fp16") model = AutoModel.from_pretrained("microsoft/bloom-deepspeed-inference-fp16", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Update README.md
648a475
-
mp-size -> tp-size
348b3ad -
add dtype to config
51dd2ae -
update inference config json
5923650 -
rename ckpts/json to remove bloom name
0b20ff8 -
rename non-tp
0e68911
-
add ds config
5400cf7 -
bloom configs
3db7b8c -
add tokenizer into lfs
969cf21
-
adding ckpt 7 7
1164ac4 -
adding ckpt 7 6
8e36620 -
adding ckpt 7 5
a9dbca7 -
adding ckpt 7 4
626e3b3 -
adding ckpt 7 3
b91248b -
adding ckpt 7 2
7f45441 -
adding ckpt 7 1
82c4ef6 -
adding ckpt 7 0
c49f720 -
adding ckpt 6 7
fa22c44 -
adding ckpt 6 6
58f8677 -
adding ckpt 6 5
aedd6fc -
adding ckpt 6 4
1dd3ebf -
adding ckpt 6 3
74efcf1 -
adding ckpt 6 2
698a6c2 -
adding ckpt 6 1
c085bef -
adding ckpt 6 0
4d29497 -
adding ckpt 5 7
6c30b82 -
adding ckpt 5 6
b21f01b -
adding ckpt 5 5
be80983 -
adding ckpt 5 4
220df79 -
adding ckpt 5 3
8fd47fa -
adding ckpt 5 2
62769b4 -
adding ckpt 5 1
9c77e67 -
adding ckpt 5 0
e585a33 -
adding ckpt 4 7
498f02d -
adding ckpt 4 6
051f56b -
adding ckpt 4 5
8620691 -
adding ckpt 4 4
b2219a2 -
adding ckpt 4 3
769fbae -
adding ckpt 4 2
60f434e -
adding ckpt 4 1
08254e7 -
adding ckpt 4 0
e7a5ecd -
adding ckpt 3 7
70e4bd6 -
adding ckpt 3 6
2c058df -
adding ckpt 3 5
5eab6aa -
adding ckpt 3 4
e2c52b0 -
adding ckpt 3 3
bfb5e53 -
adding ckpt 3 2
4c3c889 -
adding ckpt 3 1
929ad62 -
adding ckpt 3 0
6bed131 -
adding ckpt 2 7
281f1ac