Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
File size: 2,683 Bytes
4a224d8 20ac6a9 4a224d8 20ac6a9 4a224d8 a655f9e 4a224d8 536a915 4a224d8 0200b62 4a224d8 0200b62 4a224d8 0a1bf22 4a224d8 536a915 4a224d8 6cb7cc9 4a224d8 536a915 4a224d8 6cb7cc9 4a224d8 472cdd0 4a224d8 536a915 4a224d8 536a915 4a224d8 4051715 4a224d8 536a915 4a224d8 536a915 4a224d8 536a915 4a224d8 536a915 4a224d8 472cdd0 4a224d8 472cdd0 4a224d8 472cdd0 4a224d8 536a915 4a224d8 0aa20cd 4a224d8 472cdd0 536a915 4a224d8 536a915 4a224d8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | -lee este texto:
"Daring Pilot
At age 23, Barrington Irving became the youngest person to fly alone around the world—in a small plane he built himself. Since then, he started a company called Experience Aviation. It helps students learn about flying and other subjects, like science and math. Students can study in the “Flying Classroom” or learn how to build cars.
Here, Barrington Irving talks about his love of flying and his flight around the world.
Why do you fly?
I love being in the air between the sky and the Earth. It’s exciting, but I also feel very calm.
Are you ever afraid?
Yes, I was very afraid one time. It was during my flight around the world. I had to fly from Japan to a small
island called Shemya, near Alaska. The flight was challenging. It was very windy, and then there was a bad storm. At one point, the radio pilots on the ground sent me a message. They said, “You have to go back to Japan.” And I said, “I don’t have enough fuel to go back.” It was risky to continue, but I took a chance and kept going. The radio pilots asked for my parents’ phone number. It was the end for me, they thought, but I didn’t give up. When I got to Shemya Island, I had 12 minutes of fuel left.
What a frightening experience!
Yes, it was, but I got to Shemya safely. I made it. And I learned a lot about flying.""
1-responde estas preguntas y escoge a o b, cual es la correcta:
Irving flies because __________.
a-it's a good job
b-he likes the feeling of flying
Irving didn't go back to Japan because __________.
a-his plane stopped
b-he didn't have enough gas
The radio pilots asked for Irving’s parents' phone number because they __________.
a-wanted to help Irving
b-thought Irving might die
Irving says the difficult flying experience __________.
a-was exciting
b-taught him a lot
Which sentence would Irving agree with?
a-When things are hard, keep trying. You can succeed.
b-It's always better to play it safe. You don't always have to succeed.
2-tanbien ordena estas palabras : but I had to take a chance and keep going.
but I got to Shayma safely.
and then there was a bad storm.
It's exciting,
It was very windy,
It was risky to continue,
It was the end for me, they thought,
Yes it was,
but I didn't give up.
but I also feel very calm.
3- Infer meaning. Complete the sentences with the best answers, escoge a ,b o c.
In paragraph 4, sentence 4, challenging means __________.
a-exciting
b-long
c-hard
In paragraph 4, sentence 11, give up means __________.
a-talk
b-continue
c-quit
In line paragraph 5, sentence 2, Irving says I made it. This means "I did something __________ ."
a-successfully
b-slowly
c-poorly
|