Instructions to use iamplus/bloomz-7b1-cot-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use iamplus/bloomz-7b1-cot-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="iamplus/bloomz-7b1-cot-v1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("iamplus/bloomz-7b1-cot-v1")
model = AutoModelForCausalLM.from_pretrained("iamplus/bloomz-7b1-cot-v1")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use iamplus/bloomz-7b1-cot-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "iamplus/bloomz-7b1-cot-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamplus/bloomz-7b1-cot-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/iamplus/bloomz-7b1-cot-v1

SGLang

How to use iamplus/bloomz-7b1-cot-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "iamplus/bloomz-7b1-cot-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamplus/bloomz-7b1-cot-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "iamplus/bloomz-7b1-cot-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamplus/bloomz-7b1-cot-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use iamplus/bloomz-7b1-cot-v1 with Docker Model Runner:
```
docker model run hf.co/iamplus/bloomz-7b1-cot-v1
```

bloomz-7b1-cot-v1

Commit History

Update README.md

139589e

manojpreveen commited on Aug 18, 2023

Update README.md

3a40223

manojpreveen commited on Aug 18, 2023

Delete tokenizer.json

1995a87

manojpreveen commited on Apr 5, 2023

Delete pytorch_model.bin

a1c000b

manojpreveen commited on Apr 5, 2023

Update README.md

bcb87b0

manojpreveen commited on Mar 20, 2023

Update README.md

1a774fa

manojpreveen commited on Mar 17, 2023

Update README.md

754acd6

manojpreveen commited on Mar 15, 2023

Update README.md

8d786b3

manojpreveen commited on Mar 15, 2023

Update README.md

e917184

manojpreveen commited on Mar 15, 2023

Update README.md

faa7786

manojpreveen commited on Mar 15, 2023

Update README.md

168636a

manojpreveen commited on Mar 15, 2023

Update README.md

615b123

manojpreveen commited on Mar 13, 2023

Model Update

bd11436

manojpreveen commited on Mar 13, 2023

Delete pytorch_model.bin

4eb667f

manojpreveen commited on Mar 13, 2023

Update README.md

f78442e

manojpreveen commited on Mar 12, 2023

Create README.md

82f49f6

manojpreveen commited on Mar 12, 2023

Models

8fe1b8a

manojpreveen commited on Mar 12, 2023

initial commit

ee9ed5d

Manoj Preveen commited on Mar 12, 2023

Commit History

Update README.md 139589e

Update README.md 3a40223

Delete tokenizer.json 1995a87

Delete pytorch_model.bin a1c000b

Update README.md bcb87b0

Update README.md 1a774fa

Update README.md 754acd6

Update README.md 8d786b3

Update README.md e917184

Update README.md faa7786

Update README.md 168636a

Update README.md 615b123

Model Update bd11436

Delete pytorch_model.bin 4eb667f

Update README.md f78442e

Create README.md 82f49f6

Models 8fe1b8a

initial commit ee9ed5d

Update README.md

139589e

Update README.md

3a40223

Delete tokenizer.json

1995a87

Delete pytorch_model.bin

a1c000b

Update README.md

bcb87b0

Update README.md

1a774fa

Update README.md

754acd6

Update README.md

8d786b3

Update README.md

e917184

Update README.md

faa7786

Update README.md

168636a

Update README.md

615b123

Model Update

bd11436

Delete pytorch_model.bin

4eb667f

Update README.md

f78442e

Create README.md

82f49f6

Models

8fe1b8a

initial commit

ee9ed5d