Instructions to use TigerResearch/tigerbot-7b-base-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TigerResearch/tigerbot-7b-base-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TigerResearch/tigerbot-7b-base-v1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TigerResearch/tigerbot-7b-base-v1")
model = AutoModelForCausalLM.from_pretrained("TigerResearch/tigerbot-7b-base-v1")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use TigerResearch/tigerbot-7b-base-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TigerResearch/tigerbot-7b-base-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TigerResearch/tigerbot-7b-base-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TigerResearch/tigerbot-7b-base-v1

SGLang

How to use TigerResearch/tigerbot-7b-base-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TigerResearch/tigerbot-7b-base-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TigerResearch/tigerbot-7b-base-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TigerResearch/tigerbot-7b-base-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TigerResearch/tigerbot-7b-base-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TigerResearch/tigerbot-7b-base-v1 with Docker Model Runner:
```
docker model run hf.co/TigerResearch/tigerbot-7b-base-v1
```

tigerbot-7b-base-v1

Commit History

Update README.md

5b57c4c

i4never commited on Jul 8, 2023

remove old file

5f34fd2

cong.fu commited on Jun 7, 2023

Use bfloat16

6b13273

cong.fu commited on Jun 7, 2023

Update README.md

bca079b

vivicai commited on Jun 5, 2023

Update config.json

d0d17b0

i4never commited on Jun 5, 2023

Create README.md

3714b66

i4never commited on Jun 2, 2023

Upload BloomForCausalLM

0fa04ab

i4never commited on Jun 1, 2023

Upload BloomForCausalLM

4e03989

i4never commited on Jun 1, 2023

Upload tokenizer

21858a8

i4never commited on Jun 1, 2023

initial commit

d6d43e4

FC commited on May 31, 2023

Commit History

Update README.md 5b57c4c

remove old file 5f34fd2

Use bfloat16 6b13273

Update README.md bca079b

Update config.json d0d17b0

Create README.md 3714b66

Upload BloomForCausalLM 0fa04ab

Upload BloomForCausalLM 4e03989

Upload tokenizer 21858a8

initial commit d6d43e4

Update README.md

5b57c4c

remove old file

5f34fd2

Use bfloat16

6b13273

Update README.md

bca079b

Update config.json

d0d17b0

Create README.md

3714b66

Upload BloomForCausalLM

0fa04ab

Upload BloomForCausalLM

4e03989

Upload tokenizer

21858a8

initial commit

d6d43e4