Instructions to use MoeZilla/Chatbot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MoeZilla/Chatbot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MoeZilla/Chatbot")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MoeZilla/Chatbot")
model = AutoModelForCausalLM.from_pretrained("MoeZilla/Chatbot")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MoeZilla/Chatbot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MoeZilla/Chatbot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoeZilla/Chatbot",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MoeZilla/Chatbot

SGLang

How to use MoeZilla/Chatbot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MoeZilla/Chatbot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoeZilla/Chatbot",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MoeZilla/Chatbot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MoeZilla/Chatbot",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MoeZilla/Chatbot with Docker Model Runner:
```
docker model run hf.co/MoeZilla/Chatbot
```

Chatbot

Commit History

Create README.md

2f37a5b

MoeZilla commited on Jul 20, 2021

Create tokenizer_config.json

67ef92e

MoeZilla commited on Jul 20, 2021

Create special_tokens_map.json

01a75b4

MoeZilla commited on Jul 20, 2021

Create pytorch_model.bin

8fa0407

MoeZilla commited on Jul 20, 2021

Create config.json

cc5374c

MoeZilla commited on Jul 20, 2021

initial commit

d491175

system HF Staff commited on Jul 20, 2021

Commit History

Create README.md 2f37a5b

Create tokenizer_config.json 67ef92e

Create special_tokens_map.json 01a75b4

Create pytorch_model.bin 8fa0407

Create config.json cc5374c

initial commit d491175

Create README.md

2f37a5b

Create tokenizer_config.json

67ef92e

Create special_tokens_map.json

01a75b4

Create pytorch_model.bin

8fa0407

Create config.json

cc5374c

initial commit

d491175