Instructions to use kuanhuggingface/speech-chatpgpt-base-nar-only-codec with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kuanhuggingface/speech-chatpgpt-base-nar-only-codec with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kuanhuggingface/speech-chatpgpt-base-nar-only-codec")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("kuanhuggingface/speech-chatpgpt-base-nar-only-codec")
model = AutoModelForSeq2SeqLM.from_pretrained("kuanhuggingface/speech-chatpgpt-base-nar-only-codec")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kuanhuggingface/speech-chatpgpt-base-nar-only-codec with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kuanhuggingface/speech-chatpgpt-base-nar-only-codec"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kuanhuggingface/speech-chatpgpt-base-nar-only-codec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kuanhuggingface/speech-chatpgpt-base-nar-only-codec

SGLang

How to use kuanhuggingface/speech-chatpgpt-base-nar-only-codec with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kuanhuggingface/speech-chatpgpt-base-nar-only-codec" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kuanhuggingface/speech-chatpgpt-base-nar-only-codec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kuanhuggingface/speech-chatpgpt-base-nar-only-codec" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kuanhuggingface/speech-chatpgpt-base-nar-only-codec",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kuanhuggingface/speech-chatpgpt-base-nar-only-codec with Docker Model Runner:
```
docker model run hf.co/kuanhuggingface/speech-chatpgpt-base-nar-only-codec
```

speech-chatpgpt-base-nar-only-codec

Commit History

Training in progress, step 80000

2882ee9

kuanhuggingface commited on Sep 10, 2023

Training in progress, step 70000

22233b5

kuanhuggingface commited on Sep 9, 2023

Training in progress, step 60000

2b12aa7

kuanhuggingface commited on Sep 9, 2023

Training in progress, step 50000

709db84

kuanhuggingface commited on Sep 9, 2023

Training in progress, step 40000

5b38006

kuanhuggingface commited on Sep 8, 2023

Training in progress, step 30000

d2567a2

kuanhuggingface commited on Sep 8, 2023

Training in progress, step 20000

d6eea7e

kuanhuggingface commited on Sep 8, 2023

Training in progress, step 10000

267f927

kuanhuggingface commited on Sep 8, 2023

initial commit

1e4b567

kuanyi commited on Sep 8, 2023

Commit History

Training in progress, step 80000 2882ee9

Training in progress, step 70000 22233b5

Training in progress, step 60000 2b12aa7

Training in progress, step 50000 709db84

Training in progress, step 40000 5b38006

Training in progress, step 30000 d2567a2

Training in progress, step 20000 d6eea7e

Training in progress, step 10000 267f927

initial commit 1e4b567

Training in progress, step 80000

2882ee9

Training in progress, step 70000

22233b5

Training in progress, step 60000

2b12aa7

Training in progress, step 50000

709db84

Training in progress, step 40000

5b38006

Training in progress, step 30000

d2567a2

Training in progress, step 20000

d6eea7e

Training in progress, step 10000

267f927

initial commit

1e4b567