Instructions to use ToddGoldfarb/Cadet-Tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ToddGoldfarb/Cadet-Tiny with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ToddGoldfarb/Cadet-Tiny")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("ToddGoldfarb/Cadet-Tiny")
model = AutoModelForSeq2SeqLM.from_pretrained("ToddGoldfarb/Cadet-Tiny")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ToddGoldfarb/Cadet-Tiny with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ToddGoldfarb/Cadet-Tiny"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToddGoldfarb/Cadet-Tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ToddGoldfarb/Cadet-Tiny

SGLang

How to use ToddGoldfarb/Cadet-Tiny with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ToddGoldfarb/Cadet-Tiny" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToddGoldfarb/Cadet-Tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ToddGoldfarb/Cadet-Tiny" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ToddGoldfarb/Cadet-Tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ToddGoldfarb/Cadet-Tiny with Docker Model Runner:
```
docker model run hf.co/ToddGoldfarb/Cadet-Tiny
```

Cadet-Tiny

Commit History

Update README.md

04d1210

ToddGoldfarb commited on May 12, 2023

Upload 3 files

9d9788d

ToddGoldfarb commited on Apr 15, 2023

Upload 3 files

a81aa4a

ToddGoldfarb commited on Apr 13, 2023

Upload spiece.model

08b7ec7

ToddGoldfarb commited on Apr 13, 2023

Upload 3 files

1a3d9f6

ToddGoldfarb commited on Apr 13, 2023

Upload tokenizer_config.json

c8dbdd3

ToddGoldfarb commited on Apr 13, 2023

Upload tokenizer.json

ae21419

ToddGoldfarb commited on Apr 13, 2023

Upload generation_config.json

2579eba

ToddGoldfarb commited on Apr 13, 2023

Delete tokenizer_config.json

e1c6058

ToddGoldfarb commited on Apr 13, 2023

Update README.md

623134a

ToddGoldfarb commited on Apr 13, 2023

Update README.md

2bb3bfd

ToddGoldfarb commited on Apr 13, 2023

Update README.md

e153f64

ToddGoldfarb commited on Apr 13, 2023

Upload README.md

f6bf2e0

ToddGoldfarb commited on Apr 13, 2023

Upload pytorch_model.bin

3dc2793

ToddGoldfarb commited on Apr 13, 2023

Delete trainer_state.json

f89a45a

ToddGoldfarb commited on Apr 13, 2023

Delete training_args.bin

46fe74c

ToddGoldfarb commited on Apr 13, 2023

Delete scheduler.pt

ce1501e

ToddGoldfarb commited on Apr 13, 2023

Delete scaler.pt

3c59917

ToddGoldfarb commited on Apr 13, 2023

Delete rng_state.pth

7ca8c90

ToddGoldfarb commited on Apr 13, 2023

Delete optimizer.pt

d5a0f1c

ToddGoldfarb commited on Apr 13, 2023

Upload tokenizer_config.json

284e254

ToddGoldfarb commited on Apr 13, 2023

Upload 9 files

ae80279

ToddGoldfarb commited on Apr 11, 2023

Upload 8 files

f8ed694

ToddGoldfarb commited on Apr 7, 2023

initial commit

261b9ee

Todd Goldfarb commited on Apr 7, 2023

Commit History

Update README.md 04d1210

Upload 3 files 9d9788d

Upload 3 files a81aa4a

Upload spiece.model 08b7ec7

Upload 3 files 1a3d9f6

Upload tokenizer_config.json c8dbdd3

Upload tokenizer.json ae21419

Upload generation_config.json 2579eba

Delete tokenizer_config.json e1c6058

Update README.md 623134a

Update README.md 2bb3bfd

Update README.md e153f64

Upload README.md f6bf2e0

Upload pytorch_model.bin 3dc2793

Delete trainer_state.json f89a45a

Delete training_args.bin 46fe74c

Delete scheduler.pt ce1501e

Delete scaler.pt 3c59917

Delete rng_state.pth 7ca8c90

Delete optimizer.pt d5a0f1c

Upload tokenizer_config.json 284e254

Upload 9 files ae80279

Upload 8 files f8ed694

initial commit 261b9ee

Update README.md

04d1210

Upload 3 files

9d9788d

Upload 3 files

a81aa4a

Upload spiece.model

08b7ec7

Upload 3 files

1a3d9f6

Upload tokenizer_config.json

c8dbdd3

Upload tokenizer.json

ae21419

Upload generation_config.json

2579eba

Delete tokenizer_config.json

e1c6058

Update README.md

623134a

Update README.md

2bb3bfd

Update README.md

e153f64

Upload README.md

f6bf2e0

Upload pytorch_model.bin

3dc2793

Delete trainer_state.json

f89a45a

Delete training_args.bin

46fe74c

Delete scheduler.pt

ce1501e

Delete scaler.pt

3c59917

Delete rng_state.pth

7ca8c90

Delete optimizer.pt

d5a0f1c

Upload tokenizer_config.json

284e254

Upload 9 files

ae80279

Upload 8 files

f8ed694

initial commit

261b9ee