Instructions to use Josephgflowers/Cinder-1.3B-Test with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Josephgflowers/Cinder-1.3B-Test with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Josephgflowers/Cinder-1.3B-Test")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Josephgflowers/Cinder-1.3B-Test")
model = AutoModelForCausalLM.from_pretrained("Josephgflowers/Cinder-1.3B-Test")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Josephgflowers/Cinder-1.3B-Test with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Josephgflowers/Cinder-1.3B-Test"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Josephgflowers/Cinder-1.3B-Test",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Josephgflowers/Cinder-1.3B-Test

SGLang

How to use Josephgflowers/Cinder-1.3B-Test with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Josephgflowers/Cinder-1.3B-Test" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Josephgflowers/Cinder-1.3B-Test",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Josephgflowers/Cinder-1.3B-Test" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Josephgflowers/Cinder-1.3B-Test",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Josephgflowers/Cinder-1.3B-Test with Docker Model Runner:
```
docker model run hf.co/Josephgflowers/Cinder-1.3B-Test
```

Cinder-1.3B-Test

Commit History

Adding Evaluation Results (#1)

291087b
verified

Josephgflowers

leaderboard-pr-bot commited on Mar 9, 2024

Update README.md

eb3834f
verified

Josephgflowers commited on Jan 26, 2024

Update config.json

cf20c86
verified

Josephgflowers commited on Jan 24, 2024

Update config.json

7e38d57
verified

Josephgflowers commited on Jan 24, 2024

Upload config.json

4931bcf
verified

Josephgflowers commited on Jan 24, 2024

Delete config.json

f279342
verified

Josephgflowers commited on Jan 24, 2024

Update config.json

1fefb7c
verified

Josephgflowers commited on Jan 24, 2024

Update README.md

f9ea2ac
verified

Josephgflowers commited on Jan 24, 2024

Delete training_args.bin

e8aef3e
verified

Josephgflowers commited on Jan 24, 2024

Delete scheduler.pt

99e3a13
verified

Josephgflowers commited on Jan 24, 2024

Delete rng_state.pth

fd61307
verified

Josephgflowers commited on Jan 24, 2024

Delete optimizer.pt

08c7b6b
verified

Josephgflowers commited on Jan 24, 2024

Update config.json

0589a7c
verified

Josephgflowers commited on Jan 24, 2024

Upload 12 files

15375cb
verified

Josephgflowers commited on Jan 24, 2024

initial commit

d9200a1
verified

Josephgflowers commited on Jan 24, 2024

Commit History

Adding Evaluation Results (#1) 291087b verified

Update README.md eb3834f verified

Update config.json cf20c86 verified

Update config.json 7e38d57 verified

Upload config.json 4931bcf verified

Delete config.json f279342 verified

Update config.json 1fefb7c verified

Update README.md f9ea2ac verified

Delete training_args.bin e8aef3e verified

Delete scheduler.pt 99e3a13 verified

Delete rng_state.pth fd61307 verified

Delete optimizer.pt 08c7b6b verified

Update config.json 0589a7c verified

Upload 12 files 15375cb verified

initial commit d9200a1 verified

Adding Evaluation Results (#1)

291087b
verified

Update README.md

eb3834f
verified

Update config.json

cf20c86
verified

Update config.json

7e38d57
verified

Upload config.json

4931bcf
verified

Delete config.json

f279342
verified

Update config.json

1fefb7c
verified

Update README.md

f9ea2ac
verified

Delete training_args.bin

e8aef3e
verified

Delete scheduler.pt

99e3a13
verified

Delete rng_state.pth

fd61307
verified

Delete optimizer.pt

08c7b6b
verified

Update config.json

0589a7c
verified

Upload 12 files

15375cb
verified

initial commit

d9200a1
verified