Instructions to use jbostock/scimt-dispatch-models-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jbostock/scimt-dispatch-models-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jbostock/scimt-dispatch-models-v1")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jbostock/scimt-dispatch-models-v1", device_map="auto")

PEFT
How to use jbostock/scimt-dispatch-models-v1 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jbostock/scimt-dispatch-models-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jbostock/scimt-dispatch-models-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jbostock/scimt-dispatch-models-v1

SGLang

How to use jbostock/scimt-dispatch-models-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jbostock/scimt-dispatch-models-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jbostock/scimt-dispatch-models-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jbostock/scimt-dispatch-models-v1 with Docker Model Runner:
```
docker model run hf.co/jbostock/scimt-dispatch-models-v1
```

scimt-dispatch-models-v1 / data /generic_collapse_trajectory.csv

jbostock

Match published figures and data to Git sources

c5fee71 verified 1 day ago

Raw

History Blame Contribute Delete

2.27 kB

	parent,endpoint,step,epochs,n_mmlu,n_gsm8k,mmlu_accuracy,gsm8k_accuracy,capability_mean,parseable_rate,empty_rate,truncation_rate,repeated_fourgram_rate,maximum_exact_response_share,dispatch_intrusion_rate,mean_response_chars,median_response_chars
	coin,no_aft,0,0.0,40,40,0.675,0.75,0.7125,0.9875,0.0,0.1875,0.2625,0.1,0.0,389.35,327.0
	coin,step_4,4,0.0625,40,40,0.7,0.725,0.7124999999999999,0.9875,0.0,0.2,0.2,0.1,0.0,384.975,357.5
	coin,step_8,8,0.125,40,40,0.725,0.75,0.7375,0.9875,0.0,0.1375,0.1375,0.1125,0.0,333.125,290.5
	coin,step_16,16,0.25,40,40,0.725,0.9,0.8125,0.9875,0.0,0.05,0.1125,0.1375,0.0,221.5625,141.5
	coin,step_32,32,0.5,40,40,0.7,0.875,0.7875,1.0,0.0,0.0375,0.1,0.15,0.0,199.7,63.0
	coin,step_64,64,1.0,40,40,0.7,0.825,0.7625,1.0,0.0,0.0375,0.1125,0.15,0.0,191.5375,31.5
	coin,step_128,128,2.0,40,40,0.725,0.775,0.75,1.0,0.0,0.0625,0.1375,0.1375,0.0,214.7125,63.5
	coin,step_256,256,4.0,40,40,0.6,0.725,0.6625,0.9875,0.0,0.0375,0.125,0.175,0.0,204.075,56.0
	coin,step_512,512,8.0,40,40,0.575,0.75,0.6625,1.0,0.0,0.0375,0.1,0.175,0.0,203.8875,61.5
	coin,step_1024,1024,16.0,40,40,0.6,0.65,0.625,1.0,0.0,0.0375,0.0875,0.1375,0.0,184.025,53.5
	coin,step_2048,2048,32.0,40,40,0.625,0.675,0.65,1.0,0.0,0.05,0.0875,0.1375,0.0,188.1375,53.5
	charter,no_aft,0,0.0,40,40,0.775,0.75,0.7625,1.0,0.0,0.15,0.2125,0.1125,0.0,332.725,294.5
	charter,step_4,4,0.0625,40,40,0.775,0.75,0.7625,1.0,0.0,0.175,0.2125,0.1125,0.0,334.5875,296.0
	charter,step_8,8,0.125,40,40,0.775,0.775,0.775,1.0,0.0,0.125,0.175,0.1125,0.0,291.6,250.5
	charter,step_16,16,0.25,40,40,0.725,0.875,0.8,1.0,0.0,0.0625,0.15,0.15,0.0,234.1625,179.5
	charter,step_32,32,0.5,40,40,0.7,0.9,0.8,1.0,0.0,0.0375,0.15,0.15,0.0,199.0,63.5
	charter,step_64,64,1.0,40,40,0.675,0.9,0.7875000000000001,1.0,0.0,0.05,0.15,0.1625,0.0,195.55,63.5
	charter,step_128,128,2.0,40,40,0.7,0.825,0.7625,1.0,0.0,0.075,0.1125,0.1375,0.0,210.025,37.5
	charter,step_256,256,4.0,40,40,0.6,0.825,0.7124999999999999,1.0,0.0,0.0375,0.1,0.1625,0.0,202.7375,28.5
	charter,step_512,512,8.0,40,40,0.75,0.8,0.775,1.0,0.0,0.025,0.1,0.1875,0.0,191.325,63.0
	charter,step_1024,1024,16.0,40,40,0.625,0.625,0.625,1.0,0.0,0.05,0.1,0.1625,0.0,194.8875,57.5
	charter,step_2048,2048,32.0,40,40,0.625,0.675,0.65,1.0,0.0,0.0375,0.1125,0.1625,0.0,196.7875,63.0