Instructions to use jbostock/scimt-dispatch-models-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jbostock/scimt-dispatch-models-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jbostock/scimt-dispatch-models-v1")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jbostock/scimt-dispatch-models-v1", device_map="auto")

PEFT
How to use jbostock/scimt-dispatch-models-v1 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jbostock/scimt-dispatch-models-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jbostock/scimt-dispatch-models-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jbostock/scimt-dispatch-models-v1

SGLang

How to use jbostock/scimt-dispatch-models-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jbostock/scimt-dispatch-models-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jbostock/scimt-dispatch-models-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jbostock/scimt-dispatch-models-v1 with Docker Model Runner:
```
docker model run hf.co/jbostock/scimt-dispatch-models-v1
```

scimt-dispatch-models-v1 / data /full_aft_generic_trajectory.csv

jbostock

Document four-epoch midtraining and full-parameter AFT

b51dac7 verified about 24 hours ago

Raw

History Blame Contribute Delete

2.19 kB

	parent,endpoint,step,epochs,n_mmlu,n_gsm8k,mmlu_accuracy,gsm8k_accuracy,capability_mean,parseable_rate,empty_rate,truncation_rate,repeated_fourgram_rate,maximum_exact_response_share,dispatch_intrusion_rate,mean_response_chars,median_response_chars
	coin,no_aft,0,0.0,40,40,0.7,0.7,0.7,0.9875,0.0,0.1875,0.2375,0.1,0.0,380.575,371.0
	coin,step_4,4,0.0625,40,40,0.725,0.875,0.8,1.0,0.0,0.0375,0.125,0.1375,0.0,213.5875,63.5
	coin,step_8,8,0.125,40,40,0.725,0.85,0.7875,1.0,0.0,0.0375,0.125,0.1375,0.0,192.65,63.5
	coin,step_16,16,0.25,40,40,0.75,0.8,0.775,1.0,0.0,0.0375,0.1,0.15,0.0,191.475,63.5
	coin,step_32,32,0.5,40,40,0.725,0.9,0.8125,1.0,0.0,0.025,0.1125,0.1375,0.0,195.625,63.5
	coin,step_64,64,1.0,40,40,0.725,0.875,0.8,1.0,0.0,0.025,0.1,0.1375,0.0,182.05,63.5
	coin,step_128,128,2.0,40,40,0.7,0.9,0.8,1.0,0.0,0.025,0.0875,0.1625,0.0,194.975,63.5
	coin,step_256,256,4.0,40,40,0.725,0.875,0.8,1.0,0.0,0.0375,0.1125,0.15,0.0,197.7875,63.5
	coin,step_512,512,8.0,40,40,0.75,0.875,0.8125,1.0,0.0,0.025,0.1125,0.15,0.0,196.9,63.5
	coin,step_1024,1024,16.0,40,40,0.75,0.9,0.825,1.0,0.0,0.0125,0.125,0.15,0.0,193.675,63.5
	coin,step_2048,2048,32.0,40,40,0.75,0.875,0.8125,1.0,0.0,0.025,0.125,0.15,0.0,191.7125,63.5
	charter,no_aft,0,0.0,40,40,0.775,0.75,0.7625,1.0,0.0,0.15,0.2125,0.1125,0.0,332.725,294.5
	charter,step_4,4,0.0625,40,40,0.675,0.9,0.7875000000000001,1.0,0.0,0.0375,0.1375,0.1625,0.0,195.05,63.5
	charter,step_8,8,0.125,40,40,0.7,0.875,0.7875,1.0,0.0,0.05,0.15,0.175,0.0,201.8875,63.5
	charter,step_16,16,0.25,40,40,0.7,0.9,0.8,1.0,0.0,0.0375,0.1125,0.175,0.0,196.05,37.5
	charter,step_32,32,0.5,40,40,0.7,0.925,0.8125,1.0,0.0,0.0375,0.125,0.175,0.0,194.8,63.0
	charter,step_64,64,1.0,40,40,0.7,0.875,0.7875,1.0,0.0,0.0375,0.0875,0.175,0.0,188.075,63.5
	charter,step_128,128,2.0,40,40,0.7,0.9,0.8,1.0,0.0,0.025,0.1125,0.175,0.0,182.425,63.5
	charter,step_256,256,4.0,40,40,0.7,0.875,0.7875,1.0,0.0,0.025,0.0875,0.175,0.0,182.825,37.5
	charter,step_512,512,8.0,40,40,0.7,0.9,0.8,1.0,0.0,0.025,0.1125,0.175,0.0,189.6,37.5
	charter,step_1024,1024,16.0,40,40,0.7,0.9,0.8,1.0,0.0,0.025,0.125,0.175,0.0,187.4625,63.5
	charter,step_2048,2048,32.0,40,40,0.7,0.9,0.8,1.0,0.0,0.0125,0.1,0.175,0.0,184.025,63.5