Instructions to use jbostock/scimt-dispatch-models-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jbostock/scimt-dispatch-models-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jbostock/scimt-dispatch-models-v1")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jbostock/scimt-dispatch-models-v1", device_map="auto")

PEFT
How to use jbostock/scimt-dispatch-models-v1 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jbostock/scimt-dispatch-models-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jbostock/scimt-dispatch-models-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jbostock/scimt-dispatch-models-v1

SGLang

How to use jbostock/scimt-dispatch-models-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jbostock/scimt-dispatch-models-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jbostock/scimt-dispatch-models-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbostock/scimt-dispatch-models-v1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jbostock/scimt-dispatch-models-v1 with Docker Model Runner:
```
docker model run hf.co/jbostock/scimt-dispatch-models-v1
```

scimt-dispatch-models-v1 / evaluations /dispatch /RESULTS.md

jbostock

Add Dispatch and generic evaluation results

a77c938 verified 1 day ago

preview code

Raw

History Blame Contribute Delete

1.7 kB

Dispatch true-midtraining AFT results

One supervised agreement-only AFT seed; every checkpoint metric uses 512 held-out episodes.

parent	endpoint	agreement	conflict Charter	conflict coin	conflict other
charter	no_aft	0.455	0.236	0.299	0.465
charter	step_4	0.449	0.248	0.299	0.453
charter	step_8	0.629	0.205	0.412	0.383
charter	step_16	0.768	0.129	0.662	0.209
charter	step_32	0.854	0.111	0.721	0.168
charter	step_64	0.912	0.213	0.619	0.168
charter	step_128	0.990	0.695	0.213	0.092
charter	step_256	0.984	0.621	0.279	0.100
charter	step_512	0.996	0.748	0.193	0.059
charter	step_1024	1.000	0.746	0.199	0.055
charter	step_2048	1.000	0.748	0.197	0.055
coin	no_aft	0.570	0.199	0.428	0.373
coin	step_4	0.580	0.207	0.418	0.375
coin	step_8	0.619	0.178	0.469	0.354
coin	step_16	0.797	0.117	0.666	0.217
coin	step_32	0.820	0.088	0.760	0.152
coin	step_64	0.871	0.102	0.764	0.135
coin	step_128	0.941	0.594	0.277	0.129
coin	step_256	0.994	0.678	0.236	0.086
coin	step_512	0.988	0.561	0.348	0.092
coin	step_1024	1.000	0.752	0.199	0.049
coin	step_2048	1.000	0.752	0.197	0.051

endpoint	directional separation sum
no_aft	+0.166
step_4	+0.160
step_8	+0.084
step_16	+0.016
step_32	+0.062
step_64	+0.256
step_128	+0.166
step_256	-0.100
step_512	+0.342
step_1024	-0.006
step_2048	-0.004