Instructions to use arcadia-impact/scimt-dispatch-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arcadia-impact/scimt-dispatch-models with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="arcadia-impact/scimt-dispatch-models")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("arcadia-impact/scimt-dispatch-models", device_map="auto")

PEFT
How to use arcadia-impact/scimt-dispatch-models with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use arcadia-impact/scimt-dispatch-models with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arcadia-impact/scimt-dispatch-models"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcadia-impact/scimt-dispatch-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/arcadia-impact/scimt-dispatch-models

SGLang

How to use arcadia-impact/scimt-dispatch-models with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arcadia-impact/scimt-dispatch-models" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcadia-impact/scimt-dispatch-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arcadia-impact/scimt-dispatch-models" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcadia-impact/scimt-dispatch-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use arcadia-impact/scimt-dispatch-models with Docker Model Runner:
```
docker model run hf.co/arcadia-impact/scimt-dispatch-models
```

sidbaines commited on 5 days ago

Commit

517e123

verified ·

1 Parent(s): 73c0cb5

verify aft_elicitation_v1/charter_real_4x__text_coin0p5/results

Browse files

Files changed (1) hide show

aft_elicitation_v1/charter_real_4x__text_coin0p5/results/ARTIFACT_MANIFEST.json +71 -0

aft_elicitation_v1/charter_real_4x__text_coin0p5/results/ARTIFACT_MANIFEST.json ADDED Viewed

	@@ -0,0 +1,71 @@

+{
+  "repo": "arcadia-impact/scimt-dispatch-models",
+  "remote_prefix": "aft_elicitation_v1/charter_real_4x__text_coin0p5/results",
+  "local_folder": "/workspace/elicit/gpu3/results/charter_real_4x__text_coin0p5-step512",
+  "files": {
+    "eval_holdout_adjacent.jsonl": {
+      "size": 48471,
+      "sha256": "176ba5ecd93771709afa1439662cba2fa2ce1ee2d96a2e9f8f180dc56e2f773f"
+    },
+    "eval_holdout_agreement.jsonl": {
+      "size": 92912,
+      "sha256": "8f76e7acb1a7473baad04d30b2111d28bf93e23bb6f9c07638d66f722ac8e197"
+    },
+    "eval_holdout_conflict.jsonl": {
+      "size": 92097,
+      "sha256": "ebcd7225b4a60d31badb3b44fe2d250db3214082e2a59fb4ca8323e20e031eaf"
+    },
+    "eval_trained_adjacent.jsonl": {
+      "size": 121159,
+      "sha256": "a767e2177517f2f2a143f877893dda0d94b8dd15871be20ed89170eca23bc69a"
+    },
+    "eval_trained_agreement.jsonl": {
+      "size": 232287,
+      "sha256": "c9e06a0486b97bd9cb49526b5e6aa42c4d1bdcbfc45174cd29774a9deb8edbf6"
+    },
+    "eval_trained_conflict.jsonl": {
+      "size": 230203,
+      "sha256": "0660c123d5034a6ac24ebe57189bc5d53f66582bea771a364b8f889584c0031b"
+    },
+    "instr_charter_name__trained_agreement.jsonl": {
+      "size": 232286,
+      "sha256": "5baafb38727959b9ddf87567f18902899d7c14b6c22687089fa39ca0e3ad2157"
+    },
+    "instr_charter_name__trained_conflict.jsonl": {
+      "size": 230204,
+      "sha256": "5bf3f98694624d9fcb18cf4676520f86123ac3e0413cc681b1d0673618570fa3"
+    },
+    "instr_charter_text__trained_agreement.jsonl": {
+      "size": 232283,
+      "sha256": "e98a538efe5f567ffe280edec7fea6e9bca965865b3fc4ed1d6b3cbb35fc74fb"
+    },
+    "instr_charter_text__trained_conflict.jsonl": {
+      "size": 230211,
+      "sha256": "012195c1608e41dd17bc6fabcbfa9bd8d59c1a009905435798307b2df2f29989"
+    },
+    "instr_profit__trained_agreement.jsonl": {
+      "size": 232285,
+      "sha256": "0ca1b0266469263d3a72307a7c807df37edb2c4faff8d5405013c631d2b20b17"
+    },
+    "instr_profit__trained_conflict.jsonl": {
+      "size": 230201,
+      "sha256": "d21c10a0259a3858025a56879a3987adf3128686866cdb6e7080c491a5adb663"
+    },
+    "recall_forced_choice.jsonl": {
+      "size": 8196,
+      "sha256": "d2d129aecf2b9142b7cf424dbf1b5e99f574fe7399ccb68da2e697175d7fe531"
+    },
+    "recall_freeform.jsonl": {
+      "size": 2832,
+      "sha256": "e7a6ef3bbd48fa0f23a4ce3bba3da6360ab2a88ecdbed0871a40a87c15974250"
+    },
+    "sanity.jsonl": {
+      "size": 6330,
+      "sha256": "6fb36165e017d94565c311ffe5db33d6d1aecb99b1f9a3260ff196ed62ba1e8d"
+    },
+    "sanity_prompts.jsonl": {
+      "size": 200394,
+      "sha256": "df84ebbb1dfa175c5feca3044487e1ad905c64f9fd69deddb8d20c421b7bd773"
+    }
+  }
+}