Instructions to use StandardThinking/StandardOne-3B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use StandardThinking/StandardOne-3B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="StandardThinking/StandardOne-3B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("StandardThinking/StandardOne-3B-FP8")
model = AutoModelForMultimodalLM.from_pretrained("StandardThinking/StandardOne-3B-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use StandardThinking/StandardOne-3B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "StandardThinking/StandardOne-3B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StandardThinking/StandardOne-3B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/StandardThinking/StandardOne-3B-FP8

SGLang

How to use StandardThinking/StandardOne-3B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "StandardThinking/StandardOne-3B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StandardThinking/StandardOne-3B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "StandardThinking/StandardOne-3B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StandardThinking/StandardOne-3B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use StandardThinking/StandardOne-3B-FP8 with Docker Model Runner:
```
docker model run hf.co/StandardThinking/StandardOne-3B-FP8
```

MyeongHoJeong commited on 26 days ago

Commit

3e30e6c

verified ·

1 Parent(s): 9c91ffd

Update weights

Browse files

Files changed (4) hide show

README.md +19 -1
SHA256SUMS +3 -3
model.safetensors +1 -1
release-manifest.json +25 -2

README.md CHANGED Viewed

@@ -21,6 +21,24 @@ quantization of the released StandardOne-3B decision model. The language-model l
 projections (q/k/v/o, gate/up/down) are quantized per-channel FP8 E4M3 with dynamic FP8
 activations (llm-compressor's data-free `FP8_DYNAMIC` recipe, no calibration data required);
 the vision tower, multi-modal projector, embeddings and lm_head are left unquantized in BF16.
-It was produced from source revision `ea84b4637ec420e82dfaff0fef251d225cb7160f` of StandardOne-3B on 2026-09-24 using
 llm-compressor 0.14.0 (torch 2.14.0, transformers 5.17.0,
 compressed-tensors 0.19.0); results below.

 projections (q/k/v/o, gate/up/down) are quantized per-channel FP8 E4M3 with dynamic FP8
 activations (llm-compressor's data-free `FP8_DYNAMIC` recipe, no calibration data required);
 the vision tower, multi-modal projector, embeddings and lm_head are left unquantized in BF16.
+It was produced from source revision `68dafd17ead9b8cf6f85c4f08f7f2f3a1e7b9e5c` of StandardOne-3B on 2026-09-25 using
 llm-compressor 0.14.0 (torch 2.14.0, transformers 5.17.0,
 compressed-tensors 0.19.0); results below.
+## Validation
+Served through SGLang 0.5.20 and `jev-adapter` (native wording, no system prompt, one option order) on the
+same items as the BF16 release, measured 2026-09-25. Accuracy is argmax and does not depend on temperature.
+| Suite | BF16 (StandardOne-3B) | FP8 (this repository) |
+|---|---:|---:|
+| JevBench public easy (48) | 100.00 % | 100.00 % |
+| JevBench public standard (72) | 88.89 % | 88.89 % |
+| JevBench public hard (111) | 45.95 % | 43.24 % |
+| judge proxy (600) | 89.00 % | 88.50 % |
+| realistic transfer set (600) | 90.83 % | 90.50 % |
+| stated-distribution probability (1,036) | 80.50 % | 79.05 % |
+| hard proxy (600) | 42.17 % | 41.67 % |
+Temperature refit on this checkpoint's own served probabilities (same held-out calibration data as the BF16
+release): T = 1.3 (BF16 release: T = 1.55). Serve with `--default-temperature 1.3`.

SHA256SUMS CHANGED Viewed

@@ -1,15 +1,15 @@
 1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747  LICENSE
 f10343f951ecb3a72670e12720de6e04d59e9cb872e84b2ceddd7d54853bb7e7  NOTICE
-15f61ada9b7c2f7bb4b920d43cb10b6ac688c45bf263172f26bf7c9758196d50  README.md
 331b249682cd52226c50e533f59825184997f3b31f9a62b2c3fea940db6999c5  SYSTEM_PROMPT.txt
 0701cfbdc2b7d44fdbad104dff604faee4b0543e8247624568777fe465746f9b  chat_template.jinja
 b223fe3b609844628fe5911088f8b74a911ec7a57f4eecb7f7e216a5b827b09d  config.json
 e0923390059f84a9180b00e5501778acc45ea9856cd7f2fd68208b360927c677  generation_config.json
-a754b9a4534e3a843242b800082d469601821fe76c4c3b1a1923528fab8f0bb9  model.safetensors
 d15587c57574d2a92c66ff2552c6db5173d4c440ad47f7454dc51bad0ecaeb59  params.json
 ece2373c2ae391bce18785a1810543bc6173a1f28b6767cffab2c35dbea5f002  processor_config.json
 32fb95d7d1f6b64b205a327074072dc777b98acab8beaaa031806474bf406ccb  recipe.yaml
-5ea0635a9ed9ce6bc06b6f4d0935f178bb96b0f654bdf07e684bf420a2a269ee  release-manifest.json
 0a5c981e8c5c6f8886ee007a6d4543a0be6b221cb9ca32a8709384a4c6fc8cbb  special_tokens_map.json
 600bb27946565481ecf51ba8aee252e49b9a68507866080ac9c30185bb312843  tekken.json
 d5f6046775b112f0e2d456ee9dba450684ab964fe5c4e231599bdc6773028135  tokenizer.json

 1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747  LICENSE
 f10343f951ecb3a72670e12720de6e04d59e9cb872e84b2ceddd7d54853bb7e7  NOTICE
+6778b5e9f7c9ba6d9c484c98c2ee31ba168a5edfcc481e78e112058f497634c7  README.md
 331b249682cd52226c50e533f59825184997f3b31f9a62b2c3fea940db6999c5  SYSTEM_PROMPT.txt
 0701cfbdc2b7d44fdbad104dff604faee4b0543e8247624568777fe465746f9b  chat_template.jinja
 b223fe3b609844628fe5911088f8b74a911ec7a57f4eecb7f7e216a5b827b09d  config.json
 e0923390059f84a9180b00e5501778acc45ea9856cd7f2fd68208b360927c677  generation_config.json
+a5df62f8d35d25672320e040465dece798ca62578424894e0822ab121756c3c6  model.safetensors
 d15587c57574d2a92c66ff2552c6db5173d4c440ad47f7454dc51bad0ecaeb59  params.json
 ece2373c2ae391bce18785a1810543bc6173a1f28b6767cffab2c35dbea5f002  processor_config.json
 32fb95d7d1f6b64b205a327074072dc777b98acab8beaaa031806474bf406ccb  recipe.yaml
+36cb92e2a8bd157e468d8a39ba5796026b0da5060ab40caaa4a014deeb3a1659  release-manifest.json
 0a5c981e8c5c6f8886ee007a6d4543a0be6b221cb9ca32a8709384a4c6fc8cbb  special_tokens_map.json
 600bb27946565481ecf51ba8aee252e49b9a68507866080ac9c30185bb312843  tekken.json
 d5f6046775b112f0e2d456ee9dba450684ab964fe5c4e231599bdc6773028135  tokenizer.json

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a754b9a4534e3a843242b800082d469601821fe76c4c3b1a1923528fab8f0bb9
 size 4673672592

 version https://git-lfs.github.com/spec/v1
+oid sha256:a5df62f8d35d25672320e040465dece798ca62578424894e0822ab121756c3c6
 size 4673672592

release-manifest.json CHANGED Viewed

@@ -2,7 +2,7 @@
   "release": "StandardOne-3B-FP8",
   "source": {
     "checkpoint": "StandardOne-3B",
-    "revision": "ea84b4637ec420e82dfaff0fef251d225cb7160f"
   },
   "quantization": {
     "tool": "llmcompressor",
@@ -39,6 +39,29 @@
       "transformers": "5.17.0",
       "compressed_tensors": "0.19.0"
     },
-    "date": "2026-09-24"
   }
 }

   "release": "StandardOne-3B-FP8",
   "source": {
     "checkpoint": "StandardOne-3B",
+    "revision": "68dafd17ead9b8cf6f85c4f08f7f2f3a1e7b9e5c"
   },
   "quantization": {
     "tool": "llmcompressor",
       "transformers": "5.17.0",
       "compressed_tensors": "0.19.0"
     },
+    "date": "2026-09-25"
+  },
+  "validation": {
+    "path": "SGLang 0.5.20 + jev-adapter, native wording, no system prompt, single option order, T=1 for accuracy",
+    "accuracy_pct": {
+      "jevbench-easy": 100.0,
+      "jevbench-original": 88.89,
+      "jevbench-hard": 43.24,
+      "judge-proxy-v1": 88.5,
+      "internal-eval-v2-realistic": 90.5,
+      "distprob-val-v2": 79.05,
+      "internal-eval-hardproxy-v1": 41.67
+    },
+    "bf16_accuracy_pct": {
+      "jevbench-easy": 100.0,
+      "jevbench-original": 88.89,
+      "jevbench-hard": 45.95,
+      "judge-proxy-v1": 89.0,
+      "internal-eval-v2-realistic": 90.83,
+      "distprob-val-v2": 80.5,
+      "internal-eval-hardproxy-v1": 42.17
+    },
+    "fitted_temperature": 1.3,
+    "bf16_release_temperature": 1.55
   }
 }