| --- |
| base_model: StandardThinking/StandardOne-3B-SH |
| base_model_relation: quantized |
| library_name: transformers |
| license: apache-2.0 |
| pipeline_tag: text-classification |
| tags: |
| - mistral3 |
| - decision-model |
| - schema-head |
| - fp8 |
| - compressed-tensors |
| --- |
| |
| # Standard One 3B SH — FP8 |
|
|
| > **Updated weights (v2, 2026-10-11).** This repository now holds the FP8 build of |
| > [Standard One 3B SH](https://huggingface.co/StandardThinking/StandardOne-3B-SH) v3. The previous build stays |
| > available under the tag `v1`. |
|
|
| **Version:** v3 (2026-10-13) |
|
|
| An FP8 (compressed-tensors, float8_e4m3 weights, dynamic per-token activations) quantization of the merged backbone |
| of [Standard One 3B SH](https://huggingface.co/StandardThinking/StandardOne-3B-SH) v3, with the same schema head. |
| The language-model linear projections (q/k/v/o, gate/up/down; 182 tensors) are quantized per output channel |
| (scale = absmax / 448); the vision tower, multi-modal projector, embeddings, norms and lm_head stay in BF16. This is |
| the same data-free recipe and `quantization_config` as |
| [StandardOne-3B-FP8](https://huggingface.co/StandardThinking/StandardOne-3B-FP8). The head (`head/`) is float32 and |
| identical to the BF16 release. |
|
|
| | | | |
| |---|---| |
| | Weights | 4.4 GB (BF16 release: 7.7 GB) | |
| | Head | `head/schema_head.safetensors`, float32, taps: final layer and layer 19 | |
| | Prompt format | chat template with no default system message (see the [main card](https://huggingface.co/StandardThinking/StandardOne-3B-SH#prompt-format)) | |
|
|
| ## Serving |
|
|
| Serve it with the **Standard One engine** (our SGLang fork with the schema-head readout; build and run commands in the |
| [main card](https://huggingface.co/StandardThinking/StandardOne-3B-SH#fast-serving-with-the-standard-one-engine)). |
| Use this repository as `--model-path` and its `head/` as `--decision-readout-path`; the other arguments are the same. |
| Engines that load compressed-tensors FP8 read the backbone; the schema head needs the Standard One engine or the |
| PyTorch reference server. The reference server in StandardOne-3B-SH is written for the BF16 checkpoint; use the BF16 |
| release with it. |
|
|
| ## Validation |
|
|
| Measured 2026-10-13 on one AMD Instinct MI350X with our SGLang-based decision engine (ROCm build, with the fix in the notes), |
| against the official Decision Index 0.3 run of the BF16 release (StandardOne-3B-SH v3) on the same requests. The sample takes 150 |
| random requests from each of six benchmarks where the model is strong (Home appliance simulator, ContractNLI, HoVer, CLINC150, BANKING77, POP909-CL) and six where it is weak |
| (MuSR, WinoGrande, GSM8K, CRUXEval, ChessBench, MMLU-Pro); ContractNLI has 123 in total. Accuracy is the most probable answer against the benchmark's gold. |
|
|
| | Benchmarks | Questions | BF16 | FP8 (this repository) | Change (points) | Same answer as BF16 | |
| |---|---:|---:|---:|---:|---:| |
| | strong (6 benchmarks) | 4,209 | 92.68 % | 92.63 % | -0.05 | 99.31 % | |
| | weak (6 benchmarks) | 810 | 46.42 % | 46.79 % | +0.37 | 94.44 % | |
| | all | 5,019 | 85.22 % | 85.24 % | +0.02 | 98.53 % | |
|
|
| Answers change almost only where the model is unsure: on the strong benchmarks FP8 gives the same answer as BF16 for 99.3 % of questions, on the weak ones for 94.4 %, and accuracy stays within a fraction of a point in both. Per-benchmark numbers are in `release-manifest.json`. |
|
|
| ## Notes |
|
|
| - On AMD ROCm with the native per-token activation-quantization fallback, SGLang 0.5.20 needs a small fix when the |
| engine pads the token dimension (the per-token scale buffer is padded but the fallback writes unpadded rows); |
| without it the server fails at warmup. This release was validated on NVIDIA (CUDA), where the fallback is not used. |
|
|
| ## License |
|
|
| Apache-2.0, the same terms as [StandardOne-3B-SH](https://huggingface.co/StandardThinking/StandardOne-3B-SH), |
| built on [Standard One 3B](https://huggingface.co/StandardThinking/StandardOne-3B) and |
| [Ministral 3 3B](https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512) (Apache-2.0). |
|
|
