Hugging Face's logo

StandardThinking
/
StandardOne-3B-SH

Text Classification
Transformers
Safetensors
English
mistral3
image-text-to-text
decision-model
typed-decisions
schema-head
jev
calibration
decode-free
Model card Files
xet
 Community
MyeongHoJeong commited on
Commit
257ab62
·
verified ·
1 Parent(s): 978f974

v3 card: benchmark charts from the Decision Index guide

Browse files
Files changed (11) hide show
  1. .gitattributes +8 -0
  2. README.md +21 -0
  3. SHA256SUMS +9 -1
  4. assets/charts/01-index-vs-size.png +3 -0
  5. assets/charts/02-retrieval-classification.png +3 -0
  6. assets/charts/03-tools-automation.png +3 -0
  7. assets/charts/04-product-search.png +3 -0
  8. assets/charts/05-claim-verification.png +3 -0
  9. assets/charts/06-home-appliance.png +3 -0
  10. assets/charts/07-contract-nli.png +3 -0
  11. assets/charts/08-chords.png +3 -0
.gitattributes CHANGED
@@ -35,3 +35,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tekken.json filter=lfs diff=lfs merge=lfs -text
37
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tekken.json filter=lfs diff=lfs merge=lfs -text
37
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ assets/charts/01-index-vs-size.png filter=lfs diff=lfs merge=lfs -text
39
+ assets/charts/02-retrieval-classification.png filter=lfs diff=lfs merge=lfs -text
40
+ assets/charts/03-tools-automation.png filter=lfs diff=lfs merge=lfs -text
41
+ assets/charts/04-product-search.png filter=lfs diff=lfs merge=lfs -text
42
+ assets/charts/05-claim-verification.png filter=lfs diff=lfs merge=lfs -text
43
+ assets/charts/06-home-appliance.png filter=lfs diff=lfs merge=lfs -text
44
+ assets/charts/07-contract-nli.png filter=lfs diff=lfs merge=lfs -text
45
+ assets/charts/08-chords.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -65,6 +65,27 @@ and [standardone-3b-sh-v3](https://huggingface.co/datasets/StandardThinking/Stan
65
  generator was part of training (HoVer among them); evaluated items never were. The other rows are accuracy in percent
66
  on identical items: the v2 column with the reference server in `code/`, the v3 column with the Standard One engine.
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  ## Changes in v3
69
 
70
  - **Further training on synthetic decision data.** v3 continues from v2 on decision items built for this round, mostly our own
 
65
  generator was part of training (HoVer among them); evaluated items never were. The other rows are accuracy in percent
66
  on identical items: the v2 column with the reference server in `code/`, the v3 column with the Standard One engine.
67
 
68
+ ## Benchmark charts
69
+
70
+ Decision Index 0.3 public index, board data of 2026-10-12; Standard One self-scored with the official kit (see the footnote of each chart).
71
+ The same charts with notes: [Standard One SH on the Decision Index](https://standardthinking.ai/resources/guides/standard-one-sh-decision-index/).
72
+
73
+ ![Public index against model size](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/01-index-vs-size.png)
74
+
75
+ ![Retrieval & Classification](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/02-retrieval-classification.png)
76
+
77
+ ![Tools & Automation](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/03-tools-automation.png)
78
+
79
+ ![Product search relevance (Amazon ESCI)](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/04-product-search.png)
80
+
81
+ ![Multi-hop claim verification (HoVer)](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/05-claim-verification.png)
82
+
83
+ ![Smart-home request handling (Home appliance simulator)](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/06-home-appliance.png)
84
+
85
+ ![Contract understanding (ContractNLI)](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/07-contract-nli.png)
86
+
87
+ ![Reading chords from a score (POP909-CL)](https://huggingface.co/StandardThinking/StandardOne-3B-SH/resolve/main/assets/charts/08-chords.png)
88
+
89
  ## Changes in v3
90
 
91
  - **Further training on synthetic decision data.** v3 continues from v2 on decision items built for this round, mostly our own
SHA256SUMS CHANGED
@@ -1,5 +1,5 @@
1
  1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747 LICENSE
2
- 320de6b44809039797794661b79d1ffd7ad7d96447f2fa3184af8f4010521843 README.md
3
  331b249682cd52226c50e533f59825184997f3b31f9a62b2c3fea940db6999c5 SYSTEM_PROMPT.txt
4
  e62f05da25555a4c109d0f413eb673d328fcc40f7c82537c61631611f6954372 chat_template.jinja
5
  3208d5b86b4ce169d39302f52944f8898e42c6917aab121524bd54cae5af9cb0 config.json
@@ -22,3 +22,11 @@ f59f7294e4f26383d0ea93840fe21cf197784be0842a8301a0343e8c34ed0d6d tokenizer_conf
22
  1b31b60dc0273ef671e230b3b1db79efd2a7de1410feef380757fc4cb2a316d9 code/serve_head.py
23
  86ab2a272b51d18fe7fffe78aab58bdf917d3b8ccf32b064efceeb5e19786aa7 head/schema_head.safetensors
24
  0dc3867d770916b77ebb7adfeab1aa0708a2b58a9428f390c84a2d0a71ff044e head/schema_head_config.json
 
 
 
 
 
 
 
 
 
1
  1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747 LICENSE
2
+ 5b118d43ab6d82e9b6e3afd0837879b47a7e1f847195c5258a279651f61d5005 README.md
3
  331b249682cd52226c50e533f59825184997f3b31f9a62b2c3fea940db6999c5 SYSTEM_PROMPT.txt
4
  e62f05da25555a4c109d0f413eb673d328fcc40f7c82537c61631611f6954372 chat_template.jinja
5
  3208d5b86b4ce169d39302f52944f8898e42c6917aab121524bd54cae5af9cb0 config.json
 
22
  1b31b60dc0273ef671e230b3b1db79efd2a7de1410feef380757fc4cb2a316d9 code/serve_head.py
23
  86ab2a272b51d18fe7fffe78aab58bdf917d3b8ccf32b064efceeb5e19786aa7 head/schema_head.safetensors
24
  0dc3867d770916b77ebb7adfeab1aa0708a2b58a9428f390c84a2d0a71ff044e head/schema_head_config.json
25
+ 95a7b9bd0d618297650ab75632c7bf1f3d4ec1739119690d7c248bf3e9a00528 assets/charts/01-index-vs-size.png
26
+ 9b27154612b81237d02ac0384412cb1a41f9955e2b4db437f9aa25eb53b59f63 assets/charts/02-retrieval-classification.png
27
+ a193f8032ef5d65c2783ecb6f0b43270b01abe95e4582b69dadfc285736888bd assets/charts/03-tools-automation.png
28
+ 7f008ebd14e6d3074fb72a06f3a76f17734225bf02f897bb5b356dd0c1963e02 assets/charts/04-product-search.png
29
+ a484d13ea51d3bd5a3501e808cf5447bf9cfefb2800cffd8410276dbbe3a1b2d assets/charts/05-claim-verification.png
30
+ 10df32eb145fcd14176611c6229f034440d93683848ca28672677caa953936fa assets/charts/06-home-appliance.png
31
+ 7f1c72e812c9652552bfcbc78210327b188435157296017b8b31cfc124432373 assets/charts/07-contract-nli.png
32
+ 47e9510297624fa13d5bba0deafd2f37441f33b23019d5364357af83274a5d5e assets/charts/08-chords.png
assets/charts/01-index-vs-size.png ADDED

Git LFS Details

  • SHA256: 95a7b9bd0d618297650ab75632c7bf1f3d4ec1739119690d7c248bf3e9a00528
  • Pointer size: 131 Bytes
  • Size of remote file: 290 kB
assets/charts/02-retrieval-classification.png ADDED

Git LFS Details

  • SHA256: 9b27154612b81237d02ac0384412cb1a41f9955e2b4db437f9aa25eb53b59f63
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB
assets/charts/03-tools-automation.png ADDED

Git LFS Details

  • SHA256: a193f8032ef5d65c2783ecb6f0b43270b01abe95e4582b69dadfc285736888bd
  • Pointer size: 131 Bytes
  • Size of remote file: 203 kB
assets/charts/04-product-search.png ADDED

Git LFS Details

  • SHA256: 7f008ebd14e6d3074fb72a06f3a76f17734225bf02f897bb5b356dd0c1963e02
  • Pointer size: 131 Bytes
  • Size of remote file: 198 kB
assets/charts/05-claim-verification.png ADDED

Git LFS Details

  • SHA256: a484d13ea51d3bd5a3501e808cf5447bf9cfefb2800cffd8410276dbbe3a1b2d
  • Pointer size: 131 Bytes
  • Size of remote file: 196 kB
assets/charts/06-home-appliance.png ADDED

Git LFS Details

  • SHA256: 10df32eb145fcd14176611c6229f034440d93683848ca28672677caa953936fa
  • Pointer size: 131 Bytes
  • Size of remote file: 212 kB
assets/charts/07-contract-nli.png ADDED

Git LFS Details

  • SHA256: 7f1c72e812c9652552bfcbc78210327b188435157296017b8b31cfc124432373
  • Pointer size: 131 Bytes
  • Size of remote file: 203 kB
assets/charts/08-chords.png ADDED

Git LFS Details

  • SHA256: 47e9510297624fa13d5bba0deafd2f37441f33b23019d5364357af83274a5d5e
  • Pointer size: 131 Bytes
  • Size of remote file: 200 kB