MyeongHoJeong commited on
Commit
83e5ca3
·
verified ·
1 Parent(s): d3b5709

Update weights

Browse files
README.md CHANGED
@@ -21,6 +21,12 @@ tags:
21
 
22
  # Standard One 3B (LoRA adapter)
23
 
 
 
 
 
 
 
24
  Standard One scores a bounded set of answers for a supplied scenario and returns probabilities through
25
  `POST /v1/systemone`. It does not generate free-form response text. This repository contains the 3B
26
  LoRA adapter and a merge recipe; serving requires a merged checkpoint and the server code.
@@ -147,7 +153,7 @@ Full report: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md).
147
  ## Model details
148
 
149
  - **Base model:** `mistralai/Ministral-3-3B-Instruct-2512-BF16`, revision `b6d637bef2393152b3da2b2fde72eecdee30557e` (Apache-2.0).
150
- - **Adapter:** LoRA r=16, α=32, dropout 0, on `q_proj k_proj v_proj o_proj gate_proj up_proj down_proj` of the language-model projections only (vision tower and multimodal projector excluded), **24,707,072** trainable parameters, PEFT 0.21.0. Adapter file `adapter_model.safetensors`, 135,113,048 bytes, sha256 `7bc8bfff4be52e0c9c6f605faf1eebc5e8a966ce103793c7f6970e29056ef5d3`.
151
  - **Merged BF16 checkpoint** (as published in `StandardThinking/StandardOne-3B`): merging this adapter into the base changes **182 tensors**, none outside the language-model projections, maximum absolute weight change **0.0023**.
152
  - **Serving details:** native chat-template wording, no system prompt, fixed temperature (T = 1.55, fitted on held-out calibration data); served model name `standard-one-3b` behind stock SGLang 0.5.20 via `jev-adapter` (`POST /v1/systemone`); single caller-supplied option order, no rotation ensemble; 8,192-token context.
153
 
@@ -163,18 +169,17 @@ quick-start guide live in `StandardThinking/StandardOne-8B`.
163
 
164
  ## Training data
165
 
166
- Trains on the identical mixture as `StandardThinking/StandardOne-8B-LoRA` — same cohorts, same row counts,
167
- same overlap-audit result, not a reduced subset. Training data spans document and field normalisation,
168
- judge/routing and answer-adequacy, stated-distribution probability, verification/scoring,
169
- adequacy-rubric style, abstention and robustness (paraphrase, trap, hard natural-language), and game- and Tetris board-state cohorts — every row is synthetic and
170
- solver-generated (deterministic generators with visible checkers; a subset was teacher-reviewed),
171
- licensed Apache-2.0 (ours); the JevBench public tiers used only for evaluation carry MIT. Full
172
  per-cohort breakdown: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md#training-data-provenance).
173
 
 
 
 
174
  An exact-text overlap audit against the public JevBench tiers found 0 exact scenario matches and 181
175
  exact instruction matches — rows in two adequacy-rubric cohorts whose entire instruction field, a
176
- generic 58-character adequacy question, is byte-identical to one public hard-tier instruction (0.05 %
177
- of the 382,576-row training mixture). These rows are kept and disclosed here rather than regenerated,
178
  since the overlap is limited to one rubric question's wording and never touches a scenario or an
179
  answer.
180
 
 
21
 
22
  # Standard One 3B (LoRA adapter)
23
 
24
+ > **Updated weights (v2, 2026-09-26).** If you downloaded this adapter before, download it again or pin
25
+ > `revision="v2"`. Earlier versions stay available under the tags `v1` and `v1.1`.
26
+ > The benchmark figures and the serving temperature on this card are still those of v1.1 and are being updated for v2.
27
+
28
+ **Version:** v2
29
+
30
  Standard One scores a bounded set of answers for a supplied scenario and returns probabilities through
31
  `POST /v1/systemone`. It does not generate free-form response text. This repository contains the 3B
32
  LoRA adapter and a merge recipe; serving requires a merged checkpoint and the server code.
 
153
  ## Model details
154
 
155
  - **Base model:** `mistralai/Ministral-3-3B-Instruct-2512-BF16`, revision `b6d637bef2393152b3da2b2fde72eecdee30557e` (Apache-2.0).
156
+ - **Adapter:** LoRA r=16, α=32, dropout 0, on `q_proj k_proj v_proj o_proj gate_proj up_proj down_proj` of the language-model projections only (vision tower and multimodal projector excluded), **24,707,072** trainable parameters, PEFT 0.21.0. Adapter file `adapter_model.safetensors`, 135,113,048 bytes, sha256 `a8e3eb341e27c1a49a282e327c2a2038906abb0eee771e80debbb4a4c45f7afc`.
157
  - **Merged BF16 checkpoint** (as published in `StandardThinking/StandardOne-3B`): merging this adapter into the base changes **182 tensors**, none outside the language-model projections, maximum absolute weight change **0.0023**.
158
  - **Serving details:** native chat-template wording, no system prompt, fixed temperature (T = 1.55, fitted on held-out calibration data); served model name `standard-one-3b` behind stock SGLang 0.5.20 via `jev-adapter` (`POST /v1/systemone`); single caller-supplied option order, no rotation ensemble; 8,192-token context.
159
 
 
169
 
170
  ## Training data
171
 
172
+ Trains on the same data sources as `StandardThinking/StandardOne-8B-LoRA`, not a reduced subset. Training data is synthetic and format-augmented decision data plus decision items converted from public
173
+ datasets (listed below); the JevBench public tiers used only for evaluation carry MIT. Full
 
 
 
 
174
  per-cohort breakdown: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md#training-data-provenance).
175
 
176
+ Public datasets used (train splits where the dataset has one; licence as stated by each dataset; labels come from the
177
+ datasets, distractor options are generated by code): SQuAD 2.0 (CC BY-SA 4.0), ARC (CC BY-SA 4.0), BoolQ (CC BY-SA 3.0), CommonsenseQA (MIT), HellaSwag (MIT), Banking77 (CC BY 4.0), Bias in Bios (MIT), Bitext customer support (CDLA-Sharing-1.0), CLINC150 (CC BY 3.0), Amazon Counterfactual (CC BY 4.0), DBpedia-14 (CC BY-SA 3.0), Dolly 15k (CC BY-SA 3.0), GoEmotions (Apache-2.0), MASSIVE (CC BY 4.0), Twitter Financial News Sentiment (MIT), HelpSteer3 (CC BY 4.0), HelpSteer2 (CC BY 4.0), 2WikiMultihopQA (Apache-2.0), HotpotQA (CC BY-SA 4.0), MuSiQue (CC BY 4.0), QASC (CC BY 4.0), DROP (CC BY-SA 4.0), GSM8K (MIT), TempReason (CC BY-SA 3.0), MultiNLI (OANC / CC BY-SA 3.0 / CC BY 3.0), PAWS (Google terms, free for any purpose), PAWS-X (Google terms, free for any purpose), SNLI (CC BY-SA 4.0), WANLI (CC BY 4.0), ContractNLI (CC BY 4.0), CUAD (CC BY 4.0), ShARC (CC BY-SA 3.0), Jailbreak classification (Apache-2.0), Prompt injections (Apache-2.0), Aegis AI Content Safety 2.0 (CC BY 4.0), Jigsaw Toxic Comment Classification (mirror of the Kaggle data) (CC0 (data); comment text CC BY-SA 3.0 (Wikipedia)), Measuring Hate Speech (CC BY 4.0), Image safety classes (MIT). Upstream ids and the cohort each one feeds: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md#training-data-provenance).
178
+
179
  An exact-text overlap audit against the public JevBench tiers found 0 exact scenario matches and 181
180
  exact instruction matches — rows in two adequacy-rubric cohorts whose entire instruction field, a
181
+ generic 58-character adequacy question, is byte-identical to one public hard-tier instruction (0.03 %
182
+ of the 520,754-row training mixture). These rows are kept and disclosed here rather than regenerated,
183
  since the overlap is limited to one rubric question's wording and never touches a scenario or an
184
  answer.
185
 
SHA256SUMS CHANGED
@@ -1,9 +1,9 @@
1
  1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747 LICENSE
2
  caa6c931be76b0a3ed8d4901f520e3d71010ffbabe00cdd6a3de56563ffd26a3 NOTICE
3
- 2207c0ff6402b9a3244964fc4b604da9e49ac73126aed53fd2785d2916c91200 README.md
4
- 43b99a646a535b15e0f8bceebe2cfff03dd8cc519f5e21f68856b885657d7179 adapter_config.json
5
- 7bc8bfff4be52e0c9c6f605faf1eebc5e8a966ce103793c7f6970e29056ef5d3 adapter_model.safetensors
6
- 868cb61534ddc6f57b8718113c54391b2d7fae46d9b72b6ae59b3c96d4994d2c docs/BENCHMARKS.md
7
  96127670ad1e5bdd336a33118d6246330c27ac9e4fcb3f4f22767033809cd889 docs/assets/00-benchmark-card.png
8
  28c67a8c861f421fae5889af259d0850e7fb6eadb11f247adef72b94bf00b187 docs/assets/00-benchmark-card.svg
9
  7313f2b0645ee401d082572986dcee0c08d3daed1ae79fa650cfa685570ad7e2 docs/assets/02-latency-vs-qwen.png
 
1
  1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747 LICENSE
2
  caa6c931be76b0a3ed8d4901f520e3d71010ffbabe00cdd6a3de56563ffd26a3 NOTICE
3
+ 14ec17438881a77f2c305357854f35c64f516899894bb270d4609f6bbad0b374 README.md
4
+ 0d617456701ecb5bfb91d9fef119c3cfa9ce1f3d0c26a39d71893402c4928f3e adapter_config.json
5
+ a8e3eb341e27c1a49a282e327c2a2038906abb0eee771e80debbb4a4c45f7afc adapter_model.safetensors
6
+ 141067bb117600df9e09a46098e6a1f26e685ea5eab77f73b20352bce6d503d6 docs/BENCHMARKS.md
7
  96127670ad1e5bdd336a33118d6246330c27ac9e4fcb3f4f22767033809cd889 docs/assets/00-benchmark-card.png
8
  28c67a8c861f421fae5889af259d0850e7fb6eadb11f247adef72b94bf00b187 docs/assets/00-benchmark-card.svg
9
  7313f2b0645ee401d082572986dcee0c08d3daed1ae79fa650cfa685570ad7e2 docs/assets/02-latency-vs-qwen.png
adapter_config.json CHANGED
@@ -32,13 +32,13 @@
32
  "rank_pattern": {},
33
  "revision": "b6d637bef2393152b3da2b2fde72eecdee30557e",
34
  "target_modules": [
35
- "q_proj",
36
  "v_proj",
 
37
  "down_proj",
 
38
  "o_proj",
39
  "gate_proj",
40
- "k_proj",
41
- "up_proj"
42
  ],
43
  "target_parameters": null,
44
  "task_type": "CAUSAL_LM",
 
32
  "rank_pattern": {},
33
  "revision": "b6d637bef2393152b3da2b2fde72eecdee30557e",
34
  "target_modules": [
 
35
  "v_proj",
36
+ "up_proj",
37
  "down_proj",
38
+ "k_proj",
39
  "o_proj",
40
  "gate_proj",
41
+ "q_proj"
 
42
  ],
43
  "target_parameters": null,
44
  "task_type": "CAUSAL_LM",
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7bc8bfff4be52e0c9c6f605faf1eebc5e8a966ce103793c7f6970e29056ef5d3
3
  size 135113048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8e3eb341e27c1a49a282e327c2a2038906abb0eee771e80debbb4a4c45f7afc
3
  size 135113048
docs/BENCHMARKS.md CHANGED
@@ -1,5 +1,8 @@
1
  # Standard One — full benchmark report
2
 
 
 
 
3
  This report covers both released systems, **Standard One 8B** (`StandardThinking/StandardOne-8B` / `-LoRA`) and
4
  **Standard One 3B** (`StandardThinking/StandardOne-3B` / `-LoRA`). It is our own measurement, not an official JevBench
5
  leaderboard score. The only comparisons in this report are: the untuned
@@ -280,9 +283,8 @@ and in `release-manifest.json`, identical for both candidates.
280
 
281
  ## Training data provenance
282
 
283
- The following cohort families make up the identical 34-cohort, 382,576-row training mixture used by
284
- both Standard One 8B and Standard One 3B. Every row is synthetic and solver-generated (deterministic generators with
285
- visible checkers; a subset was teacher-reviewed). No JevBench evaluation scenario appears in any
286
  training cohort.
287
 
288
  | Cohort family | What it covers | Licence |
@@ -295,11 +297,57 @@ training cohort.
295
  | Abstention & robustness (`abstain`, `paraphrase`, `trap`, `hard-natural`) | Abstaining when the state does not settle the question, paraphrased and adversarial wording, harder natural-language decisions | Apache-2.0 (ours) |
296
  | Game & board-state (`games-text-train`, `games-harness-train`, `games-harness-multilingual`) | Othello, chess, tic-tac-toe, snake and graph-problem decisions | Apache-2.0 (ours) |
297
  | Tetris board-state (`games-text-train-tetris`, `games-harness-tetris`, `tetris-drought-train`) | Tetris-specific board-state decisions | Apache-2.0 (ours) |
 
 
298
  | JevBench public tiers (evaluation only, not training) | Easy / standard / hard held-out accuracy measurement | MIT |
299
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
300
  ## Public-overlap audit and disclosure
301
 
302
- Run on the exact final 34-cohort, 382,576-row mixture that **both candidates train on**:
303
 
304
  - **0 exact state matches** — no training row's state/scenario duplicates any public JevBench item.
305
  - **181 exact instruction matches**: `hardstyle-train-v1` (87 of 3,900 rows) and `hardstyle-train-v2`
@@ -307,13 +355,10 @@ Run on the exact final 34-cohort, 382,576-row mixture that **both candidates tra
307
  one public `jevbench-hard` instruction — a 58-character generic adequacy-rubric question ("does the
308
  response fully and correctly satisfy the request?"), reused as the complete instruction rather than
309
  as a substring.
310
- - 16 distinct shared word 8-grams (12 distinct phrases, 9,756 rows), all in three known families: the
311
- rubric sentence above (also in `judge-realistic-train-v3`, `judge-multilingual-train-v1`,
312
- `judge-proxy-v1`), an insurance "cancellation takes effect" boilerplate clause, and a "coverage
313
- reviewer" claims-adjudication phrase.
314
- - All other 32 cohorts: 0 exact state and 0 exact instruction matches.
315
 
316
- **These 181 rows are kept and disclosed here, not regenerated.** They are 0.05 % of the mixture, and
317
  the exposure is limited to one hard-tier *question's* generic wording, never a *state* or an answer.
318
 
319
  ## Evidence file hashes
 
1
  # Standard One — full benchmark report
2
 
3
+ > **Note.** The benchmark figures in this report are for v1.1 and are being updated for v2. The training-data provenance and
4
+ > overlap-audit sections below already describe the v2 weights.
5
+
6
  This report covers both released systems, **Standard One 8B** (`StandardThinking/StandardOne-8B` / `-LoRA`) and
7
  **Standard One 3B** (`StandardThinking/StandardOne-3B` / `-LoRA`). It is our own measurement, not an official JevBench
8
  leaderboard score. The only comparisons in this report are: the untuned
 
283
 
284
  ## Training data provenance
285
 
286
+ The following cohort families make up the 46-cohort, 520,754-row training mixture used by
287
+ both Standard One 8B and Standard One 3B. Apart from the public-dataset items listed below, every row is synthetic (deterministic generators with visible checkers; a subset was teacher-reviewed) or format-augmented. No JevBench evaluation scenario appears in any
 
288
  training cohort.
289
 
290
  | Cohort family | What it covers | Licence |
 
297
  | Abstention & robustness (`abstain`, `paraphrase`, `trap`, `hard-natural`) | Abstaining when the state does not settle the question, paraphrased and adversarial wording, harder natural-language decisions | Apache-2.0 (ours) |
298
  | Game & board-state (`games-text-train`, `games-harness-train`, `games-harness-multilingual`) | Othello, chess, tic-tac-toe, snake and graph-problem decisions | Apache-2.0 (ours) |
299
  | Tetris board-state (`games-text-train-tetris`, `games-harness-tetris`, `tetris-drought-train`) | Tetris-specific board-state decisions | Apache-2.0 (ours) |
300
+ | Synthetic and format-augmented decision data (v2) | Decision items in varied layouts | Apache-2.0 (ours) |
301
+ | Public-dataset decision items | Decision items converted from public datasets (table below) | Each dataset's own licence (table below) |
302
  | JevBench public tiers (evaluation only, not training) | Easy / standard / hard held-out accuracy measurement | MIT |
303
 
304
+ Public datasets used (train splits where a dataset has one; CUAD: its single release file). Labels come from the datasets;
305
+ distractor options and derived labels are generated by code. Licences as stated by each dataset; attribution to the dataset authors.
306
+
307
+ | Dataset | Upstream id | Licence | Cohort |
308
+ |---|---|---|---|
309
+ | SQuAD 2.0 | `rajpurkar/squad_v2` | CC BY-SA 4.0 | `pub-abstain-v1` |
310
+ | ARC | `allenai/ai2_arc` | CC BY-SA 4.0 | `pub-breadth-v1` |
311
+ | BoolQ | `google/boolq` | CC BY-SA 3.0 | `pub-breadth-v1` |
312
+ | CommonsenseQA | `tau/commonsense_qa` | MIT | `pub-breadth-v1` |
313
+ | HellaSwag | `Rowan/hellaswag` | MIT | `pub-breadth-v1` |
314
+ | Banking77 | `PolyAI/banking77` | CC BY 4.0 | `pub-classify-v1` |
315
+ | Bias in Bios | `LabHC/bias_in_bios` | MIT | `pub-classify-v1` |
316
+ | Bitext customer support | `bitext/Bitext-customer-support-llm-chatbot-training-dataset` | CDLA-Sharing-1.0 | `pub-classify-v1` |
317
+ | CLINC150 | `clinc/clinc_oos` | CC BY 3.0 | `pub-classify-v1` |
318
+ | Amazon Counterfactual | `mteb/amazon_counterfactual` | CC BY 4.0 | `pub-classify-v1` |
319
+ | DBpedia-14 | `fancyzhx/dbpedia_14` | CC BY-SA 3.0 | `pub-classify-v1` |
320
+ | Dolly 15k | `databricks/databricks-dolly-15k` | CC BY-SA 3.0 | `pub-classify-v1`, `safety-judge-v1` |
321
+ | GoEmotions | `google-research-datasets/go_emotions` | Apache-2.0 | `pub-classify-v1` |
322
+ | MASSIVE | `AmazonScience/massive` | CC BY 4.0 | `pub-classify-v1` |
323
+ | Twitter Financial News Sentiment | `zeroshot/twitter-financial-news-sentiment` | MIT | `pub-classify-v1` |
324
+ | HelpSteer3 | `nvidia/HelpSteer3` | CC BY 4.0 | `pub-judge-v1` |
325
+ | HelpSteer2 | `nvidia/HelpSteer2` | CC BY 4.0 | `pub-judge-v1` |
326
+ | 2WikiMultihopQA | `framolfese/2WikiMultihopQA` | Apache-2.0 | `pub-multihop-v1` |
327
+ | HotpotQA | `hotpotqa/hotpot_qa` | CC BY-SA 4.0 | `pub-multihop-v1` |
328
+ | MuSiQue | `dgslibisey/MuSiQue` | CC BY 4.0 | `pub-multihop-v1` |
329
+ | QASC | `allenai/qasc` | CC BY 4.0 | `pub-multihop-v1` |
330
+ | DROP | `ucinlp/drop` | CC BY-SA 4.0 | `pub-numeric-v1` |
331
+ | GSM8K | `openai/gsm8k` | MIT | `pub-numeric-v1` |
332
+ | TempReason | `tonytan48/TempReason` | CC BY-SA 3.0 | `pub-numeric-v1` |
333
+ | MultiNLI | `nyu-mll/multi_nli` | OANC / CC BY-SA 3.0 / CC BY 3.0 | `pub-paraphrase-v1` |
334
+ | PAWS | `google-research-datasets/paws` | Google terms, free for any purpose | `pub-paraphrase-v1` |
335
+ | PAWS-X | `google-research-datasets/paws-x` | Google terms, free for any purpose | `pub-paraphrase-v1` |
336
+ | SNLI | `stanfordnlp/snli` | CC BY-SA 4.0 | `pub-paraphrase-v1` |
337
+ | WANLI | `alisawuffles/WANLI` | CC BY 4.0 | `pub-paraphrase-v1` |
338
+ | ContractNLI | `stanfordnlp.github.io/contract-nli` | CC BY 4.0 | `pub-rules-v1` |
339
+ | CUAD | `theatticusproject/cuad` | CC BY 4.0 | `pub-rules-v1` |
340
+ | ShARC | `sharc-data.github.io` | CC BY-SA 3.0 | `pub-rules-v1` |
341
+ | Jailbreak classification | `jackhhao/jailbreak-classification` | Apache-2.0 | `safety-judge-v1` |
342
+ | Prompt injections | `deepset/prompt-injections` | Apache-2.0 | `safety-judge-v1` |
343
+ | Aegis AI Content Safety 2.0 | `nvidia/Aegis-AI-Content-Safety-Dataset-2.0` | CC BY 4.0 | `content-safety-text-v1` |
344
+ | Jigsaw Toxic Comment Classification (mirror of the Kaggle data) | `tasksource/jigsaw_toxicity` | CC0 (data); comment text CC BY-SA 3.0 (Wikipedia) | `content-safety-text-v1` |
345
+ | Measuring Hate Speech | `ucberkeley-dlab/measuring-hate-speech` | CC BY 4.0 | `content-safety-text-v1` |
346
+ | Image safety classes | `deepghs/nsfw_detect` | MIT | `content-safety-image-v1`, `content-safety-image-2d-v1` |
347
+
348
  ## Public-overlap audit and disclosure
349
 
350
+ Run on the 46-cohort, 520,754-row training data that **both candidates train on**:
351
 
352
  - **0 exact state matches** — no training row's state/scenario duplicates any public JevBench item.
353
  - **181 exact instruction matches**: `hardstyle-train-v1` (87 of 3,900 rows) and `hardstyle-train-v2`
 
355
  one public `jevbench-hard` instruction — a 58-character generic adequacy-rubric question ("does the
356
  response fully and correctly satisfy the request?"), reused as the complete instruction rather than
357
  as a substring.
358
+ - 24 distinct shared word 8-grams (20,400 rows), all in known families: the rubric sentence above (also in `judge-realistic-train-v3`, `judge-multilingual-train-v1`, `judge-proxy-v1` and the format-augmented cohort), an insurance "cancellation takes effect" boilerplate clause, a "coverage reviewer" claims-adjudication phrase, and generic contract boilerplate (for example "controlled by or under common control with") in one public-dataset cohort.
359
+ - All other 44 cohorts: 0 exact state and 0 exact instruction matches.
 
 
 
360
 
361
+ **These 181 rows are kept and disclosed here, not regenerated.** They are 0.03 % of the mixture, and
362
  the exposure is limited to one hard-tier *question's* generic wording, never a *state* or an answer.
363
 
364
  ## Evidence file hashes