helmo commited on
Commit
bfdb831
·
verified ·
1 Parent(s): b411df4

Card: remove the note about an earlier dataset version

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -112,7 +112,7 @@ These come from what the model did well and badly in the testbench:
112
 
113
  - **Objective:** supervised fine-tuning for 1,500 steps on target distributions (one hard label per synthetic record, soft distributions for typed-decisions), with cross-entropy plus 0.5 times a Brier term to keep the distributions calibrated.
114
  - **Data:** 11,079 records, expanded to 15,879 training examples by using every question of each case.
115
- - 9,879 synthetic records (3,282 choice, 3,304 score, 3,293 yes/no, one question each), published under the MIT licence as [helmo/synthetic-typed-decisions](https://huggingface.co/datasets/helmo/synthetic-typed-decisions). They cover 207 topic domains and were written by [deepseek-ai/DeepSeek-V4-Flash-0731](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) through OpenRouter. Each output was parsed with a strict schema and rejected if a label was invalid, and rows in non-Latin scripts were dropped. **The labels are written by that model, not by people.** An earlier, larger version of this set contained corrupted labels and was discarded before this model was trained.
116
  - 1,200 training cases (five questions each) from LocalLLaMA/typed-decisions (Apache-2.0).
117
  - **Held out:** the 400 test cases of LocalLLaMA/typed-decisions were never used for training, calibration or choosing a checkpoint.
118
  - **Hosted decision models:** we did not call TypeSafe Jev, or use its outputs, to build the synthetic data or to train this model. The benchmark's gold labels come from its authors' teacher model, as described on the dataset card.
 
112
 
113
  - **Objective:** supervised fine-tuning for 1,500 steps on target distributions (one hard label per synthetic record, soft distributions for typed-decisions), with cross-entropy plus 0.5 times a Brier term to keep the distributions calibrated.
114
  - **Data:** 11,079 records, expanded to 15,879 training examples by using every question of each case.
115
+ - 9,879 synthetic records (3,282 choice, 3,304 score, 3,293 yes/no, one question each), published under the MIT licence as [helmo/synthetic-typed-decisions](https://huggingface.co/datasets/helmo/synthetic-typed-decisions). They cover 207 topic domains and were written by [deepseek-ai/DeepSeek-V4-Flash-0731](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) through OpenRouter. Each output was parsed with a strict schema and rejected if a label was invalid, and rows in non-Latin scripts were dropped. **The labels are written by that model, not by people.**
116
  - 1,200 training cases (five questions each) from LocalLLaMA/typed-decisions (Apache-2.0).
117
  - **Held out:** the 400 test cases of LocalLLaMA/typed-decisions were never used for training, calibration or choosing a checkpoint.
118
  - **Hosted decision models:** we did not call TypeSafe Jev, or use its outputs, to build the synthetic data or to train this model. The benchmark's gold labels come from its authors' teacher model, as described on the dataset card.