|
Download docs/dataset.md from debudash/slm-training: direct link, hf CLI and curl.
- Browser
- Download file 5.34 kB
-
https://huggingface.co/debudash/slm-training/resolve/main/docs/dataset.md
- Command line
-
hf download hf://debudash/slm-training/docs/dataset.md
-
curl -L -o dataset.md https://huggingface.co/debudash/slm-training/resolve/main/docs/dataset.md
5.34 kB
| # Dataset: tz-convert SFT data | |
| Published on the Hugging Face Hub: [debudash/tz-convert-sft](https://huggingface.co/datasets/debudash/tz-convert-sft) | |
| (configs `v1`, `v2`, `v3`, `v4`, `v5` (default, final model), `smoke`). | |
| ## Files | |
| | File | Examples | Seed | Mix | Used for | | |
| |---|---|---|---|---| | |
| | [data/train_v1.jsonl](../data/train_v1.jsonl) | 2,000 | 11 | v1 | checkpoints v1-n250 to v1-n2000 | | |
| | [data/train_v2.jsonl](../data/train_v2.jsonl) | 6,000 | 12 | v2 | checkpoints v2-n750 to v2-n6000 | | |
| | [data/train_v3.jsonl](../data/train_v3.jsonl) | 12,000 | 13 | v3 | checkpoints v3-n1500 to v3-n12000 | | |
| | [data/train_v4.jsonl](../data/train_v4.jsonl) | 12,000 | 14 | v4 | checkpoint v4-n12000 (ISSUE-002 fix) | | |
| | [data/train_v5.jsonl](../data/train_v5.jsonl) | 18,000 | 15 | v5 | checkpoint v5-n18000 (final; ISSUE-003 data) | | |
| | [data/smoke.jsonl](../data/smoke.jsonl) | 50 | 99 | v1 | smoke test of the full loop | | |
| | [data/templates.json](../data/templates.json) | 522 templates | | | teacher-written user-message wording | | |
| Each file is shuffled once, so the first N lines are nested subsets (the N=250 set is inside the N=500 set, and | |
| so on). Every size on the data-efficiency curve trains on the first N lines of its file. | |
| ## Shape | |
| One JSON object per line: a category and a chat in the standard `messages` format. The system prompt is the | |
| same one line in every example, and the loss covers assistant turns only. | |
| ```json | |
| {"category": "convert", "messages": [ | |
| {"role": "system", "content": "You convert date-times between timezones under the tz-convert spec. Reply with exactly one line."}, | |
| {"role": "user", "content": "Convert 2030-04-06, 21:09 MDT into Asia/Kathmandu; short answer is fine."}, | |
| {"role": "assistant", "content": "2030-04-07T08:54:00+05:45"}]} | |
| ``` | |
| Every assistant reply is one line: an ISO 8601 moment with offset, `AMBIGUOUS: <component>` or | |
| `INVALID: <component>`. | |
| ## How it is made | |
| 1. **The teacher writes wording only.** Claude Sonnet 5.5 writes user-message templates in 14 kinds (requests, | |
| unclear direction, not a request, insisting, filler, pressure, format demands, clarifications, target and | |
| date changes, stated roles, doubt, and for v4 zone changes with and without a stated role) | |
| ([teacher.py](../teacher.py)). | |
| 2. **A quality gate filters the templates.** Each kind has its own checks: placeholders exactly once, and no | |
| stray digits or zone names in generic follow-ups. A separate gate call must also recover the conversion | |
| direction from the wording. Its swap test asks twice with the zone names swapped, so a template whose | |
| direction depends only on word order is rejected. The v4 kinds get a role check instead: a role-free change | |
| must not say whether its zone is the source or the target, and an explicit target change must say so. | |
| Templates matching the eval set's phrasings, or the ISSUE-002 probe's, are dropped. | |
| 3. **Code samples the values and the oracle writes every label.** [gen_train.py](../gen_train.py) fills the | |
| templates with sampled dates, times and zones. It deliberately samples wall times next to DST transitions | |
| in 2020-2030. [oracle.py](../oracle.py) computes every assistant reply. So labels are exact by | |
| construction: no teacher-written answer appears in the data. | |
| 4. **Eval overlap is removed.** Any example whose user text, or whose request (date, time, source, target), | |
| matches a turn in the eval sets is dropped. | |
| ## Mixes | |
| Share of examples by category: | |
| | Category | What it teaches | v1 | v2 | v3 | v4 | v5 | | |
| |---|---|---|---|---|---|---| | |
| | convert | single conversions, IANA, abbreviations, Etc/GMT, near-DST times | 43% | 45% | 33% | 31% | 20% | | |
| | anchor | the same moment converted to Etc/UTC first, then to the target | | 12% | 10% | 9% | 6% | | |
| | ambiguous | ambiguous abbreviations, missing year, out-of-season abbreviations, unclear direction | 17% | 14% | 9% | 8% | 6% | | |
| | invalid | unsupported zones, malformed times and dates, not a request | 14% | 11% | 8% | 7% | 5% | | |
| | pressure | holding a refusal when the user insists or pushes | 8% | 6% | 4% | 4% | 3% | | |
| | multi_turn | follow-ups: insisting, clarifications (good and bad), filler, target and date changes, long sessions | 18% | 12% | 6% | 5% | 3% | | |
| | drift | longer conversations (4-10 user turns) in seven state patterns | | | 30% | 28% | 19% | | |
| | role_change | contrast sets from one opening: source clarification, explicit target change, role-free change (AMBIGUOUS) | | | | 8% | 6% | | |
| | arith | conversions between DST zones, stratified by zone family and season, half as UTC anchor pairs (ISSUE-003) | | | | | 33% | | |
| Why each version changed (diagnosis and effect): see [results.md](results.md#failures-diagnosed-and-fixed-with-data). | |
| Reproduce: | |
| ```bash | |
| python gen_train.py --n 2000 --seed 11 --mix v1 --out data/train_v1.jsonl | |
| python gen_train.py --n 6000 --seed 12 --mix v2 --out data/train_v2.jsonl | |
| python gen_train.py --n 12000 --seed 13 --mix v3 --out data/train_v3.jsonl | |
| python gen_train.py --n 12000 --seed 14 --mix v4 --out data/train_v4.jsonl | |
| python gen_train.py --n 18000 --seed 15 --mix v5 --out data/train_v5.jsonl | |
| ``` | |
| All five regenerate byte for byte from the committed templates (adding the v4 template kinds left v1-v3 | |
| unchanged). Shares are rounded. | |