Fix dataset link, repoint restore maps to in-repo evidence, record supplementary files in ARTIFACTS.json
0e9c9fa verified | license: apache-2.0 | |
| library_name: transformers | |
| base_model: | |
| - Qwen/Qwen3.5-4B | |
| - Qwen/Qwen3.5-4B-Base | |
| tags: | |
| - longworld | |
| - qwen3.5 | |
| - full-sft | |
| - long-context | |
| - 128k | |
| # LongWorld Checkpoints | |
| Three complete, final step-680 SFT weight exports used as LongWorld related-work | |
| baselines. Each subfolder contains its own model shards, index, configuration, | |
| tokenizer and processor files. The variants retain independent learned weights. | |
| | Subfolder | Starting model | SFT data | | |
| |---|---|---| | |
| | `acc-step680` | Qwen3.5-4B Instruct | ACC | | |
| | `base-acc-step680` | Qwen3.5-4B-Base | ACC | | |
| | `base-longtrace-step680` | Qwen3.5-4B-Base | LongTraceRL | | |
| These are the final aligned step-680 exports, not a selection of the best | |
| validation checkpoint. The Base LongTrace run was trained independently from | |
| the Base ACC run. The training inputs are in the [128k baseline dataset](https://huggingface.co/datasets/Xnhyacinth/LongWorld-128K-Baselines). | |
| They are related-work baselines, not checkpoints trained on the LongWorld | |
| workflow-probe product. | |
| ## Download one variant | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| root = snapshot_download( | |
| "Xnhyacinth/LongWorld-Checkpoints", | |
| allow_patterns=["base-acc-step680/*"], | |
| local_dir="longworld-checkpoints", | |
| ) | |
| ``` | |
| Use the selected subfolder with a Qwen3.5-compatible inference stack. Structural | |
| validation covers the complete safetensors headers, shard index and loading | |
| files; inference has not been executed as part of this publication review. | |
| `ARTIFACTS.json` records the exact published paths, sizes and SHA-256 identities. | |
| Each subfolder carries the weight and loading files plus that variant's own | |
| training recipe and log (see [Standalone model cards](#standalone-model-cards) | |
| below). Optimizer/RNG state, historical working snapshots and the removed | |
| `training_args.bin` pickles are excluded. The original file hashes are retained | |
| for the selected exports. Shared tokenizer files appear in each subfolder so | |
| each variant can be downloaded independently; no storage-saving claim is made. | |
| The source exports and the upstream [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) | |
| and [Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base) cards declare | |
| Apache-2.0. The companion training datasets retain their own terms. | |
| ## Standalone model cards | |
| Each subfolder is a complete, independently loadable model card: it carries the | |
| variant's own `args.json` (full training recipe), `trainer_state.json` (per-step | |
| train loss and the seven periodic eval-loss points), `CLEANUP_20260930_RESTORE_MAP.json` | |
| (restore map for the intentionally removed `training_args.bin` pickle) and a | |
| `README.md` with the run summary. Loading a subfolder needs no other repository. | |
| ```python | |
| from transformers import AutoProcessor, AutoModelForCausalLM | |
| repo = "Xnhyacinth/LongWorld-Checkpoints" | |
| subfolder = "base-acc-step680" # or "acc-step680" / "base-longtrace-step680" | |
| processor = AutoProcessor.from_pretrained(repo, subfolder=subfolder) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| repo, subfolder=subfolder, dtype="auto", device_map="auto" | |
| ) | |
| ``` | |
| All three runs trained for 680 steps at `max_length` 133120. The two ACC runs | |
| report both train and eval loss; the LongTrace run reports eval loss only. | |
| Train loss is the mean over the 137 logged steps unless stated otherwise. | |
| | Variant | Starting model | Data | Train loss | Eval loss (step 680) | | |
| |---|---|---|---|---| | |
| | `acc-step680` | Qwen3.5-4B (instruct) | ACC | 0.3645 (final step; run mean 0.4327) | 0.37456 | | |
| | `base-acc-step680` | Qwen3.5-4B-Base | ACC | 0.4382 (run mean; final step 0.3725) | 0.37545 | | |
| | `base-longtrace-step680` | Qwen3.5-4B-Base | LongTraceRL | 0.0623 (run mean; final step 0.0040) | 0.17741 | | |
| Notes carried over from the standalone cards: | |
| - `acc-step680` — GBS 16, 8 GPU (SP=4, DP=2), lr 1e-5 cosine, `max_length` | |
| 133120, 680 steps; source `data/sft/swift_ext_acc/v6-20260825-153450`. | |
| - `base-acc-step680` — GBS 16, SP 4, micro 1, accum 16, lr 1e-5 cosine, | |
| `max_length` 133120, 680 steps; source `data/sft/swift_ext_acc_base/v1-20260903-163953`. | |
| - `base-longtrace-step680` — same recipe as Base ACC; trained independently of | |
| ACC (not continued from it); resumed from checkpoint-300 after a 24h session | |
| abort. Best eval during the run was checkpoint-200 (0.129); this subfolder is | |
| the aligned step-680 export, not the best-validation checkpoint. | |
| All three omit optimizer/RNG state and the `training_args.bin` pickle. | |