|
Download README.md from anonymous-md/test: direct link, hf CLI and curl.
- Browser
- Download file 1.78 kB
-
https://huggingface.co/anonymous-md/test/resolve/main/README.md
- Command line
-
hf download hf://anonymous-md/test/README.md
-
curl -L -o README.md https://huggingface.co/anonymous-md/test/resolve/main/README.md
1.78 kB
| license: cc-by-nc-4.0 | |
| # Qwen3-1.7B on Nemotron-CC (low-quality tier): ablation checkpoints | |
| Four base language models with the Qwen3-1.7B architecture (1.72B parameters). Each was pretrained from scratch for one epoch on 34.4B tokens of the low-quality (LQ) tier of Nemotron-CC, Common Crawl snapshots 2013–2015. | |
| | Folder | Training data | Seed | | |
| |---|---|---| | |
| | `v2-lq-seed1234/` | LQ after V2 repair (prose-only repair + linguistic repair) | 1234 | | |
| | `raw-lq-seed1234/` | LQ, unrepaired | 1234 | | |
| | `raw-lq-seed1337/` | LQ, unrepaired | 1337 | | |
| | `raw-lq-seed42/` | LQ, unrepaired | 42 | | |
| - **Seeds:** the seed sets the weight initialization only. Every run used the same data-shuffling seed (1234), so the three raw runs saw identical batches in identical order. `v2-lq-seed1234/` and `raw-lq-seed1234/` share the same configuration and seed and differ only in training data. | |
| - **Training:** 16,406 steps × 512 sequences × 4,096 tokens, BF16, AdamW (β = 0.9/0.95, weight decay 0.1), learning rate 3e-4 → 3e-5 (warmup–stable–decay). | |
| - **Base models only:** no instruction tuning. | |
| ## Format and compatibility | |
| Each folder is a standard Hugging Face checkpoint: BF16 `model.safetensors`, tokenizer, and generation config. The checkpoints were exported with transformers 5.3. `config.json` and `tokenizer_config.json` also carry the older field names, so transformers 4.x (4.51 or newer) loads the same RoPE base (1,000,000) and the same tokenizer. | |
| ## Usage | |
| ```bash | |
| hf download anonymous-md/test --include "v2-lq-seed1234/*" --local-dir ./qwen3-lq | |
| ``` | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| path = "./qwen3-lq/v2-lq-seed1234" | |
| tokenizer = AutoTokenizer.from_pretrained(path) | |
| model = AutoModelForCausalLM.from_pretrained(path) | |
| ``` | |