| license: apache-2.0 | |
| library_name: pytorch | |
| tags: | |
| - hrm | |
| - hierarchical-reasoning | |
| - text2sql | |
| # HRM-XL Base Checkpoint | |
| A 1.2B-parameter hierarchical reasoning language model. Used as the base for fine-tuning in the [HRM-Text](https://github.com/sapientinc/HRM-Text) tutorial. | |
| ## Architecture | |
| - 16 layers, hidden size 1536, 12 heads (head dim 128) | |
| - H_cycles=2, L_cycles=3 (dual-timescale recurrence) | |
| - RoPE positional encoding | |
| - Precision: bf16 | |
| ## Usage | |
| ```bash | |
| huggingface-cli download SapientIntelligence/HRM-XL-base --local-dir ./ckpts/base | |
| python -u pretrain.py --config-name cfg_finetune_demo resume_from=./ckpts/base | |
| ``` | |
| See [github.com/sapientinc/HRM-Text](https://github.com/sapientinc/HRM-Text) for the full tutorial. | |