metadata
license: apache-2.0
library_name: pytorch
tags:
- hrm
- hierarchical-reasoning
- text2sql
HRM-XL Base Checkpoint
A 1.2B-parameter hierarchical reasoning language model. Used as the base for fine-tuning in the HRM-Text tutorial.
Architecture
- 16 layers, hidden size 1536, 12 heads (head dim 128)
- H_cycles=2, L_cycles=3 (dual-timescale recurrence)
- RoPE positional encoding
- Precision: bf16
Usage
huggingface-cli download SapientIntelligence/HRM-XL-base --local-dir ./ckpts/base
python -u pretrain.py --config-name cfg_finetune_demo resume_from=./ckpts/base
See github.com/sapientinc/HRM-Text for the full tutorial.