File size: 744 Bytes
4a3b431 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | ---
license: apache-2.0
library_name: pytorch
tags:
- hrm
- hierarchical-reasoning
- text2sql
---
# HRM-XL Base Checkpoint
A 1.2B-parameter hierarchical reasoning language model. Used as the base for fine-tuning in the [HRM-Text](https://github.com/sapientinc/HRM-Text) tutorial.
## Architecture
- 16 layers, hidden size 1536, 12 heads (head dim 128)
- H_cycles=2, L_cycles=3 (dual-timescale recurrence)
- RoPE positional encoding
- Precision: bf16
## Usage
```bash
huggingface-cli download SapientIntelligence/HRM-XL-base --local-dir ./ckpts/base
python -u pretrain.py --config-name cfg_finetune_demo resume_from=./ckpts/base
```
See [github.com/sapientinc/HRM-Text](https://github.com/sapientinc/HRM-Text) for the full tutorial.
|