---
license: apache-2.0
library_name: pytorch
tags:
  - hrm
  - hierarchical-reasoning
  - text2sql
---

# HRM-XL Base Checkpoint

A 1.2B-parameter hierarchical reasoning language model. Used as the base for fine-tuning in the [HRM-Text](https://github.com/sapientinc/HRM-Text) tutorial.

## Architecture

- 16 layers, hidden size 1536, 12 heads (head dim 128)
- H_cycles=2, L_cycles=3 (dual-timescale recurrence)
- RoPE positional encoding
- Precision: bf16

## Usage

```bash
huggingface-cli download SapientIntelligence/HRM-XL-base --local-dir ./ckpts/base
python -u pretrain.py --config-name cfg_finetune_demo resume_from=./ckpts/base
```

See [github.com/sapientinc/HRM-Text](https://github.com/sapientinc/HRM-Text) for the full tutorial.