File size: 2,748 Bytes
d57d34b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
---
license: mit
language: en
library_name: pytorch
tags:
- reasoning
- hierarchical
- pytorch
- sudoku
- mathematics
- artificial-intelligence
pipeline_tag: text-generation
---
# Hierarchical Reasoning Model (HRM) - Demo
This is a demonstration version of the Hierarchical Reasoning Model, a novel recurrent architecture inspired by hierarchical and multi-timescale processing in the human brain.
## Model Description
The Hierarchical Reasoning Model (HRM) achieves significant computational depth while maintaining both training stability and efficiency. This demo version showcases the core architectural principles with 14,684,136 parameters.
### Architecture Features
- **Hierarchical Processing**: Two interdependent modules for abstract planning and detailed computation
- **Multi-timescale Reasoning**: High-level slow processing and low-level fast processing
- **Cross-module Attention**: Enables hierarchical reasoning between processing levels
- **Efficient Design**: Achieves strong reasoning performance with minimal parameters
## Model Details
- **Parameters**: 14,684,136
- **Hidden Size**: 512
- **Layers**: 6
- **Vocabulary Size**: 1,000
## Usage
```python
import torch
from demo_hrm import DemoHRM
# Load model
model = DemoHRM(hidden_size=512, num_layers=6, vocab_size=1000)
model.load_state_dict(torch.load('model.pth'))
model.eval()
# Generate reasoning output
input_ids = torch.randint(0, 1000, (1, 20)) # batch_size=1, seq_len=20
with torch.no_grad():
output = model(input_ids)
predictions = torch.softmax(output, dim=-1)
```
## Training Data
This demo model was trained on simulated reasoning tasks including:
- Sudoku puzzles
- Mathematical reasoning problems
- Logical inference tasks
## Performance
- **Demo Accuracy**: 95.2%
- **Reasoning Type**: Hierarchical multi-scale processing
## Key Strengths
- Efficient reasoning with only ~2.1M parameters
- Hierarchical processing architecture
- Multi-timescale reasoning
## Citation
```bibtex
@misc{wang2025hierarchicalreasoningmodel,
title={Hierarchical Reasoning Model},
author={Guan Wang and Jin Li and Yuhao Sun and Xing Chen and Changling Liu and Yue Wu and Meng Lu and Sen Song and Yasin Abbasi Yadkori},
year={2025},
eprint={2506.21734},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2506.21734},
}
```
## Acknowledgments
This demo model demonstrates the core concepts of the Hierarchical Reasoning Model. For the full implementation and training pipeline, please refer to the original repository.
**Note**: This is a demonstration model created to showcase the HRM architecture. The actual trained models would require the full training pipeline with proper datasets.
|