---
license: mit
language:
- en
tags:
- code-generation
- codefeedback
- hrm-text
base_model: sapientai/HRM-Text-1B
---

# HRM-Text-1B-Code-Feedback

Fine-tuned version of [HRM-Text-1B](https://huggingface.co/sapientai/HRM-Text-1B) on the CodeFeedback dataset for code generation.

## Model Details

- **Base Model:** sapientai/HRM-Text-1B (1B parameters, hierarchical reasoning model)
- **Training Data:** CodeFeedback dataset (~131k samples, filtered to <= 4096 tokens)
- **Training:** 2 epochs, ~8 hours on L40S GPU
- **Architecture:** Hierarchical Reasoning Model with H_cycles=2, L_cycles=3

## Training Data Distribution

| Language | Samples |
|----------|---------|
| Python | ~80k |
| JavaScript | ~7.6k |
| React | ~550 |

## Performance

| Task | Base | Fine-tuned |
|------|------|------------|
| C++ factorial | Broken (repeating includes) | Correct |
| JS reverse | Wrong syntax | Correct syntax |
| Java max | Wrong type | Better structure |

## Usage



## Training Details

- **Framework:** PyTorch with FlashAttention 3
- **Loss:** Cross-entropy
- **Hardware:** AWS L40S GPU
- **Training Time:** ~8 hours

## Limitations

- Maximum sequence length: 4096 tokens
- Requires FlashAttention 3 for inference (Ada Lovelace or newer GPUs)
- Limited React/TypeScript performance due to small training data
- Best performance on Python code generation

## License

MIT License