README.md

---
base_model: sapientinc/HRM-Text-1B
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:sapientinc/HRM-Text-1B
- lora
- code
- python
- humaneval
- mbpp
---

# HRM-Text-1B-sft-code-LoRA

LoRA adapter for `sapientinc/HRM-Text-1B`.

`sapientinc/HRM-Text-1B` is a pretrained-only HRM text model. This adapter is the code post-training release built on top of it.

The release uses supervised LoRA post-training for coding tasks. It is the adapter artifact; the merged model is:

[`josephmayo/HRM-Text-1B-sft-code`](https://huggingface.co/josephmayo/HRM-Text-1B-sft-code)

## Training

- Base model: `sapientinc/HRM-Text-1B`
- Method: supervised LoRA post-training
- Training rows: `384`
- Max steps: `120`
- LoRA rank: `64`
- Learning rate: `8e-6`
- Final train loss: `0.3275703112284342`

## Validation

Local code validation:

- Base model score: `5/100`
- Adapter score: `24/100`
- Absolute improvement: `+19/100`
- Relative improvement: `4.8x` over base
- HumanEval slice: `14/50`
- MBPP slice: `10/50`

The score above is the local validation result used for this release.

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "sapientinc/HRM-Text-1B"
adapter_id = "josephmayo/HRM-Text-1B-sft-code-LoRA"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(base_id, trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()
```

## Notes

- This is an adapter, not a standalone merged model.
- This is the LoRA adapter. Use the merged model for standalone loading.