README.md

---
pipeline_tag: text-generation
---

# Parcae-medium-370m

Parcae is a novel stable, looped language model architecture introduced in the paper [Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946).

Looped architectures scale compute by passing activations through a block of layers multiple times. Parcae addresses common instability issues in looped models (such as residual explosion and loss spikes) by recasting looping as a nonlinear time-variant dynamical system and constraining the spectral norm of injection parameters.

- **Paper:** [https://arxiv.org/abs/2604.12946](https://arxiv.org/abs/2604.12946)
- **Repository:** [https://github.com/sandyresearch/parcae](https://github.com/sandyresearch/parcae)
- **Project Page:** [https://sandyresearch.github.io/parcae/](https://sandyresearch.github.io/parcae/)

## Installation

To use this model, you can install the `parcae-lm` package:

```bash
pip install parcae-lm
```

## Usage

You can instantiate the model and load the pretrained weights using the following code:

```python
import parcae_lm

# Load the pretrained model from HuggingFace
model = parcae_lm.from_pretrained("SandyResearch/parcae-medium-370m")
```

## Model Dimensions

| Model | Parameters | Prelude | Core | Coda | Model dim. | Recurrence |
|-------|-----------|---------|------|------|-----------|------------|
| Parcae-medium-370M | 370M | 4 | 4 | 4 | 1024 | 8 |

## Citation

```bibtex
@misc{prairie2026parcaescalinglawsstable,
      title={Parcae: Scaling Laws For Stable Looped Language Models}, 
      author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
      year={2026},
      eprint={2604.12946},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.12946}, 
}
```