File size: 1,823 Bytes
4392844 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | ---
pipeline_tag: text-generation
---
# Parcae-medium-370m
Parcae is a novel stable, looped language model architecture introduced in the paper [Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946).
Looped architectures scale compute by passing activations through a block of layers multiple times. Parcae addresses common instability issues in looped models (such as residual explosion and loss spikes) by recasting looping as a nonlinear time-variant dynamical system and constraining the spectral norm of injection parameters.
- **Paper:** [https://arxiv.org/abs/2604.12946](https://arxiv.org/abs/2604.12946)
- **Repository:** [https://github.com/sandyresearch/parcae](https://github.com/sandyresearch/parcae)
- **Project Page:** [https://sandyresearch.github.io/parcae/](https://sandyresearch.github.io/parcae/)
## Installation
To use this model, you can install the `parcae-lm` package:
```bash
pip install parcae-lm
```
## Usage
You can instantiate the model and load the pretrained weights using the following code:
```python
import parcae_lm
# Load the pretrained model from HuggingFace
model = parcae_lm.from_pretrained("SandyResearch/parcae-medium-370m")
```
## Model Dimensions
| Model | Parameters | Prelude | Core | Coda | Model dim. | Recurrence |
|-------|-----------|---------|------|------|-----------|------------|
| Parcae-medium-370M | 370M | 4 | 4 | 4 | 1024 | 8 |
## Citation
```bibtex
@misc{prairie2026parcaescalinglawsstable,
title={Parcae: Scaling Laws For Stable Looped Language Models},
author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
year={2026},
eprint={2604.12946},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2604.12946},
}
``` |