metadata
pipeline_tag: text-generation
Parcae-medium-370m
Parcae is a novel stable, looped language model architecture introduced in the paper Parcae: Scaling Laws For Stable Looped Language Models.
Looped architectures scale compute by passing activations through a block of layers multiple times. Parcae addresses common instability issues in looped models (such as residual explosion and loss spikes) by recasting looping as a nonlinear time-variant dynamical system and constraining the spectral norm of injection parameters.
- Paper: https://arxiv.org/abs/2604.12946
- Repository: https://github.com/sandyresearch/parcae
- Project Page: https://sandyresearch.github.io/parcae/
Installation
To use this model, you can install the parcae-lm package:
pip install parcae-lm
Usage
You can instantiate the model and load the pretrained weights using the following code:
import parcae_lm
# Load the pretrained model from HuggingFace
model = parcae_lm.from_pretrained("SandyResearch/parcae-medium-370m")
Model Dimensions
| Model | Parameters | Prelude | Core | Coda | Model dim. | Recurrence |
|---|---|---|---|---|---|---|
| Parcae-medium-370M | 370M | 4 | 4 | 4 | 1024 | 8 |
Citation
@misc{prairie2026parcaescalinglawsstable,
title={Parcae: Scaling Laws For Stable Looped Language Models},
author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
year={2026},
eprint={2604.12946},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2604.12946},
}