nielsr HF Staff commited on
Commit
5131dba
·
verified ·
1 Parent(s): 072747c

Add model card for Parcae-large-770m

Browse files

This PR adds a comprehensive model card for the Parcae-large-770m model.
- It includes the `pipeline_tag: text-generation` metadata.
- It provides links to the paper "[Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946)", the official GitHub repository, and the project page.
- It includes installation and sample usage instructions based on the official repository.
- It provides architectural details for the 770M variant.

Files changed (1) hide show
  1. README.md +52 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ ---
4
+
5
+ # Parcae-Large-770M
6
+
7
+ Parcae is a novel stable, looped architecture that addresses instability issues in prior looped language models by recasting looping as a nonlinear time-variant dynamical system over the residual stream. This specific checkpoint is the 770M parameter version, trained on the FineWeb-Edu dataset.
8
+
9
+ - **Paper:** [Parcae: Scaling Laws For Stable Looped Language Models](https://arxiv.org/abs/2604.12946)
10
+ - **Repository:** [https://github.com/sandyresearch/parcae](https://github.com/sandyresearch/parcae)
11
+ - **Project Page:** [https://sandyresearch.github.io/parcae/](https://sandyresearch.github.io/parcae/)
12
+
13
+ ## Installation
14
+
15
+ To use this model, you can install the official package:
16
+
17
+ ```bash
18
+ pip install parcae-lm
19
+ ```
20
+
21
+ ## Usage
22
+
23
+ You can load the pretrained weights and instantiate the model using the following code:
24
+
25
+ ```python
26
+ import parcae_lm
27
+
28
+ # Load the pretrained model from HuggingFace
29
+ model = parcae_lm.from_pretrained("SandyResearch/parcae-large-770m")
30
+ ```
31
+
32
+ ## Model Details
33
+
34
+ This model variant has the following configuration:
35
+
36
+ | Model | Parameters | Prelude Layers | Core Layers | Coda Layers | Model Dim | Default Recurrence |
37
+ |-------|-----------|---------|------|------|-----------|------------|
38
+ | Parcae-770M | 770M | 6 | 6 | 6 | 1280 | 8 |
39
+
40
+ ## Citation
41
+
42
+ ```bibtex
43
+ @misc{prairie2026parcaescalinglawsstable,
44
+ title={Parcae: Scaling Laws For Stable Looped Language Models},
45
+ author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
46
+ year={2026},
47
+ eprint={2604.12946},
48
+ archivePrefix={arXiv},
49
+ primaryClass={cs.LG},
50
+ url={https://arxiv.org/abs/2604.12946},
51
+ }
52
+ ```