ykyao commited on
Commit
4f76ea0
路
verified 路
1 Parent(s): ab4a275

Add paper links, Figure 1 and citation

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +17 -1
  3. assets/figure1.png +3 -0
.gitattributes CHANGED
@@ -1 +1,2 @@
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
 
 
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ assets/figure1.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -13,11 +13,13 @@ tags:
13
 
14
  # ARLoop-0.6B
15
 
 
 
16
  ARLoop is a fully-open pretrained looped transformer with attention residuals. This release
17
  contains the final checkpoint of the 0.6B, 180B-token training run.
18
  It is a **base pretrained model**, intended for text completion, which can be used for mid-training and post-training.
19
 
20
- Training code: [ykyaol7/ARLoop](https://github.com/ykyaol7/ARLoop).
21
 
22
  ## Load and generate
23
 
@@ -71,3 +73,17 @@ Model release: [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
71
  | Optimizer step | Training tokens | Revision |
72
  | ---: | ---: | --- |
73
  | 171,660 (final) | 179,998,556,160 | [main](https://huggingface.co/osunlp/ARLoop-0.6B/tree/main) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  # ARLoop-0.6B
15
 
16
+ 馃摎 [Paper](https://arxiv.org/abs/2611.05234) 路 馃捇 [Code](https://github.com/OSU-NLP-Group/ARLoop)
17
+
18
  ARLoop is a fully-open pretrained looped transformer with attention residuals. This release
19
  contains the final checkpoint of the 0.6B, 180B-token training run.
20
  It is a **base pretrained model**, intended for text completion, which can be used for mid-training and post-training.
21
 
22
+ ![Figure 1: ARLoop benchmark results](assets/figure1.png)
23
 
24
  ## Load and generate
25
 
 
73
  | Optimizer step | Training tokens | Revision |
74
  | ---: | ---: | --- |
75
  | 171,660 (final) | 179,998,556,160 | [main](https://huggingface.co/osunlp/ARLoop-0.6B/tree/main) |
76
+
77
+ ## Citation
78
+
79
+ ```bibtex
80
+ @misc{yao2026arloop,
81
+ title={{ARLoop}: Fully-Open Pre-trained Looped Transformers with Attention Residuals},
82
+ author={Yuekun Yao and Yupei Du and Dawei Zhu and Huan Sun},
83
+ year={2026},
84
+ eprint={2611.05234},
85
+ archivePrefix={arXiv},
86
+ primaryClass={cs.CL},
87
+ url={https://arxiv.org/abs/2611.05234}
88
+ }
89
+ ```
assets/figure1.png ADDED

Git LFS Details

  • SHA256: 2bd2c2012dc1ad598b4560a07e3008ec28a8f91b091904be781b490945d5ef7e
  • Pointer size: 131 Bytes
  • Size of remote file: 335 kB