Add pipeline tag, library name and Github link
Browse filesThis PR adds the `pipeline_tag` and `library_name` to the model card metadata. The `pipeline_tag` is set to `text-generation` because PLM is a causal language model. The `library_name` is set to `transformers` because the provided usage examples utilize the `transformers` library.
This PR adds an explicit link to the Github code.
README.md
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
datasets:
|
| 4 |
- HuggingFaceFW/fineweb-edu
|
| 5 |
- mlfoundations/dclm-baseline-1.0
|
|
@@ -7,6 +6,9 @@ datasets:
|
|
| 7 |
language:
|
| 8 |
- en
|
| 9 |
- zh
|
|
|
|
|
|
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
<center>
|
|
@@ -23,6 +25,8 @@ language:
|
|
| 23 |
|
| 24 |
</center>
|
| 25 |
|
|
|
|
|
|
|
| 26 |
---
|
| 27 |
|
| 28 |
The PLM (Peripheral Language Model) series introduces a novel model architecture to peripheral computing by delivering powerful language capabilities within the constraints of resource-limited devices. Through modeling and system co-design strategy, PLM optimizes model performance and fits edge system requirements, PLM employs **Multi-head Latent Attention** and **squared ReLU** activation to achieve sparsity, significantly reducing memory footprint and computational demands. Coupled with a meticulously crafted training regimen using curated datasets and a Warmup-Stable-Decay-Constant learning rate scheduler, PLM demonstrates superior performance compared to existing small language models, all while maintaining the lowest activated parameters, making it ideally suited for deployment on diverse peripheral platforms like mobile phones and Raspberry Pis.
|
|
@@ -179,4 +183,4 @@ If you find **Project PLM** helpful for your research or applications, please ci
|
|
| 179 |
archivePrefix={arXiv},
|
| 180 |
primaryClass={cs.CL},
|
| 181 |
}
|
| 182 |
-
```
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
datasets:
|
| 3 |
- HuggingFaceFW/fineweb-edu
|
| 4 |
- mlfoundations/dclm-baseline-1.0
|
|
|
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
- zh
|
| 9 |
+
license: mit
|
| 10 |
+
pipeline_tag: text-generation
|
| 11 |
+
library_name: transformers
|
| 12 |
---
|
| 13 |
|
| 14 |
<center>
|
|
|
|
| 25 |
|
| 26 |
</center>
|
| 27 |
|
| 28 |
+
Code: https://github.com/plm-team/PLM
|
| 29 |
+
|
| 30 |
---
|
| 31 |
|
| 32 |
The PLM (Peripheral Language Model) series introduces a novel model architecture to peripheral computing by delivering powerful language capabilities within the constraints of resource-limited devices. Through modeling and system co-design strategy, PLM optimizes model performance and fits edge system requirements, PLM employs **Multi-head Latent Attention** and **squared ReLU** activation to achieve sparsity, significantly reducing memory footprint and computational demands. Coupled with a meticulously crafted training regimen using curated datasets and a Warmup-Stable-Decay-Constant learning rate scheduler, PLM demonstrates superior performance compared to existing small language models, all while maintaining the lowest activated parameters, making it ideally suited for deployment on diverse peripheral platforms like mobile phones and Raspberry Pis.
|
|
|
|
| 183 |
archivePrefix={arXiv},
|
| 184 |
primaryClass={cs.CL},
|
| 185 |
}
|
| 186 |
+
```
|