| license: apache-2.0 | |
| library_name: transformers | |
| # EarlyCheckpoint | |
| <!-- markdownlint-disable first-line-h1 --> | |
| <!-- markdownlint-disable html --> | |
| <!-- markdownlint-disable no-duplicate-header --> | |
| <div align="center"> | |
| <img src="figures/fig1.png" width="60%" alt="EarlyCheckpoint" /> | |
| </div> | |
| <hr> | |
| <div align="center" style="line-height: 1;"> | |
| <a href="LICENSE" style="margin: 2px;"> | |
| <img alt="License" src="figures/fig2.png" style="display: inline-block; vertical-align: middle;"/> | |
| </a> | |
| </div> | |
| ## 1. Introduction | |
| EarlyCheckpoint is the first saved checkpoint from our training run, captured at the very beginning of training. It serves as a baseline for comparing training progress. | |
| <p align="center"> | |
| <img width="80%" src="figures/fig3.png"> | |
| </p> | |
| This model represents the initial state of training and is useful for ablation studies and understanding training dynamics. | |
| ## 2. Model Information | |
| | Property | Value | | |
| |---|---| | |
| | Architecture | BERT | | |
| | Training Step | step_100 | | |
| | License | Apache-2.0 | | |
| ## 3. How to Use | |
| ```python | |
| from transformers import AutoModel, AutoTokenizer | |
| model = AutoModel.from_pretrained("EarlyCheckpoint-v1") | |
| tokenizer = AutoTokenizer.from_pretrained("EarlyCheckpoint-v1") | |
| ``` | |
| ## 4. License | |
| This model is licensed under the [Apache-2.0 License](LICENSE). | |
| ## 5. Contact | |
| Open an issue on our GitHub for questions. | |