File size: 2,270 Bytes
aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 aad889b 92b3ea9 c0fc7f7 252b56f 0a26fdd d9dbcf1 cd40e90 cc4a96b 24c36dd cc811c4 f73f9ef 5e17d96 6fe4697 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
language: en
tags:
- fill-mask
---
## Environmental Impact (CODE CARBON DEFAULT)
| Metric | Value |
|--------------------------|---------------------------------|
| Duration (in seconds) | [More Information Needed] |
| Emissions (Co2eq in kg) | [More Information Needed] |
| CPU power (W) | [NO CPU] |
| GPU power (W) | [No GPU] |
| RAM power (W) | [More Information Needed] |
| CPU energy (kWh) | [No CPU] |
| GPU energy (kWh) | [No GPU] |
| RAM energy (kWh) | [More Information Needed] |
| Consumed energy (kWh) | [More Information Needed] |
| Country name | [More Information Needed] |
| Cloud provider | [No Cloud] |
| Cloud region | [No Cloud] |
| CPU count | [No CPU] |
| CPU model | [No CPU] |
| GPU count | [No GPU] |
| GPU model | [No GPU] |
## Environmental Impact (for one core)
| Metric | Value |
|--------------------------|---------------------------------|
| CPU energy (kWh) | [No CPU] |
| Emissions (Co2eq in kg) | [More Information Needed] |
## Note
5 juillet 2024 !
## My Config
| Config | Value |
|--------------------------|-----------------|
| checkpoint | albert-base-v2 |
| model_name | fp_bs8_lr2e4_x2 |
| sequence_length | 400 |
| num_epoch | 6 |
| learning_rate | 0.0002 |
| batch_size | 8 |
| weight_decay | 0.0 |
| warm_up_prop | 0.0 |
| drop_out_prob | 0.1 |
| packing_length | 100 |
| train_test_split | 0.2 |
| num_steps | 83270 |
## Training and Testing steps
Epoch | Train Loss | Test Loss
---|---|---
| 0.0 | 18.083176 | 11.266975 |
| 0.5 | 7.142726 | 7.066952 |
| 1.0 | 7.035545 | 7.021196 |
| 1.5 | 6.995318 | 7.026562 |
| 2.0 | 6.978820 | 6.988426 |
| 2.5 | 6.975003 | 6.972350 |
| 3.0 | 6.974181 | 6.971103 |
| 3.5 | 6.967837 | 6.964300 |
| 4.0 | 6.965952 | 6.966316 |
| 4.5 | 6.964535 | 6.966458 |
| 5.0 | 6.956336 | 6.965864 |
| 5.5 | 6.958633 | 6.957286 |
|