Hugging Face's logo

kaizerBox
/
retnet-summarization_small

Text Generation
Transformers
TensorBoard
Safetensors
retnet
Generated from Trainer
Model card Files
xet
Metrics Community
kaizerBox commited on
Commit
1696782
·
1 Parent(s): eb4eafb

retnet-summarization-small

Browse files
Files changed (5) hide show
  1. README.md +11 -9
  2. config.json +1 -0
  3. model.safetensors +1 -1
  4. runs/Nov13_00-50-05_b14393ba9044/events.out.tfevents.1699836605.b14393ba9044.648.0 +3 -0
  5. training_args.bin +1 -1
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
  datasets:
@@ -13,9 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  # retnet-summarization_small
15
 
16
- This model is a fine-tuned version of [](https://huggingface.co/) on the xsum dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 5.1335
19
 
20
  ## Model description
21
 
@@ -35,22 +36,23 @@ More information needed
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 0.0006
38
- - train_batch_size: 4
39
- - eval_batch_size: 4
40
  - seed: 42
41
  - gradient_accumulation_steps: 4
42
- - total_train_batch_size: 16
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: cosine
45
  - lr_scheduler_warmup_steps: 10
46
- - num_epochs: 1
47
  - mixed_precision_training: Native AMP
48
 
49
  ### Training results
50
 
51
- | Training Loss | Epoch | Step | Validation Loss |
52
- |:-------------:|:-----:|:-----:|:---------------:|
53
- | 5.606 | 1.0 | 11525 | 5.1335 |
 
54
 
55
 
56
  ### Framework versions
 
1
  ---
2
+ base_model: kaizerBox/retnet-summarization_small
3
  tags:
4
  - generated_from_trainer
5
  datasets:
 
14
 
15
  # retnet-summarization_small
16
 
17
+ This model is a fine-tuned version of [kaizerBox/retnet-summarization_small](https://huggingface.co/kaizerBox/retnet-summarization_small) on the xsum dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 4.8029
20
 
21
  ## Model description
22
 
 
36
 
37
  The following hyperparameters were used during training:
38
  - learning_rate: 0.0006
39
+ - train_batch_size: 10
40
+ - eval_batch_size: 10
41
  - seed: 42
42
  - gradient_accumulation_steps: 4
43
+ - total_train_batch_size: 40
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: cosine
46
  - lr_scheduler_warmup_steps: 10
47
+ - num_epochs: 2
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
+ | Training Loss | Epoch | Step | Validation Loss |
53
+ |:-------------:|:-----:|:----:|:---------------:|
54
+ | 5.1425 | 1.0 | 4610 | 4.8581 |
55
+ | 4.9561 | 2.0 | 9220 | 4.8029 |
56
 
57
 
58
  ### Framework versions
config.json CHANGED
@@ -1,4 +1,5 @@
1
  {
 
2
  "activation_dropout": 0.0,
3
  "activation_fn": "swish",
4
  "architectures": [
 
1
  {
2
+ "_name_or_path": "kaizerBox/retnet-summarization_small",
3
  "activation_dropout": 0.0,
4
  "activation_fn": "swish",
5
  "architectures": [
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:25ae8604af53a3ee320071c5154d6e1d82bde7ce62cfd5a1bdd154646d54e5a3
3
  size 53857264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f55c789d7d551778764f51a1a6d28020dd1e3963c48a44304561a21d55b5fc6
3
  size 53857264
runs/Nov13_00-50-05_b14393ba9044/events.out.tfevents.1699836605.b14393ba9044.648.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4fa2c6cba6cd38461c6718e67e64980d1ba28094bf2f6f847aa91bd910fc0f0c
3
+ size 5811
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7cf246cae8cc6d428c85463086983fbe3337dec9b6b9912a24e85fc82cbd653b
3
  size 4600
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90af1830ea759ee67ea81e2d31abd11854174c66ef2b9be8c8e99971f6ca87fd
3
  size 4600