retnet-summarization

README.md

@@ -1,4 +1,5 @@
 ---
+base_model: kaizerBox/retnet-summarization
 tags:
 - generated_from_trainer
 datasets:
@@ -13,9 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # retnet-summarization
 
-This model is a fine-tuned version of [](https://huggingface.co/) on the xsum dataset.
+This model is a fine-tuned version of [kaizerBox/retnet-summarization](https://huggingface.co/kaizerBox/retnet-summarization) on the xsum dataset.
 It achieves the following results on the evaluation set:
-- Loss: 3.8453
+- Loss: 3.5369
 
 ## Model description
 
@@ -35,11 +36,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0006
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 4
+- eval_batch_size: 4
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
@@ -50,7 +51,7 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step  | Validation Loss |
 |:-------------:|:-----:|:-----:|:---------------:|
-| 4.3722        | 1.0   | 12500 | 3.8453          |
+| 3.8204        | 1.0   | 11525 | 3.5369          |
 
 
 ### Framework versions

config.json

@@ -1,4 +1,5 @@
 {
+  "_name_or_path": "kaizerBox/retnet-summarization",
   "activation_dropout": 0.0,
   "activation_fn": "swish",
   "architectures": [

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:621100b7d82b0c756b370ff4e4ad210f296a81bc20b0903f644049236b779c35
+oid sha256:a41442c27bf6db723cfe05a19d82cdb0d30fecf265fb84941eaee9a4b6303bf1
 size 282181632

runs/Nov05_18-25-27_8ad2684ce4a5/events.out.tfevents.1699208727.8ad2684ce4a5.288.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b218e96bf256f97537ca7888e882b375f5a4547b523ea62d9dbd1816784cd2a
+size 5355

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6758a222c5a8fb23ed432a5ecdcba3b77e388b262c786457940802f927879730
+oid sha256:10be0b4f194b73d381f41e5bb8505f054bdf8254997b9d3c7da91c5f92748c5f
 size 4600