Model save

README.md

@@ -0,0 +1,70 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- generated_from_trainer
+base_model: Falconsai/text_summarization
+metrics:
+- rouge
+model-index:
+- name: model
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# model
+
+This model is a fine-tuned version of [Falconsai/text_summarization](https://huggingface.co/Falconsai/text_summarization) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.5828
+- Rouge1: 0.066
+- Rouge2: 0.0111
+- Rougel: 0.0546
+- Rougelsum: 0.0546
+- Gen Len: 20.0
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
+| 4.3163        | 1.0   | 600  | 3.6773          | 0.0623 | 0.0095 | 0.0514 | 0.0513    | 20.0    |
+| 4.0156        | 2.0   | 1200 | 3.6124          | 0.0646 | 0.0105 | 0.0534 | 0.0533    | 20.0    |
+| 3.9641        | 3.0   | 1800 | 3.5887          | 0.0659 | 0.0111 | 0.0546 | 0.0545    | 20.0    |
+| 3.931         | 4.0   | 2400 | 3.5828          | 0.066  | 0.0111 | 0.0546 | 0.0546    | 20.0    |
+
+
+### Framework versions
+
+- PEFT 0.10.1.dev0
+- Transformers 4.39.1
+- Pytorch 2.2.1
+- Datasets 2.18.0
+- Tokenizers 0.15.2

adapter_config.json

@@ -20,10 +20,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "v",
-    "o",
+    "k",
     "q",
-    "k"
+    "v",
+    "o"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4bb5057e2badb417ee089339af56f1a6ab82fd1d69ed996d448a1c6d83cdd989
+oid sha256:814e787ab3afbf14f475e652e06419d5f097585d552aee220f7240caad1f2dac
 size 9457304

tokenizer.json

@@ -2,7 +2,7 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 128,
+    "max_length": 512,
     "strategy": "LongestFirst",
     "stride": 0
   },

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:440635b622791efa901046be5414e9b48fe8cd34e36bc9b30471cf76dedf8a21
+oid sha256:23190c98d14958fd7d0c9050bad987a8aa8792dd8af315f2c1fabea65798727d
 size 5048