Training in progress, step 10

24_10_23_config_test_4.csv

@@ -3,7 +3,7 @@ run_number,comment,peformed_already,num_train_epochs,gradient_accumulation_steps
 2,num_train_epochs set 1,True,1,4,100,0.8,False,30
 3,num_train_epochs set 4,True,4,4,100,0.8,False,30
 4,gradient_accumulation_steps set 2,True,2,2,100,0.8,False,30
-5,gradient_accumulation_steps set 8,False,2,8,100,0.8,False,30
+5,gradient_accumulation_steps set 8,True,2,8,100,0.8,False,30
 6,temperature set 0.1,False,2,4,100,0.1,False,30
 7,temperature set 0.5,False,2,4,100,0.5,False,30
 8,temperature set 0.95,False,2,4,100,0.95,False,30

24_10_23_config_test_5.csv

@@ -0,0 +1,12 @@
+run_number,comment,peformed_already,num_train_epochs,max_tokens,temperature,stop_token
+1,no variations,False,2,100,0.8,False
+2,num_train_epochs set 1,False,1,100,0.8,False
+3,num_train_epochs set 4,False,4,100,0.8,False
+4,temperature set 0.1,False,2,100,0.1,False
+5,temperature set 0.5,False,2,100,0.5,False
+6,temperature set 0.95,False,2,100,0.95,False
+7,max_tokens set 10,False,2,10,0.8,False
+8,max_tokens set 30,False,2,30,0.8,False
+9,max_tokens set 300,False,2,300,0.8,False
+10,max_tokens set 500,False,2,500,0.8,False
+11,stop_token set True,False,2,100,0.8,True

24_10_23_results_1st_group.csv

@@ -8,4 +8,4 @@ run_number,items_per_minute,changed_settings,total_time_taken,rouge_scores_unnes
 7,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
 8,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
 9,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
-10,0.0,0,0.0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+10,1525.1022178667051,gradient_accumulation_steps set 8,17.310315132141113,0,0.132317782968674,0.207878097570942,0.1524098451996827,0.1626083307083888,0.2414802543435919,0.1813134900045184,0.1979383827408445,0.2753852049253097,0.2149063654245163,0.0290823619146671,0.0393260680197094,0.0318314354838573,0.0436344688525079,0.0564713765649392,0.0469607663722504,0.0595208106477057,0.0747255707316406,0.063041904733996

24_10_23_results_real.csv

@@ -0,0 +1,4 @@
+run_number,items_per_minute,changed_settings,total_time_taken,rouge_scores_unnest,rouge1 low Precision,rouge1 low Recall,rouge1 low F1 Score,rouge1 mid Precision,rouge1 mid Recall,rouge1 mid F1 Score,rouge1 high Precision,rouge1 high Recall,rouge1 high F1 Score,rouge2 low Precision,rouge2 low Recall,rouge2 low F1 Score,rouge2 mid Precision,rouge2 mid Recall,rouge2 mid F1 Score,rouge2 high Precision,rouge2 high Recall,rouge2 high F1 Score
+1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

README.md

@@ -436,6 +436,18 @@ The following `bitsandbytes` quantization config was used during training:
 - bnb_4bit_use_double_quant: True
 - bnb_4bit_compute_dtype: bfloat16
 
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: bitsandbytes
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: bfloat16
+
 The following `bitsandbytes` quantization config was used during training:
 - quant_method: bitsandbytes
 - load_in_8bit: False
@@ -485,5 +497,6 @@ The following `bitsandbytes` quantization config was used during training:
 - PEFT 0.5.0
 - PEFT 0.5.0
 - PEFT 0.5.0
+- PEFT 0.5.0
 
 - PEFT 0.5.0

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a395cfa9f9a17c2146cd44791f7fc08a7fd2e43cf660ec73ed33df0641e92c7a
+oid sha256:ce50a99489007cb683aaab24b3e9c744000be082472a49e05d45be207fa7b561
 size 100733709

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:87b9ed10847317ae86f52d8d365efbb83dbd73a0b37e8742fa26e66cbe3c2a37
+oid sha256:568aec43b2cefbad0e93f274bc26965dd7f3b69ac59a3cd86f73b9b69f5e6665
 size 100690288

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c63567f2a9a248c3669fa87952ba8df0323c154efb70cf9491c3a4e9945bbaad
+oid sha256:42c5b5d430ba0688eb0074439f7fa811ba56ece135f1cdc9eaf721376238a565
 size 4283