error577
/

c66869bd-b0db-4fd7-92a8-55ec8caaeca5

@@ -42,7 +42,7 @@ deepspeed: null
 early_stopping_patience: null
 eval_max_new_tokens: 128
 eval_table_size: null
-evals_per_epoch: 2
 flash_attention: false
 fp16: null
 fsdp: null
@@ -54,7 +54,7 @@ hub_model_id: error577/c66869bd-b0db-4fd7-92a8-55ec8caaeca5
 hub_repo: null
 hub_strategy: checkpoint
 hub_token: null
-learning_rate: 0.0001
 load_in_4bit: true
 load_in_8bit: false
 local_rank: null
@@ -66,7 +66,7 @@ lora_model_dir: null
 lora_r: 32
 lora_target_linear: true
 lr_scheduler: cosine
-max_steps: 100
 micro_batch_size: 1
 mlflow_experiment_name: /tmp/4216a06bafbd436d_train_data.json
 model_type: AutoModelForCausalLM
@@ -86,7 +86,7 @@ tf32: false
 tokenizer_type: AutoTokenizer
 train_on_inputs: false
 trust_remote_code: true
-val_set_size: 0.02
 wandb_entity: null
 wandb_mode: online
 wandb_name: b3db347c-a6bb-4682-9904-41bfd73f101a
@@ -105,7 +105,7 @@ xformers_attention: null
 This model is a fine-tuned version of [tiiuae/falcon-rw-1b](https://huggingface.co/tiiuae/falcon-rw-1b) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.0347
 ## Model description
@@ -124,7 +124,7 @@ More information needed
 ### Training hyperparameters
 The following hyperparameters were used during training:
-- learning_rate: 0.0001
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
@@ -133,15 +133,13 @@ The following hyperparameters were used during training:
 - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
-- training_steps: 100
 ### Training results
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 14.038        | 0.0003 | 1    | 4.1484          |
-| 6.194         | 0.0125 | 50   | 1.1684          |
-| 3.5887        | 0.0250 | 100  | 1.0347          |
 ### Framework versions

 early_stopping_patience: null
 eval_max_new_tokens: 128
 eval_table_size: null
+evals_per_epoch: 1
 flash_attention: false
 fp16: null
 fsdp: null
 hub_repo: null
 hub_strategy: checkpoint
 hub_token: null
+learning_rate: 0.0002
 load_in_4bit: true
 load_in_8bit: false
 local_rank: null
 lora_r: 32
 lora_target_linear: true
 lr_scheduler: cosine
+max_steps: 1000
 micro_batch_size: 1
 mlflow_experiment_name: /tmp/4216a06bafbd436d_train_data.json
 model_type: AutoModelForCausalLM
 tokenizer_type: AutoTokenizer
 train_on_inputs: false
 trust_remote_code: true
+val_set_size: 0.05
 wandb_entity: null
 wandb_mode: online
 wandb_name: b3db347c-a6bb-4682-9904-41bfd73f101a
 This model is a fine-tuned version of [tiiuae/falcon-rw-1b](https://huggingface.co/tiiuae/falcon-rw-1b) on the None dataset.
 It achieves the following results on the evaluation set:
+- Loss: 0.5828
 ## Model description
 ### Training hyperparameters
 The following hyperparameters were used during training:
+- learning_rate: 0.0002
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
 - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
+- training_steps: 1000
 ### Training results
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
+| 2.3709        | 0.2582 | 1000 | 0.5828          |
 ### Framework versions

adapter_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7f9564c5fe6c5c328759972eb5608fbe1be7c189865d00f053d876799cfd6454
 size 100734154

 version https://git-lfs.github.com/spec/v1
+oid sha256:e8ff9d4f0dc66f82ab6dd3907a11368d36b9eaf5ad5914f46e1648b5a900f436
 size 100734154