Upload folder using huggingface_hub

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91c06f737106f7e57ee2455e1f5d0063689f933e83cc030ce2e8a1b6ce5cfc8e
+size 80755416

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:917b99f233db5b167482674308604e0441a695b290de0dcab412ac18805a08df
+size 80755416

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cd7cdcf30e6a4fbfcdea1f8d846b8634f2c2c8d914aa69a0a7e2dd0f782b26e
+size 41136570

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebeededf8ffaf82b2dbfb12a803cb32cf9145afe1eef718cec919724550e9ddd
+size 14244

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d948d1bf3ec64239322ccdba33ef9035020aa64f970968ec5c4a6170c3bb88ce
+size 1064

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/trainer_state.json

@@ -0,0 +1,795 @@
+{
+  "best_metric": 1.2224010229110718,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-343",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 1029,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05830903790087463,
+      "grad_norm": 0.27093306183815,
+      "learning_rate": 0.0002,
+      "loss": 1.6238,
+      "step": 10
+    },
+    {
+      "epoch": 0.11661807580174927,
+      "grad_norm": 0.22656577825546265,
+      "learning_rate": 0.0002,
+      "loss": 1.3942,
+      "step": 20
+    },
+    {
+      "epoch": 0.1749271137026239,
+      "grad_norm": 0.2181931734085083,
+      "learning_rate": 0.0002,
+      "loss": 1.4224,
+      "step": 30
+    },
+    {
+      "epoch": 0.23323615160349853,
+      "grad_norm": 0.23278485238552094,
+      "learning_rate": 0.0002,
+      "loss": 1.3609,
+      "step": 40
+    },
+    {
+      "epoch": 0.2915451895043732,
+      "grad_norm": 0.28298142552375793,
+      "learning_rate": 0.0002,
+      "loss": 1.3352,
+      "step": 50
+    },
+    {
+      "epoch": 0.3498542274052478,
+      "grad_norm": 0.2782207727432251,
+      "learning_rate": 0.0002,
+      "loss": 1.3061,
+      "step": 60
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.24569624662399292,
+      "learning_rate": 0.0002,
+      "loss": 1.2937,
+      "step": 70
+    },
+    {
+      "epoch": 0.46647230320699706,
+      "grad_norm": 0.27325960993766785,
+      "learning_rate": 0.0002,
+      "loss": 1.2724,
+      "step": 80
+    },
+    {
+      "epoch": 0.5247813411078717,
+      "grad_norm": 0.29733362793922424,
+      "learning_rate": 0.0002,
+      "loss": 1.3331,
+      "step": 90
+    },
+    {
+      "epoch": 0.5830903790087464,
+      "grad_norm": 0.24717216193675995,
+      "learning_rate": 0.0002,
+      "loss": 1.2704,
+      "step": 100
+    },
+    {
+      "epoch": 0.641399416909621,
+      "grad_norm": 0.30963554978370667,
+      "learning_rate": 0.0002,
+      "loss": 1.2832,
+      "step": 110
+    },
+    {
+      "epoch": 0.6997084548104956,
+      "grad_norm": 0.23280981183052063,
+      "learning_rate": 0.0002,
+      "loss": 1.2379,
+      "step": 120
+    },
+    {
+      "epoch": 0.7580174927113703,
+      "grad_norm": 0.25881555676460266,
+      "learning_rate": 0.0002,
+      "loss": 1.2787,
+      "step": 130
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.33120882511138916,
+      "learning_rate": 0.0002,
+      "loss": 1.3106,
+      "step": 140
+    },
+    {
+      "epoch": 0.8746355685131195,
+      "grad_norm": 0.2893398702144623,
+      "learning_rate": 0.0002,
+      "loss": 1.3491,
+      "step": 150
+    },
+    {
+      "epoch": 0.9329446064139941,
+      "grad_norm": 0.34311336278915405,
+      "learning_rate": 0.0002,
+      "loss": 1.3122,
+      "step": 160
+    },
+    {
+      "epoch": 0.9912536443148688,
+      "grad_norm": 0.3196379840373993,
+      "learning_rate": 0.0002,
+      "loss": 1.2457,
+      "step": 170
+    },
+    {
+      "epoch": 0.9970845481049563,
+      "eval_loss": 1.249076008796692,
+      "eval_runtime": 14.9505,
+      "eval_samples_per_second": 6.421,
+      "eval_steps_per_second": 0.803,
+      "step": 171
+    },
+    {
+      "epoch": 1.0495626822157433,
+      "grad_norm": 0.3274891972541809,
+      "learning_rate": 0.0002,
+      "loss": 1.272,
+      "step": 180
+    },
+    {
+      "epoch": 1.1078717201166182,
+      "grad_norm": 0.3209025263786316,
+      "learning_rate": 0.0002,
+      "loss": 1.2585,
+      "step": 190
+    },
+    {
+      "epoch": 1.1661807580174928,
+      "grad_norm": 0.29988062381744385,
+      "learning_rate": 0.0002,
+      "loss": 1.2164,
+      "step": 200
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.3197144567966461,
+      "learning_rate": 0.0002,
+      "loss": 1.219,
+      "step": 210
+    },
+    {
+      "epoch": 1.282798833819242,
+      "grad_norm": 0.3738149404525757,
+      "learning_rate": 0.0002,
+      "loss": 1.2431,
+      "step": 220
+    },
+    {
+      "epoch": 1.3411078717201166,
+      "grad_norm": 0.3867924213409424,
+      "learning_rate": 0.0002,
+      "loss": 1.1926,
+      "step": 230
+    },
+    {
+      "epoch": 1.3994169096209912,
+      "grad_norm": 0.31422024965286255,
+      "learning_rate": 0.0002,
+      "loss": 1.1673,
+      "step": 240
+    },
+    {
+      "epoch": 1.4577259475218658,
+      "grad_norm": 0.34369608759880066,
+      "learning_rate": 0.0002,
+      "loss": 1.1594,
+      "step": 250
+    },
+    {
+      "epoch": 1.5160349854227406,
+      "grad_norm": 0.3204766809940338,
+      "learning_rate": 0.0002,
+      "loss": 1.2399,
+      "step": 260
+    },
+    {
+      "epoch": 1.574344023323615,
+      "grad_norm": 0.3328384757041931,
+      "learning_rate": 0.0002,
+      "loss": 1.2248,
+      "step": 270
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.30920523405075073,
+      "learning_rate": 0.0002,
+      "loss": 1.2094,
+      "step": 280
+    },
+    {
+      "epoch": 1.6909620991253644,
+      "grad_norm": 0.3138664662837982,
+      "learning_rate": 0.0002,
+      "loss": 1.2603,
+      "step": 290
+    },
+    {
+      "epoch": 1.749271137026239,
+      "grad_norm": 0.32747137546539307,
+      "learning_rate": 0.0002,
+      "loss": 1.1932,
+      "step": 300
+    },
+    {
+      "epoch": 1.8075801749271136,
+      "grad_norm": 0.31668511033058167,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 310
+    },
+    {
+      "epoch": 1.8658892128279883,
+      "grad_norm": 0.3314826488494873,
+      "learning_rate": 0.0002,
+      "loss": 1.216,
+      "step": 320
+    },
+    {
+      "epoch": 1.924198250728863,
+      "grad_norm": 0.3154251277446747,
+      "learning_rate": 0.0002,
+      "loss": 1.1739,
+      "step": 330
+    },
+    {
+      "epoch": 1.9825072886297375,
+      "grad_norm": 0.3600097596645355,
+      "learning_rate": 0.0002,
+      "loss": 1.2476,
+      "step": 340
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2224010229110718,
+      "eval_runtime": 14.9032,
+      "eval_samples_per_second": 6.442,
+      "eval_steps_per_second": 0.805,
+      "step": 343
+    },
+    {
+      "epoch": 2.0408163265306123,
+      "grad_norm": 0.3134561777114868,
+      "learning_rate": 0.0002,
+      "loss": 1.1687,
+      "step": 350
+    },
+    {
+      "epoch": 2.0991253644314867,
+      "grad_norm": 0.3430805802345276,
+      "learning_rate": 0.0002,
+      "loss": 1.1658,
+      "step": 360
+    },
+    {
+      "epoch": 2.1574344023323615,
+      "grad_norm": 0.4015083909034729,
+      "learning_rate": 0.0002,
+      "loss": 1.0947,
+      "step": 370
+    },
+    {
+      "epoch": 2.2157434402332363,
+      "grad_norm": 0.43478450179100037,
+      "learning_rate": 0.0002,
+      "loss": 1.1208,
+      "step": 380
+    },
+    {
+      "epoch": 2.2740524781341107,
+      "grad_norm": 0.3541523814201355,
+      "learning_rate": 0.0002,
+      "loss": 1.1555,
+      "step": 390
+    },
+    {
+      "epoch": 2.3323615160349855,
+      "grad_norm": 0.40292826294898987,
+      "learning_rate": 0.0002,
+      "loss": 1.175,
+      "step": 400
+    },
+    {
+      "epoch": 2.39067055393586,
+      "grad_norm": 0.3743220567703247,
+      "learning_rate": 0.0002,
+      "loss": 1.0685,
+      "step": 410
+    },
+    {
+      "epoch": 2.4489795918367347,
+      "grad_norm": 0.3321108818054199,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 420
+    },
+    {
+      "epoch": 2.5072886297376096,
+      "grad_norm": 0.3650798499584198,
+      "learning_rate": 0.0002,
+      "loss": 1.1441,
+      "step": 430
+    },
+    {
+      "epoch": 2.565597667638484,
+      "grad_norm": 0.3603045642375946,
+      "learning_rate": 0.0002,
+      "loss": 1.157,
+      "step": 440
+    },
+    {
+      "epoch": 2.6239067055393583,
+      "grad_norm": 0.36125364899635315,
+      "learning_rate": 0.0002,
+      "loss": 1.0995,
+      "step": 450
+    },
+    {
+      "epoch": 2.682215743440233,
+      "grad_norm": 0.4070495367050171,
+      "learning_rate": 0.0002,
+      "loss": 1.1364,
+      "step": 460
+    },
+    {
+      "epoch": 2.740524781341108,
+      "grad_norm": 0.42682793736457825,
+      "learning_rate": 0.0002,
+      "loss": 1.1407,
+      "step": 470
+    },
+    {
+      "epoch": 2.7988338192419824,
+      "grad_norm": 0.3968621790409088,
+      "learning_rate": 0.0002,
+      "loss": 1.1808,
+      "step": 480
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.37401914596557617,
+      "learning_rate": 0.0002,
+      "loss": 1.1252,
+      "step": 490
+    },
+    {
+      "epoch": 2.9154518950437316,
+      "grad_norm": 0.42643091082572937,
+      "learning_rate": 0.0002,
+      "loss": 1.1094,
+      "step": 500
+    },
+    {
+      "epoch": 2.9737609329446064,
+      "grad_norm": 0.48727893829345703,
+      "learning_rate": 0.0002,
+      "loss": 1.1587,
+      "step": 510
+    },
+    {
+      "epoch": 2.997084548104956,
+      "eval_loss": 1.2232840061187744,
+      "eval_runtime": 14.6841,
+      "eval_samples_per_second": 6.538,
+      "eval_steps_per_second": 0.817,
+      "step": 514
+    },
+    {
+      "epoch": 3.0320699708454812,
+      "grad_norm": 0.4159005880355835,
+      "learning_rate": 0.0002,
+      "loss": 1.0599,
+      "step": 520
+    },
+    {
+      "epoch": 3.0903790087463556,
+      "grad_norm": 0.4462415874004364,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 530
+    },
+    {
+      "epoch": 3.1486880466472305,
+      "grad_norm": 0.47459903359413147,
+      "learning_rate": 0.0002,
+      "loss": 1.0135,
+      "step": 540
+    },
+    {
+      "epoch": 3.206997084548105,
+      "grad_norm": 0.47449398040771484,
+      "learning_rate": 0.0002,
+      "loss": 1.0083,
+      "step": 550
+    },
+    {
+      "epoch": 3.2653061224489797,
+      "grad_norm": 0.48261985182762146,
+      "learning_rate": 0.0002,
+      "loss": 1.0698,
+      "step": 560
+    },
+    {
+      "epoch": 3.323615160349854,
+      "grad_norm": 0.48770663142204285,
+      "learning_rate": 0.0002,
+      "loss": 1.0553,
+      "step": 570
+    },
+    {
+      "epoch": 3.381924198250729,
+      "grad_norm": 0.478135883808136,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 580
+    },
+    {
+      "epoch": 3.4402332361516033,
+      "grad_norm": 0.47697558999061584,
+      "learning_rate": 0.0002,
+      "loss": 1.0214,
+      "step": 590
+    },
+    {
+      "epoch": 3.498542274052478,
+      "grad_norm": 0.4764975607395172,
+      "learning_rate": 0.0002,
+      "loss": 1.0644,
+      "step": 600
+    },
+    {
+      "epoch": 3.556851311953353,
+      "grad_norm": 0.5096248388290405,
+      "learning_rate": 0.0002,
+      "loss": 1.0729,
+      "step": 610
+    },
+    {
+      "epoch": 3.6151603498542273,
+      "grad_norm": 0.42080581188201904,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 620
+    },
+    {
+      "epoch": 3.673469387755102,
+      "grad_norm": 0.4736129641532898,
+      "learning_rate": 0.0002,
+      "loss": 1.0424,
+      "step": 630
+    },
+    {
+      "epoch": 3.7317784256559765,
+      "grad_norm": 0.46505600214004517,
+      "learning_rate": 0.0002,
+      "loss": 1.0474,
+      "step": 640
+    },
+    {
+      "epoch": 3.7900874635568513,
+      "grad_norm": 0.44950205087661743,
+      "learning_rate": 0.0002,
+      "loss": 1.0535,
+      "step": 650
+    },
+    {
+      "epoch": 3.848396501457726,
+      "grad_norm": 0.4537162780761719,
+      "learning_rate": 0.0002,
+      "loss": 1.0677,
+      "step": 660
+    },
+    {
+      "epoch": 3.9067055393586005,
+      "grad_norm": 0.4871664345264435,
+      "learning_rate": 0.0002,
+      "loss": 1.0871,
+      "step": 670
+    },
+    {
+      "epoch": 3.9650145772594754,
+      "grad_norm": 0.42579060792922974,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 680
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.235727071762085,
+      "eval_runtime": 14.6921,
+      "eval_samples_per_second": 6.534,
+      "eval_steps_per_second": 0.817,
+      "step": 686
+    },
+    {
+      "epoch": 4.02332361516035,
+      "grad_norm": 0.39996492862701416,
+      "learning_rate": 0.0002,
+      "loss": 1.064,
+      "step": 690
+    },
+    {
+      "epoch": 4.081632653061225,
+      "grad_norm": 0.6311050057411194,
+      "learning_rate": 0.0002,
+      "loss": 0.9566,
+      "step": 700
+    },
+    {
+      "epoch": 4.139941690962099,
+      "grad_norm": 0.5494613647460938,
+      "learning_rate": 0.0002,
+      "loss": 0.9549,
+      "step": 710
+    },
+    {
+      "epoch": 4.198250728862973,
+      "grad_norm": 0.689102828502655,
+      "learning_rate": 0.0002,
+      "loss": 0.9251,
+      "step": 720
+    },
+    {
+      "epoch": 4.256559766763848,
+      "grad_norm": 0.4727540910243988,
+      "learning_rate": 0.0002,
+      "loss": 0.9362,
+      "step": 730
+    },
+    {
+      "epoch": 4.314868804664723,
+      "grad_norm": 0.5162383317947388,
+      "learning_rate": 0.0002,
+      "loss": 1.0063,
+      "step": 740
+    },
+    {
+      "epoch": 4.373177842565598,
+      "grad_norm": 0.5838354229927063,
+      "learning_rate": 0.0002,
+      "loss": 0.9488,
+      "step": 750
+    },
+    {
+      "epoch": 4.431486880466473,
+      "grad_norm": 0.5268456935882568,
+      "learning_rate": 0.0002,
+      "loss": 0.9397,
+      "step": 760
+    },
+    {
+      "epoch": 4.489795918367347,
+      "grad_norm": 0.6695316433906555,
+      "learning_rate": 0.0002,
+      "loss": 0.9586,
+      "step": 770
+    },
+    {
+      "epoch": 4.548104956268221,
+      "grad_norm": 0.5652480721473694,
+      "learning_rate": 0.0002,
+      "loss": 0.9637,
+      "step": 780
+    },
+    {
+      "epoch": 4.606413994169096,
+      "grad_norm": 0.4949062168598175,
+      "learning_rate": 0.0002,
+      "loss": 0.9678,
+      "step": 790
+    },
+    {
+      "epoch": 4.664723032069971,
+      "grad_norm": 0.5157344341278076,
+      "learning_rate": 0.0002,
+      "loss": 0.9487,
+      "step": 800
+    },
+    {
+      "epoch": 4.723032069970845,
+      "grad_norm": 0.5152610540390015,
+      "learning_rate": 0.0002,
+      "loss": 0.972,
+      "step": 810
+    },
+    {
+      "epoch": 4.78134110787172,
+      "grad_norm": 0.540663480758667,
+      "learning_rate": 0.0002,
+      "loss": 0.9762,
+      "step": 820
+    },
+    {
+      "epoch": 4.839650145772595,
+      "grad_norm": 0.6327324509620667,
+      "learning_rate": 0.0002,
+      "loss": 0.9884,
+      "step": 830
+    },
+    {
+      "epoch": 4.8979591836734695,
+      "grad_norm": 0.5663048028945923,
+      "learning_rate": 0.0002,
+      "loss": 0.9656,
+      "step": 840
+    },
+    {
+      "epoch": 4.956268221574344,
+      "grad_norm": 0.5561565160751343,
+      "learning_rate": 0.0002,
+      "loss": 0.9956,
+      "step": 850
+    },
+    {
+      "epoch": 4.997084548104956,
+      "eval_loss": 1.2652570009231567,
+      "eval_runtime": 14.6946,
+      "eval_samples_per_second": 6.533,
+      "eval_steps_per_second": 0.817,
+      "step": 857
+    },
+    {
+      "epoch": 5.014577259475218,
+      "grad_norm": 0.6206780672073364,
+      "learning_rate": 0.0002,
+      "loss": 0.9756,
+      "step": 860
+    },
+    {
+      "epoch": 5.072886297376093,
+      "grad_norm": 0.7620121836662292,
+      "learning_rate": 0.0002,
+      "loss": 0.8336,
+      "step": 870
+    },
+    {
+      "epoch": 5.131195335276968,
+      "grad_norm": 0.744425892829895,
+      "learning_rate": 0.0002,
+      "loss": 0.8825,
+      "step": 880
+    },
+    {
+      "epoch": 5.189504373177843,
+      "grad_norm": 0.6208587288856506,
+      "learning_rate": 0.0002,
+      "loss": 0.8503,
+      "step": 890
+    },
+    {
+      "epoch": 5.247813411078718,
+      "grad_norm": 0.6400560736656189,
+      "learning_rate": 0.0002,
+      "loss": 0.8387,
+      "step": 900
+    },
+    {
+      "epoch": 5.3061224489795915,
+      "grad_norm": 0.6354245543479919,
+      "learning_rate": 0.0002,
+      "loss": 0.8454,
+      "step": 910
+    },
+    {
+      "epoch": 5.364431486880466,
+      "grad_norm": 0.6441580057144165,
+      "learning_rate": 0.0002,
+      "loss": 0.8158,
+      "step": 920
+    },
+    {
+      "epoch": 5.422740524781341,
+      "grad_norm": 0.8845874667167664,
+      "learning_rate": 0.0002,
+      "loss": 0.8509,
+      "step": 930
+    },
+    {
+      "epoch": 5.481049562682216,
+      "grad_norm": 0.7521125078201294,
+      "learning_rate": 0.0002,
+      "loss": 0.9311,
+      "step": 940
+    },
+    {
+      "epoch": 5.539358600583091,
+      "grad_norm": 0.6190000772476196,
+      "learning_rate": 0.0002,
+      "loss": 0.852,
+      "step": 950
+    },
+    {
+      "epoch": 5.597667638483965,
+      "grad_norm": 0.6528187990188599,
+      "learning_rate": 0.0002,
+      "loss": 0.9208,
+      "step": 960
+    },
+    {
+      "epoch": 5.65597667638484,
+      "grad_norm": 0.7539762258529663,
+      "learning_rate": 0.0002,
+      "loss": 0.886,
+      "step": 970
+    },
+    {
+      "epoch": 5.714285714285714,
+      "grad_norm": 0.6197044253349304,
+      "learning_rate": 0.0002,
+      "loss": 0.8631,
+      "step": 980
+    },
+    {
+      "epoch": 5.772594752186589,
+      "grad_norm": 0.569410502910614,
+      "learning_rate": 0.0002,
+      "loss": 0.8266,
+      "step": 990
+    },
+    {
+      "epoch": 5.830903790087463,
+      "grad_norm": 0.6476598978042603,
+      "learning_rate": 0.0002,
+      "loss": 0.8832,
+      "step": 1000
+    },
+    {
+      "epoch": 5.889212827988338,
+      "grad_norm": 0.6748721599578857,
+      "learning_rate": 0.0002,
+      "loss": 0.8817,
+      "step": 1010
+    },
+    {
+      "epoch": 5.947521865889213,
+      "grad_norm": 0.67653489112854,
+      "learning_rate": 0.0002,
+      "loss": 0.9131,
+      "step": 1020
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3123321533203125,
+      "eval_runtime": 14.6876,
+      "eval_samples_per_second": 6.536,
+      "eval_steps_per_second": 0.817,
+      "step": 1029
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1368,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.482931228567142e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1029/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:558d8475370dfee2c47b98ed05aca969bb536c25048d37c81a11582fa4f0e57c
+size 5560

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e7724c121f7b51b4e71001fc652cb9e93e89324beed2c229a849dab25014fc6
+size 80755416

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52add32659cacaec5607b2a33f7acbce316ea365be5275a0a0f9cc3c079a891f
+size 41136570

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ea16b1bc2c8109467436d51df832f0aac3da5eb702e5409270c356f9b9061ce
+size 14244

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2316ad5425e5b3ab31a35a8544e3379804230ad21119fcc05a3d896d9d05594f
+size 1064

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/trainer_state.json

@@ -0,0 +1,929 @@
+{
+  "best_metric": 1.2224010229110718,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-343",
+  "epoch": 6.997084548104956,
+  "eval_steps": 10,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05830903790087463,
+      "grad_norm": 0.27093306183815,
+      "learning_rate": 0.0002,
+      "loss": 1.6238,
+      "step": 10
+    },
+    {
+      "epoch": 0.11661807580174927,
+      "grad_norm": 0.22656577825546265,
+      "learning_rate": 0.0002,
+      "loss": 1.3942,
+      "step": 20
+    },
+    {
+      "epoch": 0.1749271137026239,
+      "grad_norm": 0.2181931734085083,
+      "learning_rate": 0.0002,
+      "loss": 1.4224,
+      "step": 30
+    },
+    {
+      "epoch": 0.23323615160349853,
+      "grad_norm": 0.23278485238552094,
+      "learning_rate": 0.0002,
+      "loss": 1.3609,
+      "step": 40
+    },
+    {
+      "epoch": 0.2915451895043732,
+      "grad_norm": 0.28298142552375793,
+      "learning_rate": 0.0002,
+      "loss": 1.3352,
+      "step": 50
+    },
+    {
+      "epoch": 0.3498542274052478,
+      "grad_norm": 0.2782207727432251,
+      "learning_rate": 0.0002,
+      "loss": 1.3061,
+      "step": 60
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.24569624662399292,
+      "learning_rate": 0.0002,
+      "loss": 1.2937,
+      "step": 70
+    },
+    {
+      "epoch": 0.46647230320699706,
+      "grad_norm": 0.27325960993766785,
+      "learning_rate": 0.0002,
+      "loss": 1.2724,
+      "step": 80
+    },
+    {
+      "epoch": 0.5247813411078717,
+      "grad_norm": 0.29733362793922424,
+      "learning_rate": 0.0002,
+      "loss": 1.3331,
+      "step": 90
+    },
+    {
+      "epoch": 0.5830903790087464,
+      "grad_norm": 0.24717216193675995,
+      "learning_rate": 0.0002,
+      "loss": 1.2704,
+      "step": 100
+    },
+    {
+      "epoch": 0.641399416909621,
+      "grad_norm": 0.30963554978370667,
+      "learning_rate": 0.0002,
+      "loss": 1.2832,
+      "step": 110
+    },
+    {
+      "epoch": 0.6997084548104956,
+      "grad_norm": 0.23280981183052063,
+      "learning_rate": 0.0002,
+      "loss": 1.2379,
+      "step": 120
+    },
+    {
+      "epoch": 0.7580174927113703,
+      "grad_norm": 0.25881555676460266,
+      "learning_rate": 0.0002,
+      "loss": 1.2787,
+      "step": 130
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.33120882511138916,
+      "learning_rate": 0.0002,
+      "loss": 1.3106,
+      "step": 140
+    },
+    {
+      "epoch": 0.8746355685131195,
+      "grad_norm": 0.2893398702144623,
+      "learning_rate": 0.0002,
+      "loss": 1.3491,
+      "step": 150
+    },
+    {
+      "epoch": 0.9329446064139941,
+      "grad_norm": 0.34311336278915405,
+      "learning_rate": 0.0002,
+      "loss": 1.3122,
+      "step": 160
+    },
+    {
+      "epoch": 0.9912536443148688,
+      "grad_norm": 0.3196379840373993,
+      "learning_rate": 0.0002,
+      "loss": 1.2457,
+      "step": 170
+    },
+    {
+      "epoch": 0.9970845481049563,
+      "eval_loss": 1.249076008796692,
+      "eval_runtime": 14.9505,
+      "eval_samples_per_second": 6.421,
+      "eval_steps_per_second": 0.803,
+      "step": 171
+    },
+    {
+      "epoch": 1.0495626822157433,
+      "grad_norm": 0.3274891972541809,
+      "learning_rate": 0.0002,
+      "loss": 1.272,
+      "step": 180
+    },
+    {
+      "epoch": 1.1078717201166182,
+      "grad_norm": 0.3209025263786316,
+      "learning_rate": 0.0002,
+      "loss": 1.2585,
+      "step": 190
+    },
+    {
+      "epoch": 1.1661807580174928,
+      "grad_norm": 0.29988062381744385,
+      "learning_rate": 0.0002,
+      "loss": 1.2164,
+      "step": 200
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.3197144567966461,
+      "learning_rate": 0.0002,
+      "loss": 1.219,
+      "step": 210
+    },
+    {
+      "epoch": 1.282798833819242,
+      "grad_norm": 0.3738149404525757,
+      "learning_rate": 0.0002,
+      "loss": 1.2431,
+      "step": 220
+    },
+    {
+      "epoch": 1.3411078717201166,
+      "grad_norm": 0.3867924213409424,
+      "learning_rate": 0.0002,
+      "loss": 1.1926,
+      "step": 230
+    },
+    {
+      "epoch": 1.3994169096209912,
+      "grad_norm": 0.31422024965286255,
+      "learning_rate": 0.0002,
+      "loss": 1.1673,
+      "step": 240
+    },
+    {
+      "epoch": 1.4577259475218658,
+      "grad_norm": 0.34369608759880066,
+      "learning_rate": 0.0002,
+      "loss": 1.1594,
+      "step": 250
+    },
+    {
+      "epoch": 1.5160349854227406,
+      "grad_norm": 0.3204766809940338,
+      "learning_rate": 0.0002,
+      "loss": 1.2399,
+      "step": 260
+    },
+    {
+      "epoch": 1.574344023323615,
+      "grad_norm": 0.3328384757041931,
+      "learning_rate": 0.0002,
+      "loss": 1.2248,
+      "step": 270
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.30920523405075073,
+      "learning_rate": 0.0002,
+      "loss": 1.2094,
+      "step": 280
+    },
+    {
+      "epoch": 1.6909620991253644,
+      "grad_norm": 0.3138664662837982,
+      "learning_rate": 0.0002,
+      "loss": 1.2603,
+      "step": 290
+    },
+    {
+      "epoch": 1.749271137026239,
+      "grad_norm": 0.32747137546539307,
+      "learning_rate": 0.0002,
+      "loss": 1.1932,
+      "step": 300
+    },
+    {
+      "epoch": 1.8075801749271136,
+      "grad_norm": 0.31668511033058167,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 310
+    },
+    {
+      "epoch": 1.8658892128279883,
+      "grad_norm": 0.3314826488494873,
+      "learning_rate": 0.0002,
+      "loss": 1.216,
+      "step": 320
+    },
+    {
+      "epoch": 1.924198250728863,
+      "grad_norm": 0.3154251277446747,
+      "learning_rate": 0.0002,
+      "loss": 1.1739,
+      "step": 330
+    },
+    {
+      "epoch": 1.9825072886297375,
+      "grad_norm": 0.3600097596645355,
+      "learning_rate": 0.0002,
+      "loss": 1.2476,
+      "step": 340
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2224010229110718,
+      "eval_runtime": 14.9032,
+      "eval_samples_per_second": 6.442,
+      "eval_steps_per_second": 0.805,
+      "step": 343
+    },
+    {
+      "epoch": 2.0408163265306123,
+      "grad_norm": 0.3134561777114868,
+      "learning_rate": 0.0002,
+      "loss": 1.1687,
+      "step": 350
+    },
+    {
+      "epoch": 2.0991253644314867,
+      "grad_norm": 0.3430805802345276,
+      "learning_rate": 0.0002,
+      "loss": 1.1658,
+      "step": 360
+    },
+    {
+      "epoch": 2.1574344023323615,
+      "grad_norm": 0.4015083909034729,
+      "learning_rate": 0.0002,
+      "loss": 1.0947,
+      "step": 370
+    },
+    {
+      "epoch": 2.2157434402332363,
+      "grad_norm": 0.43478450179100037,
+      "learning_rate": 0.0002,
+      "loss": 1.1208,
+      "step": 380
+    },
+    {
+      "epoch": 2.2740524781341107,
+      "grad_norm": 0.3541523814201355,
+      "learning_rate": 0.0002,
+      "loss": 1.1555,
+      "step": 390
+    },
+    {
+      "epoch": 2.3323615160349855,
+      "grad_norm": 0.40292826294898987,
+      "learning_rate": 0.0002,
+      "loss": 1.175,
+      "step": 400
+    },
+    {
+      "epoch": 2.39067055393586,
+      "grad_norm": 0.3743220567703247,
+      "learning_rate": 0.0002,
+      "loss": 1.0685,
+      "step": 410
+    },
+    {
+      "epoch": 2.4489795918367347,
+      "grad_norm": 0.3321108818054199,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 420
+    },
+    {
+      "epoch": 2.5072886297376096,
+      "grad_norm": 0.3650798499584198,
+      "learning_rate": 0.0002,
+      "loss": 1.1441,
+      "step": 430
+    },
+    {
+      "epoch": 2.565597667638484,
+      "grad_norm": 0.3603045642375946,
+      "learning_rate": 0.0002,
+      "loss": 1.157,
+      "step": 440
+    },
+    {
+      "epoch": 2.6239067055393583,
+      "grad_norm": 0.36125364899635315,
+      "learning_rate": 0.0002,
+      "loss": 1.0995,
+      "step": 450
+    },
+    {
+      "epoch": 2.682215743440233,
+      "grad_norm": 0.4070495367050171,
+      "learning_rate": 0.0002,
+      "loss": 1.1364,
+      "step": 460
+    },
+    {
+      "epoch": 2.740524781341108,
+      "grad_norm": 0.42682793736457825,
+      "learning_rate": 0.0002,
+      "loss": 1.1407,
+      "step": 470
+    },
+    {
+      "epoch": 2.7988338192419824,
+      "grad_norm": 0.3968621790409088,
+      "learning_rate": 0.0002,
+      "loss": 1.1808,
+      "step": 480
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.37401914596557617,
+      "learning_rate": 0.0002,
+      "loss": 1.1252,
+      "step": 490
+    },
+    {
+      "epoch": 2.9154518950437316,
+      "grad_norm": 0.42643091082572937,
+      "learning_rate": 0.0002,
+      "loss": 1.1094,
+      "step": 500
+    },
+    {
+      "epoch": 2.9737609329446064,
+      "grad_norm": 0.48727893829345703,
+      "learning_rate": 0.0002,
+      "loss": 1.1587,
+      "step": 510
+    },
+    {
+      "epoch": 2.997084548104956,
+      "eval_loss": 1.2232840061187744,
+      "eval_runtime": 14.6841,
+      "eval_samples_per_second": 6.538,
+      "eval_steps_per_second": 0.817,
+      "step": 514
+    },
+    {
+      "epoch": 3.0320699708454812,
+      "grad_norm": 0.4159005880355835,
+      "learning_rate": 0.0002,
+      "loss": 1.0599,
+      "step": 520
+    },
+    {
+      "epoch": 3.0903790087463556,
+      "grad_norm": 0.4462415874004364,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 530
+    },
+    {
+      "epoch": 3.1486880466472305,
+      "grad_norm": 0.47459903359413147,
+      "learning_rate": 0.0002,
+      "loss": 1.0135,
+      "step": 540
+    },
+    {
+      "epoch": 3.206997084548105,
+      "grad_norm": 0.47449398040771484,
+      "learning_rate": 0.0002,
+      "loss": 1.0083,
+      "step": 550
+    },
+    {
+      "epoch": 3.2653061224489797,
+      "grad_norm": 0.48261985182762146,
+      "learning_rate": 0.0002,
+      "loss": 1.0698,
+      "step": 560
+    },
+    {
+      "epoch": 3.323615160349854,
+      "grad_norm": 0.48770663142204285,
+      "learning_rate": 0.0002,
+      "loss": 1.0553,
+      "step": 570
+    },
+    {
+      "epoch": 3.381924198250729,
+      "grad_norm": 0.478135883808136,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 580
+    },
+    {
+      "epoch": 3.4402332361516033,
+      "grad_norm": 0.47697558999061584,
+      "learning_rate": 0.0002,
+      "loss": 1.0214,
+      "step": 590
+    },
+    {
+      "epoch": 3.498542274052478,
+      "grad_norm": 0.4764975607395172,
+      "learning_rate": 0.0002,
+      "loss": 1.0644,
+      "step": 600
+    },
+    {
+      "epoch": 3.556851311953353,
+      "grad_norm": 0.5096248388290405,
+      "learning_rate": 0.0002,
+      "loss": 1.0729,
+      "step": 610
+    },
+    {
+      "epoch": 3.6151603498542273,
+      "grad_norm": 0.42080581188201904,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 620
+    },
+    {
+      "epoch": 3.673469387755102,
+      "grad_norm": 0.4736129641532898,
+      "learning_rate": 0.0002,
+      "loss": 1.0424,
+      "step": 630
+    },
+    {
+      "epoch": 3.7317784256559765,
+      "grad_norm": 0.46505600214004517,
+      "learning_rate": 0.0002,
+      "loss": 1.0474,
+      "step": 640
+    },
+    {
+      "epoch": 3.7900874635568513,
+      "grad_norm": 0.44950205087661743,
+      "learning_rate": 0.0002,
+      "loss": 1.0535,
+      "step": 650
+    },
+    {
+      "epoch": 3.848396501457726,
+      "grad_norm": 0.4537162780761719,
+      "learning_rate": 0.0002,
+      "loss": 1.0677,
+      "step": 660
+    },
+    {
+      "epoch": 3.9067055393586005,
+      "grad_norm": 0.4871664345264435,
+      "learning_rate": 0.0002,
+      "loss": 1.0871,
+      "step": 670
+    },
+    {
+      "epoch": 3.9650145772594754,
+      "grad_norm": 0.42579060792922974,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 680
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.235727071762085,
+      "eval_runtime": 14.6921,
+      "eval_samples_per_second": 6.534,
+      "eval_steps_per_second": 0.817,
+      "step": 686
+    },
+    {
+      "epoch": 4.02332361516035,
+      "grad_norm": 0.39996492862701416,
+      "learning_rate": 0.0002,
+      "loss": 1.064,
+      "step": 690
+    },
+    {
+      "epoch": 4.081632653061225,
+      "grad_norm": 0.6311050057411194,
+      "learning_rate": 0.0002,
+      "loss": 0.9566,
+      "step": 700
+    },
+    {
+      "epoch": 4.139941690962099,
+      "grad_norm": 0.5494613647460938,
+      "learning_rate": 0.0002,
+      "loss": 0.9549,
+      "step": 710
+    },
+    {
+      "epoch": 4.198250728862973,
+      "grad_norm": 0.689102828502655,
+      "learning_rate": 0.0002,
+      "loss": 0.9251,
+      "step": 720
+    },
+    {
+      "epoch": 4.256559766763848,
+      "grad_norm": 0.4727540910243988,
+      "learning_rate": 0.0002,
+      "loss": 0.9362,
+      "step": 730
+    },
+    {
+      "epoch": 4.314868804664723,
+      "grad_norm": 0.5162383317947388,
+      "learning_rate": 0.0002,
+      "loss": 1.0063,
+      "step": 740
+    },
+    {
+      "epoch": 4.373177842565598,
+      "grad_norm": 0.5838354229927063,
+      "learning_rate": 0.0002,
+      "loss": 0.9488,
+      "step": 750
+    },
+    {
+      "epoch": 4.431486880466473,
+      "grad_norm": 0.5268456935882568,
+      "learning_rate": 0.0002,
+      "loss": 0.9397,
+      "step": 760
+    },
+    {
+      "epoch": 4.489795918367347,
+      "grad_norm": 0.6695316433906555,
+      "learning_rate": 0.0002,
+      "loss": 0.9586,
+      "step": 770
+    },
+    {
+      "epoch": 4.548104956268221,
+      "grad_norm": 0.5652480721473694,
+      "learning_rate": 0.0002,
+      "loss": 0.9637,
+      "step": 780
+    },
+    {
+      "epoch": 4.606413994169096,
+      "grad_norm": 0.4949062168598175,
+      "learning_rate": 0.0002,
+      "loss": 0.9678,
+      "step": 790
+    },
+    {
+      "epoch": 4.664723032069971,
+      "grad_norm": 0.5157344341278076,
+      "learning_rate": 0.0002,
+      "loss": 0.9487,
+      "step": 800
+    },
+    {
+      "epoch": 4.723032069970845,
+      "grad_norm": 0.5152610540390015,
+      "learning_rate": 0.0002,
+      "loss": 0.972,
+      "step": 810
+    },
+    {
+      "epoch": 4.78134110787172,
+      "grad_norm": 0.540663480758667,
+      "learning_rate": 0.0002,
+      "loss": 0.9762,
+      "step": 820
+    },
+    {
+      "epoch": 4.839650145772595,
+      "grad_norm": 0.6327324509620667,
+      "learning_rate": 0.0002,
+      "loss": 0.9884,
+      "step": 830
+    },
+    {
+      "epoch": 4.8979591836734695,
+      "grad_norm": 0.5663048028945923,
+      "learning_rate": 0.0002,
+      "loss": 0.9656,
+      "step": 840
+    },
+    {
+      "epoch": 4.956268221574344,
+      "grad_norm": 0.5561565160751343,
+      "learning_rate": 0.0002,
+      "loss": 0.9956,
+      "step": 850
+    },
+    {
+      "epoch": 4.997084548104956,
+      "eval_loss": 1.2652570009231567,
+      "eval_runtime": 14.6946,
+      "eval_samples_per_second": 6.533,
+      "eval_steps_per_second": 0.817,
+      "step": 857
+    },
+    {
+      "epoch": 5.014577259475218,
+      "grad_norm": 0.6206780672073364,
+      "learning_rate": 0.0002,
+      "loss": 0.9756,
+      "step": 860
+    },
+    {
+      "epoch": 5.072886297376093,
+      "grad_norm": 0.7620121836662292,
+      "learning_rate": 0.0002,
+      "loss": 0.8336,
+      "step": 870
+    },
+    {
+      "epoch": 5.131195335276968,
+      "grad_norm": 0.744425892829895,
+      "learning_rate": 0.0002,
+      "loss": 0.8825,
+      "step": 880
+    },
+    {
+      "epoch": 5.189504373177843,
+      "grad_norm": 0.6208587288856506,
+      "learning_rate": 0.0002,
+      "loss": 0.8503,
+      "step": 890
+    },
+    {
+      "epoch": 5.247813411078718,
+      "grad_norm": 0.6400560736656189,
+      "learning_rate": 0.0002,
+      "loss": 0.8387,
+      "step": 900
+    },
+    {
+      "epoch": 5.3061224489795915,
+      "grad_norm": 0.6354245543479919,
+      "learning_rate": 0.0002,
+      "loss": 0.8454,
+      "step": 910
+    },
+    {
+      "epoch": 5.364431486880466,
+      "grad_norm": 0.6441580057144165,
+      "learning_rate": 0.0002,
+      "loss": 0.8158,
+      "step": 920
+    },
+    {
+      "epoch": 5.422740524781341,
+      "grad_norm": 0.8845874667167664,
+      "learning_rate": 0.0002,
+      "loss": 0.8509,
+      "step": 930
+    },
+    {
+      "epoch": 5.481049562682216,
+      "grad_norm": 0.7521125078201294,
+      "learning_rate": 0.0002,
+      "loss": 0.9311,
+      "step": 940
+    },
+    {
+      "epoch": 5.539358600583091,
+      "grad_norm": 0.6190000772476196,
+      "learning_rate": 0.0002,
+      "loss": 0.852,
+      "step": 950
+    },
+    {
+      "epoch": 5.597667638483965,
+      "grad_norm": 0.6528187990188599,
+      "learning_rate": 0.0002,
+      "loss": 0.9208,
+      "step": 960
+    },
+    {
+      "epoch": 5.65597667638484,
+      "grad_norm": 0.7539762258529663,
+      "learning_rate": 0.0002,
+      "loss": 0.886,
+      "step": 970
+    },
+    {
+      "epoch": 5.714285714285714,
+      "grad_norm": 0.6197044253349304,
+      "learning_rate": 0.0002,
+      "loss": 0.8631,
+      "step": 980
+    },
+    {
+      "epoch": 5.772594752186589,
+      "grad_norm": 0.569410502910614,
+      "learning_rate": 0.0002,
+      "loss": 0.8266,
+      "step": 990
+    },
+    {
+      "epoch": 5.830903790087463,
+      "grad_norm": 0.6476598978042603,
+      "learning_rate": 0.0002,
+      "loss": 0.8832,
+      "step": 1000
+    },
+    {
+      "epoch": 5.889212827988338,
+      "grad_norm": 0.6748721599578857,
+      "learning_rate": 0.0002,
+      "loss": 0.8817,
+      "step": 1010
+    },
+    {
+      "epoch": 5.947521865889213,
+      "grad_norm": 0.67653489112854,
+      "learning_rate": 0.0002,
+      "loss": 0.9131,
+      "step": 1020
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3123321533203125,
+      "eval_runtime": 14.6876,
+      "eval_samples_per_second": 6.536,
+      "eval_steps_per_second": 0.817,
+      "step": 1029
+    },
+    {
+      "epoch": 6.005830903790088,
+      "grad_norm": 0.8456128239631653,
+      "learning_rate": 0.0002,
+      "loss": 0.9302,
+      "step": 1030
+    },
+    {
+      "epoch": 6.0641399416909625,
+      "grad_norm": 0.7655927538871765,
+      "learning_rate": 0.0002,
+      "loss": 0.7582,
+      "step": 1040
+    },
+    {
+      "epoch": 6.122448979591836,
+      "grad_norm": 0.8100736737251282,
+      "learning_rate": 0.0002,
+      "loss": 0.7671,
+      "step": 1050
+    },
+    {
+      "epoch": 6.180758017492711,
+      "grad_norm": 0.7127841114997864,
+      "learning_rate": 0.0002,
+      "loss": 0.7702,
+      "step": 1060
+    },
+    {
+      "epoch": 6.239067055393586,
+      "grad_norm": 0.7305651307106018,
+      "learning_rate": 0.0002,
+      "loss": 0.7767,
+      "step": 1070
+    },
+    {
+      "epoch": 6.297376093294461,
+      "grad_norm": 0.7976569533348083,
+      "learning_rate": 0.0002,
+      "loss": 0.7635,
+      "step": 1080
+    },
+    {
+      "epoch": 6.355685131195335,
+      "grad_norm": 0.9411352276802063,
+      "learning_rate": 0.0002,
+      "loss": 0.7743,
+      "step": 1090
+    },
+    {
+      "epoch": 6.41399416909621,
+      "grad_norm": 0.7893549203872681,
+      "learning_rate": 0.0002,
+      "loss": 0.7708,
+      "step": 1100
+    },
+    {
+      "epoch": 6.4723032069970845,
+      "grad_norm": 0.837874174118042,
+      "learning_rate": 0.0002,
+      "loss": 0.7866,
+      "step": 1110
+    },
+    {
+      "epoch": 6.530612244897959,
+      "grad_norm": 0.8557870388031006,
+      "learning_rate": 0.0002,
+      "loss": 0.7942,
+      "step": 1120
+    },
+    {
+      "epoch": 6.588921282798834,
+      "grad_norm": 0.7962378859519958,
+      "learning_rate": 0.0002,
+      "loss": 0.808,
+      "step": 1130
+    },
+    {
+      "epoch": 6.647230320699708,
+      "grad_norm": 0.654983639717102,
+      "learning_rate": 0.0002,
+      "loss": 0.7776,
+      "step": 1140
+    },
+    {
+      "epoch": 6.705539358600583,
+      "grad_norm": 0.7596574425697327,
+      "learning_rate": 0.0002,
+      "loss": 0.77,
+      "step": 1150
+    },
+    {
+      "epoch": 6.763848396501458,
+      "grad_norm": 0.7047041058540344,
+      "learning_rate": 0.0002,
+      "loss": 0.7873,
+      "step": 1160
+    },
+    {
+      "epoch": 6.822157434402333,
+      "grad_norm": 0.7320792078971863,
+      "learning_rate": 0.0002,
+      "loss": 0.7999,
+      "step": 1170
+    },
+    {
+      "epoch": 6.8804664723032065,
+      "grad_norm": 0.6893225908279419,
+      "learning_rate": 0.0002,
+      "loss": 0.7942,
+      "step": 1180
+    },
+    {
+      "epoch": 6.938775510204081,
+      "grad_norm": 0.7593189477920532,
+      "learning_rate": 0.0002,
+      "loss": 0.7977,
+      "step": 1190
+    },
+    {
+      "epoch": 6.997084548104956,
+      "grad_norm": 0.9252594113349915,
+      "learning_rate": 0.0002,
+      "loss": 0.7975,
+      "step": 1200
+    },
+    {
+      "epoch": 6.997084548104956,
+      "eval_loss": 1.3805131912231445,
+      "eval_runtime": 14.6665,
+      "eval_samples_per_second": 6.546,
+      "eval_steps_per_second": 0.818,
+      "step": 1200
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1368,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.230086433328333e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:558d8475370dfee2c47b98ed05aca969bb536c25048d37c81a11582fa4f0e57c
+size 5560

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e406eb4b2c974cdc3aa06bec5a56c3c81c97d75a3d9185283310e99958ec2edd
+size 80755416

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa0389ed4d2f2113fd702bc7b3634d2c421d189405e07e88e55d79169a45a577
+size 41136570

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acbc5c912232c32d373788bcb1aed2175894a7956239e7fbe2c984e9f16254d5
+size 14244

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9430bcc8729bbc647c2620523e26a08b1e5bdd26bf18e131c0cfef389edd6a3
+size 1064

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/trainer_state.json

@@ -0,0 +1,1049 @@
+{
+  "best_metric": 1.2224010229110718,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-343",
+  "epoch": 7.97667638483965,
+  "eval_steps": 10,
+  "global_step": 1368,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05830903790087463,
+      "grad_norm": 0.27093306183815,
+      "learning_rate": 0.0002,
+      "loss": 1.6238,
+      "step": 10
+    },
+    {
+      "epoch": 0.11661807580174927,
+      "grad_norm": 0.22656577825546265,
+      "learning_rate": 0.0002,
+      "loss": 1.3942,
+      "step": 20
+    },
+    {
+      "epoch": 0.1749271137026239,
+      "grad_norm": 0.2181931734085083,
+      "learning_rate": 0.0002,
+      "loss": 1.4224,
+      "step": 30
+    },
+    {
+      "epoch": 0.23323615160349853,
+      "grad_norm": 0.23278485238552094,
+      "learning_rate": 0.0002,
+      "loss": 1.3609,
+      "step": 40
+    },
+    {
+      "epoch": 0.2915451895043732,
+      "grad_norm": 0.28298142552375793,
+      "learning_rate": 0.0002,
+      "loss": 1.3352,
+      "step": 50
+    },
+    {
+      "epoch": 0.3498542274052478,
+      "grad_norm": 0.2782207727432251,
+      "learning_rate": 0.0002,
+      "loss": 1.3061,
+      "step": 60
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.24569624662399292,
+      "learning_rate": 0.0002,
+      "loss": 1.2937,
+      "step": 70
+    },
+    {
+      "epoch": 0.46647230320699706,
+      "grad_norm": 0.27325960993766785,
+      "learning_rate": 0.0002,
+      "loss": 1.2724,
+      "step": 80
+    },
+    {
+      "epoch": 0.5247813411078717,
+      "grad_norm": 0.29733362793922424,
+      "learning_rate": 0.0002,
+      "loss": 1.3331,
+      "step": 90
+    },
+    {
+      "epoch": 0.5830903790087464,
+      "grad_norm": 0.24717216193675995,
+      "learning_rate": 0.0002,
+      "loss": 1.2704,
+      "step": 100
+    },
+    {
+      "epoch": 0.641399416909621,
+      "grad_norm": 0.30963554978370667,
+      "learning_rate": 0.0002,
+      "loss": 1.2832,
+      "step": 110
+    },
+    {
+      "epoch": 0.6997084548104956,
+      "grad_norm": 0.23280981183052063,
+      "learning_rate": 0.0002,
+      "loss": 1.2379,
+      "step": 120
+    },
+    {
+      "epoch": 0.7580174927113703,
+      "grad_norm": 0.25881555676460266,
+      "learning_rate": 0.0002,
+      "loss": 1.2787,
+      "step": 130
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.33120882511138916,
+      "learning_rate": 0.0002,
+      "loss": 1.3106,
+      "step": 140
+    },
+    {
+      "epoch": 0.8746355685131195,
+      "grad_norm": 0.2893398702144623,
+      "learning_rate": 0.0002,
+      "loss": 1.3491,
+      "step": 150
+    },
+    {
+      "epoch": 0.9329446064139941,
+      "grad_norm": 0.34311336278915405,
+      "learning_rate": 0.0002,
+      "loss": 1.3122,
+      "step": 160
+    },
+    {
+      "epoch": 0.9912536443148688,
+      "grad_norm": 0.3196379840373993,
+      "learning_rate": 0.0002,
+      "loss": 1.2457,
+      "step": 170
+    },
+    {
+      "epoch": 0.9970845481049563,
+      "eval_loss": 1.249076008796692,
+      "eval_runtime": 14.9505,
+      "eval_samples_per_second": 6.421,
+      "eval_steps_per_second": 0.803,
+      "step": 171
+    },
+    {
+      "epoch": 1.0495626822157433,
+      "grad_norm": 0.3274891972541809,
+      "learning_rate": 0.0002,
+      "loss": 1.272,
+      "step": 180
+    },
+    {
+      "epoch": 1.1078717201166182,
+      "grad_norm": 0.3209025263786316,
+      "learning_rate": 0.0002,
+      "loss": 1.2585,
+      "step": 190
+    },
+    {
+      "epoch": 1.1661807580174928,
+      "grad_norm": 0.29988062381744385,
+      "learning_rate": 0.0002,
+      "loss": 1.2164,
+      "step": 200
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.3197144567966461,
+      "learning_rate": 0.0002,
+      "loss": 1.219,
+      "step": 210
+    },
+    {
+      "epoch": 1.282798833819242,
+      "grad_norm": 0.3738149404525757,
+      "learning_rate": 0.0002,
+      "loss": 1.2431,
+      "step": 220
+    },
+    {
+      "epoch": 1.3411078717201166,
+      "grad_norm": 0.3867924213409424,
+      "learning_rate": 0.0002,
+      "loss": 1.1926,
+      "step": 230
+    },
+    {
+      "epoch": 1.3994169096209912,
+      "grad_norm": 0.31422024965286255,
+      "learning_rate": 0.0002,
+      "loss": 1.1673,
+      "step": 240
+    },
+    {
+      "epoch": 1.4577259475218658,
+      "grad_norm": 0.34369608759880066,
+      "learning_rate": 0.0002,
+      "loss": 1.1594,
+      "step": 250
+    },
+    {
+      "epoch": 1.5160349854227406,
+      "grad_norm": 0.3204766809940338,
+      "learning_rate": 0.0002,
+      "loss": 1.2399,
+      "step": 260
+    },
+    {
+      "epoch": 1.574344023323615,
+      "grad_norm": 0.3328384757041931,
+      "learning_rate": 0.0002,
+      "loss": 1.2248,
+      "step": 270
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.30920523405075073,
+      "learning_rate": 0.0002,
+      "loss": 1.2094,
+      "step": 280
+    },
+    {
+      "epoch": 1.6909620991253644,
+      "grad_norm": 0.3138664662837982,
+      "learning_rate": 0.0002,
+      "loss": 1.2603,
+      "step": 290
+    },
+    {
+      "epoch": 1.749271137026239,
+      "grad_norm": 0.32747137546539307,
+      "learning_rate": 0.0002,
+      "loss": 1.1932,
+      "step": 300
+    },
+    {
+      "epoch": 1.8075801749271136,
+      "grad_norm": 0.31668511033058167,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 310
+    },
+    {
+      "epoch": 1.8658892128279883,
+      "grad_norm": 0.3314826488494873,
+      "learning_rate": 0.0002,
+      "loss": 1.216,
+      "step": 320
+    },
+    {
+      "epoch": 1.924198250728863,
+      "grad_norm": 0.3154251277446747,
+      "learning_rate": 0.0002,
+      "loss": 1.1739,
+      "step": 330
+    },
+    {
+      "epoch": 1.9825072886297375,
+      "grad_norm": 0.3600097596645355,
+      "learning_rate": 0.0002,
+      "loss": 1.2476,
+      "step": 340
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2224010229110718,
+      "eval_runtime": 14.9032,
+      "eval_samples_per_second": 6.442,
+      "eval_steps_per_second": 0.805,
+      "step": 343
+    },
+    {
+      "epoch": 2.0408163265306123,
+      "grad_norm": 0.3134561777114868,
+      "learning_rate": 0.0002,
+      "loss": 1.1687,
+      "step": 350
+    },
+    {
+      "epoch": 2.0991253644314867,
+      "grad_norm": 0.3430805802345276,
+      "learning_rate": 0.0002,
+      "loss": 1.1658,
+      "step": 360
+    },
+    {
+      "epoch": 2.1574344023323615,
+      "grad_norm": 0.4015083909034729,
+      "learning_rate": 0.0002,
+      "loss": 1.0947,
+      "step": 370
+    },
+    {
+      "epoch": 2.2157434402332363,
+      "grad_norm": 0.43478450179100037,
+      "learning_rate": 0.0002,
+      "loss": 1.1208,
+      "step": 380
+    },
+    {
+      "epoch": 2.2740524781341107,
+      "grad_norm": 0.3541523814201355,
+      "learning_rate": 0.0002,
+      "loss": 1.1555,
+      "step": 390
+    },
+    {
+      "epoch": 2.3323615160349855,
+      "grad_norm": 0.40292826294898987,
+      "learning_rate": 0.0002,
+      "loss": 1.175,
+      "step": 400
+    },
+    {
+      "epoch": 2.39067055393586,
+      "grad_norm": 0.3743220567703247,
+      "learning_rate": 0.0002,
+      "loss": 1.0685,
+      "step": 410
+    },
+    {
+      "epoch": 2.4489795918367347,
+      "grad_norm": 0.3321108818054199,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 420
+    },
+    {
+      "epoch": 2.5072886297376096,
+      "grad_norm": 0.3650798499584198,
+      "learning_rate": 0.0002,
+      "loss": 1.1441,
+      "step": 430
+    },
+    {
+      "epoch": 2.565597667638484,
+      "grad_norm": 0.3603045642375946,
+      "learning_rate": 0.0002,
+      "loss": 1.157,
+      "step": 440
+    },
+    {
+      "epoch": 2.6239067055393583,
+      "grad_norm": 0.36125364899635315,
+      "learning_rate": 0.0002,
+      "loss": 1.0995,
+      "step": 450
+    },
+    {
+      "epoch": 2.682215743440233,
+      "grad_norm": 0.4070495367050171,
+      "learning_rate": 0.0002,
+      "loss": 1.1364,
+      "step": 460
+    },
+    {
+      "epoch": 2.740524781341108,
+      "grad_norm": 0.42682793736457825,
+      "learning_rate": 0.0002,
+      "loss": 1.1407,
+      "step": 470
+    },
+    {
+      "epoch": 2.7988338192419824,
+      "grad_norm": 0.3968621790409088,
+      "learning_rate": 0.0002,
+      "loss": 1.1808,
+      "step": 480
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.37401914596557617,
+      "learning_rate": 0.0002,
+      "loss": 1.1252,
+      "step": 490
+    },
+    {
+      "epoch": 2.9154518950437316,
+      "grad_norm": 0.42643091082572937,
+      "learning_rate": 0.0002,
+      "loss": 1.1094,
+      "step": 500
+    },
+    {
+      "epoch": 2.9737609329446064,
+      "grad_norm": 0.48727893829345703,
+      "learning_rate": 0.0002,
+      "loss": 1.1587,
+      "step": 510
+    },
+    {
+      "epoch": 2.997084548104956,
+      "eval_loss": 1.2232840061187744,
+      "eval_runtime": 14.6841,
+      "eval_samples_per_second": 6.538,
+      "eval_steps_per_second": 0.817,
+      "step": 514
+    },
+    {
+      "epoch": 3.0320699708454812,
+      "grad_norm": 0.4159005880355835,
+      "learning_rate": 0.0002,
+      "loss": 1.0599,
+      "step": 520
+    },
+    {
+      "epoch": 3.0903790087463556,
+      "grad_norm": 0.4462415874004364,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 530
+    },
+    {
+      "epoch": 3.1486880466472305,
+      "grad_norm": 0.47459903359413147,
+      "learning_rate": 0.0002,
+      "loss": 1.0135,
+      "step": 540
+    },
+    {
+      "epoch": 3.206997084548105,
+      "grad_norm": 0.47449398040771484,
+      "learning_rate": 0.0002,
+      "loss": 1.0083,
+      "step": 550
+    },
+    {
+      "epoch": 3.2653061224489797,
+      "grad_norm": 0.48261985182762146,
+      "learning_rate": 0.0002,
+      "loss": 1.0698,
+      "step": 560
+    },
+    {
+      "epoch": 3.323615160349854,
+      "grad_norm": 0.48770663142204285,
+      "learning_rate": 0.0002,
+      "loss": 1.0553,
+      "step": 570
+    },
+    {
+      "epoch": 3.381924198250729,
+      "grad_norm": 0.478135883808136,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 580
+    },
+    {
+      "epoch": 3.4402332361516033,
+      "grad_norm": 0.47697558999061584,
+      "learning_rate": 0.0002,
+      "loss": 1.0214,
+      "step": 590
+    },
+    {
+      "epoch": 3.498542274052478,
+      "grad_norm": 0.4764975607395172,
+      "learning_rate": 0.0002,
+      "loss": 1.0644,
+      "step": 600
+    },
+    {
+      "epoch": 3.556851311953353,
+      "grad_norm": 0.5096248388290405,
+      "learning_rate": 0.0002,
+      "loss": 1.0729,
+      "step": 610
+    },
+    {
+      "epoch": 3.6151603498542273,
+      "grad_norm": 0.42080581188201904,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 620
+    },
+    {
+      "epoch": 3.673469387755102,
+      "grad_norm": 0.4736129641532898,
+      "learning_rate": 0.0002,
+      "loss": 1.0424,
+      "step": 630
+    },
+    {
+      "epoch": 3.7317784256559765,
+      "grad_norm": 0.46505600214004517,
+      "learning_rate": 0.0002,
+      "loss": 1.0474,
+      "step": 640
+    },
+    {
+      "epoch": 3.7900874635568513,
+      "grad_norm": 0.44950205087661743,
+      "learning_rate": 0.0002,
+      "loss": 1.0535,
+      "step": 650
+    },
+    {
+      "epoch": 3.848396501457726,
+      "grad_norm": 0.4537162780761719,
+      "learning_rate": 0.0002,
+      "loss": 1.0677,
+      "step": 660
+    },
+    {
+      "epoch": 3.9067055393586005,
+      "grad_norm": 0.4871664345264435,
+      "learning_rate": 0.0002,
+      "loss": 1.0871,
+      "step": 670
+    },
+    {
+      "epoch": 3.9650145772594754,
+      "grad_norm": 0.42579060792922974,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 680
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.235727071762085,
+      "eval_runtime": 14.6921,
+      "eval_samples_per_second": 6.534,
+      "eval_steps_per_second": 0.817,
+      "step": 686
+    },
+    {
+      "epoch": 4.02332361516035,
+      "grad_norm": 0.39996492862701416,
+      "learning_rate": 0.0002,
+      "loss": 1.064,
+      "step": 690
+    },
+    {
+      "epoch": 4.081632653061225,
+      "grad_norm": 0.6311050057411194,
+      "learning_rate": 0.0002,
+      "loss": 0.9566,
+      "step": 700
+    },
+    {
+      "epoch": 4.139941690962099,
+      "grad_norm": 0.5494613647460938,
+      "learning_rate": 0.0002,
+      "loss": 0.9549,
+      "step": 710
+    },
+    {
+      "epoch": 4.198250728862973,
+      "grad_norm": 0.689102828502655,
+      "learning_rate": 0.0002,
+      "loss": 0.9251,
+      "step": 720
+    },
+    {
+      "epoch": 4.256559766763848,
+      "grad_norm": 0.4727540910243988,
+      "learning_rate": 0.0002,
+      "loss": 0.9362,
+      "step": 730
+    },
+    {
+      "epoch": 4.314868804664723,
+      "grad_norm": 0.5162383317947388,
+      "learning_rate": 0.0002,
+      "loss": 1.0063,
+      "step": 740
+    },
+    {
+      "epoch": 4.373177842565598,
+      "grad_norm": 0.5838354229927063,
+      "learning_rate": 0.0002,
+      "loss": 0.9488,
+      "step": 750
+    },
+    {
+      "epoch": 4.431486880466473,
+      "grad_norm": 0.5268456935882568,
+      "learning_rate": 0.0002,
+      "loss": 0.9397,
+      "step": 760
+    },
+    {
+      "epoch": 4.489795918367347,
+      "grad_norm": 0.6695316433906555,
+      "learning_rate": 0.0002,
+      "loss": 0.9586,
+      "step": 770
+    },
+    {
+      "epoch": 4.548104956268221,
+      "grad_norm": 0.5652480721473694,
+      "learning_rate": 0.0002,
+      "loss": 0.9637,
+      "step": 780
+    },
+    {
+      "epoch": 4.606413994169096,
+      "grad_norm": 0.4949062168598175,
+      "learning_rate": 0.0002,
+      "loss": 0.9678,
+      "step": 790
+    },
+    {
+      "epoch": 4.664723032069971,
+      "grad_norm": 0.5157344341278076,
+      "learning_rate": 0.0002,
+      "loss": 0.9487,
+      "step": 800
+    },
+    {
+      "epoch": 4.723032069970845,
+      "grad_norm": 0.5152610540390015,
+      "learning_rate": 0.0002,
+      "loss": 0.972,
+      "step": 810
+    },
+    {
+      "epoch": 4.78134110787172,
+      "grad_norm": 0.540663480758667,
+      "learning_rate": 0.0002,
+      "loss": 0.9762,
+      "step": 820
+    },
+    {
+      "epoch": 4.839650145772595,
+      "grad_norm": 0.6327324509620667,
+      "learning_rate": 0.0002,
+      "loss": 0.9884,
+      "step": 830
+    },
+    {
+      "epoch": 4.8979591836734695,
+      "grad_norm": 0.5663048028945923,
+      "learning_rate": 0.0002,
+      "loss": 0.9656,
+      "step": 840
+    },
+    {
+      "epoch": 4.956268221574344,
+      "grad_norm": 0.5561565160751343,
+      "learning_rate": 0.0002,
+      "loss": 0.9956,
+      "step": 850
+    },
+    {
+      "epoch": 4.997084548104956,
+      "eval_loss": 1.2652570009231567,
+      "eval_runtime": 14.6946,
+      "eval_samples_per_second": 6.533,
+      "eval_steps_per_second": 0.817,
+      "step": 857
+    },
+    {
+      "epoch": 5.014577259475218,
+      "grad_norm": 0.6206780672073364,
+      "learning_rate": 0.0002,
+      "loss": 0.9756,
+      "step": 860
+    },
+    {
+      "epoch": 5.072886297376093,
+      "grad_norm": 0.7620121836662292,
+      "learning_rate": 0.0002,
+      "loss": 0.8336,
+      "step": 870
+    },
+    {
+      "epoch": 5.131195335276968,
+      "grad_norm": 0.744425892829895,
+      "learning_rate": 0.0002,
+      "loss": 0.8825,
+      "step": 880
+    },
+    {
+      "epoch": 5.189504373177843,
+      "grad_norm": 0.6208587288856506,
+      "learning_rate": 0.0002,
+      "loss": 0.8503,
+      "step": 890
+    },
+    {
+      "epoch": 5.247813411078718,
+      "grad_norm": 0.6400560736656189,
+      "learning_rate": 0.0002,
+      "loss": 0.8387,
+      "step": 900
+    },
+    {
+      "epoch": 5.3061224489795915,
+      "grad_norm": 0.6354245543479919,
+      "learning_rate": 0.0002,
+      "loss": 0.8454,
+      "step": 910
+    },
+    {
+      "epoch": 5.364431486880466,
+      "grad_norm": 0.6441580057144165,
+      "learning_rate": 0.0002,
+      "loss": 0.8158,
+      "step": 920
+    },
+    {
+      "epoch": 5.422740524781341,
+      "grad_norm": 0.8845874667167664,
+      "learning_rate": 0.0002,
+      "loss": 0.8509,
+      "step": 930
+    },
+    {
+      "epoch": 5.481049562682216,
+      "grad_norm": 0.7521125078201294,
+      "learning_rate": 0.0002,
+      "loss": 0.9311,
+      "step": 940
+    },
+    {
+      "epoch": 5.539358600583091,
+      "grad_norm": 0.6190000772476196,
+      "learning_rate": 0.0002,
+      "loss": 0.852,
+      "step": 950
+    },
+    {
+      "epoch": 5.597667638483965,
+      "grad_norm": 0.6528187990188599,
+      "learning_rate": 0.0002,
+      "loss": 0.9208,
+      "step": 960
+    },
+    {
+      "epoch": 5.65597667638484,
+      "grad_norm": 0.7539762258529663,
+      "learning_rate": 0.0002,
+      "loss": 0.886,
+      "step": 970
+    },
+    {
+      "epoch": 5.714285714285714,
+      "grad_norm": 0.6197044253349304,
+      "learning_rate": 0.0002,
+      "loss": 0.8631,
+      "step": 980
+    },
+    {
+      "epoch": 5.772594752186589,
+      "grad_norm": 0.569410502910614,
+      "learning_rate": 0.0002,
+      "loss": 0.8266,
+      "step": 990
+    },
+    {
+      "epoch": 5.830903790087463,
+      "grad_norm": 0.6476598978042603,
+      "learning_rate": 0.0002,
+      "loss": 0.8832,
+      "step": 1000
+    },
+    {
+      "epoch": 5.889212827988338,
+      "grad_norm": 0.6748721599578857,
+      "learning_rate": 0.0002,
+      "loss": 0.8817,
+      "step": 1010
+    },
+    {
+      "epoch": 5.947521865889213,
+      "grad_norm": 0.67653489112854,
+      "learning_rate": 0.0002,
+      "loss": 0.9131,
+      "step": 1020
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3123321533203125,
+      "eval_runtime": 14.6876,
+      "eval_samples_per_second": 6.536,
+      "eval_steps_per_second": 0.817,
+      "step": 1029
+    },
+    {
+      "epoch": 6.005830903790088,
+      "grad_norm": 0.8456128239631653,
+      "learning_rate": 0.0002,
+      "loss": 0.9302,
+      "step": 1030
+    },
+    {
+      "epoch": 6.0641399416909625,
+      "grad_norm": 0.7655927538871765,
+      "learning_rate": 0.0002,
+      "loss": 0.7582,
+      "step": 1040
+    },
+    {
+      "epoch": 6.122448979591836,
+      "grad_norm": 0.8100736737251282,
+      "learning_rate": 0.0002,
+      "loss": 0.7671,
+      "step": 1050
+    },
+    {
+      "epoch": 6.180758017492711,
+      "grad_norm": 0.7127841114997864,
+      "learning_rate": 0.0002,
+      "loss": 0.7702,
+      "step": 1060
+    },
+    {
+      "epoch": 6.239067055393586,
+      "grad_norm": 0.7305651307106018,
+      "learning_rate": 0.0002,
+      "loss": 0.7767,
+      "step": 1070
+    },
+    {
+      "epoch": 6.297376093294461,
+      "grad_norm": 0.7976569533348083,
+      "learning_rate": 0.0002,
+      "loss": 0.7635,
+      "step": 1080
+    },
+    {
+      "epoch": 6.355685131195335,
+      "grad_norm": 0.9411352276802063,
+      "learning_rate": 0.0002,
+      "loss": 0.7743,
+      "step": 1090
+    },
+    {
+      "epoch": 6.41399416909621,
+      "grad_norm": 0.7893549203872681,
+      "learning_rate": 0.0002,
+      "loss": 0.7708,
+      "step": 1100
+    },
+    {
+      "epoch": 6.4723032069970845,
+      "grad_norm": 0.837874174118042,
+      "learning_rate": 0.0002,
+      "loss": 0.7866,
+      "step": 1110
+    },
+    {
+      "epoch": 6.530612244897959,
+      "grad_norm": 0.8557870388031006,
+      "learning_rate": 0.0002,
+      "loss": 0.7942,
+      "step": 1120
+    },
+    {
+      "epoch": 6.588921282798834,
+      "grad_norm": 0.7962378859519958,
+      "learning_rate": 0.0002,
+      "loss": 0.808,
+      "step": 1130
+    },
+    {
+      "epoch": 6.647230320699708,
+      "grad_norm": 0.654983639717102,
+      "learning_rate": 0.0002,
+      "loss": 0.7776,
+      "step": 1140
+    },
+    {
+      "epoch": 6.705539358600583,
+      "grad_norm": 0.7596574425697327,
+      "learning_rate": 0.0002,
+      "loss": 0.77,
+      "step": 1150
+    },
+    {
+      "epoch": 6.763848396501458,
+      "grad_norm": 0.7047041058540344,
+      "learning_rate": 0.0002,
+      "loss": 0.7873,
+      "step": 1160
+    },
+    {
+      "epoch": 6.822157434402333,
+      "grad_norm": 0.7320792078971863,
+      "learning_rate": 0.0002,
+      "loss": 0.7999,
+      "step": 1170
+    },
+    {
+      "epoch": 6.8804664723032065,
+      "grad_norm": 0.6893225908279419,
+      "learning_rate": 0.0002,
+      "loss": 0.7942,
+      "step": 1180
+    },
+    {
+      "epoch": 6.938775510204081,
+      "grad_norm": 0.7593189477920532,
+      "learning_rate": 0.0002,
+      "loss": 0.7977,
+      "step": 1190
+    },
+    {
+      "epoch": 6.997084548104956,
+      "grad_norm": 0.9252594113349915,
+      "learning_rate": 0.0002,
+      "loss": 0.7975,
+      "step": 1200
+    },
+    {
+      "epoch": 6.997084548104956,
+      "eval_loss": 1.3805131912231445,
+      "eval_runtime": 14.6665,
+      "eval_samples_per_second": 6.546,
+      "eval_steps_per_second": 0.818,
+      "step": 1200
+    },
+    {
+      "epoch": 7.055393586005831,
+      "grad_norm": 1.183172345161438,
+      "learning_rate": 0.0002,
+      "loss": 0.7001,
+      "step": 1210
+    },
+    {
+      "epoch": 7.113702623906706,
+      "grad_norm": 0.763805627822876,
+      "learning_rate": 0.0002,
+      "loss": 0.6594,
+      "step": 1220
+    },
+    {
+      "epoch": 7.17201166180758,
+      "grad_norm": 0.9782307147979736,
+      "learning_rate": 0.0002,
+      "loss": 0.6947,
+      "step": 1230
+    },
+    {
+      "epoch": 7.230320699708455,
+      "grad_norm": 0.8512941002845764,
+      "learning_rate": 0.0002,
+      "loss": 0.6697,
+      "step": 1240
+    },
+    {
+      "epoch": 7.288629737609329,
+      "grad_norm": 0.9969114065170288,
+      "learning_rate": 0.0002,
+      "loss": 0.696,
+      "step": 1250
+    },
+    {
+      "epoch": 7.346938775510204,
+      "grad_norm": 0.8895878791809082,
+      "learning_rate": 0.0002,
+      "loss": 0.7046,
+      "step": 1260
+    },
+    {
+      "epoch": 7.405247813411079,
+      "grad_norm": 0.8417901396751404,
+      "learning_rate": 0.0002,
+      "loss": 0.6843,
+      "step": 1270
+    },
+    {
+      "epoch": 7.463556851311953,
+      "grad_norm": 0.848461389541626,
+      "learning_rate": 0.0002,
+      "loss": 0.6958,
+      "step": 1280
+    },
+    {
+      "epoch": 7.521865889212828,
+      "grad_norm": 0.8924468755722046,
+      "learning_rate": 0.0002,
+      "loss": 0.6577,
+      "step": 1290
+    },
+    {
+      "epoch": 7.580174927113703,
+      "grad_norm": 0.8705193400382996,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 1300
+    },
+    {
+      "epoch": 7.6384839650145775,
+      "grad_norm": 0.8253726363182068,
+      "learning_rate": 0.0002,
+      "loss": 0.7104,
+      "step": 1310
+    },
+    {
+      "epoch": 7.696793002915452,
+      "grad_norm": 0.8224742412567139,
+      "learning_rate": 0.0002,
+      "loss": 0.76,
+      "step": 1320
+    },
+    {
+      "epoch": 7.755102040816326,
+      "grad_norm": 0.9237377047538757,
+      "learning_rate": 0.0002,
+      "loss": 0.6781,
+      "step": 1330
+    },
+    {
+      "epoch": 7.813411078717201,
+      "grad_norm": 0.7956030368804932,
+      "learning_rate": 0.0002,
+      "loss": 0.6874,
+      "step": 1340
+    },
+    {
+      "epoch": 7.871720116618076,
+      "grad_norm": 0.8239567279815674,
+      "learning_rate": 0.0002,
+      "loss": 0.7018,
+      "step": 1350
+    },
+    {
+      "epoch": 7.930029154518951,
+      "grad_norm": 1.1445919275283813,
+      "learning_rate": 0.0002,
+      "loss": 0.7432,
+      "step": 1360
+    },
+    {
+      "epoch": 7.97667638483965,
+      "eval_loss": 1.4571114778518677,
+      "eval_runtime": 14.6461,
+      "eval_samples_per_second": 6.555,
+      "eval_steps_per_second": 0.819,
+      "step": 1368
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1368,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.959815277628621e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-1368/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:558d8475370dfee2c47b98ed05aca969bb536c25048d37c81a11582fa4f0e57c
+size 5560

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-171/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-171/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-171/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc678b997b2fb80901a9e0ed681e8300b9e4c2fdb51f6990d019028c82ebca0b
+size 80755416

Qwen2-7B-Instruct_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-2790-sd-1/checkpoint-171/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}