Upload folder using huggingface_hub

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e5905c5082b7d2b89d41d2c049454e25d02cc467e2836c76b4f9a7fbb5db3d9
+size 80755416

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e928e8a6a5225ad01e25ddedd62219168a7fb4c0b7a2694bf63f0846a8c3a2f8
+size 80755416

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0accfa910bf95b0438bc032c1226fb4233e3c7bbf0f6aa40391daee4986b4934
+size 41136570

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e4d8acb0b4146e742095d18ad10ac6a7258664d3bf30d404bd637011d111799
+size 14244

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd63c2c32c0c712ee205fa0b0e45fbc01f06929d093fa262218286d69efc2ec3
+size 1064

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-10136/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ea6b9e0f2ffc74d13ed62dee09419738d933df3bba81dd517190763f6d34686
+size 5560

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c1b9fc09c46ebd398a9ca9570cc1893be0b0ba106a56aef34b4f5057f9340d0
+size 80755416

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a73abdc4a065bb9804c3d036b055bafe38b37611a9413abaa25b6b373da2a7a5
+size 41136570

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e02f5bfd6e0ee0815e4891dacc360cb303584891cd91b5cbf535ea32965541bc
+size 14244

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b81b990422fb0fe80e5db50b31090ee2d61c8e34587acc656ffb46b1af7fb5f1
+size 1064

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-11584/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ea6b9e0f2ffc74d13ed62dee09419738d933df3bba81dd517190763f6d34686
+size 5560

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bae71ea7a0984c6ff17df14baf692f7e84d2d2bf705555e2755cb99ae6bc0a2
+size 80755416

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eac569460db1bea57a5c5db704cbc997ba0725e80899e8fc59a128890d868d64
+size 41136570

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e893e543afb598132a0b809b15f14129f1886c66ed8281d771d6883881e5e95f
+size 14244

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:032e0abf78de568a0b81156504f01409b6f7e03912c91371302afa894e32ae05
+size 1064

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/trainer_state.json

@@ -0,0 +1,1049 @@
+{
+  "best_metric": 1.1255040168762207,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448",
+  "epoch": 1.0,
+  "eval_steps": 10,
+  "global_step": 1448,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.006906077348066298,
+      "grad_norm": 0.2756422460079193,
+      "learning_rate": 0.0002,
+      "loss": 1.7827,
+      "step": 10
+    },
+    {
+      "epoch": 0.013812154696132596,
+      "grad_norm": 0.24023374915122986,
+      "learning_rate": 0.0002,
+      "loss": 1.4709,
+      "step": 20
+    },
+    {
+      "epoch": 0.020718232044198894,
+      "grad_norm": 0.2553252577781677,
+      "learning_rate": 0.0002,
+      "loss": 1.2645,
+      "step": 30
+    },
+    {
+      "epoch": 0.027624309392265192,
+      "grad_norm": 0.21703363955020905,
+      "learning_rate": 0.0002,
+      "loss": 1.3009,
+      "step": 40
+    },
+    {
+      "epoch": 0.034530386740331494,
+      "grad_norm": 0.24089135229587555,
+      "learning_rate": 0.0002,
+      "loss": 1.3019,
+      "step": 50
+    },
+    {
+      "epoch": 0.04143646408839779,
+      "grad_norm": 0.19410525262355804,
+      "learning_rate": 0.0002,
+      "loss": 1.1807,
+      "step": 60
+    },
+    {
+      "epoch": 0.04834254143646409,
+      "grad_norm": 0.23195500671863556,
+      "learning_rate": 0.0002,
+      "loss": 1.2095,
+      "step": 70
+    },
+    {
+      "epoch": 0.055248618784530384,
+      "grad_norm": 0.3088529109954834,
+      "learning_rate": 0.0002,
+      "loss": 1.2373,
+      "step": 80
+    },
+    {
+      "epoch": 0.062154696132596686,
+      "grad_norm": 0.21218286454677582,
+      "learning_rate": 0.0002,
+      "loss": 1.2693,
+      "step": 90
+    },
+    {
+      "epoch": 0.06906077348066299,
+      "grad_norm": 0.26224467158317566,
+      "learning_rate": 0.0002,
+      "loss": 1.073,
+      "step": 100
+    },
+    {
+      "epoch": 0.07596685082872928,
+      "grad_norm": 0.26181790232658386,
+      "learning_rate": 0.0002,
+      "loss": 1.1634,
+      "step": 110
+    },
+    {
+      "epoch": 0.08287292817679558,
+      "grad_norm": 0.21940255165100098,
+      "learning_rate": 0.0002,
+      "loss": 1.1445,
+      "step": 120
+    },
+    {
+      "epoch": 0.08977900552486189,
+      "grad_norm": 0.2149958610534668,
+      "learning_rate": 0.0002,
+      "loss": 1.0716,
+      "step": 130
+    },
+    {
+      "epoch": 0.09668508287292818,
+      "grad_norm": 0.23623034358024597,
+      "learning_rate": 0.0002,
+      "loss": 1.1216,
+      "step": 140
+    },
+    {
+      "epoch": 0.10359116022099447,
+      "grad_norm": 0.3235692083835602,
+      "learning_rate": 0.0002,
+      "loss": 1.1945,
+      "step": 150
+    },
+    {
+      "epoch": 0.11049723756906077,
+      "grad_norm": 0.2900988459587097,
+      "learning_rate": 0.0002,
+      "loss": 1.2782,
+      "step": 160
+    },
+    {
+      "epoch": 0.11740331491712708,
+      "grad_norm": 0.21585266292095184,
+      "learning_rate": 0.0002,
+      "loss": 1.3508,
+      "step": 170
+    },
+    {
+      "epoch": 0.12430939226519337,
+      "grad_norm": 0.23717999458312988,
+      "learning_rate": 0.0002,
+      "loss": 1.111,
+      "step": 180
+    },
+    {
+      "epoch": 0.13121546961325967,
+      "grad_norm": 0.23339584469795227,
+      "learning_rate": 0.0002,
+      "loss": 1.1339,
+      "step": 190
+    },
+    {
+      "epoch": 0.13812154696132597,
+      "grad_norm": 0.21428699791431427,
+      "learning_rate": 0.0002,
+      "loss": 1.0896,
+      "step": 200
+    },
+    {
+      "epoch": 0.14502762430939226,
+      "grad_norm": 0.2187514454126358,
+      "learning_rate": 0.0002,
+      "loss": 1.0733,
+      "step": 210
+    },
+    {
+      "epoch": 0.15193370165745856,
+      "grad_norm": 0.21560987830162048,
+      "learning_rate": 0.0002,
+      "loss": 1.1038,
+      "step": 220
+    },
+    {
+      "epoch": 0.15883977900552487,
+      "grad_norm": 0.21689754724502563,
+      "learning_rate": 0.0002,
+      "loss": 1.1225,
+      "step": 230
+    },
+    {
+      "epoch": 0.16574585635359115,
+      "grad_norm": 0.20956148207187653,
+      "learning_rate": 0.0002,
+      "loss": 1.1505,
+      "step": 240
+    },
+    {
+      "epoch": 0.17265193370165746,
+      "grad_norm": 0.25023630261421204,
+      "learning_rate": 0.0002,
+      "loss": 1.3259,
+      "step": 250
+    },
+    {
+      "epoch": 0.17955801104972377,
+      "grad_norm": 0.20211918652057648,
+      "learning_rate": 0.0002,
+      "loss": 1.0401,
+      "step": 260
+    },
+    {
+      "epoch": 0.18646408839779005,
+      "grad_norm": 0.21812127530574799,
+      "learning_rate": 0.0002,
+      "loss": 1.1022,
+      "step": 270
+    },
+    {
+      "epoch": 0.19337016574585636,
+      "grad_norm": 0.25310298800468445,
+      "learning_rate": 0.0002,
+      "loss": 1.3994,
+      "step": 280
+    },
+    {
+      "epoch": 0.20027624309392264,
+      "grad_norm": 0.27219468355178833,
+      "learning_rate": 0.0002,
+      "loss": 1.2534,
+      "step": 290
+    },
+    {
+      "epoch": 0.20718232044198895,
+      "grad_norm": 0.22803102433681488,
+      "learning_rate": 0.0002,
+      "loss": 1.1339,
+      "step": 300
+    },
+    {
+      "epoch": 0.21408839779005526,
+      "grad_norm": 0.21374811232089996,
+      "learning_rate": 0.0002,
+      "loss": 1.104,
+      "step": 310
+    },
+    {
+      "epoch": 0.22099447513812154,
+      "grad_norm": 0.2415817230939865,
+      "learning_rate": 0.0002,
+      "loss": 1.1239,
+      "step": 320
+    },
+    {
+      "epoch": 0.22790055248618785,
+      "grad_norm": 0.25831377506256104,
+      "learning_rate": 0.0002,
+      "loss": 1.3162,
+      "step": 330
+    },
+    {
+      "epoch": 0.23480662983425415,
+      "grad_norm": 0.21656718850135803,
+      "learning_rate": 0.0002,
+      "loss": 1.0214,
+      "step": 340
+    },
+    {
+      "epoch": 0.24171270718232044,
+      "grad_norm": 0.3059662878513336,
+      "learning_rate": 0.0002,
+      "loss": 1.2289,
+      "step": 350
+    },
+    {
+      "epoch": 0.24861878453038674,
+      "grad_norm": 0.27221348881721497,
+      "learning_rate": 0.0002,
+      "loss": 1.2495,
+      "step": 360
+    },
+    {
+      "epoch": 0.255524861878453,
+      "grad_norm": 0.20476201176643372,
+      "learning_rate": 0.0002,
+      "loss": 1.132,
+      "step": 370
+    },
+    {
+      "epoch": 0.26243093922651933,
+      "grad_norm": 0.23827573657035828,
+      "learning_rate": 0.0002,
+      "loss": 1.0487,
+      "step": 380
+    },
+    {
+      "epoch": 0.26933701657458564,
+      "grad_norm": 0.5039234161376953,
+      "learning_rate": 0.0002,
+      "loss": 1.0669,
+      "step": 390
+    },
+    {
+      "epoch": 0.27624309392265195,
+      "grad_norm": 0.207203209400177,
+      "learning_rate": 0.0002,
+      "loss": 1.0976,
+      "step": 400
+    },
+    {
+      "epoch": 0.28314917127071826,
+      "grad_norm": 0.24646668136119843,
+      "learning_rate": 0.0002,
+      "loss": 1.1001,
+      "step": 410
+    },
+    {
+      "epoch": 0.2900552486187845,
+      "grad_norm": 0.28638091683387756,
+      "learning_rate": 0.0002,
+      "loss": 1.236,
+      "step": 420
+    },
+    {
+      "epoch": 0.2969613259668508,
+      "grad_norm": 0.2341601401567459,
+      "learning_rate": 0.0002,
+      "loss": 1.2807,
+      "step": 430
+    },
+    {
+      "epoch": 0.30386740331491713,
+      "grad_norm": 0.20526322722434998,
+      "learning_rate": 0.0002,
+      "loss": 1.1774,
+      "step": 440
+    },
+    {
+      "epoch": 0.31077348066298344,
+      "grad_norm": 0.2122214436531067,
+      "learning_rate": 0.0002,
+      "loss": 1.0352,
+      "step": 450
+    },
+    {
+      "epoch": 0.31767955801104975,
+      "grad_norm": 0.27404525876045227,
+      "learning_rate": 0.0002,
+      "loss": 1.1159,
+      "step": 460
+    },
+    {
+      "epoch": 0.324585635359116,
+      "grad_norm": 0.5660505294799805,
+      "learning_rate": 0.0002,
+      "loss": 1.1764,
+      "step": 470
+    },
+    {
+      "epoch": 0.3314917127071823,
+      "grad_norm": 0.18058662116527557,
+      "learning_rate": 0.0002,
+      "loss": 1.144,
+      "step": 480
+    },
+    {
+      "epoch": 0.3383977900552486,
+      "grad_norm": 0.21312153339385986,
+      "learning_rate": 0.0002,
+      "loss": 1.2643,
+      "step": 490
+    },
+    {
+      "epoch": 0.3453038674033149,
+      "grad_norm": 0.2062060385942459,
+      "learning_rate": 0.0002,
+      "loss": 1.0834,
+      "step": 500
+    },
+    {
+      "epoch": 0.35220994475138123,
+      "grad_norm": 0.18960623443126678,
+      "learning_rate": 0.0002,
+      "loss": 1.0357,
+      "step": 510
+    },
+    {
+      "epoch": 0.35911602209944754,
+      "grad_norm": 0.1784827709197998,
+      "learning_rate": 0.0002,
+      "loss": 1.1298,
+      "step": 520
+    },
+    {
+      "epoch": 0.3660220994475138,
+      "grad_norm": 0.24883900582790375,
+      "learning_rate": 0.0002,
+      "loss": 1.1989,
+      "step": 530
+    },
+    {
+      "epoch": 0.3729281767955801,
+      "grad_norm": 0.18481262028217316,
+      "learning_rate": 0.0002,
+      "loss": 1.1444,
+      "step": 540
+    },
+    {
+      "epoch": 0.3798342541436464,
+      "grad_norm": 0.20193436741828918,
+      "learning_rate": 0.0002,
+      "loss": 1.0792,
+      "step": 550
+    },
+    {
+      "epoch": 0.3867403314917127,
+      "grad_norm": 0.23905101418495178,
+      "learning_rate": 0.0002,
+      "loss": 1.1019,
+      "step": 560
+    },
+    {
+      "epoch": 0.393646408839779,
+      "grad_norm": 0.2091626226902008,
+      "learning_rate": 0.0002,
+      "loss": 1.0779,
+      "step": 570
+    },
+    {
+      "epoch": 0.4005524861878453,
+      "grad_norm": 0.37083542346954346,
+      "learning_rate": 0.0002,
+      "loss": 1.1142,
+      "step": 580
+    },
+    {
+      "epoch": 0.4074585635359116,
+      "grad_norm": 0.22831453382968903,
+      "learning_rate": 0.0002,
+      "loss": 1.2294,
+      "step": 590
+    },
+    {
+      "epoch": 0.4143646408839779,
+      "grad_norm": 0.19660364091396332,
+      "learning_rate": 0.0002,
+      "loss": 1.0477,
+      "step": 600
+    },
+    {
+      "epoch": 0.4212707182320442,
+      "grad_norm": 0.24261155724525452,
+      "learning_rate": 0.0002,
+      "loss": 1.0534,
+      "step": 610
+    },
+    {
+      "epoch": 0.4281767955801105,
+      "grad_norm": 0.20413841307163239,
+      "learning_rate": 0.0002,
+      "loss": 1.0877,
+      "step": 620
+    },
+    {
+      "epoch": 0.4350828729281768,
+      "grad_norm": 0.15334749221801758,
+      "learning_rate": 0.0002,
+      "loss": 1.0738,
+      "step": 630
+    },
+    {
+      "epoch": 0.4419889502762431,
+      "grad_norm": 0.31286001205444336,
+      "learning_rate": 0.0002,
+      "loss": 1.2146,
+      "step": 640
+    },
+    {
+      "epoch": 0.4488950276243094,
+      "grad_norm": 0.1957162618637085,
+      "learning_rate": 0.0002,
+      "loss": 1.2343,
+      "step": 650
+    },
+    {
+      "epoch": 0.4558011049723757,
+      "grad_norm": 0.1765211671590805,
+      "learning_rate": 0.0002,
+      "loss": 1.1704,
+      "step": 660
+    },
+    {
+      "epoch": 0.462707182320442,
+      "grad_norm": 0.2002234011888504,
+      "learning_rate": 0.0002,
+      "loss": 1.2657,
+      "step": 670
+    },
+    {
+      "epoch": 0.4696132596685083,
+      "grad_norm": 0.1707477867603302,
+      "learning_rate": 0.0002,
+      "loss": 1.0751,
+      "step": 680
+    },
+    {
+      "epoch": 0.47651933701657456,
+      "grad_norm": 0.22710315883159637,
+      "learning_rate": 0.0002,
+      "loss": 1.08,
+      "step": 690
+    },
+    {
+      "epoch": 0.48342541436464087,
+      "grad_norm": 0.21569234132766724,
+      "learning_rate": 0.0002,
+      "loss": 0.9985,
+      "step": 700
+    },
+    {
+      "epoch": 0.4903314917127072,
+      "grad_norm": 0.17725816369056702,
+      "learning_rate": 0.0002,
+      "loss": 1.2896,
+      "step": 710
+    },
+    {
+      "epoch": 0.4972375690607735,
+      "grad_norm": 0.21359217166900635,
+      "learning_rate": 0.0002,
+      "loss": 1.0929,
+      "step": 720
+    },
+    {
+      "epoch": 0.5041436464088398,
+      "grad_norm": 0.22619548439979553,
+      "learning_rate": 0.0002,
+      "loss": 1.0851,
+      "step": 730
+    },
+    {
+      "epoch": 0.511049723756906,
+      "grad_norm": 0.20815977454185486,
+      "learning_rate": 0.0002,
+      "loss": 1.1034,
+      "step": 740
+    },
+    {
+      "epoch": 0.5179558011049724,
+      "grad_norm": 0.23183371126651764,
+      "learning_rate": 0.0002,
+      "loss": 1.2466,
+      "step": 750
+    },
+    {
+      "epoch": 0.5248618784530387,
+      "grad_norm": 0.225651815533638,
+      "learning_rate": 0.0002,
+      "loss": 1.2491,
+      "step": 760
+    },
+    {
+      "epoch": 0.5317679558011049,
+      "grad_norm": 0.23851291835308075,
+      "learning_rate": 0.0002,
+      "loss": 1.2037,
+      "step": 770
+    },
+    {
+      "epoch": 0.5386740331491713,
+      "grad_norm": 0.20826849341392517,
+      "learning_rate": 0.0002,
+      "loss": 1.0815,
+      "step": 780
+    },
+    {
+      "epoch": 0.5455801104972375,
+      "grad_norm": 0.1769036501646042,
+      "learning_rate": 0.0002,
+      "loss": 1.1869,
+      "step": 790
+    },
+    {
+      "epoch": 0.5524861878453039,
+      "grad_norm": 0.21358650922775269,
+      "learning_rate": 0.0002,
+      "loss": 1.0401,
+      "step": 800
+    },
+    {
+      "epoch": 0.5593922651933702,
+      "grad_norm": 0.2848151624202728,
+      "learning_rate": 0.0002,
+      "loss": 1.1141,
+      "step": 810
+    },
+    {
+      "epoch": 0.5662983425414365,
+      "grad_norm": 0.1954721212387085,
+      "learning_rate": 0.0002,
+      "loss": 1.0497,
+      "step": 820
+    },
+    {
+      "epoch": 0.5732044198895028,
+      "grad_norm": 0.23684093356132507,
+      "learning_rate": 0.0002,
+      "loss": 1.0026,
+      "step": 830
+    },
+    {
+      "epoch": 0.580110497237569,
+      "grad_norm": 0.23327535390853882,
+      "learning_rate": 0.0002,
+      "loss": 1.175,
+      "step": 840
+    },
+    {
+      "epoch": 0.5870165745856354,
+      "grad_norm": 0.20750251412391663,
+      "learning_rate": 0.0002,
+      "loss": 1.1243,
+      "step": 850
+    },
+    {
+      "epoch": 0.5939226519337016,
+      "grad_norm": 0.20300744473934174,
+      "learning_rate": 0.0002,
+      "loss": 1.0817,
+      "step": 860
+    },
+    {
+      "epoch": 0.600828729281768,
+      "grad_norm": 0.2262803614139557,
+      "learning_rate": 0.0002,
+      "loss": 1.2388,
+      "step": 870
+    },
+    {
+      "epoch": 0.6077348066298343,
+      "grad_norm": 0.19671843945980072,
+      "learning_rate": 0.0002,
+      "loss": 1.0605,
+      "step": 880
+    },
+    {
+      "epoch": 0.6146408839779005,
+      "grad_norm": 0.2702012360095978,
+      "learning_rate": 0.0002,
+      "loss": 1.1471,
+      "step": 890
+    },
+    {
+      "epoch": 0.6215469613259669,
+      "grad_norm": 0.2425249218940735,
+      "learning_rate": 0.0002,
+      "loss": 1.0883,
+      "step": 900
+    },
+    {
+      "epoch": 0.6284530386740331,
+      "grad_norm": 0.24542637169361115,
+      "learning_rate": 0.0002,
+      "loss": 1.0984,
+      "step": 910
+    },
+    {
+      "epoch": 0.6353591160220995,
+      "grad_norm": 0.2246340662240982,
+      "learning_rate": 0.0002,
+      "loss": 1.083,
+      "step": 920
+    },
+    {
+      "epoch": 0.6422651933701657,
+      "grad_norm": 0.2105531394481659,
+      "learning_rate": 0.0002,
+      "loss": 1.1466,
+      "step": 930
+    },
+    {
+      "epoch": 0.649171270718232,
+      "grad_norm": 0.22141265869140625,
+      "learning_rate": 0.0002,
+      "loss": 1.0162,
+      "step": 940
+    },
+    {
+      "epoch": 0.6560773480662984,
+      "grad_norm": 0.21778923273086548,
+      "learning_rate": 0.0002,
+      "loss": 1.2384,
+      "step": 950
+    },
+    {
+      "epoch": 0.6629834254143646,
+      "grad_norm": 0.23456862568855286,
+      "learning_rate": 0.0002,
+      "loss": 1.1523,
+      "step": 960
+    },
+    {
+      "epoch": 0.669889502762431,
+      "grad_norm": 0.3211105763912201,
+      "learning_rate": 0.0002,
+      "loss": 1.1379,
+      "step": 970
+    },
+    {
+      "epoch": 0.6767955801104972,
+      "grad_norm": 0.19384050369262695,
+      "learning_rate": 0.0002,
+      "loss": 1.0192,
+      "step": 980
+    },
+    {
+      "epoch": 0.6837016574585635,
+      "grad_norm": 0.22961807250976562,
+      "learning_rate": 0.0002,
+      "loss": 1.1487,
+      "step": 990
+    },
+    {
+      "epoch": 0.6906077348066298,
+      "grad_norm": 0.17578937113285065,
+      "learning_rate": 0.0002,
+      "loss": 1.0838,
+      "step": 1000
+    },
+    {
+      "epoch": 0.6975138121546961,
+      "grad_norm": 0.19182339310646057,
+      "learning_rate": 0.0002,
+      "loss": 1.1003,
+      "step": 1010
+    },
+    {
+      "epoch": 0.7044198895027625,
+      "grad_norm": 0.20082128047943115,
+      "learning_rate": 0.0002,
+      "loss": 1.0888,
+      "step": 1020
+    },
+    {
+      "epoch": 0.7113259668508287,
+      "grad_norm": 0.17474359273910522,
+      "learning_rate": 0.0002,
+      "loss": 1.0657,
+      "step": 1030
+    },
+    {
+      "epoch": 0.7182320441988951,
+      "grad_norm": 0.196266770362854,
+      "learning_rate": 0.0002,
+      "loss": 1.1528,
+      "step": 1040
+    },
+    {
+      "epoch": 0.7251381215469613,
+      "grad_norm": 0.176788792014122,
+      "learning_rate": 0.0002,
+      "loss": 0.923,
+      "step": 1050
+    },
+    {
+      "epoch": 0.7320441988950276,
+      "grad_norm": 0.24255797266960144,
+      "learning_rate": 0.0002,
+      "loss": 1.0158,
+      "step": 1060
+    },
+    {
+      "epoch": 0.738950276243094,
+      "grad_norm": 0.19366870820522308,
+      "learning_rate": 0.0002,
+      "loss": 0.9921,
+      "step": 1070
+    },
+    {
+      "epoch": 0.7458563535911602,
+      "grad_norm": 0.20018735527992249,
+      "learning_rate": 0.0002,
+      "loss": 1.1521,
+      "step": 1080
+    },
+    {
+      "epoch": 0.7527624309392266,
+      "grad_norm": 0.18794989585876465,
+      "learning_rate": 0.0002,
+      "loss": 0.959,
+      "step": 1090
+    },
+    {
+      "epoch": 0.7596685082872928,
+      "grad_norm": 0.181159108877182,
+      "learning_rate": 0.0002,
+      "loss": 1.0171,
+      "step": 1100
+    },
+    {
+      "epoch": 0.7665745856353591,
+      "grad_norm": 0.19768114387989044,
+      "learning_rate": 0.0002,
+      "loss": 1.1975,
+      "step": 1110
+    },
+    {
+      "epoch": 0.7734806629834254,
+      "grad_norm": 0.2164643406867981,
+      "learning_rate": 0.0002,
+      "loss": 1.1263,
+      "step": 1120
+    },
+    {
+      "epoch": 0.7803867403314917,
+      "grad_norm": 0.16902120411396027,
+      "learning_rate": 0.0002,
+      "loss": 1.0438,
+      "step": 1130
+    },
+    {
+      "epoch": 0.787292817679558,
+      "grad_norm": 0.19257378578186035,
+      "learning_rate": 0.0002,
+      "loss": 1.1698,
+      "step": 1140
+    },
+    {
+      "epoch": 0.7941988950276243,
+      "grad_norm": 0.3037569522857666,
+      "learning_rate": 0.0002,
+      "loss": 1.1421,
+      "step": 1150
+    },
+    {
+      "epoch": 0.8011049723756906,
+      "grad_norm": 0.2325952649116516,
+      "learning_rate": 0.0002,
+      "loss": 1.2362,
+      "step": 1160
+    },
+    {
+      "epoch": 0.8080110497237569,
+      "grad_norm": 0.234732985496521,
+      "learning_rate": 0.0002,
+      "loss": 1.2024,
+      "step": 1170
+    },
+    {
+      "epoch": 0.8149171270718232,
+      "grad_norm": 0.18733015656471252,
+      "learning_rate": 0.0002,
+      "loss": 1.0745,
+      "step": 1180
+    },
+    {
+      "epoch": 0.8218232044198895,
+      "grad_norm": 0.25279903411865234,
+      "learning_rate": 0.0002,
+      "loss": 1.1617,
+      "step": 1190
+    },
+    {
+      "epoch": 0.8287292817679558,
+      "grad_norm": 0.2609824538230896,
+      "learning_rate": 0.0002,
+      "loss": 1.0834,
+      "step": 1200
+    },
+    {
+      "epoch": 0.835635359116022,
+      "grad_norm": 0.21643219888210297,
+      "learning_rate": 0.0002,
+      "loss": 1.1303,
+      "step": 1210
+    },
+    {
+      "epoch": 0.8425414364640884,
+      "grad_norm": 0.16698487102985382,
+      "learning_rate": 0.0002,
+      "loss": 1.0872,
+      "step": 1220
+    },
+    {
+      "epoch": 0.8494475138121547,
+      "grad_norm": 0.19885262846946716,
+      "learning_rate": 0.0002,
+      "loss": 1.0828,
+      "step": 1230
+    },
+    {
+      "epoch": 0.856353591160221,
+      "grad_norm": 0.2277640998363495,
+      "learning_rate": 0.0002,
+      "loss": 1.2445,
+      "step": 1240
+    },
+    {
+      "epoch": 0.8632596685082873,
+      "grad_norm": 0.195664644241333,
+      "learning_rate": 0.0002,
+      "loss": 1.1022,
+      "step": 1250
+    },
+    {
+      "epoch": 0.8701657458563536,
+      "grad_norm": 0.20432411134243011,
+      "learning_rate": 0.0002,
+      "loss": 1.1499,
+      "step": 1260
+    },
+    {
+      "epoch": 0.8770718232044199,
+      "grad_norm": 0.2625722289085388,
+      "learning_rate": 0.0002,
+      "loss": 0.9963,
+      "step": 1270
+    },
+    {
+      "epoch": 0.8839779005524862,
+      "grad_norm": 0.2837998867034912,
+      "learning_rate": 0.0002,
+      "loss": 1.1232,
+      "step": 1280
+    },
+    {
+      "epoch": 0.8908839779005525,
+      "grad_norm": 0.1976904720067978,
+      "learning_rate": 0.0002,
+      "loss": 1.0358,
+      "step": 1290
+    },
+    {
+      "epoch": 0.8977900552486188,
+      "grad_norm": 0.21545688807964325,
+      "learning_rate": 0.0002,
+      "loss": 1.0575,
+      "step": 1300
+    },
+    {
+      "epoch": 0.9046961325966851,
+      "grad_norm": 0.2800905704498291,
+      "learning_rate": 0.0002,
+      "loss": 1.1985,
+      "step": 1310
+    },
+    {
+      "epoch": 0.9116022099447514,
+      "grad_norm": 0.23891423642635345,
+      "learning_rate": 0.0002,
+      "loss": 1.2247,
+      "step": 1320
+    },
+    {
+      "epoch": 0.9185082872928176,
+      "grad_norm": 0.26337701082229614,
+      "learning_rate": 0.0002,
+      "loss": 1.0509,
+      "step": 1330
+    },
+    {
+      "epoch": 0.925414364640884,
+      "grad_norm": 0.21950937807559967,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 1340
+    },
+    {
+      "epoch": 0.9323204419889503,
+      "grad_norm": 0.19111743569374084,
+      "learning_rate": 0.0002,
+      "loss": 1.0635,
+      "step": 1350
+    },
+    {
+      "epoch": 0.9392265193370166,
+      "grad_norm": 0.16557033360004425,
+      "learning_rate": 0.0002,
+      "loss": 1.1305,
+      "step": 1360
+    },
+    {
+      "epoch": 0.9461325966850829,
+      "grad_norm": 0.3363538980484009,
+      "learning_rate": 0.0002,
+      "loss": 1.0869,
+      "step": 1370
+    },
+    {
+      "epoch": 0.9530386740331491,
+      "grad_norm": 0.2307894080877304,
+      "learning_rate": 0.0002,
+      "loss": 1.027,
+      "step": 1380
+    },
+    {
+      "epoch": 0.9599447513812155,
+      "grad_norm": 0.23236998915672302,
+      "learning_rate": 0.0002,
+      "loss": 1.2408,
+      "step": 1390
+    },
+    {
+      "epoch": 0.9668508287292817,
+      "grad_norm": 0.20302017033100128,
+      "learning_rate": 0.0002,
+      "loss": 1.2403,
+      "step": 1400
+    },
+    {
+      "epoch": 0.9737569060773481,
+      "grad_norm": 0.18202321231365204,
+      "learning_rate": 0.0002,
+      "loss": 1.0732,
+      "step": 1410
+    },
+    {
+      "epoch": 0.9806629834254144,
+      "grad_norm": 0.21759551763534546,
+      "learning_rate": 0.0002,
+      "loss": 1.0514,
+      "step": 1420
+    },
+    {
+      "epoch": 0.9875690607734806,
+      "grad_norm": 0.24724246561527252,
+      "learning_rate": 0.0002,
+      "loss": 1.2664,
+      "step": 1430
+    },
+    {
+      "epoch": 0.994475138121547,
+      "grad_norm": 0.25889360904693604,
+      "learning_rate": 0.0002,
+      "loss": 1.1621,
+      "step": 1440
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.1255040168762207,
+      "eval_runtime": 32.5795,
+      "eval_samples_per_second": 14.058,
+      "eval_steps_per_second": 1.78,
+      "step": 1448
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 11584,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.308342486846669e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-1448/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ea6b9e0f2ffc74d13ed62dee09419738d933df3bba81dd517190763f6d34686
+size 5560

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-2896/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-2896/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-2896/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:883af189f053bf8640dbabdda63e87533644890eb87f62fc714d060c03646609
+size 80755416

Qwen2-7B-Instruct_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-8828-sd-42/checkpoint-2896/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}