Upload folder using huggingface_hub

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a3cf595a971f592cd10a704bc9c64b45b510e900d19a03a48be7122d740bc6
+size 80755416

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e40925b3751ae39cdd41b615f93bd8ad2ae6ce73fc5458aefaa424c27800ce6
+size 80755416

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:012fedd7fe455283743764e9a98e6c056e9060b9039d2a7d5ca2742d4a626bb8
+size 41136570

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b764063ed9737012a48d4e83df8049ca6095ed9055b326b37bc9ccd9bfa5600
+size 14244

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5000a5052c8d090ffd908477f337cfaa88890bbadf7e2e9c16cda94eee348adf
+size 1064

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/trainer_state.json

@@ -0,0 +1,926 @@
+{
+  "best_metric": 1.7614221572875977,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-615",
+  "epoch": 4.0,
+  "eval_steps": 10,
+  "global_step": 1230,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.032520325203252036,
+      "grad_norm": 0.3243047297000885,
+      "learning_rate": 0.0002,
+      "loss": 2.4549,
+      "step": 10
+    },
+    {
+      "epoch": 0.06504065040650407,
+      "grad_norm": 0.3810383975505829,
+      "learning_rate": 0.0002,
+      "loss": 2.0662,
+      "step": 20
+    },
+    {
+      "epoch": 0.0975609756097561,
+      "grad_norm": 0.3727293908596039,
+      "learning_rate": 0.0002,
+      "loss": 1.9994,
+      "step": 30
+    },
+    {
+      "epoch": 0.13008130081300814,
+      "grad_norm": 0.33587437868118286,
+      "learning_rate": 0.0002,
+      "loss": 1.8857,
+      "step": 40
+    },
+    {
+      "epoch": 0.16260162601626016,
+      "grad_norm": 0.29074037075042725,
+      "learning_rate": 0.0002,
+      "loss": 1.8232,
+      "step": 50
+    },
+    {
+      "epoch": 0.1951219512195122,
+      "grad_norm": 0.3021906316280365,
+      "learning_rate": 0.0002,
+      "loss": 1.7747,
+      "step": 60
+    },
+    {
+      "epoch": 0.22764227642276422,
+      "grad_norm": 0.29506710171699524,
+      "learning_rate": 0.0002,
+      "loss": 1.8284,
+      "step": 70
+    },
+    {
+      "epoch": 0.2601626016260163,
+      "grad_norm": 0.32335522770881653,
+      "learning_rate": 0.0002,
+      "loss": 1.7864,
+      "step": 80
+    },
+    {
+      "epoch": 0.2926829268292683,
+      "grad_norm": 0.245526984333992,
+      "learning_rate": 0.0002,
+      "loss": 1.8362,
+      "step": 90
+    },
+    {
+      "epoch": 0.3252032520325203,
+      "grad_norm": 0.260805606842041,
+      "learning_rate": 0.0002,
+      "loss": 1.7142,
+      "step": 100
+    },
+    {
+      "epoch": 0.35772357723577236,
+      "grad_norm": 0.2878130376338959,
+      "learning_rate": 0.0002,
+      "loss": 1.7665,
+      "step": 110
+    },
+    {
+      "epoch": 0.3902439024390244,
+      "grad_norm": 0.263217568397522,
+      "learning_rate": 0.0002,
+      "loss": 1.6984,
+      "step": 120
+    },
+    {
+      "epoch": 0.42276422764227645,
+      "grad_norm": 0.37357720732688904,
+      "learning_rate": 0.0002,
+      "loss": 1.8168,
+      "step": 130
+    },
+    {
+      "epoch": 0.45528455284552843,
+      "grad_norm": 0.2695927023887634,
+      "learning_rate": 0.0002,
+      "loss": 1.7649,
+      "step": 140
+    },
+    {
+      "epoch": 0.4878048780487805,
+      "grad_norm": 0.3133211135864258,
+      "learning_rate": 0.0002,
+      "loss": 1.7631,
+      "step": 150
+    },
+    {
+      "epoch": 0.5203252032520326,
+      "grad_norm": 0.2665450870990753,
+      "learning_rate": 0.0002,
+      "loss": 1.7223,
+      "step": 160
+    },
+    {
+      "epoch": 0.5528455284552846,
+      "grad_norm": 0.2663995921611786,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 170
+    },
+    {
+      "epoch": 0.5853658536585366,
+      "grad_norm": 0.2748648226261139,
+      "learning_rate": 0.0002,
+      "loss": 1.7395,
+      "step": 180
+    },
+    {
+      "epoch": 0.6178861788617886,
+      "grad_norm": 0.2700922191143036,
+      "learning_rate": 0.0002,
+      "loss": 1.8011,
+      "step": 190
+    },
+    {
+      "epoch": 0.6504065040650406,
+      "grad_norm": 0.2584550678730011,
+      "learning_rate": 0.0002,
+      "loss": 1.7561,
+      "step": 200
+    },
+    {
+      "epoch": 0.6829268292682927,
+      "grad_norm": 0.23968899250030518,
+      "learning_rate": 0.0002,
+      "loss": 1.6472,
+      "step": 210
+    },
+    {
+      "epoch": 0.7154471544715447,
+      "grad_norm": 0.23806197941303253,
+      "learning_rate": 0.0002,
+      "loss": 1.7167,
+      "step": 220
+    },
+    {
+      "epoch": 0.7479674796747967,
+      "grad_norm": 0.23665426671504974,
+      "learning_rate": 0.0002,
+      "loss": 1.7429,
+      "step": 230
+    },
+    {
+      "epoch": 0.7804878048780488,
+      "grad_norm": 0.2510588467121124,
+      "learning_rate": 0.0002,
+      "loss": 1.7044,
+      "step": 240
+    },
+    {
+      "epoch": 0.8130081300813008,
+      "grad_norm": 0.2480899840593338,
+      "learning_rate": 0.0002,
+      "loss": 1.7843,
+      "step": 250
+    },
+    {
+      "epoch": 0.8455284552845529,
+      "grad_norm": 0.2802377939224243,
+      "learning_rate": 0.0002,
+      "loss": 1.7034,
+      "step": 260
+    },
+    {
+      "epoch": 0.8780487804878049,
+      "grad_norm": 0.2634681165218353,
+      "learning_rate": 0.0002,
+      "loss": 1.7354,
+      "step": 270
+    },
+    {
+      "epoch": 0.9105691056910569,
+      "grad_norm": 0.2952527403831482,
+      "learning_rate": 0.0002,
+      "loss": 1.7428,
+      "step": 280
+    },
+    {
+      "epoch": 0.943089430894309,
+      "grad_norm": 0.22973614931106567,
+      "learning_rate": 0.0002,
+      "loss": 1.6869,
+      "step": 290
+    },
+    {
+      "epoch": 0.975609756097561,
+      "grad_norm": 0.23497417569160461,
+      "learning_rate": 0.0002,
+      "loss": 1.7412,
+      "step": 300
+    },
+    {
+      "epoch": 0.9983739837398374,
+      "eval_loss": 1.7698818445205688,
+      "eval_runtime": 38.0302,
+      "eval_samples_per_second": 13.989,
+      "eval_steps_per_second": 1.762,
+      "step": 307
+    },
+    {
+      "epoch": 1.008130081300813,
+      "grad_norm": 0.2552054822444916,
+      "learning_rate": 0.0002,
+      "loss": 1.7849,
+      "step": 310
+    },
+    {
+      "epoch": 1.040650406504065,
+      "grad_norm": 0.21663406491279602,
+      "learning_rate": 0.0002,
+      "loss": 1.7233,
+      "step": 320
+    },
+    {
+      "epoch": 1.0731707317073171,
+      "grad_norm": 0.2413843721151352,
+      "learning_rate": 0.0002,
+      "loss": 1.7069,
+      "step": 330
+    },
+    {
+      "epoch": 1.1056910569105691,
+      "grad_norm": 0.2675042450428009,
+      "learning_rate": 0.0002,
+      "loss": 1.6097,
+      "step": 340
+    },
+    {
+      "epoch": 1.1382113821138211,
+      "grad_norm": 0.2572104334831238,
+      "learning_rate": 0.0002,
+      "loss": 1.6697,
+      "step": 350
+    },
+    {
+      "epoch": 1.170731707317073,
+      "grad_norm": 0.2535107731819153,
+      "learning_rate": 0.0002,
+      "loss": 1.6972,
+      "step": 360
+    },
+    {
+      "epoch": 1.203252032520325,
+      "grad_norm": 0.2549455165863037,
+      "learning_rate": 0.0002,
+      "loss": 1.7133,
+      "step": 370
+    },
+    {
+      "epoch": 1.2357723577235773,
+      "grad_norm": 0.27836501598358154,
+      "learning_rate": 0.0002,
+      "loss": 1.757,
+      "step": 380
+    },
+    {
+      "epoch": 1.2682926829268293,
+      "grad_norm": 0.26294031739234924,
+      "learning_rate": 0.0002,
+      "loss": 1.652,
+      "step": 390
+    },
+    {
+      "epoch": 1.3008130081300813,
+      "grad_norm": 0.25705304741859436,
+      "learning_rate": 0.0002,
+      "loss": 1.6738,
+      "step": 400
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.2765929698944092,
+      "learning_rate": 0.0002,
+      "loss": 1.6396,
+      "step": 410
+    },
+    {
+      "epoch": 1.3658536585365852,
+      "grad_norm": 0.2620247006416321,
+      "learning_rate": 0.0002,
+      "loss": 1.7496,
+      "step": 420
+    },
+    {
+      "epoch": 1.3983739837398375,
+      "grad_norm": 0.28890499472618103,
+      "learning_rate": 0.0002,
+      "loss": 1.7139,
+      "step": 430
+    },
+    {
+      "epoch": 1.4308943089430894,
+      "grad_norm": 0.524259090423584,
+      "learning_rate": 0.0002,
+      "loss": 1.6881,
+      "step": 440
+    },
+    {
+      "epoch": 1.4634146341463414,
+      "grad_norm": 0.279712438583374,
+      "learning_rate": 0.0002,
+      "loss": 1.6994,
+      "step": 450
+    },
+    {
+      "epoch": 1.4959349593495934,
+      "grad_norm": 0.2770076096057892,
+      "learning_rate": 0.0002,
+      "loss": 1.6853,
+      "step": 460
+    },
+    {
+      "epoch": 1.5284552845528454,
+      "grad_norm": 0.2519102394580841,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 470
+    },
+    {
+      "epoch": 1.5609756097560976,
+      "grad_norm": 0.2679072916507721,
+      "learning_rate": 0.0002,
+      "loss": 1.6689,
+      "step": 480
+    },
+    {
+      "epoch": 1.5934959349593496,
+      "grad_norm": 0.2848704159259796,
+      "learning_rate": 0.0002,
+      "loss": 1.6938,
+      "step": 490
+    },
+    {
+      "epoch": 1.6260162601626016,
+      "grad_norm": 0.2666904032230377,
+      "learning_rate": 0.0002,
+      "loss": 1.6752,
+      "step": 500
+    },
+    {
+      "epoch": 1.6585365853658538,
+      "grad_norm": 0.24341872334480286,
+      "learning_rate": 0.0002,
+      "loss": 1.7231,
+      "step": 510
+    },
+    {
+      "epoch": 1.6910569105691056,
+      "grad_norm": 0.24182367324829102,
+      "learning_rate": 0.0002,
+      "loss": 1.6827,
+      "step": 520
+    },
+    {
+      "epoch": 1.7235772357723578,
+      "grad_norm": 0.266201913356781,
+      "learning_rate": 0.0002,
+      "loss": 1.7059,
+      "step": 530
+    },
+    {
+      "epoch": 1.7560975609756098,
+      "grad_norm": 0.26361775398254395,
+      "learning_rate": 0.0002,
+      "loss": 1.7196,
+      "step": 540
+    },
+    {
+      "epoch": 1.7886178861788617,
+      "grad_norm": 0.2974611818790436,
+      "learning_rate": 0.0002,
+      "loss": 1.698,
+      "step": 550
+    },
+    {
+      "epoch": 1.821138211382114,
+      "grad_norm": 0.26264363527297974,
+      "learning_rate": 0.0002,
+      "loss": 1.6913,
+      "step": 560
+    },
+    {
+      "epoch": 1.8536585365853657,
+      "grad_norm": 0.31887590885162354,
+      "learning_rate": 0.0002,
+      "loss": 1.7373,
+      "step": 570
+    },
+    {
+      "epoch": 1.886178861788618,
+      "grad_norm": 0.25847622752189636,
+      "learning_rate": 0.0002,
+      "loss": 1.7269,
+      "step": 580
+    },
+    {
+      "epoch": 1.91869918699187,
+      "grad_norm": 0.2770395576953888,
+      "learning_rate": 0.0002,
+      "loss": 1.7126,
+      "step": 590
+    },
+    {
+      "epoch": 1.951219512195122,
+      "grad_norm": 0.24041323363780975,
+      "learning_rate": 0.0002,
+      "loss": 1.6897,
+      "step": 600
+    },
+    {
+      "epoch": 1.9837398373983741,
+      "grad_norm": 0.25060364603996277,
+      "learning_rate": 0.0002,
+      "loss": 1.6904,
+      "step": 610
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.7614221572875977,
+      "eval_runtime": 38.0401,
+      "eval_samples_per_second": 13.985,
+      "eval_steps_per_second": 1.761,
+      "step": 615
+    },
+    {
+      "epoch": 2.016260162601626,
+      "grad_norm": 0.25516635179519653,
+      "learning_rate": 0.0002,
+      "loss": 1.7299,
+      "step": 620
+    },
+    {
+      "epoch": 2.048780487804878,
+      "grad_norm": 0.31203776597976685,
+      "learning_rate": 0.0002,
+      "loss": 1.6142,
+      "step": 630
+    },
+    {
+      "epoch": 2.08130081300813,
+      "grad_norm": 0.27818965911865234,
+      "learning_rate": 0.0002,
+      "loss": 1.6519,
+      "step": 640
+    },
+    {
+      "epoch": 2.113821138211382,
+      "grad_norm": 0.2781952917575836,
+      "learning_rate": 0.0002,
+      "loss": 1.6678,
+      "step": 650
+    },
+    {
+      "epoch": 2.1463414634146343,
+      "grad_norm": 0.2937796413898468,
+      "learning_rate": 0.0002,
+      "loss": 1.5723,
+      "step": 660
+    },
+    {
+      "epoch": 2.178861788617886,
+      "grad_norm": 0.3559171259403229,
+      "learning_rate": 0.0002,
+      "loss": 1.6118,
+      "step": 670
+    },
+    {
+      "epoch": 2.2113821138211383,
+      "grad_norm": 0.3274490237236023,
+      "learning_rate": 0.0002,
+      "loss": 1.6044,
+      "step": 680
+    },
+    {
+      "epoch": 2.2439024390243905,
+      "grad_norm": 0.34521982073783875,
+      "learning_rate": 0.0002,
+      "loss": 1.6521,
+      "step": 690
+    },
+    {
+      "epoch": 2.2764227642276422,
+      "grad_norm": 0.30293744802474976,
+      "learning_rate": 0.0002,
+      "loss": 1.6075,
+      "step": 700
+    },
+    {
+      "epoch": 2.3089430894308944,
+      "grad_norm": 0.31012430787086487,
+      "learning_rate": 0.0002,
+      "loss": 1.6843,
+      "step": 710
+    },
+    {
+      "epoch": 2.341463414634146,
+      "grad_norm": 0.3424604535102844,
+      "learning_rate": 0.0002,
+      "loss": 1.6524,
+      "step": 720
+    },
+    {
+      "epoch": 2.3739837398373984,
+      "grad_norm": 0.3361012041568756,
+      "learning_rate": 0.0002,
+      "loss": 1.6404,
+      "step": 730
+    },
+    {
+      "epoch": 2.40650406504065,
+      "grad_norm": 0.308322012424469,
+      "learning_rate": 0.0002,
+      "loss": 1.6335,
+      "step": 740
+    },
+    {
+      "epoch": 2.4390243902439024,
+      "grad_norm": 0.3367919921875,
+      "learning_rate": 0.0002,
+      "loss": 1.6867,
+      "step": 750
+    },
+    {
+      "epoch": 2.4715447154471546,
+      "grad_norm": 0.3119719922542572,
+      "learning_rate": 0.0002,
+      "loss": 1.6332,
+      "step": 760
+    },
+    {
+      "epoch": 2.5040650406504064,
+      "grad_norm": 0.31164902448654175,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 770
+    },
+    {
+      "epoch": 2.5365853658536586,
+      "grad_norm": 0.3472500443458557,
+      "learning_rate": 0.0002,
+      "loss": 1.5806,
+      "step": 780
+    },
+    {
+      "epoch": 2.569105691056911,
+      "grad_norm": 0.32276982069015503,
+      "learning_rate": 0.0002,
+      "loss": 1.6171,
+      "step": 790
+    },
+    {
+      "epoch": 2.6016260162601625,
+      "grad_norm": 0.2904753088951111,
+      "learning_rate": 0.0002,
+      "loss": 1.5879,
+      "step": 800
+    },
+    {
+      "epoch": 2.6341463414634148,
+      "grad_norm": 0.3249073624610901,
+      "learning_rate": 0.0002,
+      "loss": 1.5967,
+      "step": 810
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.35803690552711487,
+      "learning_rate": 0.0002,
+      "loss": 1.7066,
+      "step": 820
+    },
+    {
+      "epoch": 2.6991869918699187,
+      "grad_norm": 0.3210543990135193,
+      "learning_rate": 0.0002,
+      "loss": 1.6163,
+      "step": 830
+    },
+    {
+      "epoch": 2.7317073170731705,
+      "grad_norm": 0.330085813999176,
+      "learning_rate": 0.0002,
+      "loss": 1.6599,
+      "step": 840
+    },
+    {
+      "epoch": 2.7642276422764227,
+      "grad_norm": 0.35581567883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.6338,
+      "step": 850
+    },
+    {
+      "epoch": 2.796747967479675,
+      "grad_norm": 0.3423900008201599,
+      "learning_rate": 0.0002,
+      "loss": 1.6033,
+      "step": 860
+    },
+    {
+      "epoch": 2.8292682926829267,
+      "grad_norm": 0.33455702662467957,
+      "learning_rate": 0.0002,
+      "loss": 1.6406,
+      "step": 870
+    },
+    {
+      "epoch": 2.861788617886179,
+      "grad_norm": 0.37637296319007874,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 880
+    },
+    {
+      "epoch": 2.894308943089431,
+      "grad_norm": 0.3715825080871582,
+      "learning_rate": 0.0002,
+      "loss": 1.6166,
+      "step": 890
+    },
+    {
+      "epoch": 2.926829268292683,
+      "grad_norm": 0.3526729345321655,
+      "learning_rate": 0.0002,
+      "loss": 1.7131,
+      "step": 900
+    },
+    {
+      "epoch": 2.959349593495935,
+      "grad_norm": 0.33071836829185486,
+      "learning_rate": 0.0002,
+      "loss": 1.5714,
+      "step": 910
+    },
+    {
+      "epoch": 2.991869918699187,
+      "grad_norm": 0.2956294119358063,
+      "learning_rate": 0.0002,
+      "loss": 1.6251,
+      "step": 920
+    },
+    {
+      "epoch": 2.9983739837398375,
+      "eval_loss": 1.7697627544403076,
+      "eval_runtime": 38.0483,
+      "eval_samples_per_second": 13.982,
+      "eval_steps_per_second": 1.761,
+      "step": 922
+    },
+    {
+      "epoch": 3.024390243902439,
+      "grad_norm": 0.35567474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.5265,
+      "step": 930
+    },
+    {
+      "epoch": 3.0569105691056913,
+      "grad_norm": 0.36280205845832825,
+      "learning_rate": 0.0002,
+      "loss": 1.5131,
+      "step": 940
+    },
+    {
+      "epoch": 3.089430894308943,
+      "grad_norm": 0.39025646448135376,
+      "learning_rate": 0.0002,
+      "loss": 1.5492,
+      "step": 950
+    },
+    {
+      "epoch": 3.1219512195121952,
+      "grad_norm": 0.4128074049949646,
+      "learning_rate": 0.0002,
+      "loss": 1.6059,
+      "step": 960
+    },
+    {
+      "epoch": 3.154471544715447,
+      "grad_norm": 0.37436643242836,
+      "learning_rate": 0.0002,
+      "loss": 1.5675,
+      "step": 970
+    },
+    {
+      "epoch": 3.186991869918699,
+      "grad_norm": 0.3950212895870209,
+      "learning_rate": 0.0002,
+      "loss": 1.5018,
+      "step": 980
+    },
+    {
+      "epoch": 3.2195121951219514,
+      "grad_norm": 0.3334033787250519,
+      "learning_rate": 0.0002,
+      "loss": 1.4856,
+      "step": 990
+    },
+    {
+      "epoch": 3.252032520325203,
+      "grad_norm": 0.41348570585250854,
+      "learning_rate": 0.0002,
+      "loss": 1.5288,
+      "step": 1000
+    },
+    {
+      "epoch": 3.2845528455284554,
+      "grad_norm": 0.48985755443573,
+      "learning_rate": 0.0002,
+      "loss": 1.565,
+      "step": 1010
+    },
+    {
+      "epoch": 3.317073170731707,
+      "grad_norm": 0.36514127254486084,
+      "learning_rate": 0.0002,
+      "loss": 1.5463,
+      "step": 1020
+    },
+    {
+      "epoch": 3.3495934959349594,
+      "grad_norm": 0.4296952188014984,
+      "learning_rate": 0.0002,
+      "loss": 1.5135,
+      "step": 1030
+    },
+    {
+      "epoch": 3.3821138211382116,
+      "grad_norm": 0.40017712116241455,
+      "learning_rate": 0.0002,
+      "loss": 1.5086,
+      "step": 1040
+    },
+    {
+      "epoch": 3.4146341463414633,
+      "grad_norm": 0.4218886196613312,
+      "learning_rate": 0.0002,
+      "loss": 1.6026,
+      "step": 1050
+    },
+    {
+      "epoch": 3.4471544715447155,
+      "grad_norm": 0.42598757147789,
+      "learning_rate": 0.0002,
+      "loss": 1.5337,
+      "step": 1060
+    },
+    {
+      "epoch": 3.4796747967479673,
+      "grad_norm": 0.4137382209300995,
+      "learning_rate": 0.0002,
+      "loss": 1.5487,
+      "step": 1070
+    },
+    {
+      "epoch": 3.5121951219512195,
+      "grad_norm": 0.3774033784866333,
+      "learning_rate": 0.0002,
+      "loss": 1.5132,
+      "step": 1080
+    },
+    {
+      "epoch": 3.5447154471544717,
+      "grad_norm": 0.40397852659225464,
+      "learning_rate": 0.0002,
+      "loss": 1.5842,
+      "step": 1090
+    },
+    {
+      "epoch": 3.5772357723577235,
+      "grad_norm": 0.4346853792667389,
+      "learning_rate": 0.0002,
+      "loss": 1.583,
+      "step": 1100
+    },
+    {
+      "epoch": 3.6097560975609757,
+      "grad_norm": 0.40098339319229126,
+      "learning_rate": 0.0002,
+      "loss": 1.5963,
+      "step": 1110
+    },
+    {
+      "epoch": 3.642276422764228,
+      "grad_norm": 0.4212099313735962,
+      "learning_rate": 0.0002,
+      "loss": 1.5225,
+      "step": 1120
+    },
+    {
+      "epoch": 3.6747967479674797,
+      "grad_norm": 0.4408110976219177,
+      "learning_rate": 0.0002,
+      "loss": 1.5987,
+      "step": 1130
+    },
+    {
+      "epoch": 3.7073170731707314,
+      "grad_norm": 0.3728943169116974,
+      "learning_rate": 0.0002,
+      "loss": 1.5813,
+      "step": 1140
+    },
+    {
+      "epoch": 3.7398373983739837,
+      "grad_norm": 0.3994787633419037,
+      "learning_rate": 0.0002,
+      "loss": 1.6103,
+      "step": 1150
+    },
+    {
+      "epoch": 3.772357723577236,
+      "grad_norm": 0.4318053126335144,
+      "learning_rate": 0.0002,
+      "loss": 1.5812,
+      "step": 1160
+    },
+    {
+      "epoch": 3.8048780487804876,
+      "grad_norm": 0.40556737780570984,
+      "learning_rate": 0.0002,
+      "loss": 1.6173,
+      "step": 1170
+    },
+    {
+      "epoch": 3.83739837398374,
+      "grad_norm": 0.38010650873184204,
+      "learning_rate": 0.0002,
+      "loss": 1.6099,
+      "step": 1180
+    },
+    {
+      "epoch": 3.869918699186992,
+      "grad_norm": 0.4114062190055847,
+      "learning_rate": 0.0002,
+      "loss": 1.4842,
+      "step": 1190
+    },
+    {
+      "epoch": 3.902439024390244,
+      "grad_norm": 0.4016933739185333,
+      "learning_rate": 0.0002,
+      "loss": 1.6047,
+      "step": 1200
+    },
+    {
+      "epoch": 3.934959349593496,
+      "grad_norm": 0.45411986112594604,
+      "learning_rate": 0.0002,
+      "loss": 1.497,
+      "step": 1210
+    },
+    {
+      "epoch": 3.9674796747967482,
+      "grad_norm": 0.3914777934551239,
+      "learning_rate": 0.0002,
+      "loss": 1.5838,
+      "step": 1220
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.42423349618911743,
+      "learning_rate": 0.0002,
+      "loss": 1.541,
+      "step": 1230
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.7998414039611816,
+      "eval_runtime": 38.0517,
+      "eval_samples_per_second": 13.981,
+      "eval_steps_per_second": 1.761,
+      "step": 1230
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2456,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.358605841727488e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1230/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a6ec6940b5d2cf34a18e1e3cb0fe34c3805cbaea884830888b0239757f74df
+size 5560

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9ba386369415bc4aa1c56b307a968e437c73df680231f093883c3c9ba858de8
+size 80755416

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64d4545239e533fed5829e6f1335f3a9f9b5a72c152dde58a8907e20deb3cedf
+size 41136570

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1a2e1be99a73392bce09d7ea19c2f8049c9cb88ba1505c0c1fe86978beaf730
+size 14244

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26031a56dc00724157cb06dd0e54a7fe7ad468381ff4a212cbc495e5c60f2ae7
+size 1064

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/trainer_state.json

@@ -0,0 +1,1144 @@
+{
+  "best_metric": 1.7614221572875977,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-615",
+  "epoch": 4.998373983739837,
+  "eval_steps": 10,
+  "global_step": 1537,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.032520325203252036,
+      "grad_norm": 0.3243047297000885,
+      "learning_rate": 0.0002,
+      "loss": 2.4549,
+      "step": 10
+    },
+    {
+      "epoch": 0.06504065040650407,
+      "grad_norm": 0.3810383975505829,
+      "learning_rate": 0.0002,
+      "loss": 2.0662,
+      "step": 20
+    },
+    {
+      "epoch": 0.0975609756097561,
+      "grad_norm": 0.3727293908596039,
+      "learning_rate": 0.0002,
+      "loss": 1.9994,
+      "step": 30
+    },
+    {
+      "epoch": 0.13008130081300814,
+      "grad_norm": 0.33587437868118286,
+      "learning_rate": 0.0002,
+      "loss": 1.8857,
+      "step": 40
+    },
+    {
+      "epoch": 0.16260162601626016,
+      "grad_norm": 0.29074037075042725,
+      "learning_rate": 0.0002,
+      "loss": 1.8232,
+      "step": 50
+    },
+    {
+      "epoch": 0.1951219512195122,
+      "grad_norm": 0.3021906316280365,
+      "learning_rate": 0.0002,
+      "loss": 1.7747,
+      "step": 60
+    },
+    {
+      "epoch": 0.22764227642276422,
+      "grad_norm": 0.29506710171699524,
+      "learning_rate": 0.0002,
+      "loss": 1.8284,
+      "step": 70
+    },
+    {
+      "epoch": 0.2601626016260163,
+      "grad_norm": 0.32335522770881653,
+      "learning_rate": 0.0002,
+      "loss": 1.7864,
+      "step": 80
+    },
+    {
+      "epoch": 0.2926829268292683,
+      "grad_norm": 0.245526984333992,
+      "learning_rate": 0.0002,
+      "loss": 1.8362,
+      "step": 90
+    },
+    {
+      "epoch": 0.3252032520325203,
+      "grad_norm": 0.260805606842041,
+      "learning_rate": 0.0002,
+      "loss": 1.7142,
+      "step": 100
+    },
+    {
+      "epoch": 0.35772357723577236,
+      "grad_norm": 0.2878130376338959,
+      "learning_rate": 0.0002,
+      "loss": 1.7665,
+      "step": 110
+    },
+    {
+      "epoch": 0.3902439024390244,
+      "grad_norm": 0.263217568397522,
+      "learning_rate": 0.0002,
+      "loss": 1.6984,
+      "step": 120
+    },
+    {
+      "epoch": 0.42276422764227645,
+      "grad_norm": 0.37357720732688904,
+      "learning_rate": 0.0002,
+      "loss": 1.8168,
+      "step": 130
+    },
+    {
+      "epoch": 0.45528455284552843,
+      "grad_norm": 0.2695927023887634,
+      "learning_rate": 0.0002,
+      "loss": 1.7649,
+      "step": 140
+    },
+    {
+      "epoch": 0.4878048780487805,
+      "grad_norm": 0.3133211135864258,
+      "learning_rate": 0.0002,
+      "loss": 1.7631,
+      "step": 150
+    },
+    {
+      "epoch": 0.5203252032520326,
+      "grad_norm": 0.2665450870990753,
+      "learning_rate": 0.0002,
+      "loss": 1.7223,
+      "step": 160
+    },
+    {
+      "epoch": 0.5528455284552846,
+      "grad_norm": 0.2663995921611786,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 170
+    },
+    {
+      "epoch": 0.5853658536585366,
+      "grad_norm": 0.2748648226261139,
+      "learning_rate": 0.0002,
+      "loss": 1.7395,
+      "step": 180
+    },
+    {
+      "epoch": 0.6178861788617886,
+      "grad_norm": 0.2700922191143036,
+      "learning_rate": 0.0002,
+      "loss": 1.8011,
+      "step": 190
+    },
+    {
+      "epoch": 0.6504065040650406,
+      "grad_norm": 0.2584550678730011,
+      "learning_rate": 0.0002,
+      "loss": 1.7561,
+      "step": 200
+    },
+    {
+      "epoch": 0.6829268292682927,
+      "grad_norm": 0.23968899250030518,
+      "learning_rate": 0.0002,
+      "loss": 1.6472,
+      "step": 210
+    },
+    {
+      "epoch": 0.7154471544715447,
+      "grad_norm": 0.23806197941303253,
+      "learning_rate": 0.0002,
+      "loss": 1.7167,
+      "step": 220
+    },
+    {
+      "epoch": 0.7479674796747967,
+      "grad_norm": 0.23665426671504974,
+      "learning_rate": 0.0002,
+      "loss": 1.7429,
+      "step": 230
+    },
+    {
+      "epoch": 0.7804878048780488,
+      "grad_norm": 0.2510588467121124,
+      "learning_rate": 0.0002,
+      "loss": 1.7044,
+      "step": 240
+    },
+    {
+      "epoch": 0.8130081300813008,
+      "grad_norm": 0.2480899840593338,
+      "learning_rate": 0.0002,
+      "loss": 1.7843,
+      "step": 250
+    },
+    {
+      "epoch": 0.8455284552845529,
+      "grad_norm": 0.2802377939224243,
+      "learning_rate": 0.0002,
+      "loss": 1.7034,
+      "step": 260
+    },
+    {
+      "epoch": 0.8780487804878049,
+      "grad_norm": 0.2634681165218353,
+      "learning_rate": 0.0002,
+      "loss": 1.7354,
+      "step": 270
+    },
+    {
+      "epoch": 0.9105691056910569,
+      "grad_norm": 0.2952527403831482,
+      "learning_rate": 0.0002,
+      "loss": 1.7428,
+      "step": 280
+    },
+    {
+      "epoch": 0.943089430894309,
+      "grad_norm": 0.22973614931106567,
+      "learning_rate": 0.0002,
+      "loss": 1.6869,
+      "step": 290
+    },
+    {
+      "epoch": 0.975609756097561,
+      "grad_norm": 0.23497417569160461,
+      "learning_rate": 0.0002,
+      "loss": 1.7412,
+      "step": 300
+    },
+    {
+      "epoch": 0.9983739837398374,
+      "eval_loss": 1.7698818445205688,
+      "eval_runtime": 38.0302,
+      "eval_samples_per_second": 13.989,
+      "eval_steps_per_second": 1.762,
+      "step": 307
+    },
+    {
+      "epoch": 1.008130081300813,
+      "grad_norm": 0.2552054822444916,
+      "learning_rate": 0.0002,
+      "loss": 1.7849,
+      "step": 310
+    },
+    {
+      "epoch": 1.040650406504065,
+      "grad_norm": 0.21663406491279602,
+      "learning_rate": 0.0002,
+      "loss": 1.7233,
+      "step": 320
+    },
+    {
+      "epoch": 1.0731707317073171,
+      "grad_norm": 0.2413843721151352,
+      "learning_rate": 0.0002,
+      "loss": 1.7069,
+      "step": 330
+    },
+    {
+      "epoch": 1.1056910569105691,
+      "grad_norm": 0.2675042450428009,
+      "learning_rate": 0.0002,
+      "loss": 1.6097,
+      "step": 340
+    },
+    {
+      "epoch": 1.1382113821138211,
+      "grad_norm": 0.2572104334831238,
+      "learning_rate": 0.0002,
+      "loss": 1.6697,
+      "step": 350
+    },
+    {
+      "epoch": 1.170731707317073,
+      "grad_norm": 0.2535107731819153,
+      "learning_rate": 0.0002,
+      "loss": 1.6972,
+      "step": 360
+    },
+    {
+      "epoch": 1.203252032520325,
+      "grad_norm": 0.2549455165863037,
+      "learning_rate": 0.0002,
+      "loss": 1.7133,
+      "step": 370
+    },
+    {
+      "epoch": 1.2357723577235773,
+      "grad_norm": 0.27836501598358154,
+      "learning_rate": 0.0002,
+      "loss": 1.757,
+      "step": 380
+    },
+    {
+      "epoch": 1.2682926829268293,
+      "grad_norm": 0.26294031739234924,
+      "learning_rate": 0.0002,
+      "loss": 1.652,
+      "step": 390
+    },
+    {
+      "epoch": 1.3008130081300813,
+      "grad_norm": 0.25705304741859436,
+      "learning_rate": 0.0002,
+      "loss": 1.6738,
+      "step": 400
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.2765929698944092,
+      "learning_rate": 0.0002,
+      "loss": 1.6396,
+      "step": 410
+    },
+    {
+      "epoch": 1.3658536585365852,
+      "grad_norm": 0.2620247006416321,
+      "learning_rate": 0.0002,
+      "loss": 1.7496,
+      "step": 420
+    },
+    {
+      "epoch": 1.3983739837398375,
+      "grad_norm": 0.28890499472618103,
+      "learning_rate": 0.0002,
+      "loss": 1.7139,
+      "step": 430
+    },
+    {
+      "epoch": 1.4308943089430894,
+      "grad_norm": 0.524259090423584,
+      "learning_rate": 0.0002,
+      "loss": 1.6881,
+      "step": 440
+    },
+    {
+      "epoch": 1.4634146341463414,
+      "grad_norm": 0.279712438583374,
+      "learning_rate": 0.0002,
+      "loss": 1.6994,
+      "step": 450
+    },
+    {
+      "epoch": 1.4959349593495934,
+      "grad_norm": 0.2770076096057892,
+      "learning_rate": 0.0002,
+      "loss": 1.6853,
+      "step": 460
+    },
+    {
+      "epoch": 1.5284552845528454,
+      "grad_norm": 0.2519102394580841,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 470
+    },
+    {
+      "epoch": 1.5609756097560976,
+      "grad_norm": 0.2679072916507721,
+      "learning_rate": 0.0002,
+      "loss": 1.6689,
+      "step": 480
+    },
+    {
+      "epoch": 1.5934959349593496,
+      "grad_norm": 0.2848704159259796,
+      "learning_rate": 0.0002,
+      "loss": 1.6938,
+      "step": 490
+    },
+    {
+      "epoch": 1.6260162601626016,
+      "grad_norm": 0.2666904032230377,
+      "learning_rate": 0.0002,
+      "loss": 1.6752,
+      "step": 500
+    },
+    {
+      "epoch": 1.6585365853658538,
+      "grad_norm": 0.24341872334480286,
+      "learning_rate": 0.0002,
+      "loss": 1.7231,
+      "step": 510
+    },
+    {
+      "epoch": 1.6910569105691056,
+      "grad_norm": 0.24182367324829102,
+      "learning_rate": 0.0002,
+      "loss": 1.6827,
+      "step": 520
+    },
+    {
+      "epoch": 1.7235772357723578,
+      "grad_norm": 0.266201913356781,
+      "learning_rate": 0.0002,
+      "loss": 1.7059,
+      "step": 530
+    },
+    {
+      "epoch": 1.7560975609756098,
+      "grad_norm": 0.26361775398254395,
+      "learning_rate": 0.0002,
+      "loss": 1.7196,
+      "step": 540
+    },
+    {
+      "epoch": 1.7886178861788617,
+      "grad_norm": 0.2974611818790436,
+      "learning_rate": 0.0002,
+      "loss": 1.698,
+      "step": 550
+    },
+    {
+      "epoch": 1.821138211382114,
+      "grad_norm": 0.26264363527297974,
+      "learning_rate": 0.0002,
+      "loss": 1.6913,
+      "step": 560
+    },
+    {
+      "epoch": 1.8536585365853657,
+      "grad_norm": 0.31887590885162354,
+      "learning_rate": 0.0002,
+      "loss": 1.7373,
+      "step": 570
+    },
+    {
+      "epoch": 1.886178861788618,
+      "grad_norm": 0.25847622752189636,
+      "learning_rate": 0.0002,
+      "loss": 1.7269,
+      "step": 580
+    },
+    {
+      "epoch": 1.91869918699187,
+      "grad_norm": 0.2770395576953888,
+      "learning_rate": 0.0002,
+      "loss": 1.7126,
+      "step": 590
+    },
+    {
+      "epoch": 1.951219512195122,
+      "grad_norm": 0.24041323363780975,
+      "learning_rate": 0.0002,
+      "loss": 1.6897,
+      "step": 600
+    },
+    {
+      "epoch": 1.9837398373983741,
+      "grad_norm": 0.25060364603996277,
+      "learning_rate": 0.0002,
+      "loss": 1.6904,
+      "step": 610
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.7614221572875977,
+      "eval_runtime": 38.0401,
+      "eval_samples_per_second": 13.985,
+      "eval_steps_per_second": 1.761,
+      "step": 615
+    },
+    {
+      "epoch": 2.016260162601626,
+      "grad_norm": 0.25516635179519653,
+      "learning_rate": 0.0002,
+      "loss": 1.7299,
+      "step": 620
+    },
+    {
+      "epoch": 2.048780487804878,
+      "grad_norm": 0.31203776597976685,
+      "learning_rate": 0.0002,
+      "loss": 1.6142,
+      "step": 630
+    },
+    {
+      "epoch": 2.08130081300813,
+      "grad_norm": 0.27818965911865234,
+      "learning_rate": 0.0002,
+      "loss": 1.6519,
+      "step": 640
+    },
+    {
+      "epoch": 2.113821138211382,
+      "grad_norm": 0.2781952917575836,
+      "learning_rate": 0.0002,
+      "loss": 1.6678,
+      "step": 650
+    },
+    {
+      "epoch": 2.1463414634146343,
+      "grad_norm": 0.2937796413898468,
+      "learning_rate": 0.0002,
+      "loss": 1.5723,
+      "step": 660
+    },
+    {
+      "epoch": 2.178861788617886,
+      "grad_norm": 0.3559171259403229,
+      "learning_rate": 0.0002,
+      "loss": 1.6118,
+      "step": 670
+    },
+    {
+      "epoch": 2.2113821138211383,
+      "grad_norm": 0.3274490237236023,
+      "learning_rate": 0.0002,
+      "loss": 1.6044,
+      "step": 680
+    },
+    {
+      "epoch": 2.2439024390243905,
+      "grad_norm": 0.34521982073783875,
+      "learning_rate": 0.0002,
+      "loss": 1.6521,
+      "step": 690
+    },
+    {
+      "epoch": 2.2764227642276422,
+      "grad_norm": 0.30293744802474976,
+      "learning_rate": 0.0002,
+      "loss": 1.6075,
+      "step": 700
+    },
+    {
+      "epoch": 2.3089430894308944,
+      "grad_norm": 0.31012430787086487,
+      "learning_rate": 0.0002,
+      "loss": 1.6843,
+      "step": 710
+    },
+    {
+      "epoch": 2.341463414634146,
+      "grad_norm": 0.3424604535102844,
+      "learning_rate": 0.0002,
+      "loss": 1.6524,
+      "step": 720
+    },
+    {
+      "epoch": 2.3739837398373984,
+      "grad_norm": 0.3361012041568756,
+      "learning_rate": 0.0002,
+      "loss": 1.6404,
+      "step": 730
+    },
+    {
+      "epoch": 2.40650406504065,
+      "grad_norm": 0.308322012424469,
+      "learning_rate": 0.0002,
+      "loss": 1.6335,
+      "step": 740
+    },
+    {
+      "epoch": 2.4390243902439024,
+      "grad_norm": 0.3367919921875,
+      "learning_rate": 0.0002,
+      "loss": 1.6867,
+      "step": 750
+    },
+    {
+      "epoch": 2.4715447154471546,
+      "grad_norm": 0.3119719922542572,
+      "learning_rate": 0.0002,
+      "loss": 1.6332,
+      "step": 760
+    },
+    {
+      "epoch": 2.5040650406504064,
+      "grad_norm": 0.31164902448654175,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 770
+    },
+    {
+      "epoch": 2.5365853658536586,
+      "grad_norm": 0.3472500443458557,
+      "learning_rate": 0.0002,
+      "loss": 1.5806,
+      "step": 780
+    },
+    {
+      "epoch": 2.569105691056911,
+      "grad_norm": 0.32276982069015503,
+      "learning_rate": 0.0002,
+      "loss": 1.6171,
+      "step": 790
+    },
+    {
+      "epoch": 2.6016260162601625,
+      "grad_norm": 0.2904753088951111,
+      "learning_rate": 0.0002,
+      "loss": 1.5879,
+      "step": 800
+    },
+    {
+      "epoch": 2.6341463414634148,
+      "grad_norm": 0.3249073624610901,
+      "learning_rate": 0.0002,
+      "loss": 1.5967,
+      "step": 810
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.35803690552711487,
+      "learning_rate": 0.0002,
+      "loss": 1.7066,
+      "step": 820
+    },
+    {
+      "epoch": 2.6991869918699187,
+      "grad_norm": 0.3210543990135193,
+      "learning_rate": 0.0002,
+      "loss": 1.6163,
+      "step": 830
+    },
+    {
+      "epoch": 2.7317073170731705,
+      "grad_norm": 0.330085813999176,
+      "learning_rate": 0.0002,
+      "loss": 1.6599,
+      "step": 840
+    },
+    {
+      "epoch": 2.7642276422764227,
+      "grad_norm": 0.35581567883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.6338,
+      "step": 850
+    },
+    {
+      "epoch": 2.796747967479675,
+      "grad_norm": 0.3423900008201599,
+      "learning_rate": 0.0002,
+      "loss": 1.6033,
+      "step": 860
+    },
+    {
+      "epoch": 2.8292682926829267,
+      "grad_norm": 0.33455702662467957,
+      "learning_rate": 0.0002,
+      "loss": 1.6406,
+      "step": 870
+    },
+    {
+      "epoch": 2.861788617886179,
+      "grad_norm": 0.37637296319007874,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 880
+    },
+    {
+      "epoch": 2.894308943089431,
+      "grad_norm": 0.3715825080871582,
+      "learning_rate": 0.0002,
+      "loss": 1.6166,
+      "step": 890
+    },
+    {
+      "epoch": 2.926829268292683,
+      "grad_norm": 0.3526729345321655,
+      "learning_rate": 0.0002,
+      "loss": 1.7131,
+      "step": 900
+    },
+    {
+      "epoch": 2.959349593495935,
+      "grad_norm": 0.33071836829185486,
+      "learning_rate": 0.0002,
+      "loss": 1.5714,
+      "step": 910
+    },
+    {
+      "epoch": 2.991869918699187,
+      "grad_norm": 0.2956294119358063,
+      "learning_rate": 0.0002,
+      "loss": 1.6251,
+      "step": 920
+    },
+    {
+      "epoch": 2.9983739837398375,
+      "eval_loss": 1.7697627544403076,
+      "eval_runtime": 38.0483,
+      "eval_samples_per_second": 13.982,
+      "eval_steps_per_second": 1.761,
+      "step": 922
+    },
+    {
+      "epoch": 3.024390243902439,
+      "grad_norm": 0.35567474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.5265,
+      "step": 930
+    },
+    {
+      "epoch": 3.0569105691056913,
+      "grad_norm": 0.36280205845832825,
+      "learning_rate": 0.0002,
+      "loss": 1.5131,
+      "step": 940
+    },
+    {
+      "epoch": 3.089430894308943,
+      "grad_norm": 0.39025646448135376,
+      "learning_rate": 0.0002,
+      "loss": 1.5492,
+      "step": 950
+    },
+    {
+      "epoch": 3.1219512195121952,
+      "grad_norm": 0.4128074049949646,
+      "learning_rate": 0.0002,
+      "loss": 1.6059,
+      "step": 960
+    },
+    {
+      "epoch": 3.154471544715447,
+      "grad_norm": 0.37436643242836,
+      "learning_rate": 0.0002,
+      "loss": 1.5675,
+      "step": 970
+    },
+    {
+      "epoch": 3.186991869918699,
+      "grad_norm": 0.3950212895870209,
+      "learning_rate": 0.0002,
+      "loss": 1.5018,
+      "step": 980
+    },
+    {
+      "epoch": 3.2195121951219514,
+      "grad_norm": 0.3334033787250519,
+      "learning_rate": 0.0002,
+      "loss": 1.4856,
+      "step": 990
+    },
+    {
+      "epoch": 3.252032520325203,
+      "grad_norm": 0.41348570585250854,
+      "learning_rate": 0.0002,
+      "loss": 1.5288,
+      "step": 1000
+    },
+    {
+      "epoch": 3.2845528455284554,
+      "grad_norm": 0.48985755443573,
+      "learning_rate": 0.0002,
+      "loss": 1.565,
+      "step": 1010
+    },
+    {
+      "epoch": 3.317073170731707,
+      "grad_norm": 0.36514127254486084,
+      "learning_rate": 0.0002,
+      "loss": 1.5463,
+      "step": 1020
+    },
+    {
+      "epoch": 3.3495934959349594,
+      "grad_norm": 0.4296952188014984,
+      "learning_rate": 0.0002,
+      "loss": 1.5135,
+      "step": 1030
+    },
+    {
+      "epoch": 3.3821138211382116,
+      "grad_norm": 0.40017712116241455,
+      "learning_rate": 0.0002,
+      "loss": 1.5086,
+      "step": 1040
+    },
+    {
+      "epoch": 3.4146341463414633,
+      "grad_norm": 0.4218886196613312,
+      "learning_rate": 0.0002,
+      "loss": 1.6026,
+      "step": 1050
+    },
+    {
+      "epoch": 3.4471544715447155,
+      "grad_norm": 0.42598757147789,
+      "learning_rate": 0.0002,
+      "loss": 1.5337,
+      "step": 1060
+    },
+    {
+      "epoch": 3.4796747967479673,
+      "grad_norm": 0.4137382209300995,
+      "learning_rate": 0.0002,
+      "loss": 1.5487,
+      "step": 1070
+    },
+    {
+      "epoch": 3.5121951219512195,
+      "grad_norm": 0.3774033784866333,
+      "learning_rate": 0.0002,
+      "loss": 1.5132,
+      "step": 1080
+    },
+    {
+      "epoch": 3.5447154471544717,
+      "grad_norm": 0.40397852659225464,
+      "learning_rate": 0.0002,
+      "loss": 1.5842,
+      "step": 1090
+    },
+    {
+      "epoch": 3.5772357723577235,
+      "grad_norm": 0.4346853792667389,
+      "learning_rate": 0.0002,
+      "loss": 1.583,
+      "step": 1100
+    },
+    {
+      "epoch": 3.6097560975609757,
+      "grad_norm": 0.40098339319229126,
+      "learning_rate": 0.0002,
+      "loss": 1.5963,
+      "step": 1110
+    },
+    {
+      "epoch": 3.642276422764228,
+      "grad_norm": 0.4212099313735962,
+      "learning_rate": 0.0002,
+      "loss": 1.5225,
+      "step": 1120
+    },
+    {
+      "epoch": 3.6747967479674797,
+      "grad_norm": 0.4408110976219177,
+      "learning_rate": 0.0002,
+      "loss": 1.5987,
+      "step": 1130
+    },
+    {
+      "epoch": 3.7073170731707314,
+      "grad_norm": 0.3728943169116974,
+      "learning_rate": 0.0002,
+      "loss": 1.5813,
+      "step": 1140
+    },
+    {
+      "epoch": 3.7398373983739837,
+      "grad_norm": 0.3994787633419037,
+      "learning_rate": 0.0002,
+      "loss": 1.6103,
+      "step": 1150
+    },
+    {
+      "epoch": 3.772357723577236,
+      "grad_norm": 0.4318053126335144,
+      "learning_rate": 0.0002,
+      "loss": 1.5812,
+      "step": 1160
+    },
+    {
+      "epoch": 3.8048780487804876,
+      "grad_norm": 0.40556737780570984,
+      "learning_rate": 0.0002,
+      "loss": 1.6173,
+      "step": 1170
+    },
+    {
+      "epoch": 3.83739837398374,
+      "grad_norm": 0.38010650873184204,
+      "learning_rate": 0.0002,
+      "loss": 1.6099,
+      "step": 1180
+    },
+    {
+      "epoch": 3.869918699186992,
+      "grad_norm": 0.4114062190055847,
+      "learning_rate": 0.0002,
+      "loss": 1.4842,
+      "step": 1190
+    },
+    {
+      "epoch": 3.902439024390244,
+      "grad_norm": 0.4016933739185333,
+      "learning_rate": 0.0002,
+      "loss": 1.6047,
+      "step": 1200
+    },
+    {
+      "epoch": 3.934959349593496,
+      "grad_norm": 0.45411986112594604,
+      "learning_rate": 0.0002,
+      "loss": 1.497,
+      "step": 1210
+    },
+    {
+      "epoch": 3.9674796747967482,
+      "grad_norm": 0.3914777934551239,
+      "learning_rate": 0.0002,
+      "loss": 1.5838,
+      "step": 1220
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.42423349618911743,
+      "learning_rate": 0.0002,
+      "loss": 1.541,
+      "step": 1230
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.7998414039611816,
+      "eval_runtime": 38.0517,
+      "eval_samples_per_second": 13.981,
+      "eval_steps_per_second": 1.761,
+      "step": 1230
+    },
+    {
+      "epoch": 4.032520325203252,
+      "grad_norm": 0.5187233686447144,
+      "learning_rate": 0.0002,
+      "loss": 1.4585,
+      "step": 1240
+    },
+    {
+      "epoch": 4.065040650406504,
+      "grad_norm": 0.47915923595428467,
+      "learning_rate": 0.0002,
+      "loss": 1.3959,
+      "step": 1250
+    },
+    {
+      "epoch": 4.097560975609756,
+      "grad_norm": 0.5152283310890198,
+      "learning_rate": 0.0002,
+      "loss": 1.4296,
+      "step": 1260
+    },
+    {
+      "epoch": 4.130081300813008,
+      "grad_norm": 0.49710479378700256,
+      "learning_rate": 0.0002,
+      "loss": 1.4722,
+      "step": 1270
+    },
+    {
+      "epoch": 4.16260162601626,
+      "grad_norm": 0.5043147802352905,
+      "learning_rate": 0.0002,
+      "loss": 1.3868,
+      "step": 1280
+    },
+    {
+      "epoch": 4.195121951219512,
+      "grad_norm": 0.49862509965896606,
+      "learning_rate": 0.0002,
+      "loss": 1.4762,
+      "step": 1290
+    },
+    {
+      "epoch": 4.227642276422764,
+      "grad_norm": 0.5163412690162659,
+      "learning_rate": 0.0002,
+      "loss": 1.4187,
+      "step": 1300
+    },
+    {
+      "epoch": 4.260162601626016,
+      "grad_norm": 0.5454265475273132,
+      "learning_rate": 0.0002,
+      "loss": 1.4717,
+      "step": 1310
+    },
+    {
+      "epoch": 4.2926829268292686,
+      "grad_norm": 0.54813551902771,
+      "learning_rate": 0.0002,
+      "loss": 1.4065,
+      "step": 1320
+    },
+    {
+      "epoch": 4.32520325203252,
+      "grad_norm": 0.5753244161605835,
+      "learning_rate": 0.0002,
+      "loss": 1.5298,
+      "step": 1330
+    },
+    {
+      "epoch": 4.357723577235772,
+      "grad_norm": 0.5596114993095398,
+      "learning_rate": 0.0002,
+      "loss": 1.4372,
+      "step": 1340
+    },
+    {
+      "epoch": 4.390243902439025,
+      "grad_norm": 0.5419771671295166,
+      "learning_rate": 0.0002,
+      "loss": 1.451,
+      "step": 1350
+    },
+    {
+      "epoch": 4.4227642276422765,
+      "grad_norm": 0.5357267260551453,
+      "learning_rate": 0.0002,
+      "loss": 1.5257,
+      "step": 1360
+    },
+    {
+      "epoch": 4.455284552845528,
+      "grad_norm": 0.5426302552223206,
+      "learning_rate": 0.0002,
+      "loss": 1.4724,
+      "step": 1370
+    },
+    {
+      "epoch": 4.487804878048781,
+      "grad_norm": 0.48658859729766846,
+      "learning_rate": 0.0002,
+      "loss": 1.442,
+      "step": 1380
+    },
+    {
+      "epoch": 4.520325203252033,
+      "grad_norm": 0.46241894364356995,
+      "learning_rate": 0.0002,
+      "loss": 1.5034,
+      "step": 1390
+    },
+    {
+      "epoch": 4.5528455284552845,
+      "grad_norm": 0.47949138283729553,
+      "learning_rate": 0.0002,
+      "loss": 1.4738,
+      "step": 1400
+    },
+    {
+      "epoch": 4.585365853658536,
+      "grad_norm": 0.5619633793830872,
+      "learning_rate": 0.0002,
+      "loss": 1.4343,
+      "step": 1410
+    },
+    {
+      "epoch": 4.617886178861789,
+      "grad_norm": 0.5513656139373779,
+      "learning_rate": 0.0002,
+      "loss": 1.398,
+      "step": 1420
+    },
+    {
+      "epoch": 4.650406504065041,
+      "grad_norm": 0.5279194712638855,
+      "learning_rate": 0.0002,
+      "loss": 1.4071,
+      "step": 1430
+    },
+    {
+      "epoch": 4.682926829268292,
+      "grad_norm": 0.49925369024276733,
+      "learning_rate": 0.0002,
+      "loss": 1.5391,
+      "step": 1440
+    },
+    {
+      "epoch": 4.715447154471545,
+      "grad_norm": 0.5767236351966858,
+      "learning_rate": 0.0002,
+      "loss": 1.5315,
+      "step": 1450
+    },
+    {
+      "epoch": 4.747967479674797,
+      "grad_norm": 0.5069461464881897,
+      "learning_rate": 0.0002,
+      "loss": 1.4125,
+      "step": 1460
+    },
+    {
+      "epoch": 4.780487804878049,
+      "grad_norm": 0.5566238760948181,
+      "learning_rate": 0.0002,
+      "loss": 1.457,
+      "step": 1470
+    },
+    {
+      "epoch": 4.8130081300813,
+      "grad_norm": 0.529227077960968,
+      "learning_rate": 0.0002,
+      "loss": 1.4497,
+      "step": 1480
+    },
+    {
+      "epoch": 4.845528455284553,
+      "grad_norm": 0.5591039657592773,
+      "learning_rate": 0.0002,
+      "loss": 1.4585,
+      "step": 1490
+    },
+    {
+      "epoch": 4.878048780487805,
+      "grad_norm": 0.45602425932884216,
+      "learning_rate": 0.0002,
+      "loss": 1.5522,
+      "step": 1500
+    },
+    {
+      "epoch": 4.9105691056910565,
+      "grad_norm": 0.4569244682788849,
+      "learning_rate": 0.0002,
+      "loss": 1.4134,
+      "step": 1510
+    },
+    {
+      "epoch": 4.943089430894309,
+      "grad_norm": 0.5215898752212524,
+      "learning_rate": 0.0002,
+      "loss": 1.5061,
+      "step": 1520
+    },
+    {
+      "epoch": 4.975609756097561,
+      "grad_norm": 0.4940566122531891,
+      "learning_rate": 0.0002,
+      "loss": 1.4864,
+      "step": 1530
+    },
+    {
+      "epoch": 4.998373983739837,
+      "eval_loss": 1.8480068445205688,
+      "eval_runtime": 38.043,
+      "eval_samples_per_second": 13.984,
+      "eval_steps_per_second": 1.761,
+      "step": 1537
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2456,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.69825730215936e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1537/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a6ec6940b5d2cf34a18e1e3cb0fe34c3805cbaea884830888b0239757f74df
+size 5560

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1f03f51500959be6d14c85013e782d320e70121b32d435aa9f282d4b4fd5666
+size 80755416

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:502feb07ef08fbe0016bceab52d0b31b65b8d4921d2cb225dffd29bb2c73bb7b
+size 41136570

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7365054263e6204c36827dabcc5a439216ed777f17c212dbcb800b00f0bea168
+size 14244

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9ecf3cbe2bdeeb71fbeaf36b416466e300cc4d67381f2ab44ce5a65f3569ea2
+size 1064

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/trainer_state.json

@@ -0,0 +1,1369 @@
+{
+  "best_metric": 1.7614221572875977,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-615",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 1845,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.032520325203252036,
+      "grad_norm": 0.3243047297000885,
+      "learning_rate": 0.0002,
+      "loss": 2.4549,
+      "step": 10
+    },
+    {
+      "epoch": 0.06504065040650407,
+      "grad_norm": 0.3810383975505829,
+      "learning_rate": 0.0002,
+      "loss": 2.0662,
+      "step": 20
+    },
+    {
+      "epoch": 0.0975609756097561,
+      "grad_norm": 0.3727293908596039,
+      "learning_rate": 0.0002,
+      "loss": 1.9994,
+      "step": 30
+    },
+    {
+      "epoch": 0.13008130081300814,
+      "grad_norm": 0.33587437868118286,
+      "learning_rate": 0.0002,
+      "loss": 1.8857,
+      "step": 40
+    },
+    {
+      "epoch": 0.16260162601626016,
+      "grad_norm": 0.29074037075042725,
+      "learning_rate": 0.0002,
+      "loss": 1.8232,
+      "step": 50
+    },
+    {
+      "epoch": 0.1951219512195122,
+      "grad_norm": 0.3021906316280365,
+      "learning_rate": 0.0002,
+      "loss": 1.7747,
+      "step": 60
+    },
+    {
+      "epoch": 0.22764227642276422,
+      "grad_norm": 0.29506710171699524,
+      "learning_rate": 0.0002,
+      "loss": 1.8284,
+      "step": 70
+    },
+    {
+      "epoch": 0.2601626016260163,
+      "grad_norm": 0.32335522770881653,
+      "learning_rate": 0.0002,
+      "loss": 1.7864,
+      "step": 80
+    },
+    {
+      "epoch": 0.2926829268292683,
+      "grad_norm": 0.245526984333992,
+      "learning_rate": 0.0002,
+      "loss": 1.8362,
+      "step": 90
+    },
+    {
+      "epoch": 0.3252032520325203,
+      "grad_norm": 0.260805606842041,
+      "learning_rate": 0.0002,
+      "loss": 1.7142,
+      "step": 100
+    },
+    {
+      "epoch": 0.35772357723577236,
+      "grad_norm": 0.2878130376338959,
+      "learning_rate": 0.0002,
+      "loss": 1.7665,
+      "step": 110
+    },
+    {
+      "epoch": 0.3902439024390244,
+      "grad_norm": 0.263217568397522,
+      "learning_rate": 0.0002,
+      "loss": 1.6984,
+      "step": 120
+    },
+    {
+      "epoch": 0.42276422764227645,
+      "grad_norm": 0.37357720732688904,
+      "learning_rate": 0.0002,
+      "loss": 1.8168,
+      "step": 130
+    },
+    {
+      "epoch": 0.45528455284552843,
+      "grad_norm": 0.2695927023887634,
+      "learning_rate": 0.0002,
+      "loss": 1.7649,
+      "step": 140
+    },
+    {
+      "epoch": 0.4878048780487805,
+      "grad_norm": 0.3133211135864258,
+      "learning_rate": 0.0002,
+      "loss": 1.7631,
+      "step": 150
+    },
+    {
+      "epoch": 0.5203252032520326,
+      "grad_norm": 0.2665450870990753,
+      "learning_rate": 0.0002,
+      "loss": 1.7223,
+      "step": 160
+    },
+    {
+      "epoch": 0.5528455284552846,
+      "grad_norm": 0.2663995921611786,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 170
+    },
+    {
+      "epoch": 0.5853658536585366,
+      "grad_norm": 0.2748648226261139,
+      "learning_rate": 0.0002,
+      "loss": 1.7395,
+      "step": 180
+    },
+    {
+      "epoch": 0.6178861788617886,
+      "grad_norm": 0.2700922191143036,
+      "learning_rate": 0.0002,
+      "loss": 1.8011,
+      "step": 190
+    },
+    {
+      "epoch": 0.6504065040650406,
+      "grad_norm": 0.2584550678730011,
+      "learning_rate": 0.0002,
+      "loss": 1.7561,
+      "step": 200
+    },
+    {
+      "epoch": 0.6829268292682927,
+      "grad_norm": 0.23968899250030518,
+      "learning_rate": 0.0002,
+      "loss": 1.6472,
+      "step": 210
+    },
+    {
+      "epoch": 0.7154471544715447,
+      "grad_norm": 0.23806197941303253,
+      "learning_rate": 0.0002,
+      "loss": 1.7167,
+      "step": 220
+    },
+    {
+      "epoch": 0.7479674796747967,
+      "grad_norm": 0.23665426671504974,
+      "learning_rate": 0.0002,
+      "loss": 1.7429,
+      "step": 230
+    },
+    {
+      "epoch": 0.7804878048780488,
+      "grad_norm": 0.2510588467121124,
+      "learning_rate": 0.0002,
+      "loss": 1.7044,
+      "step": 240
+    },
+    {
+      "epoch": 0.8130081300813008,
+      "grad_norm": 0.2480899840593338,
+      "learning_rate": 0.0002,
+      "loss": 1.7843,
+      "step": 250
+    },
+    {
+      "epoch": 0.8455284552845529,
+      "grad_norm": 0.2802377939224243,
+      "learning_rate": 0.0002,
+      "loss": 1.7034,
+      "step": 260
+    },
+    {
+      "epoch": 0.8780487804878049,
+      "grad_norm": 0.2634681165218353,
+      "learning_rate": 0.0002,
+      "loss": 1.7354,
+      "step": 270
+    },
+    {
+      "epoch": 0.9105691056910569,
+      "grad_norm": 0.2952527403831482,
+      "learning_rate": 0.0002,
+      "loss": 1.7428,
+      "step": 280
+    },
+    {
+      "epoch": 0.943089430894309,
+      "grad_norm": 0.22973614931106567,
+      "learning_rate": 0.0002,
+      "loss": 1.6869,
+      "step": 290
+    },
+    {
+      "epoch": 0.975609756097561,
+      "grad_norm": 0.23497417569160461,
+      "learning_rate": 0.0002,
+      "loss": 1.7412,
+      "step": 300
+    },
+    {
+      "epoch": 0.9983739837398374,
+      "eval_loss": 1.7698818445205688,
+      "eval_runtime": 38.0302,
+      "eval_samples_per_second": 13.989,
+      "eval_steps_per_second": 1.762,
+      "step": 307
+    },
+    {
+      "epoch": 1.008130081300813,
+      "grad_norm": 0.2552054822444916,
+      "learning_rate": 0.0002,
+      "loss": 1.7849,
+      "step": 310
+    },
+    {
+      "epoch": 1.040650406504065,
+      "grad_norm": 0.21663406491279602,
+      "learning_rate": 0.0002,
+      "loss": 1.7233,
+      "step": 320
+    },
+    {
+      "epoch": 1.0731707317073171,
+      "grad_norm": 0.2413843721151352,
+      "learning_rate": 0.0002,
+      "loss": 1.7069,
+      "step": 330
+    },
+    {
+      "epoch": 1.1056910569105691,
+      "grad_norm": 0.2675042450428009,
+      "learning_rate": 0.0002,
+      "loss": 1.6097,
+      "step": 340
+    },
+    {
+      "epoch": 1.1382113821138211,
+      "grad_norm": 0.2572104334831238,
+      "learning_rate": 0.0002,
+      "loss": 1.6697,
+      "step": 350
+    },
+    {
+      "epoch": 1.170731707317073,
+      "grad_norm": 0.2535107731819153,
+      "learning_rate": 0.0002,
+      "loss": 1.6972,
+      "step": 360
+    },
+    {
+      "epoch": 1.203252032520325,
+      "grad_norm": 0.2549455165863037,
+      "learning_rate": 0.0002,
+      "loss": 1.7133,
+      "step": 370
+    },
+    {
+      "epoch": 1.2357723577235773,
+      "grad_norm": 0.27836501598358154,
+      "learning_rate": 0.0002,
+      "loss": 1.757,
+      "step": 380
+    },
+    {
+      "epoch": 1.2682926829268293,
+      "grad_norm": 0.26294031739234924,
+      "learning_rate": 0.0002,
+      "loss": 1.652,
+      "step": 390
+    },
+    {
+      "epoch": 1.3008130081300813,
+      "grad_norm": 0.25705304741859436,
+      "learning_rate": 0.0002,
+      "loss": 1.6738,
+      "step": 400
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.2765929698944092,
+      "learning_rate": 0.0002,
+      "loss": 1.6396,
+      "step": 410
+    },
+    {
+      "epoch": 1.3658536585365852,
+      "grad_norm": 0.2620247006416321,
+      "learning_rate": 0.0002,
+      "loss": 1.7496,
+      "step": 420
+    },
+    {
+      "epoch": 1.3983739837398375,
+      "grad_norm": 0.28890499472618103,
+      "learning_rate": 0.0002,
+      "loss": 1.7139,
+      "step": 430
+    },
+    {
+      "epoch": 1.4308943089430894,
+      "grad_norm": 0.524259090423584,
+      "learning_rate": 0.0002,
+      "loss": 1.6881,
+      "step": 440
+    },
+    {
+      "epoch": 1.4634146341463414,
+      "grad_norm": 0.279712438583374,
+      "learning_rate": 0.0002,
+      "loss": 1.6994,
+      "step": 450
+    },
+    {
+      "epoch": 1.4959349593495934,
+      "grad_norm": 0.2770076096057892,
+      "learning_rate": 0.0002,
+      "loss": 1.6853,
+      "step": 460
+    },
+    {
+      "epoch": 1.5284552845528454,
+      "grad_norm": 0.2519102394580841,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 470
+    },
+    {
+      "epoch": 1.5609756097560976,
+      "grad_norm": 0.2679072916507721,
+      "learning_rate": 0.0002,
+      "loss": 1.6689,
+      "step": 480
+    },
+    {
+      "epoch": 1.5934959349593496,
+      "grad_norm": 0.2848704159259796,
+      "learning_rate": 0.0002,
+      "loss": 1.6938,
+      "step": 490
+    },
+    {
+      "epoch": 1.6260162601626016,
+      "grad_norm": 0.2666904032230377,
+      "learning_rate": 0.0002,
+      "loss": 1.6752,
+      "step": 500
+    },
+    {
+      "epoch": 1.6585365853658538,
+      "grad_norm": 0.24341872334480286,
+      "learning_rate": 0.0002,
+      "loss": 1.7231,
+      "step": 510
+    },
+    {
+      "epoch": 1.6910569105691056,
+      "grad_norm": 0.24182367324829102,
+      "learning_rate": 0.0002,
+      "loss": 1.6827,
+      "step": 520
+    },
+    {
+      "epoch": 1.7235772357723578,
+      "grad_norm": 0.266201913356781,
+      "learning_rate": 0.0002,
+      "loss": 1.7059,
+      "step": 530
+    },
+    {
+      "epoch": 1.7560975609756098,
+      "grad_norm": 0.26361775398254395,
+      "learning_rate": 0.0002,
+      "loss": 1.7196,
+      "step": 540
+    },
+    {
+      "epoch": 1.7886178861788617,
+      "grad_norm": 0.2974611818790436,
+      "learning_rate": 0.0002,
+      "loss": 1.698,
+      "step": 550
+    },
+    {
+      "epoch": 1.821138211382114,
+      "grad_norm": 0.26264363527297974,
+      "learning_rate": 0.0002,
+      "loss": 1.6913,
+      "step": 560
+    },
+    {
+      "epoch": 1.8536585365853657,
+      "grad_norm": 0.31887590885162354,
+      "learning_rate": 0.0002,
+      "loss": 1.7373,
+      "step": 570
+    },
+    {
+      "epoch": 1.886178861788618,
+      "grad_norm": 0.25847622752189636,
+      "learning_rate": 0.0002,
+      "loss": 1.7269,
+      "step": 580
+    },
+    {
+      "epoch": 1.91869918699187,
+      "grad_norm": 0.2770395576953888,
+      "learning_rate": 0.0002,
+      "loss": 1.7126,
+      "step": 590
+    },
+    {
+      "epoch": 1.951219512195122,
+      "grad_norm": 0.24041323363780975,
+      "learning_rate": 0.0002,
+      "loss": 1.6897,
+      "step": 600
+    },
+    {
+      "epoch": 1.9837398373983741,
+      "grad_norm": 0.25060364603996277,
+      "learning_rate": 0.0002,
+      "loss": 1.6904,
+      "step": 610
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.7614221572875977,
+      "eval_runtime": 38.0401,
+      "eval_samples_per_second": 13.985,
+      "eval_steps_per_second": 1.761,
+      "step": 615
+    },
+    {
+      "epoch": 2.016260162601626,
+      "grad_norm": 0.25516635179519653,
+      "learning_rate": 0.0002,
+      "loss": 1.7299,
+      "step": 620
+    },
+    {
+      "epoch": 2.048780487804878,
+      "grad_norm": 0.31203776597976685,
+      "learning_rate": 0.0002,
+      "loss": 1.6142,
+      "step": 630
+    },
+    {
+      "epoch": 2.08130081300813,
+      "grad_norm": 0.27818965911865234,
+      "learning_rate": 0.0002,
+      "loss": 1.6519,
+      "step": 640
+    },
+    {
+      "epoch": 2.113821138211382,
+      "grad_norm": 0.2781952917575836,
+      "learning_rate": 0.0002,
+      "loss": 1.6678,
+      "step": 650
+    },
+    {
+      "epoch": 2.1463414634146343,
+      "grad_norm": 0.2937796413898468,
+      "learning_rate": 0.0002,
+      "loss": 1.5723,
+      "step": 660
+    },
+    {
+      "epoch": 2.178861788617886,
+      "grad_norm": 0.3559171259403229,
+      "learning_rate": 0.0002,
+      "loss": 1.6118,
+      "step": 670
+    },
+    {
+      "epoch": 2.2113821138211383,
+      "grad_norm": 0.3274490237236023,
+      "learning_rate": 0.0002,
+      "loss": 1.6044,
+      "step": 680
+    },
+    {
+      "epoch": 2.2439024390243905,
+      "grad_norm": 0.34521982073783875,
+      "learning_rate": 0.0002,
+      "loss": 1.6521,
+      "step": 690
+    },
+    {
+      "epoch": 2.2764227642276422,
+      "grad_norm": 0.30293744802474976,
+      "learning_rate": 0.0002,
+      "loss": 1.6075,
+      "step": 700
+    },
+    {
+      "epoch": 2.3089430894308944,
+      "grad_norm": 0.31012430787086487,
+      "learning_rate": 0.0002,
+      "loss": 1.6843,
+      "step": 710
+    },
+    {
+      "epoch": 2.341463414634146,
+      "grad_norm": 0.3424604535102844,
+      "learning_rate": 0.0002,
+      "loss": 1.6524,
+      "step": 720
+    },
+    {
+      "epoch": 2.3739837398373984,
+      "grad_norm": 0.3361012041568756,
+      "learning_rate": 0.0002,
+      "loss": 1.6404,
+      "step": 730
+    },
+    {
+      "epoch": 2.40650406504065,
+      "grad_norm": 0.308322012424469,
+      "learning_rate": 0.0002,
+      "loss": 1.6335,
+      "step": 740
+    },
+    {
+      "epoch": 2.4390243902439024,
+      "grad_norm": 0.3367919921875,
+      "learning_rate": 0.0002,
+      "loss": 1.6867,
+      "step": 750
+    },
+    {
+      "epoch": 2.4715447154471546,
+      "grad_norm": 0.3119719922542572,
+      "learning_rate": 0.0002,
+      "loss": 1.6332,
+      "step": 760
+    },
+    {
+      "epoch": 2.5040650406504064,
+      "grad_norm": 0.31164902448654175,
+      "learning_rate": 0.0002,
+      "loss": 1.6855,
+      "step": 770
+    },
+    {
+      "epoch": 2.5365853658536586,
+      "grad_norm": 0.3472500443458557,
+      "learning_rate": 0.0002,
+      "loss": 1.5806,
+      "step": 780
+    },
+    {
+      "epoch": 2.569105691056911,
+      "grad_norm": 0.32276982069015503,
+      "learning_rate": 0.0002,
+      "loss": 1.6171,
+      "step": 790
+    },
+    {
+      "epoch": 2.6016260162601625,
+      "grad_norm": 0.2904753088951111,
+      "learning_rate": 0.0002,
+      "loss": 1.5879,
+      "step": 800
+    },
+    {
+      "epoch": 2.6341463414634148,
+      "grad_norm": 0.3249073624610901,
+      "learning_rate": 0.0002,
+      "loss": 1.5967,
+      "step": 810
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.35803690552711487,
+      "learning_rate": 0.0002,
+      "loss": 1.7066,
+      "step": 820
+    },
+    {
+      "epoch": 2.6991869918699187,
+      "grad_norm": 0.3210543990135193,
+      "learning_rate": 0.0002,
+      "loss": 1.6163,
+      "step": 830
+    },
+    {
+      "epoch": 2.7317073170731705,
+      "grad_norm": 0.330085813999176,
+      "learning_rate": 0.0002,
+      "loss": 1.6599,
+      "step": 840
+    },
+    {
+      "epoch": 2.7642276422764227,
+      "grad_norm": 0.35581567883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.6338,
+      "step": 850
+    },
+    {
+      "epoch": 2.796747967479675,
+      "grad_norm": 0.3423900008201599,
+      "learning_rate": 0.0002,
+      "loss": 1.6033,
+      "step": 860
+    },
+    {
+      "epoch": 2.8292682926829267,
+      "grad_norm": 0.33455702662467957,
+      "learning_rate": 0.0002,
+      "loss": 1.6406,
+      "step": 870
+    },
+    {
+      "epoch": 2.861788617886179,
+      "grad_norm": 0.37637296319007874,
+      "learning_rate": 0.0002,
+      "loss": 1.6728,
+      "step": 880
+    },
+    {
+      "epoch": 2.894308943089431,
+      "grad_norm": 0.3715825080871582,
+      "learning_rate": 0.0002,
+      "loss": 1.6166,
+      "step": 890
+    },
+    {
+      "epoch": 2.926829268292683,
+      "grad_norm": 0.3526729345321655,
+      "learning_rate": 0.0002,
+      "loss": 1.7131,
+      "step": 900
+    },
+    {
+      "epoch": 2.959349593495935,
+      "grad_norm": 0.33071836829185486,
+      "learning_rate": 0.0002,
+      "loss": 1.5714,
+      "step": 910
+    },
+    {
+      "epoch": 2.991869918699187,
+      "grad_norm": 0.2956294119358063,
+      "learning_rate": 0.0002,
+      "loss": 1.6251,
+      "step": 920
+    },
+    {
+      "epoch": 2.9983739837398375,
+      "eval_loss": 1.7697627544403076,
+      "eval_runtime": 38.0483,
+      "eval_samples_per_second": 13.982,
+      "eval_steps_per_second": 1.761,
+      "step": 922
+    },
+    {
+      "epoch": 3.024390243902439,
+      "grad_norm": 0.35567474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.5265,
+      "step": 930
+    },
+    {
+      "epoch": 3.0569105691056913,
+      "grad_norm": 0.36280205845832825,
+      "learning_rate": 0.0002,
+      "loss": 1.5131,
+      "step": 940
+    },
+    {
+      "epoch": 3.089430894308943,
+      "grad_norm": 0.39025646448135376,
+      "learning_rate": 0.0002,
+      "loss": 1.5492,
+      "step": 950
+    },
+    {
+      "epoch": 3.1219512195121952,
+      "grad_norm": 0.4128074049949646,
+      "learning_rate": 0.0002,
+      "loss": 1.6059,
+      "step": 960
+    },
+    {
+      "epoch": 3.154471544715447,
+      "grad_norm": 0.37436643242836,
+      "learning_rate": 0.0002,
+      "loss": 1.5675,
+      "step": 970
+    },
+    {
+      "epoch": 3.186991869918699,
+      "grad_norm": 0.3950212895870209,
+      "learning_rate": 0.0002,
+      "loss": 1.5018,
+      "step": 980
+    },
+    {
+      "epoch": 3.2195121951219514,
+      "grad_norm": 0.3334033787250519,
+      "learning_rate": 0.0002,
+      "loss": 1.4856,
+      "step": 990
+    },
+    {
+      "epoch": 3.252032520325203,
+      "grad_norm": 0.41348570585250854,
+      "learning_rate": 0.0002,
+      "loss": 1.5288,
+      "step": 1000
+    },
+    {
+      "epoch": 3.2845528455284554,
+      "grad_norm": 0.48985755443573,
+      "learning_rate": 0.0002,
+      "loss": 1.565,
+      "step": 1010
+    },
+    {
+      "epoch": 3.317073170731707,
+      "grad_norm": 0.36514127254486084,
+      "learning_rate": 0.0002,
+      "loss": 1.5463,
+      "step": 1020
+    },
+    {
+      "epoch": 3.3495934959349594,
+      "grad_norm": 0.4296952188014984,
+      "learning_rate": 0.0002,
+      "loss": 1.5135,
+      "step": 1030
+    },
+    {
+      "epoch": 3.3821138211382116,
+      "grad_norm": 0.40017712116241455,
+      "learning_rate": 0.0002,
+      "loss": 1.5086,
+      "step": 1040
+    },
+    {
+      "epoch": 3.4146341463414633,
+      "grad_norm": 0.4218886196613312,
+      "learning_rate": 0.0002,
+      "loss": 1.6026,
+      "step": 1050
+    },
+    {
+      "epoch": 3.4471544715447155,
+      "grad_norm": 0.42598757147789,
+      "learning_rate": 0.0002,
+      "loss": 1.5337,
+      "step": 1060
+    },
+    {
+      "epoch": 3.4796747967479673,
+      "grad_norm": 0.4137382209300995,
+      "learning_rate": 0.0002,
+      "loss": 1.5487,
+      "step": 1070
+    },
+    {
+      "epoch": 3.5121951219512195,
+      "grad_norm": 0.3774033784866333,
+      "learning_rate": 0.0002,
+      "loss": 1.5132,
+      "step": 1080
+    },
+    {
+      "epoch": 3.5447154471544717,
+      "grad_norm": 0.40397852659225464,
+      "learning_rate": 0.0002,
+      "loss": 1.5842,
+      "step": 1090
+    },
+    {
+      "epoch": 3.5772357723577235,
+      "grad_norm": 0.4346853792667389,
+      "learning_rate": 0.0002,
+      "loss": 1.583,
+      "step": 1100
+    },
+    {
+      "epoch": 3.6097560975609757,
+      "grad_norm": 0.40098339319229126,
+      "learning_rate": 0.0002,
+      "loss": 1.5963,
+      "step": 1110
+    },
+    {
+      "epoch": 3.642276422764228,
+      "grad_norm": 0.4212099313735962,
+      "learning_rate": 0.0002,
+      "loss": 1.5225,
+      "step": 1120
+    },
+    {
+      "epoch": 3.6747967479674797,
+      "grad_norm": 0.4408110976219177,
+      "learning_rate": 0.0002,
+      "loss": 1.5987,
+      "step": 1130
+    },
+    {
+      "epoch": 3.7073170731707314,
+      "grad_norm": 0.3728943169116974,
+      "learning_rate": 0.0002,
+      "loss": 1.5813,
+      "step": 1140
+    },
+    {
+      "epoch": 3.7398373983739837,
+      "grad_norm": 0.3994787633419037,
+      "learning_rate": 0.0002,
+      "loss": 1.6103,
+      "step": 1150
+    },
+    {
+      "epoch": 3.772357723577236,
+      "grad_norm": 0.4318053126335144,
+      "learning_rate": 0.0002,
+      "loss": 1.5812,
+      "step": 1160
+    },
+    {
+      "epoch": 3.8048780487804876,
+      "grad_norm": 0.40556737780570984,
+      "learning_rate": 0.0002,
+      "loss": 1.6173,
+      "step": 1170
+    },
+    {
+      "epoch": 3.83739837398374,
+      "grad_norm": 0.38010650873184204,
+      "learning_rate": 0.0002,
+      "loss": 1.6099,
+      "step": 1180
+    },
+    {
+      "epoch": 3.869918699186992,
+      "grad_norm": 0.4114062190055847,
+      "learning_rate": 0.0002,
+      "loss": 1.4842,
+      "step": 1190
+    },
+    {
+      "epoch": 3.902439024390244,
+      "grad_norm": 0.4016933739185333,
+      "learning_rate": 0.0002,
+      "loss": 1.6047,
+      "step": 1200
+    },
+    {
+      "epoch": 3.934959349593496,
+      "grad_norm": 0.45411986112594604,
+      "learning_rate": 0.0002,
+      "loss": 1.497,
+      "step": 1210
+    },
+    {
+      "epoch": 3.9674796747967482,
+      "grad_norm": 0.3914777934551239,
+      "learning_rate": 0.0002,
+      "loss": 1.5838,
+      "step": 1220
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.42423349618911743,
+      "learning_rate": 0.0002,
+      "loss": 1.541,
+      "step": 1230
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.7998414039611816,
+      "eval_runtime": 38.0517,
+      "eval_samples_per_second": 13.981,
+      "eval_steps_per_second": 1.761,
+      "step": 1230
+    },
+    {
+      "epoch": 4.032520325203252,
+      "grad_norm": 0.5187233686447144,
+      "learning_rate": 0.0002,
+      "loss": 1.4585,
+      "step": 1240
+    },
+    {
+      "epoch": 4.065040650406504,
+      "grad_norm": 0.47915923595428467,
+      "learning_rate": 0.0002,
+      "loss": 1.3959,
+      "step": 1250
+    },
+    {
+      "epoch": 4.097560975609756,
+      "grad_norm": 0.5152283310890198,
+      "learning_rate": 0.0002,
+      "loss": 1.4296,
+      "step": 1260
+    },
+    {
+      "epoch": 4.130081300813008,
+      "grad_norm": 0.49710479378700256,
+      "learning_rate": 0.0002,
+      "loss": 1.4722,
+      "step": 1270
+    },
+    {
+      "epoch": 4.16260162601626,
+      "grad_norm": 0.5043147802352905,
+      "learning_rate": 0.0002,
+      "loss": 1.3868,
+      "step": 1280
+    },
+    {
+      "epoch": 4.195121951219512,
+      "grad_norm": 0.49862509965896606,
+      "learning_rate": 0.0002,
+      "loss": 1.4762,
+      "step": 1290
+    },
+    {
+      "epoch": 4.227642276422764,
+      "grad_norm": 0.5163412690162659,
+      "learning_rate": 0.0002,
+      "loss": 1.4187,
+      "step": 1300
+    },
+    {
+      "epoch": 4.260162601626016,
+      "grad_norm": 0.5454265475273132,
+      "learning_rate": 0.0002,
+      "loss": 1.4717,
+      "step": 1310
+    },
+    {
+      "epoch": 4.2926829268292686,
+      "grad_norm": 0.54813551902771,
+      "learning_rate": 0.0002,
+      "loss": 1.4065,
+      "step": 1320
+    },
+    {
+      "epoch": 4.32520325203252,
+      "grad_norm": 0.5753244161605835,
+      "learning_rate": 0.0002,
+      "loss": 1.5298,
+      "step": 1330
+    },
+    {
+      "epoch": 4.357723577235772,
+      "grad_norm": 0.5596114993095398,
+      "learning_rate": 0.0002,
+      "loss": 1.4372,
+      "step": 1340
+    },
+    {
+      "epoch": 4.390243902439025,
+      "grad_norm": 0.5419771671295166,
+      "learning_rate": 0.0002,
+      "loss": 1.451,
+      "step": 1350
+    },
+    {
+      "epoch": 4.4227642276422765,
+      "grad_norm": 0.5357267260551453,
+      "learning_rate": 0.0002,
+      "loss": 1.5257,
+      "step": 1360
+    },
+    {
+      "epoch": 4.455284552845528,
+      "grad_norm": 0.5426302552223206,
+      "learning_rate": 0.0002,
+      "loss": 1.4724,
+      "step": 1370
+    },
+    {
+      "epoch": 4.487804878048781,
+      "grad_norm": 0.48658859729766846,
+      "learning_rate": 0.0002,
+      "loss": 1.442,
+      "step": 1380
+    },
+    {
+      "epoch": 4.520325203252033,
+      "grad_norm": 0.46241894364356995,
+      "learning_rate": 0.0002,
+      "loss": 1.5034,
+      "step": 1390
+    },
+    {
+      "epoch": 4.5528455284552845,
+      "grad_norm": 0.47949138283729553,
+      "learning_rate": 0.0002,
+      "loss": 1.4738,
+      "step": 1400
+    },
+    {
+      "epoch": 4.585365853658536,
+      "grad_norm": 0.5619633793830872,
+      "learning_rate": 0.0002,
+      "loss": 1.4343,
+      "step": 1410
+    },
+    {
+      "epoch": 4.617886178861789,
+      "grad_norm": 0.5513656139373779,
+      "learning_rate": 0.0002,
+      "loss": 1.398,
+      "step": 1420
+    },
+    {
+      "epoch": 4.650406504065041,
+      "grad_norm": 0.5279194712638855,
+      "learning_rate": 0.0002,
+      "loss": 1.4071,
+      "step": 1430
+    },
+    {
+      "epoch": 4.682926829268292,
+      "grad_norm": 0.49925369024276733,
+      "learning_rate": 0.0002,
+      "loss": 1.5391,
+      "step": 1440
+    },
+    {
+      "epoch": 4.715447154471545,
+      "grad_norm": 0.5767236351966858,
+      "learning_rate": 0.0002,
+      "loss": 1.5315,
+      "step": 1450
+    },
+    {
+      "epoch": 4.747967479674797,
+      "grad_norm": 0.5069461464881897,
+      "learning_rate": 0.0002,
+      "loss": 1.4125,
+      "step": 1460
+    },
+    {
+      "epoch": 4.780487804878049,
+      "grad_norm": 0.5566238760948181,
+      "learning_rate": 0.0002,
+      "loss": 1.457,
+      "step": 1470
+    },
+    {
+      "epoch": 4.8130081300813,
+      "grad_norm": 0.529227077960968,
+      "learning_rate": 0.0002,
+      "loss": 1.4497,
+      "step": 1480
+    },
+    {
+      "epoch": 4.845528455284553,
+      "grad_norm": 0.5591039657592773,
+      "learning_rate": 0.0002,
+      "loss": 1.4585,
+      "step": 1490
+    },
+    {
+      "epoch": 4.878048780487805,
+      "grad_norm": 0.45602425932884216,
+      "learning_rate": 0.0002,
+      "loss": 1.5522,
+      "step": 1500
+    },
+    {
+      "epoch": 4.9105691056910565,
+      "grad_norm": 0.4569244682788849,
+      "learning_rate": 0.0002,
+      "loss": 1.4134,
+      "step": 1510
+    },
+    {
+      "epoch": 4.943089430894309,
+      "grad_norm": 0.5215898752212524,
+      "learning_rate": 0.0002,
+      "loss": 1.5061,
+      "step": 1520
+    },
+    {
+      "epoch": 4.975609756097561,
+      "grad_norm": 0.4940566122531891,
+      "learning_rate": 0.0002,
+      "loss": 1.4864,
+      "step": 1530
+    },
+    {
+      "epoch": 4.998373983739837,
+      "eval_loss": 1.8480068445205688,
+      "eval_runtime": 38.043,
+      "eval_samples_per_second": 13.984,
+      "eval_steps_per_second": 1.761,
+      "step": 1537
+    },
+    {
+      "epoch": 5.008130081300813,
+      "grad_norm": 0.5146103501319885,
+      "learning_rate": 0.0002,
+      "loss": 1.4122,
+      "step": 1540
+    },
+    {
+      "epoch": 5.040650406504065,
+      "grad_norm": 0.5901735424995422,
+      "learning_rate": 0.0002,
+      "loss": 1.3637,
+      "step": 1550
+    },
+    {
+      "epoch": 5.073170731707317,
+      "grad_norm": 0.55073481798172,
+      "learning_rate": 0.0002,
+      "loss": 1.3513,
+      "step": 1560
+    },
+    {
+      "epoch": 5.105691056910569,
+      "grad_norm": 0.6664248704910278,
+      "learning_rate": 0.0002,
+      "loss": 1.345,
+      "step": 1570
+    },
+    {
+      "epoch": 5.138211382113822,
+      "grad_norm": 1.6458815336227417,
+      "learning_rate": 0.0002,
+      "loss": 1.3587,
+      "step": 1580
+    },
+    {
+      "epoch": 5.170731707317073,
+      "grad_norm": 0.5578900575637817,
+      "learning_rate": 0.0002,
+      "loss": 1.4153,
+      "step": 1590
+    },
+    {
+      "epoch": 5.203252032520325,
+      "grad_norm": 0.6810198426246643,
+      "learning_rate": 0.0002,
+      "loss": 1.4132,
+      "step": 1600
+    },
+    {
+      "epoch": 5.235772357723577,
+      "grad_norm": 0.6808196902275085,
+      "learning_rate": 0.0002,
+      "loss": 1.3227,
+      "step": 1610
+    },
+    {
+      "epoch": 5.2682926829268295,
+      "grad_norm": 0.6254436373710632,
+      "learning_rate": 0.0002,
+      "loss": 1.358,
+      "step": 1620
+    },
+    {
+      "epoch": 5.300813008130081,
+      "grad_norm": 0.6863614320755005,
+      "learning_rate": 0.0002,
+      "loss": 1.3661,
+      "step": 1630
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 0.6279789209365845,
+      "learning_rate": 0.0002,
+      "loss": 1.3824,
+      "step": 1640
+    },
+    {
+      "epoch": 5.365853658536586,
+      "grad_norm": 0.7112933397293091,
+      "learning_rate": 0.0002,
+      "loss": 1.3383,
+      "step": 1650
+    },
+    {
+      "epoch": 5.3983739837398375,
+      "grad_norm": 0.6307976841926575,
+      "learning_rate": 0.0002,
+      "loss": 1.3122,
+      "step": 1660
+    },
+    {
+      "epoch": 5.430894308943089,
+      "grad_norm": 0.6683973670005798,
+      "learning_rate": 0.0002,
+      "loss": 1.3796,
+      "step": 1670
+    },
+    {
+      "epoch": 5.463414634146342,
+      "grad_norm": 0.667790949344635,
+      "learning_rate": 0.0002,
+      "loss": 1.2882,
+      "step": 1680
+    },
+    {
+      "epoch": 5.495934959349594,
+      "grad_norm": 0.6940338015556335,
+      "learning_rate": 0.0002,
+      "loss": 1.3672,
+      "step": 1690
+    },
+    {
+      "epoch": 5.528455284552845,
+      "grad_norm": 0.5552698969841003,
+      "learning_rate": 0.0002,
+      "loss": 1.3508,
+      "step": 1700
+    },
+    {
+      "epoch": 5.560975609756097,
+      "grad_norm": 0.6904126405715942,
+      "learning_rate": 0.0002,
+      "loss": 1.4224,
+      "step": 1710
+    },
+    {
+      "epoch": 5.59349593495935,
+      "grad_norm": 0.7521847486495972,
+      "learning_rate": 0.0002,
+      "loss": 1.329,
+      "step": 1720
+    },
+    {
+      "epoch": 5.626016260162602,
+      "grad_norm": 0.6844537854194641,
+      "learning_rate": 0.0002,
+      "loss": 1.3259,
+      "step": 1730
+    },
+    {
+      "epoch": 5.658536585365853,
+      "grad_norm": 0.7945169806480408,
+      "learning_rate": 0.0002,
+      "loss": 1.343,
+      "step": 1740
+    },
+    {
+      "epoch": 5.691056910569106,
+      "grad_norm": 0.6818302869796753,
+      "learning_rate": 0.0002,
+      "loss": 1.3845,
+      "step": 1750
+    },
+    {
+      "epoch": 5.723577235772358,
+      "grad_norm": 0.5638683438301086,
+      "learning_rate": 0.0002,
+      "loss": 1.3502,
+      "step": 1760
+    },
+    {
+      "epoch": 5.7560975609756095,
+      "grad_norm": 0.6028141975402832,
+      "learning_rate": 0.0002,
+      "loss": 1.3778,
+      "step": 1770
+    },
+    {
+      "epoch": 5.788617886178862,
+      "grad_norm": 0.7325388193130493,
+      "learning_rate": 0.0002,
+      "loss": 1.3879,
+      "step": 1780
+    },
+    {
+      "epoch": 5.821138211382114,
+      "grad_norm": 0.6316136717796326,
+      "learning_rate": 0.0002,
+      "loss": 1.3711,
+      "step": 1790
+    },
+    {
+      "epoch": 5.853658536585366,
+      "grad_norm": 0.7407512068748474,
+      "learning_rate": 0.0002,
+      "loss": 1.3615,
+      "step": 1800
+    },
+    {
+      "epoch": 5.886178861788618,
+      "grad_norm": 0.6717495918273926,
+      "learning_rate": 0.0002,
+      "loss": 1.3225,
+      "step": 1810
+    },
+    {
+      "epoch": 5.91869918699187,
+      "grad_norm": 0.6663013100624084,
+      "learning_rate": 0.0002,
+      "loss": 1.2845,
+      "step": 1820
+    },
+    {
+      "epoch": 5.951219512195122,
+      "grad_norm": 0.6448108553886414,
+      "learning_rate": 0.0002,
+      "loss": 1.3695,
+      "step": 1830
+    },
+    {
+      "epoch": 5.983739837398374,
+      "grad_norm": 0.6900140047073364,
+      "learning_rate": 0.0002,
+      "loss": 1.3445,
+      "step": 1840
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.9016555547714233,
+      "eval_runtime": 38.0417,
+      "eval_samples_per_second": 13.985,
+      "eval_steps_per_second": 1.761,
+      "step": 1845
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2456,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.037908762591232e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-1845/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a6ec6940b5d2cf34a18e1e3cb0fe34c3805cbaea884830888b0239757f74df
+size 5560

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-2152/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-2152/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-2152/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:174776d5c5b04987df0b033488422eab375d06b2021c5c03f4f38dc021def3be
+size 80755416

Qwen2-7B-Instruct_int4_hellaswag-routerbench-0shot_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-1405-sd-4/checkpoint-2152/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}