Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0aeb9a3fa5ca7dae97311f327fe4d9be03d768797557681128a077ed0938f911
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aef542b6b4406db11d734813ca16f878d45828a1431ce8987c81632b85d9e6d8
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fb7757782422fcb9b0497c9cd5202b82854c4d621f05daf6ee05b694ea0db97
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fa5c51ebaa58554d933cb076e63dcb1751bc3a86fe4374d083cf1d814108864
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:993b156e7cfdce550d18cf7697cd47ed8dd1120317c14cdb7c96791c6abca29f
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/trainer_state.json

@@ -0,0 +1,859 @@
+{
+  "best_metric": 1.2515619993209839,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158",
+  "epoch": 6.996845425867508,
+  "eval_steps": 10,
+  "global_step": 1109,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06309148264984227,
+      "grad_norm": 1.1088604927062988,
+      "learning_rate": 0.0002,
+      "loss": 1.893,
+      "step": 10
+    },
+    {
+      "epoch": 0.12618296529968454,
+      "grad_norm": 0.8254690170288086,
+      "learning_rate": 0.0002,
+      "loss": 1.5359,
+      "step": 20
+    },
+    {
+      "epoch": 0.1892744479495268,
+      "grad_norm": 0.7787631154060364,
+      "learning_rate": 0.0002,
+      "loss": 1.5667,
+      "step": 30
+    },
+    {
+      "epoch": 0.25236593059936907,
+      "grad_norm": 1.0321671962738037,
+      "learning_rate": 0.0002,
+      "loss": 1.4567,
+      "step": 40
+    },
+    {
+      "epoch": 0.31545741324921134,
+      "grad_norm": 1.6003938913345337,
+      "learning_rate": 0.0002,
+      "loss": 1.4166,
+      "step": 50
+    },
+    {
+      "epoch": 0.3785488958990536,
+      "grad_norm": 0.6283053159713745,
+      "learning_rate": 0.0002,
+      "loss": 1.3302,
+      "step": 60
+    },
+    {
+      "epoch": 0.4416403785488959,
+      "grad_norm": 4.210716724395752,
+      "learning_rate": 0.0002,
+      "loss": 1.3358,
+      "step": 70
+    },
+    {
+      "epoch": 0.5047318611987381,
+      "grad_norm": 0.6119468212127686,
+      "learning_rate": 0.0002,
+      "loss": 1.2831,
+      "step": 80
+    },
+    {
+      "epoch": 0.5678233438485805,
+      "grad_norm": 0.6245906949043274,
+      "learning_rate": 0.0002,
+      "loss": 1.2199,
+      "step": 90
+    },
+    {
+      "epoch": 0.6309148264984227,
+      "grad_norm": 0.7737864255905151,
+      "learning_rate": 0.0002,
+      "loss": 1.3255,
+      "step": 100
+    },
+    {
+      "epoch": 0.694006309148265,
+      "grad_norm": 0.7152376770973206,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 110
+    },
+    {
+      "epoch": 0.7570977917981072,
+      "grad_norm": 0.7520427107810974,
+      "learning_rate": 0.0002,
+      "loss": 1.1685,
+      "step": 120
+    },
+    {
+      "epoch": 0.8201892744479495,
+      "grad_norm": 0.9310459494590759,
+      "learning_rate": 0.0002,
+      "loss": 1.2923,
+      "step": 130
+    },
+    {
+      "epoch": 0.8832807570977917,
+      "grad_norm": 0.6360874772071838,
+      "learning_rate": 0.0002,
+      "loss": 1.3183,
+      "step": 140
+    },
+    {
+      "epoch": 0.9463722397476341,
+      "grad_norm": 0.5835223197937012,
+      "learning_rate": 0.0002,
+      "loss": 1.2824,
+      "step": 150
+    },
+    {
+      "epoch": 0.9968454258675079,
+      "eval_loss": 1.2515619993209839,
+      "eval_runtime": 87.7902,
+      "eval_samples_per_second": 5.194,
+      "eval_steps_per_second": 0.649,
+      "step": 158
+    },
+    {
+      "epoch": 1.0094637223974763,
+      "grad_norm": 0.459634393453598,
+      "learning_rate": 0.0002,
+      "loss": 1.225,
+      "step": 160
+    },
+    {
+      "epoch": 1.0725552050473186,
+      "grad_norm": 0.7972345948219299,
+      "learning_rate": 0.0002,
+      "loss": 1.0948,
+      "step": 170
+    },
+    {
+      "epoch": 1.135646687697161,
+      "grad_norm": 0.7474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.1488,
+      "step": 180
+    },
+    {
+      "epoch": 1.1987381703470033,
+      "grad_norm": 0.8033431172370911,
+      "learning_rate": 0.0002,
+      "loss": 1.0838,
+      "step": 190
+    },
+    {
+      "epoch": 1.2618296529968454,
+      "grad_norm": 0.7721152305603027,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 200
+    },
+    {
+      "epoch": 1.3249211356466877,
+      "grad_norm": 0.6452336311340332,
+      "learning_rate": 0.0002,
+      "loss": 1.178,
+      "step": 210
+    },
+    {
+      "epoch": 1.38801261829653,
+      "grad_norm": 0.6738545298576355,
+      "learning_rate": 0.0002,
+      "loss": 1.0702,
+      "step": 220
+    },
+    {
+      "epoch": 1.4511041009463723,
+      "grad_norm": 0.7977299094200134,
+      "learning_rate": 0.0002,
+      "loss": 1.1874,
+      "step": 230
+    },
+    {
+      "epoch": 1.5141955835962144,
+      "grad_norm": 0.688931941986084,
+      "learning_rate": 0.0002,
+      "loss": 1.0575,
+      "step": 240
+    },
+    {
+      "epoch": 1.5772870662460567,
+      "grad_norm": 0.7142922878265381,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 250
+    },
+    {
+      "epoch": 1.640378548895899,
+      "grad_norm": 0.974367082118988,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 260
+    },
+    {
+      "epoch": 1.7034700315457414,
+      "grad_norm": 0.8500529527664185,
+      "learning_rate": 0.0002,
+      "loss": 1.0401,
+      "step": 270
+    },
+    {
+      "epoch": 1.7665615141955837,
+      "grad_norm": 0.6906878352165222,
+      "learning_rate": 0.0002,
+      "loss": 1.1212,
+      "step": 280
+    },
+    {
+      "epoch": 1.8296529968454258,
+      "grad_norm": 0.9898052215576172,
+      "learning_rate": 0.0002,
+      "loss": 1.1583,
+      "step": 290
+    },
+    {
+      "epoch": 1.8927444794952681,
+      "grad_norm": 0.6456423401832581,
+      "learning_rate": 0.0002,
+      "loss": 1.1509,
+      "step": 300
+    },
+    {
+      "epoch": 1.9558359621451105,
+      "grad_norm": 0.9416996240615845,
+      "learning_rate": 0.0002,
+      "loss": 1.0231,
+      "step": 310
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2548311948776245,
+      "eval_runtime": 88.0511,
+      "eval_samples_per_second": 5.179,
+      "eval_steps_per_second": 0.647,
+      "step": 317
+    },
+    {
+      "epoch": 2.0189274447949526,
+      "grad_norm": 0.8139097094535828,
+      "learning_rate": 0.0002,
+      "loss": 1.1737,
+      "step": 320
+    },
+    {
+      "epoch": 2.082018927444795,
+      "grad_norm": 0.8165537714958191,
+      "learning_rate": 0.0002,
+      "loss": 0.9225,
+      "step": 330
+    },
+    {
+      "epoch": 2.145110410094637,
+      "grad_norm": 1.0027955770492554,
+      "learning_rate": 0.0002,
+      "loss": 0.8145,
+      "step": 340
+    },
+    {
+      "epoch": 2.2082018927444795,
+      "grad_norm": 1.1729809045791626,
+      "learning_rate": 0.0002,
+      "loss": 0.8326,
+      "step": 350
+    },
+    {
+      "epoch": 2.271293375394322,
+      "grad_norm": 1.067833423614502,
+      "learning_rate": 0.0002,
+      "loss": 0.8332,
+      "step": 360
+    },
+    {
+      "epoch": 2.334384858044164,
+      "grad_norm": 0.9201022386550903,
+      "learning_rate": 0.0002,
+      "loss": 0.8749,
+      "step": 370
+    },
+    {
+      "epoch": 2.3974763406940065,
+      "grad_norm": 0.9174187779426575,
+      "learning_rate": 0.0002,
+      "loss": 0.8903,
+      "step": 380
+    },
+    {
+      "epoch": 2.4605678233438484,
+      "grad_norm": 1.0019031763076782,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 390
+    },
+    {
+      "epoch": 2.5236593059936907,
+      "grad_norm": 0.9429060220718384,
+      "learning_rate": 0.0002,
+      "loss": 0.947,
+      "step": 400
+    },
+    {
+      "epoch": 2.586750788643533,
+      "grad_norm": 1.307620644569397,
+      "learning_rate": 0.0002,
+      "loss": 0.8577,
+      "step": 410
+    },
+    {
+      "epoch": 2.6498422712933754,
+      "grad_norm": 1.395595669746399,
+      "learning_rate": 0.0002,
+      "loss": 0.8823,
+      "step": 420
+    },
+    {
+      "epoch": 2.7129337539432177,
+      "grad_norm": 1.1542165279388428,
+      "learning_rate": 0.0002,
+      "loss": 0.9286,
+      "step": 430
+    },
+    {
+      "epoch": 2.77602523659306,
+      "grad_norm": 0.9339272975921631,
+      "learning_rate": 0.0002,
+      "loss": 0.8581,
+      "step": 440
+    },
+    {
+      "epoch": 2.8391167192429023,
+      "grad_norm": 1.0789870023727417,
+      "learning_rate": 0.0002,
+      "loss": 0.8878,
+      "step": 450
+    },
+    {
+      "epoch": 2.9022082018927446,
+      "grad_norm": 0.9041107892990112,
+      "learning_rate": 0.0002,
+      "loss": 0.777,
+      "step": 460
+    },
+    {
+      "epoch": 2.965299684542587,
+      "grad_norm": 1.1063491106033325,
+      "learning_rate": 0.0002,
+      "loss": 0.9554,
+      "step": 470
+    },
+    {
+      "epoch": 2.996845425867508,
+      "eval_loss": 1.3252698183059692,
+      "eval_runtime": 87.6199,
+      "eval_samples_per_second": 5.204,
+      "eval_steps_per_second": 0.651,
+      "step": 475
+    },
+    {
+      "epoch": 3.028391167192429,
+      "grad_norm": 0.8952794671058655,
+      "learning_rate": 0.0002,
+      "loss": 0.916,
+      "step": 480
+    },
+    {
+      "epoch": 3.091482649842271,
+      "grad_norm": 1.1680232286453247,
+      "learning_rate": 0.0002,
+      "loss": 0.6808,
+      "step": 490
+    },
+    {
+      "epoch": 3.1545741324921135,
+      "grad_norm": 1.1490330696105957,
+      "learning_rate": 0.0002,
+      "loss": 0.6603,
+      "step": 500
+    },
+    {
+      "epoch": 3.217665615141956,
+      "grad_norm": 1.807936429977417,
+      "learning_rate": 0.0002,
+      "loss": 0.5892,
+      "step": 510
+    },
+    {
+      "epoch": 3.280757097791798,
+      "grad_norm": 1.4477553367614746,
+      "learning_rate": 0.0002,
+      "loss": 0.6701,
+      "step": 520
+    },
+    {
+      "epoch": 3.3438485804416405,
+      "grad_norm": 1.7911386489868164,
+      "learning_rate": 0.0002,
+      "loss": 0.6107,
+      "step": 530
+    },
+    {
+      "epoch": 3.406940063091483,
+      "grad_norm": 1.136080026626587,
+      "learning_rate": 0.0002,
+      "loss": 0.6779,
+      "step": 540
+    },
+    {
+      "epoch": 3.470031545741325,
+      "grad_norm": 1.1295137405395508,
+      "learning_rate": 0.0002,
+      "loss": 0.6493,
+      "step": 550
+    },
+    {
+      "epoch": 3.5331230283911674,
+      "grad_norm": 1.573101282119751,
+      "learning_rate": 0.0002,
+      "loss": 0.6652,
+      "step": 560
+    },
+    {
+      "epoch": 3.5962145110410093,
+      "grad_norm": 1.403876543045044,
+      "learning_rate": 0.0002,
+      "loss": 0.6772,
+      "step": 570
+    },
+    {
+      "epoch": 3.6593059936908516,
+      "grad_norm": 1.1751461029052734,
+      "learning_rate": 0.0002,
+      "loss": 0.6655,
+      "step": 580
+    },
+    {
+      "epoch": 3.722397476340694,
+      "grad_norm": 1.6286189556121826,
+      "learning_rate": 0.0002,
+      "loss": 0.783,
+      "step": 590
+    },
+    {
+      "epoch": 3.7854889589905363,
+      "grad_norm": 1.2480772733688354,
+      "learning_rate": 0.0002,
+      "loss": 0.6854,
+      "step": 600
+    },
+    {
+      "epoch": 3.8485804416403786,
+      "grad_norm": 1.1290836334228516,
+      "learning_rate": 0.0002,
+      "loss": 0.647,
+      "step": 610
+    },
+    {
+      "epoch": 3.911671924290221,
+      "grad_norm": 1.1974420547485352,
+      "learning_rate": 0.0002,
+      "loss": 0.609,
+      "step": 620
+    },
+    {
+      "epoch": 3.9747634069400632,
+      "grad_norm": 1.318346619606018,
+      "learning_rate": 0.0002,
+      "loss": 0.6775,
+      "step": 630
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.4375206232070923,
+      "eval_runtime": 87.9488,
+      "eval_samples_per_second": 5.185,
+      "eval_steps_per_second": 0.648,
+      "step": 634
+    },
+    {
+      "epoch": 4.037854889589905,
+      "grad_norm": 1.7765438556671143,
+      "learning_rate": 0.0002,
+      "loss": 0.5632,
+      "step": 640
+    },
+    {
+      "epoch": 4.100946372239748,
+      "grad_norm": 1.1392035484313965,
+      "learning_rate": 0.0002,
+      "loss": 0.4745,
+      "step": 650
+    },
+    {
+      "epoch": 4.16403785488959,
+      "grad_norm": 2.375727653503418,
+      "learning_rate": 0.0002,
+      "loss": 0.5029,
+      "step": 660
+    },
+    {
+      "epoch": 4.2271293375394325,
+      "grad_norm": 1.537709355354309,
+      "learning_rate": 0.0002,
+      "loss": 0.4621,
+      "step": 670
+    },
+    {
+      "epoch": 4.290220820189274,
+      "grad_norm": 1.8388986587524414,
+      "learning_rate": 0.0002,
+      "loss": 0.4678,
+      "step": 680
+    },
+    {
+      "epoch": 4.353312302839116,
+      "grad_norm": 1.8100976943969727,
+      "learning_rate": 0.0002,
+      "loss": 0.4833,
+      "step": 690
+    },
+    {
+      "epoch": 4.416403785488959,
+      "grad_norm": 1.5121203660964966,
+      "learning_rate": 0.0002,
+      "loss": 0.451,
+      "step": 700
+    },
+    {
+      "epoch": 4.479495268138801,
+      "grad_norm": 2.152935743331909,
+      "learning_rate": 0.0002,
+      "loss": 0.5022,
+      "step": 710
+    },
+    {
+      "epoch": 4.542586750788644,
+      "grad_norm": 1.7240005731582642,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 720
+    },
+    {
+      "epoch": 4.605678233438486,
+      "grad_norm": 2.038640260696411,
+      "learning_rate": 0.0002,
+      "loss": 0.6016,
+      "step": 730
+    },
+    {
+      "epoch": 4.668769716088328,
+      "grad_norm": 2.1428792476654053,
+      "learning_rate": 0.0002,
+      "loss": 0.4528,
+      "step": 740
+    },
+    {
+      "epoch": 4.73186119873817,
+      "grad_norm": 1.5168582201004028,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 750
+    },
+    {
+      "epoch": 4.794952681388013,
+      "grad_norm": 1.2821815013885498,
+      "learning_rate": 0.0002,
+      "loss": 0.5586,
+      "step": 760
+    },
+    {
+      "epoch": 4.858044164037855,
+      "grad_norm": 1.3252662420272827,
+      "learning_rate": 0.0002,
+      "loss": 0.4759,
+      "step": 770
+    },
+    {
+      "epoch": 4.921135646687697,
+      "grad_norm": 1.7149165868759155,
+      "learning_rate": 0.0002,
+      "loss": 0.4706,
+      "step": 780
+    },
+    {
+      "epoch": 4.9842271293375395,
+      "grad_norm": 2.2286508083343506,
+      "learning_rate": 0.0002,
+      "loss": 0.529,
+      "step": 790
+    },
+    {
+      "epoch": 4.996845425867508,
+      "eval_loss": 1.614780306816101,
+      "eval_runtime": 87.4623,
+      "eval_samples_per_second": 5.214,
+      "eval_steps_per_second": 0.652,
+      "step": 792
+    },
+    {
+      "epoch": 5.047318611987381,
+      "grad_norm": 2.4778950214385986,
+      "learning_rate": 0.0002,
+      "loss": 0.4083,
+      "step": 800
+    },
+    {
+      "epoch": 5.110410094637224,
+      "grad_norm": 1.6371349096298218,
+      "learning_rate": 0.0002,
+      "loss": 0.4011,
+      "step": 810
+    },
+    {
+      "epoch": 5.173501577287066,
+      "grad_norm": 1.877687931060791,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 820
+    },
+    {
+      "epoch": 5.236593059936909,
+      "grad_norm": 1.5015276670455933,
+      "learning_rate": 0.0002,
+      "loss": 0.3601,
+      "step": 830
+    },
+    {
+      "epoch": 5.299684542586751,
+      "grad_norm": 2.1286396980285645,
+      "learning_rate": 0.0002,
+      "loss": 0.4043,
+      "step": 840
+    },
+    {
+      "epoch": 5.3627760252365935,
+      "grad_norm": 1.6563063859939575,
+      "learning_rate": 0.0002,
+      "loss": 0.3259,
+      "step": 850
+    },
+    {
+      "epoch": 5.425867507886435,
+      "grad_norm": 1.997558832168579,
+      "learning_rate": 0.0002,
+      "loss": 0.4065,
+      "step": 860
+    },
+    {
+      "epoch": 5.488958990536277,
+      "grad_norm": 1.8498985767364502,
+      "learning_rate": 0.0002,
+      "loss": 0.3405,
+      "step": 870
+    },
+    {
+      "epoch": 5.55205047318612,
+      "grad_norm": 1.7636688947677612,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 880
+    },
+    {
+      "epoch": 5.615141955835962,
+      "grad_norm": 1.9259185791015625,
+      "learning_rate": 0.0002,
+      "loss": 0.3898,
+      "step": 890
+    },
+    {
+      "epoch": 5.678233438485805,
+      "grad_norm": 1.917427659034729,
+      "learning_rate": 0.0002,
+      "loss": 0.3937,
+      "step": 900
+    },
+    {
+      "epoch": 5.7413249211356465,
+      "grad_norm": 1.5327430963516235,
+      "learning_rate": 0.0002,
+      "loss": 0.4001,
+      "step": 910
+    },
+    {
+      "epoch": 5.804416403785489,
+      "grad_norm": 1.8258512020111084,
+      "learning_rate": 0.0002,
+      "loss": 0.3925,
+      "step": 920
+    },
+    {
+      "epoch": 5.867507886435331,
+      "grad_norm": 1.909255862236023,
+      "learning_rate": 0.0002,
+      "loss": 0.4068,
+      "step": 930
+    },
+    {
+      "epoch": 5.930599369085174,
+      "grad_norm": 2.272733211517334,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 940
+    },
+    {
+      "epoch": 5.993690851735016,
+      "grad_norm": 2.08266019821167,
+      "learning_rate": 0.0002,
+      "loss": 0.4092,
+      "step": 950
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.7353854179382324,
+      "eval_runtime": 88.0772,
+      "eval_samples_per_second": 5.177,
+      "eval_steps_per_second": 0.647,
+      "step": 951
+    },
+    {
+      "epoch": 6.056782334384858,
+      "grad_norm": 1.4494482278823853,
+      "learning_rate": 0.0002,
+      "loss": 0.3164,
+      "step": 960
+    },
+    {
+      "epoch": 6.1198738170347005,
+      "grad_norm": 1.1357238292694092,
+      "learning_rate": 0.0002,
+      "loss": 0.2941,
+      "step": 970
+    },
+    {
+      "epoch": 6.182965299684542,
+      "grad_norm": 2.562446117401123,
+      "learning_rate": 0.0002,
+      "loss": 0.2961,
+      "step": 980
+    },
+    {
+      "epoch": 6.246056782334385,
+      "grad_norm": 1.1694231033325195,
+      "learning_rate": 0.0002,
+      "loss": 0.2832,
+      "step": 990
+    },
+    {
+      "epoch": 6.309148264984227,
+      "grad_norm": 2.090214490890503,
+      "learning_rate": 0.0002,
+      "loss": 0.3119,
+      "step": 1000
+    },
+    {
+      "epoch": 6.37223974763407,
+      "grad_norm": 1.6510322093963623,
+      "learning_rate": 0.0002,
+      "loss": 0.3246,
+      "step": 1010
+    },
+    {
+      "epoch": 6.435331230283912,
+      "grad_norm": 1.3792554140090942,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 1020
+    },
+    {
+      "epoch": 6.498422712933754,
+      "grad_norm": 1.7142070531845093,
+      "learning_rate": 0.0002,
+      "loss": 0.3241,
+      "step": 1030
+    },
+    {
+      "epoch": 6.561514195583596,
+      "grad_norm": 1.5898644924163818,
+      "learning_rate": 0.0002,
+      "loss": 0.2856,
+      "step": 1040
+    },
+    {
+      "epoch": 6.624605678233438,
+      "grad_norm": 1.8487178087234497,
+      "learning_rate": 0.0002,
+      "loss": 0.2943,
+      "step": 1050
+    },
+    {
+      "epoch": 6.687697160883281,
+      "grad_norm": 2.2733006477355957,
+      "learning_rate": 0.0002,
+      "loss": 0.3221,
+      "step": 1060
+    },
+    {
+      "epoch": 6.750788643533123,
+      "grad_norm": 1.9059652090072632,
+      "learning_rate": 0.0002,
+      "loss": 0.3026,
+      "step": 1070
+    },
+    {
+      "epoch": 6.813880126182966,
+      "grad_norm": 1.752540946006775,
+      "learning_rate": 0.0002,
+      "loss": 0.3084,
+      "step": 1080
+    },
+    {
+      "epoch": 6.8769716088328074,
+      "grad_norm": 1.3432809114456177,
+      "learning_rate": 0.0002,
+      "loss": 0.3233,
+      "step": 1090
+    },
+    {
+      "epoch": 6.94006309148265,
+      "grad_norm": 1.4755741357803345,
+      "learning_rate": 0.0002,
+      "loss": 0.299,
+      "step": 1100
+    },
+    {
+      "epoch": 6.996845425867508,
+      "eval_loss": 1.8801419734954834,
+      "eval_runtime": 87.6769,
+      "eval_samples_per_second": 5.201,
+      "eval_steps_per_second": 0.65,
+      "step": 1109
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1264,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.86790092816384e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1109/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be3fafc872bebe92f48880d252dfc7e7f3a3d91907139be4d6ac2759fb235bc1
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e339c0b6b5186c915703cdb41ab18ec3a3a34c395926b9d7ae96420d774baaa0
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cee47ad1aecc6c49989f004154b4daa1aff4e48677a0712bdfba8021c5b371b7
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f94895b0d506b9e2abc7836ae3cb24e568bcc2632f6f33de8f10df25838653af
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e7f2713bfd941b9381ca9832768e2e0e23140b201c9a07d4a92cd80bb28283a
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/trainer_state.json

@@ -0,0 +1,979 @@
+{
+  "best_metric": 1.2515619993209839,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158",
+  "epoch": 7.974763406940063,
+  "eval_steps": 10,
+  "global_step": 1264,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06309148264984227,
+      "grad_norm": 1.1088604927062988,
+      "learning_rate": 0.0002,
+      "loss": 1.893,
+      "step": 10
+    },
+    {
+      "epoch": 0.12618296529968454,
+      "grad_norm": 0.8254690170288086,
+      "learning_rate": 0.0002,
+      "loss": 1.5359,
+      "step": 20
+    },
+    {
+      "epoch": 0.1892744479495268,
+      "grad_norm": 0.7787631154060364,
+      "learning_rate": 0.0002,
+      "loss": 1.5667,
+      "step": 30
+    },
+    {
+      "epoch": 0.25236593059936907,
+      "grad_norm": 1.0321671962738037,
+      "learning_rate": 0.0002,
+      "loss": 1.4567,
+      "step": 40
+    },
+    {
+      "epoch": 0.31545741324921134,
+      "grad_norm": 1.6003938913345337,
+      "learning_rate": 0.0002,
+      "loss": 1.4166,
+      "step": 50
+    },
+    {
+      "epoch": 0.3785488958990536,
+      "grad_norm": 0.6283053159713745,
+      "learning_rate": 0.0002,
+      "loss": 1.3302,
+      "step": 60
+    },
+    {
+      "epoch": 0.4416403785488959,
+      "grad_norm": 4.210716724395752,
+      "learning_rate": 0.0002,
+      "loss": 1.3358,
+      "step": 70
+    },
+    {
+      "epoch": 0.5047318611987381,
+      "grad_norm": 0.6119468212127686,
+      "learning_rate": 0.0002,
+      "loss": 1.2831,
+      "step": 80
+    },
+    {
+      "epoch": 0.5678233438485805,
+      "grad_norm": 0.6245906949043274,
+      "learning_rate": 0.0002,
+      "loss": 1.2199,
+      "step": 90
+    },
+    {
+      "epoch": 0.6309148264984227,
+      "grad_norm": 0.7737864255905151,
+      "learning_rate": 0.0002,
+      "loss": 1.3255,
+      "step": 100
+    },
+    {
+      "epoch": 0.694006309148265,
+      "grad_norm": 0.7152376770973206,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 110
+    },
+    {
+      "epoch": 0.7570977917981072,
+      "grad_norm": 0.7520427107810974,
+      "learning_rate": 0.0002,
+      "loss": 1.1685,
+      "step": 120
+    },
+    {
+      "epoch": 0.8201892744479495,
+      "grad_norm": 0.9310459494590759,
+      "learning_rate": 0.0002,
+      "loss": 1.2923,
+      "step": 130
+    },
+    {
+      "epoch": 0.8832807570977917,
+      "grad_norm": 0.6360874772071838,
+      "learning_rate": 0.0002,
+      "loss": 1.3183,
+      "step": 140
+    },
+    {
+      "epoch": 0.9463722397476341,
+      "grad_norm": 0.5835223197937012,
+      "learning_rate": 0.0002,
+      "loss": 1.2824,
+      "step": 150
+    },
+    {
+      "epoch": 0.9968454258675079,
+      "eval_loss": 1.2515619993209839,
+      "eval_runtime": 87.7902,
+      "eval_samples_per_second": 5.194,
+      "eval_steps_per_second": 0.649,
+      "step": 158
+    },
+    {
+      "epoch": 1.0094637223974763,
+      "grad_norm": 0.459634393453598,
+      "learning_rate": 0.0002,
+      "loss": 1.225,
+      "step": 160
+    },
+    {
+      "epoch": 1.0725552050473186,
+      "grad_norm": 0.7972345948219299,
+      "learning_rate": 0.0002,
+      "loss": 1.0948,
+      "step": 170
+    },
+    {
+      "epoch": 1.135646687697161,
+      "grad_norm": 0.7474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.1488,
+      "step": 180
+    },
+    {
+      "epoch": 1.1987381703470033,
+      "grad_norm": 0.8033431172370911,
+      "learning_rate": 0.0002,
+      "loss": 1.0838,
+      "step": 190
+    },
+    {
+      "epoch": 1.2618296529968454,
+      "grad_norm": 0.7721152305603027,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 200
+    },
+    {
+      "epoch": 1.3249211356466877,
+      "grad_norm": 0.6452336311340332,
+      "learning_rate": 0.0002,
+      "loss": 1.178,
+      "step": 210
+    },
+    {
+      "epoch": 1.38801261829653,
+      "grad_norm": 0.6738545298576355,
+      "learning_rate": 0.0002,
+      "loss": 1.0702,
+      "step": 220
+    },
+    {
+      "epoch": 1.4511041009463723,
+      "grad_norm": 0.7977299094200134,
+      "learning_rate": 0.0002,
+      "loss": 1.1874,
+      "step": 230
+    },
+    {
+      "epoch": 1.5141955835962144,
+      "grad_norm": 0.688931941986084,
+      "learning_rate": 0.0002,
+      "loss": 1.0575,
+      "step": 240
+    },
+    {
+      "epoch": 1.5772870662460567,
+      "grad_norm": 0.7142922878265381,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 250
+    },
+    {
+      "epoch": 1.640378548895899,
+      "grad_norm": 0.974367082118988,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 260
+    },
+    {
+      "epoch": 1.7034700315457414,
+      "grad_norm": 0.8500529527664185,
+      "learning_rate": 0.0002,
+      "loss": 1.0401,
+      "step": 270
+    },
+    {
+      "epoch": 1.7665615141955837,
+      "grad_norm": 0.6906878352165222,
+      "learning_rate": 0.0002,
+      "loss": 1.1212,
+      "step": 280
+    },
+    {
+      "epoch": 1.8296529968454258,
+      "grad_norm": 0.9898052215576172,
+      "learning_rate": 0.0002,
+      "loss": 1.1583,
+      "step": 290
+    },
+    {
+      "epoch": 1.8927444794952681,
+      "grad_norm": 0.6456423401832581,
+      "learning_rate": 0.0002,
+      "loss": 1.1509,
+      "step": 300
+    },
+    {
+      "epoch": 1.9558359621451105,
+      "grad_norm": 0.9416996240615845,
+      "learning_rate": 0.0002,
+      "loss": 1.0231,
+      "step": 310
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2548311948776245,
+      "eval_runtime": 88.0511,
+      "eval_samples_per_second": 5.179,
+      "eval_steps_per_second": 0.647,
+      "step": 317
+    },
+    {
+      "epoch": 2.0189274447949526,
+      "grad_norm": 0.8139097094535828,
+      "learning_rate": 0.0002,
+      "loss": 1.1737,
+      "step": 320
+    },
+    {
+      "epoch": 2.082018927444795,
+      "grad_norm": 0.8165537714958191,
+      "learning_rate": 0.0002,
+      "loss": 0.9225,
+      "step": 330
+    },
+    {
+      "epoch": 2.145110410094637,
+      "grad_norm": 1.0027955770492554,
+      "learning_rate": 0.0002,
+      "loss": 0.8145,
+      "step": 340
+    },
+    {
+      "epoch": 2.2082018927444795,
+      "grad_norm": 1.1729809045791626,
+      "learning_rate": 0.0002,
+      "loss": 0.8326,
+      "step": 350
+    },
+    {
+      "epoch": 2.271293375394322,
+      "grad_norm": 1.067833423614502,
+      "learning_rate": 0.0002,
+      "loss": 0.8332,
+      "step": 360
+    },
+    {
+      "epoch": 2.334384858044164,
+      "grad_norm": 0.9201022386550903,
+      "learning_rate": 0.0002,
+      "loss": 0.8749,
+      "step": 370
+    },
+    {
+      "epoch": 2.3974763406940065,
+      "grad_norm": 0.9174187779426575,
+      "learning_rate": 0.0002,
+      "loss": 0.8903,
+      "step": 380
+    },
+    {
+      "epoch": 2.4605678233438484,
+      "grad_norm": 1.0019031763076782,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 390
+    },
+    {
+      "epoch": 2.5236593059936907,
+      "grad_norm": 0.9429060220718384,
+      "learning_rate": 0.0002,
+      "loss": 0.947,
+      "step": 400
+    },
+    {
+      "epoch": 2.586750788643533,
+      "grad_norm": 1.307620644569397,
+      "learning_rate": 0.0002,
+      "loss": 0.8577,
+      "step": 410
+    },
+    {
+      "epoch": 2.6498422712933754,
+      "grad_norm": 1.395595669746399,
+      "learning_rate": 0.0002,
+      "loss": 0.8823,
+      "step": 420
+    },
+    {
+      "epoch": 2.7129337539432177,
+      "grad_norm": 1.1542165279388428,
+      "learning_rate": 0.0002,
+      "loss": 0.9286,
+      "step": 430
+    },
+    {
+      "epoch": 2.77602523659306,
+      "grad_norm": 0.9339272975921631,
+      "learning_rate": 0.0002,
+      "loss": 0.8581,
+      "step": 440
+    },
+    {
+      "epoch": 2.8391167192429023,
+      "grad_norm": 1.0789870023727417,
+      "learning_rate": 0.0002,
+      "loss": 0.8878,
+      "step": 450
+    },
+    {
+      "epoch": 2.9022082018927446,
+      "grad_norm": 0.9041107892990112,
+      "learning_rate": 0.0002,
+      "loss": 0.777,
+      "step": 460
+    },
+    {
+      "epoch": 2.965299684542587,
+      "grad_norm": 1.1063491106033325,
+      "learning_rate": 0.0002,
+      "loss": 0.9554,
+      "step": 470
+    },
+    {
+      "epoch": 2.996845425867508,
+      "eval_loss": 1.3252698183059692,
+      "eval_runtime": 87.6199,
+      "eval_samples_per_second": 5.204,
+      "eval_steps_per_second": 0.651,
+      "step": 475
+    },
+    {
+      "epoch": 3.028391167192429,
+      "grad_norm": 0.8952794671058655,
+      "learning_rate": 0.0002,
+      "loss": 0.916,
+      "step": 480
+    },
+    {
+      "epoch": 3.091482649842271,
+      "grad_norm": 1.1680232286453247,
+      "learning_rate": 0.0002,
+      "loss": 0.6808,
+      "step": 490
+    },
+    {
+      "epoch": 3.1545741324921135,
+      "grad_norm": 1.1490330696105957,
+      "learning_rate": 0.0002,
+      "loss": 0.6603,
+      "step": 500
+    },
+    {
+      "epoch": 3.217665615141956,
+      "grad_norm": 1.807936429977417,
+      "learning_rate": 0.0002,
+      "loss": 0.5892,
+      "step": 510
+    },
+    {
+      "epoch": 3.280757097791798,
+      "grad_norm": 1.4477553367614746,
+      "learning_rate": 0.0002,
+      "loss": 0.6701,
+      "step": 520
+    },
+    {
+      "epoch": 3.3438485804416405,
+      "grad_norm": 1.7911386489868164,
+      "learning_rate": 0.0002,
+      "loss": 0.6107,
+      "step": 530
+    },
+    {
+      "epoch": 3.406940063091483,
+      "grad_norm": 1.136080026626587,
+      "learning_rate": 0.0002,
+      "loss": 0.6779,
+      "step": 540
+    },
+    {
+      "epoch": 3.470031545741325,
+      "grad_norm": 1.1295137405395508,
+      "learning_rate": 0.0002,
+      "loss": 0.6493,
+      "step": 550
+    },
+    {
+      "epoch": 3.5331230283911674,
+      "grad_norm": 1.573101282119751,
+      "learning_rate": 0.0002,
+      "loss": 0.6652,
+      "step": 560
+    },
+    {
+      "epoch": 3.5962145110410093,
+      "grad_norm": 1.403876543045044,
+      "learning_rate": 0.0002,
+      "loss": 0.6772,
+      "step": 570
+    },
+    {
+      "epoch": 3.6593059936908516,
+      "grad_norm": 1.1751461029052734,
+      "learning_rate": 0.0002,
+      "loss": 0.6655,
+      "step": 580
+    },
+    {
+      "epoch": 3.722397476340694,
+      "grad_norm": 1.6286189556121826,
+      "learning_rate": 0.0002,
+      "loss": 0.783,
+      "step": 590
+    },
+    {
+      "epoch": 3.7854889589905363,
+      "grad_norm": 1.2480772733688354,
+      "learning_rate": 0.0002,
+      "loss": 0.6854,
+      "step": 600
+    },
+    {
+      "epoch": 3.8485804416403786,
+      "grad_norm": 1.1290836334228516,
+      "learning_rate": 0.0002,
+      "loss": 0.647,
+      "step": 610
+    },
+    {
+      "epoch": 3.911671924290221,
+      "grad_norm": 1.1974420547485352,
+      "learning_rate": 0.0002,
+      "loss": 0.609,
+      "step": 620
+    },
+    {
+      "epoch": 3.9747634069400632,
+      "grad_norm": 1.318346619606018,
+      "learning_rate": 0.0002,
+      "loss": 0.6775,
+      "step": 630
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.4375206232070923,
+      "eval_runtime": 87.9488,
+      "eval_samples_per_second": 5.185,
+      "eval_steps_per_second": 0.648,
+      "step": 634
+    },
+    {
+      "epoch": 4.037854889589905,
+      "grad_norm": 1.7765438556671143,
+      "learning_rate": 0.0002,
+      "loss": 0.5632,
+      "step": 640
+    },
+    {
+      "epoch": 4.100946372239748,
+      "grad_norm": 1.1392035484313965,
+      "learning_rate": 0.0002,
+      "loss": 0.4745,
+      "step": 650
+    },
+    {
+      "epoch": 4.16403785488959,
+      "grad_norm": 2.375727653503418,
+      "learning_rate": 0.0002,
+      "loss": 0.5029,
+      "step": 660
+    },
+    {
+      "epoch": 4.2271293375394325,
+      "grad_norm": 1.537709355354309,
+      "learning_rate": 0.0002,
+      "loss": 0.4621,
+      "step": 670
+    },
+    {
+      "epoch": 4.290220820189274,
+      "grad_norm": 1.8388986587524414,
+      "learning_rate": 0.0002,
+      "loss": 0.4678,
+      "step": 680
+    },
+    {
+      "epoch": 4.353312302839116,
+      "grad_norm": 1.8100976943969727,
+      "learning_rate": 0.0002,
+      "loss": 0.4833,
+      "step": 690
+    },
+    {
+      "epoch": 4.416403785488959,
+      "grad_norm": 1.5121203660964966,
+      "learning_rate": 0.0002,
+      "loss": 0.451,
+      "step": 700
+    },
+    {
+      "epoch": 4.479495268138801,
+      "grad_norm": 2.152935743331909,
+      "learning_rate": 0.0002,
+      "loss": 0.5022,
+      "step": 710
+    },
+    {
+      "epoch": 4.542586750788644,
+      "grad_norm": 1.7240005731582642,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 720
+    },
+    {
+      "epoch": 4.605678233438486,
+      "grad_norm": 2.038640260696411,
+      "learning_rate": 0.0002,
+      "loss": 0.6016,
+      "step": 730
+    },
+    {
+      "epoch": 4.668769716088328,
+      "grad_norm": 2.1428792476654053,
+      "learning_rate": 0.0002,
+      "loss": 0.4528,
+      "step": 740
+    },
+    {
+      "epoch": 4.73186119873817,
+      "grad_norm": 1.5168582201004028,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 750
+    },
+    {
+      "epoch": 4.794952681388013,
+      "grad_norm": 1.2821815013885498,
+      "learning_rate": 0.0002,
+      "loss": 0.5586,
+      "step": 760
+    },
+    {
+      "epoch": 4.858044164037855,
+      "grad_norm": 1.3252662420272827,
+      "learning_rate": 0.0002,
+      "loss": 0.4759,
+      "step": 770
+    },
+    {
+      "epoch": 4.921135646687697,
+      "grad_norm": 1.7149165868759155,
+      "learning_rate": 0.0002,
+      "loss": 0.4706,
+      "step": 780
+    },
+    {
+      "epoch": 4.9842271293375395,
+      "grad_norm": 2.2286508083343506,
+      "learning_rate": 0.0002,
+      "loss": 0.529,
+      "step": 790
+    },
+    {
+      "epoch": 4.996845425867508,
+      "eval_loss": 1.614780306816101,
+      "eval_runtime": 87.4623,
+      "eval_samples_per_second": 5.214,
+      "eval_steps_per_second": 0.652,
+      "step": 792
+    },
+    {
+      "epoch": 5.047318611987381,
+      "grad_norm": 2.4778950214385986,
+      "learning_rate": 0.0002,
+      "loss": 0.4083,
+      "step": 800
+    },
+    {
+      "epoch": 5.110410094637224,
+      "grad_norm": 1.6371349096298218,
+      "learning_rate": 0.0002,
+      "loss": 0.4011,
+      "step": 810
+    },
+    {
+      "epoch": 5.173501577287066,
+      "grad_norm": 1.877687931060791,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 820
+    },
+    {
+      "epoch": 5.236593059936909,
+      "grad_norm": 1.5015276670455933,
+      "learning_rate": 0.0002,
+      "loss": 0.3601,
+      "step": 830
+    },
+    {
+      "epoch": 5.299684542586751,
+      "grad_norm": 2.1286396980285645,
+      "learning_rate": 0.0002,
+      "loss": 0.4043,
+      "step": 840
+    },
+    {
+      "epoch": 5.3627760252365935,
+      "grad_norm": 1.6563063859939575,
+      "learning_rate": 0.0002,
+      "loss": 0.3259,
+      "step": 850
+    },
+    {
+      "epoch": 5.425867507886435,
+      "grad_norm": 1.997558832168579,
+      "learning_rate": 0.0002,
+      "loss": 0.4065,
+      "step": 860
+    },
+    {
+      "epoch": 5.488958990536277,
+      "grad_norm": 1.8498985767364502,
+      "learning_rate": 0.0002,
+      "loss": 0.3405,
+      "step": 870
+    },
+    {
+      "epoch": 5.55205047318612,
+      "grad_norm": 1.7636688947677612,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 880
+    },
+    {
+      "epoch": 5.615141955835962,
+      "grad_norm": 1.9259185791015625,
+      "learning_rate": 0.0002,
+      "loss": 0.3898,
+      "step": 890
+    },
+    {
+      "epoch": 5.678233438485805,
+      "grad_norm": 1.917427659034729,
+      "learning_rate": 0.0002,
+      "loss": 0.3937,
+      "step": 900
+    },
+    {
+      "epoch": 5.7413249211356465,
+      "grad_norm": 1.5327430963516235,
+      "learning_rate": 0.0002,
+      "loss": 0.4001,
+      "step": 910
+    },
+    {
+      "epoch": 5.804416403785489,
+      "grad_norm": 1.8258512020111084,
+      "learning_rate": 0.0002,
+      "loss": 0.3925,
+      "step": 920
+    },
+    {
+      "epoch": 5.867507886435331,
+      "grad_norm": 1.909255862236023,
+      "learning_rate": 0.0002,
+      "loss": 0.4068,
+      "step": 930
+    },
+    {
+      "epoch": 5.930599369085174,
+      "grad_norm": 2.272733211517334,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 940
+    },
+    {
+      "epoch": 5.993690851735016,
+      "grad_norm": 2.08266019821167,
+      "learning_rate": 0.0002,
+      "loss": 0.4092,
+      "step": 950
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.7353854179382324,
+      "eval_runtime": 88.0772,
+      "eval_samples_per_second": 5.177,
+      "eval_steps_per_second": 0.647,
+      "step": 951
+    },
+    {
+      "epoch": 6.056782334384858,
+      "grad_norm": 1.4494482278823853,
+      "learning_rate": 0.0002,
+      "loss": 0.3164,
+      "step": 960
+    },
+    {
+      "epoch": 6.1198738170347005,
+      "grad_norm": 1.1357238292694092,
+      "learning_rate": 0.0002,
+      "loss": 0.2941,
+      "step": 970
+    },
+    {
+      "epoch": 6.182965299684542,
+      "grad_norm": 2.562446117401123,
+      "learning_rate": 0.0002,
+      "loss": 0.2961,
+      "step": 980
+    },
+    {
+      "epoch": 6.246056782334385,
+      "grad_norm": 1.1694231033325195,
+      "learning_rate": 0.0002,
+      "loss": 0.2832,
+      "step": 990
+    },
+    {
+      "epoch": 6.309148264984227,
+      "grad_norm": 2.090214490890503,
+      "learning_rate": 0.0002,
+      "loss": 0.3119,
+      "step": 1000
+    },
+    {
+      "epoch": 6.37223974763407,
+      "grad_norm": 1.6510322093963623,
+      "learning_rate": 0.0002,
+      "loss": 0.3246,
+      "step": 1010
+    },
+    {
+      "epoch": 6.435331230283912,
+      "grad_norm": 1.3792554140090942,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 1020
+    },
+    {
+      "epoch": 6.498422712933754,
+      "grad_norm": 1.7142070531845093,
+      "learning_rate": 0.0002,
+      "loss": 0.3241,
+      "step": 1030
+    },
+    {
+      "epoch": 6.561514195583596,
+      "grad_norm": 1.5898644924163818,
+      "learning_rate": 0.0002,
+      "loss": 0.2856,
+      "step": 1040
+    },
+    {
+      "epoch": 6.624605678233438,
+      "grad_norm": 1.8487178087234497,
+      "learning_rate": 0.0002,
+      "loss": 0.2943,
+      "step": 1050
+    },
+    {
+      "epoch": 6.687697160883281,
+      "grad_norm": 2.2733006477355957,
+      "learning_rate": 0.0002,
+      "loss": 0.3221,
+      "step": 1060
+    },
+    {
+      "epoch": 6.750788643533123,
+      "grad_norm": 1.9059652090072632,
+      "learning_rate": 0.0002,
+      "loss": 0.3026,
+      "step": 1070
+    },
+    {
+      "epoch": 6.813880126182966,
+      "grad_norm": 1.752540946006775,
+      "learning_rate": 0.0002,
+      "loss": 0.3084,
+      "step": 1080
+    },
+    {
+      "epoch": 6.8769716088328074,
+      "grad_norm": 1.3432809114456177,
+      "learning_rate": 0.0002,
+      "loss": 0.3233,
+      "step": 1090
+    },
+    {
+      "epoch": 6.94006309148265,
+      "grad_norm": 1.4755741357803345,
+      "learning_rate": 0.0002,
+      "loss": 0.299,
+      "step": 1100
+    },
+    {
+      "epoch": 6.996845425867508,
+      "eval_loss": 1.8801419734954834,
+      "eval_runtime": 87.6769,
+      "eval_samples_per_second": 5.201,
+      "eval_steps_per_second": 0.65,
+      "step": 1109
+    },
+    {
+      "epoch": 7.003154574132492,
+      "grad_norm": 1.3077971935272217,
+      "learning_rate": 0.0002,
+      "loss": 0.2955,
+      "step": 1110
+    },
+    {
+      "epoch": 7.066246056782334,
+      "grad_norm": 1.8616857528686523,
+      "learning_rate": 0.0002,
+      "loss": 0.233,
+      "step": 1120
+    },
+    {
+      "epoch": 7.129337539432177,
+      "grad_norm": 1.4871293306350708,
+      "learning_rate": 0.0002,
+      "loss": 0.2474,
+      "step": 1130
+    },
+    {
+      "epoch": 7.192429022082019,
+      "grad_norm": 1.7677148580551147,
+      "learning_rate": 0.0002,
+      "loss": 0.2329,
+      "step": 1140
+    },
+    {
+      "epoch": 7.255520504731861,
+      "grad_norm": 1.571831464767456,
+      "learning_rate": 0.0002,
+      "loss": 0.2459,
+      "step": 1150
+    },
+    {
+      "epoch": 7.318611987381703,
+      "grad_norm": 1.6635631322860718,
+      "learning_rate": 0.0002,
+      "loss": 0.2373,
+      "step": 1160
+    },
+    {
+      "epoch": 7.381703470031546,
+      "grad_norm": 1.708690881729126,
+      "learning_rate": 0.0002,
+      "loss": 0.2798,
+      "step": 1170
+    },
+    {
+      "epoch": 7.444794952681388,
+      "grad_norm": 1.5613728761672974,
+      "learning_rate": 0.0002,
+      "loss": 0.261,
+      "step": 1180
+    },
+    {
+      "epoch": 7.50788643533123,
+      "grad_norm": 1.3542531728744507,
+      "learning_rate": 0.0002,
+      "loss": 0.2496,
+      "step": 1190
+    },
+    {
+      "epoch": 7.570977917981073,
+      "grad_norm": 2.2147374153137207,
+      "learning_rate": 0.0002,
+      "loss": 0.2555,
+      "step": 1200
+    },
+    {
+      "epoch": 7.634069400630915,
+      "grad_norm": 1.9874738454818726,
+      "learning_rate": 0.0002,
+      "loss": 0.2774,
+      "step": 1210
+    },
+    {
+      "epoch": 7.697160883280757,
+      "grad_norm": 1.602935552597046,
+      "learning_rate": 0.0002,
+      "loss": 0.2734,
+      "step": 1220
+    },
+    {
+      "epoch": 7.760252365930599,
+      "grad_norm": 1.8155338764190674,
+      "learning_rate": 0.0002,
+      "loss": 0.2727,
+      "step": 1230
+    },
+    {
+      "epoch": 7.823343848580442,
+      "grad_norm": 1.6560618877410889,
+      "learning_rate": 0.0002,
+      "loss": 0.299,
+      "step": 1240
+    },
+    {
+      "epoch": 7.886435331230284,
+      "grad_norm": 1.4073737859725952,
+      "learning_rate": 0.0002,
+      "loss": 0.2593,
+      "step": 1250
+    },
+    {
+      "epoch": 7.9495268138801265,
+      "grad_norm": 1.5730996131896973,
+      "learning_rate": 0.0002,
+      "loss": 0.2686,
+      "step": 1260
+    },
+    {
+      "epoch": 7.974763406940063,
+      "eval_loss": 2.0034003257751465,
+      "eval_runtime": 87.8461,
+      "eval_samples_per_second": 5.191,
+      "eval_steps_per_second": 0.649,
+      "step": 1264
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1264,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.54576545579008e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-1264/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be3fafc872bebe92f48880d252dfc7e7f3a3d91907139be4d6ac2759fb235bc1
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0aeb9a3fa5ca7dae97311f327fe4d9be03d768797557681128a077ed0938f911
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14bf1be350ce2a1e019e51a69f671c4a8052cbfca87bb24b55a0fb3f811fa5ff
+size 55532538

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23bdb5efa780391860f39bd341b0a55edf7a5ecf8173a4a1eaf0fabd3bd80ce2
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ed380cc50068be48108841431f81f2dc587b97f74440a0f7d6675f771472707
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/trainer_state.json

@@ -0,0 +1,146 @@
+{
+  "best_metric": 1.2515619993209839,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158",
+  "epoch": 0.9968454258675079,
+  "eval_steps": 10,
+  "global_step": 158,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06309148264984227,
+      "grad_norm": 1.1088604927062988,
+      "learning_rate": 0.0002,
+      "loss": 1.893,
+      "step": 10
+    },
+    {
+      "epoch": 0.12618296529968454,
+      "grad_norm": 0.8254690170288086,
+      "learning_rate": 0.0002,
+      "loss": 1.5359,
+      "step": 20
+    },
+    {
+      "epoch": 0.1892744479495268,
+      "grad_norm": 0.7787631154060364,
+      "learning_rate": 0.0002,
+      "loss": 1.5667,
+      "step": 30
+    },
+    {
+      "epoch": 0.25236593059936907,
+      "grad_norm": 1.0321671962738037,
+      "learning_rate": 0.0002,
+      "loss": 1.4567,
+      "step": 40
+    },
+    {
+      "epoch": 0.31545741324921134,
+      "grad_norm": 1.6003938913345337,
+      "learning_rate": 0.0002,
+      "loss": 1.4166,
+      "step": 50
+    },
+    {
+      "epoch": 0.3785488958990536,
+      "grad_norm": 0.6283053159713745,
+      "learning_rate": 0.0002,
+      "loss": 1.3302,
+      "step": 60
+    },
+    {
+      "epoch": 0.4416403785488959,
+      "grad_norm": 4.210716724395752,
+      "learning_rate": 0.0002,
+      "loss": 1.3358,
+      "step": 70
+    },
+    {
+      "epoch": 0.5047318611987381,
+      "grad_norm": 0.6119468212127686,
+      "learning_rate": 0.0002,
+      "loss": 1.2831,
+      "step": 80
+    },
+    {
+      "epoch": 0.5678233438485805,
+      "grad_norm": 0.6245906949043274,
+      "learning_rate": 0.0002,
+      "loss": 1.2199,
+      "step": 90
+    },
+    {
+      "epoch": 0.6309148264984227,
+      "grad_norm": 0.7737864255905151,
+      "learning_rate": 0.0002,
+      "loss": 1.3255,
+      "step": 100
+    },
+    {
+      "epoch": 0.694006309148265,
+      "grad_norm": 0.7152376770973206,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 110
+    },
+    {
+      "epoch": 0.7570977917981072,
+      "grad_norm": 0.7520427107810974,
+      "learning_rate": 0.0002,
+      "loss": 1.1685,
+      "step": 120
+    },
+    {
+      "epoch": 0.8201892744479495,
+      "grad_norm": 0.9310459494590759,
+      "learning_rate": 0.0002,
+      "loss": 1.2923,
+      "step": 130
+    },
+    {
+      "epoch": 0.8832807570977917,
+      "grad_norm": 0.6360874772071838,
+      "learning_rate": 0.0002,
+      "loss": 1.3183,
+      "step": 140
+    },
+    {
+      "epoch": 0.9463722397476341,
+      "grad_norm": 0.5835223197937012,
+      "learning_rate": 0.0002,
+      "loss": 1.2824,
+      "step": 150
+    },
+    {
+      "epoch": 0.9968454258675079,
+      "eval_loss": 1.2515619993209839,
+      "eval_runtime": 87.7902,
+      "eval_samples_per_second": 5.194,
+      "eval_steps_per_second": 0.649,
+      "step": 158
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1264,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6954144183091200.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be3fafc872bebe92f48880d252dfc7e7f3a3d91907139be4d6ac2759fb235bc1
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e75516a3afe174a0d738db2d9628c1f7e31dc5fbc8c4056f1cc064c776e15843
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a9d3eaf9b710d4db763d31c003f77a9c66429b9e2a3cb3b7045b0f67e6adfa3
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:469d816a64a0af027350734d876a80fec8a40d3eb2071a706b536bb97589af8b
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04e570ecabf78702403371994b70d75ccf834e6fb5ef288cf1b00aaabddf2632
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-317/trainer_state.json

@@ -0,0 +1,266 @@
+{
+  "best_metric": 1.2515619993209839,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-1/checkpoint-158",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 317,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06309148264984227,
+      "grad_norm": 1.1088604927062988,
+      "learning_rate": 0.0002,
+      "loss": 1.893,
+      "step": 10
+    },
+    {
+      "epoch": 0.12618296529968454,
+      "grad_norm": 0.8254690170288086,
+      "learning_rate": 0.0002,
+      "loss": 1.5359,
+      "step": 20
+    },
+    {
+      "epoch": 0.1892744479495268,
+      "grad_norm": 0.7787631154060364,
+      "learning_rate": 0.0002,
+      "loss": 1.5667,
+      "step": 30
+    },
+    {
+      "epoch": 0.25236593059936907,
+      "grad_norm": 1.0321671962738037,
+      "learning_rate": 0.0002,
+      "loss": 1.4567,
+      "step": 40
+    },
+    {
+      "epoch": 0.31545741324921134,
+      "grad_norm": 1.6003938913345337,
+      "learning_rate": 0.0002,
+      "loss": 1.4166,
+      "step": 50
+    },
+    {
+      "epoch": 0.3785488958990536,
+      "grad_norm": 0.6283053159713745,
+      "learning_rate": 0.0002,
+      "loss": 1.3302,
+      "step": 60
+    },
+    {
+      "epoch": 0.4416403785488959,
+      "grad_norm": 4.210716724395752,
+      "learning_rate": 0.0002,
+      "loss": 1.3358,
+      "step": 70
+    },
+    {
+      "epoch": 0.5047318611987381,
+      "grad_norm": 0.6119468212127686,
+      "learning_rate": 0.0002,
+      "loss": 1.2831,
+      "step": 80
+    },
+    {
+      "epoch": 0.5678233438485805,
+      "grad_norm": 0.6245906949043274,
+      "learning_rate": 0.0002,
+      "loss": 1.2199,
+      "step": 90
+    },
+    {
+      "epoch": 0.6309148264984227,
+      "grad_norm": 0.7737864255905151,
+      "learning_rate": 0.0002,
+      "loss": 1.3255,
+      "step": 100
+    },
+    {
+      "epoch": 0.694006309148265,
+      "grad_norm": 0.7152376770973206,
+      "learning_rate": 0.0002,
+      "loss": 1.1686,
+      "step": 110
+    },
+    {
+      "epoch": 0.7570977917981072,
+      "grad_norm": 0.7520427107810974,
+      "learning_rate": 0.0002,
+      "loss": 1.1685,
+      "step": 120
+    },
+    {
+      "epoch": 0.8201892744479495,
+      "grad_norm": 0.9310459494590759,
+      "learning_rate": 0.0002,
+      "loss": 1.2923,
+      "step": 130
+    },
+    {
+      "epoch": 0.8832807570977917,
+      "grad_norm": 0.6360874772071838,
+      "learning_rate": 0.0002,
+      "loss": 1.3183,
+      "step": 140
+    },
+    {
+      "epoch": 0.9463722397476341,
+      "grad_norm": 0.5835223197937012,
+      "learning_rate": 0.0002,
+      "loss": 1.2824,
+      "step": 150
+    },
+    {
+      "epoch": 0.9968454258675079,
+      "eval_loss": 1.2515619993209839,
+      "eval_runtime": 87.7902,
+      "eval_samples_per_second": 5.194,
+      "eval_steps_per_second": 0.649,
+      "step": 158
+    },
+    {
+      "epoch": 1.0094637223974763,
+      "grad_norm": 0.459634393453598,
+      "learning_rate": 0.0002,
+      "loss": 1.225,
+      "step": 160
+    },
+    {
+      "epoch": 1.0725552050473186,
+      "grad_norm": 0.7972345948219299,
+      "learning_rate": 0.0002,
+      "loss": 1.0948,
+      "step": 170
+    },
+    {
+      "epoch": 1.135646687697161,
+      "grad_norm": 0.7474365234375,
+      "learning_rate": 0.0002,
+      "loss": 1.1488,
+      "step": 180
+    },
+    {
+      "epoch": 1.1987381703470033,
+      "grad_norm": 0.8033431172370911,
+      "learning_rate": 0.0002,
+      "loss": 1.0838,
+      "step": 190
+    },
+    {
+      "epoch": 1.2618296529968454,
+      "grad_norm": 0.7721152305603027,
+      "learning_rate": 0.0002,
+      "loss": 0.9528,
+      "step": 200
+    },
+    {
+      "epoch": 1.3249211356466877,
+      "grad_norm": 0.6452336311340332,
+      "learning_rate": 0.0002,
+      "loss": 1.178,
+      "step": 210
+    },
+    {
+      "epoch": 1.38801261829653,
+      "grad_norm": 0.6738545298576355,
+      "learning_rate": 0.0002,
+      "loss": 1.0702,
+      "step": 220
+    },
+    {
+      "epoch": 1.4511041009463723,
+      "grad_norm": 0.7977299094200134,
+      "learning_rate": 0.0002,
+      "loss": 1.1874,
+      "step": 230
+    },
+    {
+      "epoch": 1.5141955835962144,
+      "grad_norm": 0.688931941986084,
+      "learning_rate": 0.0002,
+      "loss": 1.0575,
+      "step": 240
+    },
+    {
+      "epoch": 1.5772870662460567,
+      "grad_norm": 0.7142922878265381,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 250
+    },
+    {
+      "epoch": 1.640378548895899,
+      "grad_norm": 0.974367082118988,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 260
+    },
+    {
+      "epoch": 1.7034700315457414,
+      "grad_norm": 0.8500529527664185,
+      "learning_rate": 0.0002,
+      "loss": 1.0401,
+      "step": 270
+    },
+    {
+      "epoch": 1.7665615141955837,
+      "grad_norm": 0.6906878352165222,
+      "learning_rate": 0.0002,
+      "loss": 1.1212,
+      "step": 280
+    },
+    {
+      "epoch": 1.8296529968454258,
+      "grad_norm": 0.9898052215576172,
+      "learning_rate": 0.0002,
+      "loss": 1.1583,
+      "step": 290
+    },
+    {
+      "epoch": 1.8927444794952681,
+      "grad_norm": 0.6456423401832581,
+      "learning_rate": 0.0002,
+      "loss": 1.1509,
+      "step": 300
+    },
+    {
+      "epoch": 1.9558359621451105,
+      "grad_norm": 0.9416996240615845,
+      "learning_rate": 0.0002,
+      "loss": 1.0231,
+      "step": 310
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2548311948776245,
+      "eval_runtime": 88.0511,
+      "eval_samples_per_second": 5.179,
+      "eval_steps_per_second": 0.647,
+      "step": 317
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1264,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.39082883661824e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}