Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4db5b8118d82a12f1ff25d17d5c69c3c8968ff120e815eed815af41f95c45960
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:752ee34867e75c734d91d3bf2593039d988ab9c7e7ecddf39988c5dd844dc4ef
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31f062048bf344d715d0da67cc673d5710dba98eeba0d88b535e149edb8cb932
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deb22d1e6ad9783d077d19c329ff80f57d21d98ec306f72866f9e46c43c7580f
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:391219a5b23350462f8d4a396de26f4e075a65b376831a0e39d5bf13095cfa0a
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/trainer_state.json

@@ -0,0 +1,873 @@
+{
+  "best_metric": 1.2526909112930298,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161",
+  "epoch": 7.0,
+  "eval_steps": 10,
+  "global_step": 1127,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 1.3165894746780396,
+      "learning_rate": 0.0002,
+      "loss": 1.8735,
+      "step": 10
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 1.0744154453277588,
+      "learning_rate": 0.0002,
+      "loss": 1.6236,
+      "step": 20
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.7020019292831421,
+      "learning_rate": 0.0002,
+      "loss": 1.5484,
+      "step": 30
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 1.083817958831787,
+      "learning_rate": 0.0002,
+      "loss": 1.5461,
+      "step": 40
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 0.8009080290794373,
+      "learning_rate": 0.0002,
+      "loss": 1.3413,
+      "step": 50
+    },
+    {
+      "epoch": 0.37267080745341613,
+      "grad_norm": 0.8239762783050537,
+      "learning_rate": 0.0002,
+      "loss": 1.2591,
+      "step": 60
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "grad_norm": 0.8437100052833557,
+      "learning_rate": 0.0002,
+      "loss": 1.2237,
+      "step": 70
+    },
+    {
+      "epoch": 0.4968944099378882,
+      "grad_norm": 1.0204336643218994,
+      "learning_rate": 0.0002,
+      "loss": 1.2777,
+      "step": 80
+    },
+    {
+      "epoch": 0.5590062111801242,
+      "grad_norm": 1.185208797454834,
+      "learning_rate": 0.0002,
+      "loss": 1.2178,
+      "step": 90
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 0.8615248203277588,
+      "learning_rate": 0.0002,
+      "loss": 1.3011,
+      "step": 100
+    },
+    {
+      "epoch": 0.6832298136645962,
+      "grad_norm": 0.5289493799209595,
+      "learning_rate": 0.0002,
+      "loss": 1.276,
+      "step": 110
+    },
+    {
+      "epoch": 0.7453416149068323,
+      "grad_norm": 0.6098535656929016,
+      "learning_rate": 0.0002,
+      "loss": 1.2099,
+      "step": 120
+    },
+    {
+      "epoch": 0.8074534161490683,
+      "grad_norm": 0.601532518863678,
+      "learning_rate": 0.0002,
+      "loss": 1.1765,
+      "step": 130
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.6565020680427551,
+      "learning_rate": 0.0002,
+      "loss": 1.299,
+      "step": 140
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 0.6863213181495667,
+      "learning_rate": 0.0002,
+      "loss": 1.1232,
+      "step": 150
+    },
+    {
+      "epoch": 0.9937888198757764,
+      "grad_norm": 0.6823898553848267,
+      "learning_rate": 0.0002,
+      "loss": 1.2542,
+      "step": 160
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.2526909112930298,
+      "eval_runtime": 89.0661,
+      "eval_samples_per_second": 5.12,
+      "eval_steps_per_second": 0.64,
+      "step": 161
+    },
+    {
+      "epoch": 1.0559006211180124,
+      "grad_norm": 0.562255859375,
+      "learning_rate": 0.0002,
+      "loss": 1.1205,
+      "step": 170
+    },
+    {
+      "epoch": 1.1180124223602483,
+      "grad_norm": 0.5982539653778076,
+      "learning_rate": 0.0002,
+      "loss": 1.1351,
+      "step": 180
+    },
+    {
+      "epoch": 1.1801242236024845,
+      "grad_norm": 0.6252831220626831,
+      "learning_rate": 0.0002,
+      "loss": 1.0561,
+      "step": 190
+    },
+    {
+      "epoch": 1.2422360248447206,
+      "grad_norm": 0.6803395748138428,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 200
+    },
+    {
+      "epoch": 1.3043478260869565,
+      "grad_norm": 0.8347863554954529,
+      "learning_rate": 0.0002,
+      "loss": 1.1508,
+      "step": 210
+    },
+    {
+      "epoch": 1.3664596273291925,
+      "grad_norm": 0.7230828404426575,
+      "learning_rate": 0.0002,
+      "loss": 1.0958,
+      "step": 220
+    },
+    {
+      "epoch": 1.4285714285714286,
+      "grad_norm": 0.6892024874687195,
+      "learning_rate": 0.0002,
+      "loss": 1.0728,
+      "step": 230
+    },
+    {
+      "epoch": 1.4906832298136645,
+      "grad_norm": 0.9822754263877869,
+      "learning_rate": 0.0002,
+      "loss": 1.08,
+      "step": 240
+    },
+    {
+      "epoch": 1.5527950310559007,
+      "grad_norm": 0.8614016175270081,
+      "learning_rate": 0.0002,
+      "loss": 1.0916,
+      "step": 250
+    },
+    {
+      "epoch": 1.6149068322981366,
+      "grad_norm": 0.6894949078559875,
+      "learning_rate": 0.0002,
+      "loss": 1.0036,
+      "step": 260
+    },
+    {
+      "epoch": 1.6770186335403725,
+      "grad_norm": 0.721196711063385,
+      "learning_rate": 0.0002,
+      "loss": 1.1229,
+      "step": 270
+    },
+    {
+      "epoch": 1.7391304347826086,
+      "grad_norm": 0.7291169166564941,
+      "learning_rate": 0.0002,
+      "loss": 1.0329,
+      "step": 280
+    },
+    {
+      "epoch": 1.8012422360248448,
+      "grad_norm": 0.6831535696983337,
+      "learning_rate": 0.0002,
+      "loss": 1.0795,
+      "step": 290
+    },
+    {
+      "epoch": 1.8633540372670807,
+      "grad_norm": 0.6903841495513916,
+      "learning_rate": 0.0002,
+      "loss": 1.1072,
+      "step": 300
+    },
+    {
+      "epoch": 1.9254658385093166,
+      "grad_norm": 0.8273940682411194,
+      "learning_rate": 0.0002,
+      "loss": 1.1371,
+      "step": 310
+    },
+    {
+      "epoch": 1.9875776397515528,
+      "grad_norm": 1.1644595861434937,
+      "learning_rate": 0.0002,
+      "loss": 1.0195,
+      "step": 320
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2606149911880493,
+      "eval_runtime": 89.1274,
+      "eval_samples_per_second": 5.116,
+      "eval_steps_per_second": 0.64,
+      "step": 322
+    },
+    {
+      "epoch": 2.049689440993789,
+      "grad_norm": 0.7743585705757141,
+      "learning_rate": 0.0002,
+      "loss": 0.922,
+      "step": 330
+    },
+    {
+      "epoch": 2.111801242236025,
+      "grad_norm": 0.7924496531486511,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 340
+    },
+    {
+      "epoch": 2.1739130434782608,
+      "grad_norm": 0.8791683912277222,
+      "learning_rate": 0.0002,
+      "loss": 0.8697,
+      "step": 350
+    },
+    {
+      "epoch": 2.2360248447204967,
+      "grad_norm": 0.9607393145561218,
+      "learning_rate": 0.0002,
+      "loss": 0.8225,
+      "step": 360
+    },
+    {
+      "epoch": 2.298136645962733,
+      "grad_norm": 0.7969710826873779,
+      "learning_rate": 0.0002,
+      "loss": 0.8323,
+      "step": 370
+    },
+    {
+      "epoch": 2.360248447204969,
+      "grad_norm": 0.9643616080284119,
+      "learning_rate": 0.0002,
+      "loss": 0.9034,
+      "step": 380
+    },
+    {
+      "epoch": 2.422360248447205,
+      "grad_norm": 0.9008212685585022,
+      "learning_rate": 0.0002,
+      "loss": 0.8451,
+      "step": 390
+    },
+    {
+      "epoch": 2.4844720496894412,
+      "grad_norm": 0.9994485974311829,
+      "learning_rate": 0.0002,
+      "loss": 0.8682,
+      "step": 400
+    },
+    {
+      "epoch": 2.546583850931677,
+      "grad_norm": 1.1590228080749512,
+      "learning_rate": 0.0002,
+      "loss": 0.8517,
+      "step": 410
+    },
+    {
+      "epoch": 2.608695652173913,
+      "grad_norm": 0.9434095621109009,
+      "learning_rate": 0.0002,
+      "loss": 0.8938,
+      "step": 420
+    },
+    {
+      "epoch": 2.670807453416149,
+      "grad_norm": 1.2967880964279175,
+      "learning_rate": 0.0002,
+      "loss": 0.9034,
+      "step": 430
+    },
+    {
+      "epoch": 2.732919254658385,
+      "grad_norm": 1.1672614812850952,
+      "learning_rate": 0.0002,
+      "loss": 0.7798,
+      "step": 440
+    },
+    {
+      "epoch": 2.795031055900621,
+      "grad_norm": 0.8670998215675354,
+      "learning_rate": 0.0002,
+      "loss": 0.8379,
+      "step": 450
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.9719066023826599,
+      "learning_rate": 0.0002,
+      "loss": 0.8821,
+      "step": 460
+    },
+    {
+      "epoch": 2.919254658385093,
+      "grad_norm": 1.1839795112609863,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 470
+    },
+    {
+      "epoch": 2.981366459627329,
+      "grad_norm": 1.2288304567337036,
+      "learning_rate": 0.0002,
+      "loss": 0.9273,
+      "step": 480
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.318354606628418,
+      "eval_runtime": 88.4693,
+      "eval_samples_per_second": 5.154,
+      "eval_steps_per_second": 0.644,
+      "step": 483
+    },
+    {
+      "epoch": 3.0434782608695654,
+      "grad_norm": 1.5638580322265625,
+      "learning_rate": 0.0002,
+      "loss": 0.7584,
+      "step": 490
+    },
+    {
+      "epoch": 3.1055900621118013,
+      "grad_norm": 1.0773272514343262,
+      "learning_rate": 0.0002,
+      "loss": 0.5986,
+      "step": 500
+    },
+    {
+      "epoch": 3.1677018633540373,
+      "grad_norm": 1.4005111455917358,
+      "learning_rate": 0.0002,
+      "loss": 0.6344,
+      "step": 510
+    },
+    {
+      "epoch": 3.229813664596273,
+      "grad_norm": 1.6811457872390747,
+      "learning_rate": 0.0002,
+      "loss": 0.6512,
+      "step": 520
+    },
+    {
+      "epoch": 3.291925465838509,
+      "grad_norm": 1.3982810974121094,
+      "learning_rate": 0.0002,
+      "loss": 0.6334,
+      "step": 530
+    },
+    {
+      "epoch": 3.3540372670807455,
+      "grad_norm": 1.2195545434951782,
+      "learning_rate": 0.0002,
+      "loss": 0.6347,
+      "step": 540
+    },
+    {
+      "epoch": 3.4161490683229814,
+      "grad_norm": 1.2203441858291626,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 550
+    },
+    {
+      "epoch": 3.4782608695652173,
+      "grad_norm": 1.9244214296340942,
+      "learning_rate": 0.0002,
+      "loss": 0.6634,
+      "step": 560
+    },
+    {
+      "epoch": 3.5403726708074537,
+      "grad_norm": 1.1874186992645264,
+      "learning_rate": 0.0002,
+      "loss": 0.6053,
+      "step": 570
+    },
+    {
+      "epoch": 3.6024844720496896,
+      "grad_norm": 1.512715220451355,
+      "learning_rate": 0.0002,
+      "loss": 0.6323,
+      "step": 580
+    },
+    {
+      "epoch": 3.6645962732919255,
+      "grad_norm": 1.4360668659210205,
+      "learning_rate": 0.0002,
+      "loss": 0.6808,
+      "step": 590
+    },
+    {
+      "epoch": 3.7267080745341614,
+      "grad_norm": 1.582342505455017,
+      "learning_rate": 0.0002,
+      "loss": 0.6205,
+      "step": 600
+    },
+    {
+      "epoch": 3.7888198757763973,
+      "grad_norm": 1.3575900793075562,
+      "learning_rate": 0.0002,
+      "loss": 0.6958,
+      "step": 610
+    },
+    {
+      "epoch": 3.8509316770186337,
+      "grad_norm": 1.6533644199371338,
+      "learning_rate": 0.0002,
+      "loss": 0.6818,
+      "step": 620
+    },
+    {
+      "epoch": 3.9130434782608696,
+      "grad_norm": 1.4815455675125122,
+      "learning_rate": 0.0002,
+      "loss": 0.6959,
+      "step": 630
+    },
+    {
+      "epoch": 3.9751552795031055,
+      "grad_norm": 1.2100576162338257,
+      "learning_rate": 0.0002,
+      "loss": 0.6434,
+      "step": 640
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.4600155353546143,
+      "eval_runtime": 87.3063,
+      "eval_samples_per_second": 5.223,
+      "eval_steps_per_second": 0.653,
+      "step": 644
+    },
+    {
+      "epoch": 4.037267080745342,
+      "grad_norm": 2.0771918296813965,
+      "learning_rate": 0.0002,
+      "loss": 0.5366,
+      "step": 650
+    },
+    {
+      "epoch": 4.099378881987578,
+      "grad_norm": 1.5498052835464478,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 660
+    },
+    {
+      "epoch": 4.161490683229814,
+      "grad_norm": 1.6029658317565918,
+      "learning_rate": 0.0002,
+      "loss": 0.4249,
+      "step": 670
+    },
+    {
+      "epoch": 4.22360248447205,
+      "grad_norm": 1.5245393514633179,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 680
+    },
+    {
+      "epoch": 4.285714285714286,
+      "grad_norm": 1.4152382612228394,
+      "learning_rate": 0.0002,
+      "loss": 0.5141,
+      "step": 690
+    },
+    {
+      "epoch": 4.3478260869565215,
+      "grad_norm": 2.2080681324005127,
+      "learning_rate": 0.0002,
+      "loss": 0.4887,
+      "step": 700
+    },
+    {
+      "epoch": 4.409937888198757,
+      "grad_norm": 2.016655206680298,
+      "learning_rate": 0.0002,
+      "loss": 0.521,
+      "step": 710
+    },
+    {
+      "epoch": 4.472049689440993,
+      "grad_norm": 1.6059858798980713,
+      "learning_rate": 0.0002,
+      "loss": 0.46,
+      "step": 720
+    },
+    {
+      "epoch": 4.53416149068323,
+      "grad_norm": 1.213496446609497,
+      "learning_rate": 0.0002,
+      "loss": 0.4988,
+      "step": 730
+    },
+    {
+      "epoch": 4.596273291925466,
+      "grad_norm": 1.539108157157898,
+      "learning_rate": 0.0002,
+      "loss": 0.4715,
+      "step": 740
+    },
+    {
+      "epoch": 4.658385093167702,
+      "grad_norm": 1.9071044921875,
+      "learning_rate": 0.0002,
+      "loss": 0.4718,
+      "step": 750
+    },
+    {
+      "epoch": 4.720496894409938,
+      "grad_norm": 1.1161843538284302,
+      "learning_rate": 0.0002,
+      "loss": 0.5077,
+      "step": 760
+    },
+    {
+      "epoch": 4.782608695652174,
+      "grad_norm": 1.770416259765625,
+      "learning_rate": 0.0002,
+      "loss": 0.5029,
+      "step": 770
+    },
+    {
+      "epoch": 4.84472049689441,
+      "grad_norm": 1.2480278015136719,
+      "learning_rate": 0.0002,
+      "loss": 0.4838,
+      "step": 780
+    },
+    {
+      "epoch": 4.906832298136646,
+      "grad_norm": 1.6392626762390137,
+      "learning_rate": 0.0002,
+      "loss": 0.4648,
+      "step": 790
+    },
+    {
+      "epoch": 4.9689440993788825,
+      "grad_norm": 1.656785011291504,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 800
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.6247237920761108,
+      "eval_runtime": 87.6213,
+      "eval_samples_per_second": 5.204,
+      "eval_steps_per_second": 0.651,
+      "step": 805
+    },
+    {
+      "epoch": 5.031055900621118,
+      "grad_norm": 1.9282710552215576,
+      "learning_rate": 0.0002,
+      "loss": 0.4279,
+      "step": 810
+    },
+    {
+      "epoch": 5.093167701863354,
+      "grad_norm": 1.2856742143630981,
+      "learning_rate": 0.0002,
+      "loss": 0.3677,
+      "step": 820
+    },
+    {
+      "epoch": 5.15527950310559,
+      "grad_norm": 2.040598154067993,
+      "learning_rate": 0.0002,
+      "loss": 0.3275,
+      "step": 830
+    },
+    {
+      "epoch": 5.217391304347826,
+      "grad_norm": 1.6899571418762207,
+      "learning_rate": 0.0002,
+      "loss": 0.3469,
+      "step": 840
+    },
+    {
+      "epoch": 5.279503105590062,
+      "grad_norm": 1.4350981712341309,
+      "learning_rate": 0.0002,
+      "loss": 0.3743,
+      "step": 850
+    },
+    {
+      "epoch": 5.341614906832298,
+      "grad_norm": 1.6330562829971313,
+      "learning_rate": 0.0002,
+      "loss": 0.3618,
+      "step": 860
+    },
+    {
+      "epoch": 5.403726708074534,
+      "grad_norm": 1.4449222087860107,
+      "learning_rate": 0.0002,
+      "loss": 0.3826,
+      "step": 870
+    },
+    {
+      "epoch": 5.46583850931677,
+      "grad_norm": 1.4217298030853271,
+      "learning_rate": 0.0002,
+      "loss": 0.3656,
+      "step": 880
+    },
+    {
+      "epoch": 5.527950310559007,
+      "grad_norm": 1.6559629440307617,
+      "learning_rate": 0.0002,
+      "loss": 0.4036,
+      "step": 890
+    },
+    {
+      "epoch": 5.590062111801243,
+      "grad_norm": 1.730210304260254,
+      "learning_rate": 0.0002,
+      "loss": 0.3657,
+      "step": 900
+    },
+    {
+      "epoch": 5.6521739130434785,
+      "grad_norm": 1.564587950706482,
+      "learning_rate": 0.0002,
+      "loss": 0.3702,
+      "step": 910
+    },
+    {
+      "epoch": 5.714285714285714,
+      "grad_norm": 1.7802600860595703,
+      "learning_rate": 0.0002,
+      "loss": 0.3648,
+      "step": 920
+    },
+    {
+      "epoch": 5.77639751552795,
+      "grad_norm": 1.9120898246765137,
+      "learning_rate": 0.0002,
+      "loss": 0.3823,
+      "step": 930
+    },
+    {
+      "epoch": 5.838509316770186,
+      "grad_norm": 1.728097915649414,
+      "learning_rate": 0.0002,
+      "loss": 0.3667,
+      "step": 940
+    },
+    {
+      "epoch": 5.900621118012422,
+      "grad_norm": 1.4571962356567383,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 950
+    },
+    {
+      "epoch": 5.962732919254658,
+      "grad_norm": 1.5490957498550415,
+      "learning_rate": 0.0002,
+      "loss": 0.3989,
+      "step": 960
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.7719664573669434,
+      "eval_runtime": 90.075,
+      "eval_samples_per_second": 5.062,
+      "eval_steps_per_second": 0.633,
+      "step": 966
+    },
+    {
+      "epoch": 6.024844720496894,
+      "grad_norm": 1.3889669179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.3192,
+      "step": 970
+    },
+    {
+      "epoch": 6.086956521739131,
+      "grad_norm": 1.6877079010009766,
+      "learning_rate": 0.0002,
+      "loss": 0.2789,
+      "step": 980
+    },
+    {
+      "epoch": 6.149068322981367,
+      "grad_norm": 2.135519504547119,
+      "learning_rate": 0.0002,
+      "loss": 0.277,
+      "step": 990
+    },
+    {
+      "epoch": 6.211180124223603,
+      "grad_norm": 1.2083741426467896,
+      "learning_rate": 0.0002,
+      "loss": 0.2542,
+      "step": 1000
+    },
+    {
+      "epoch": 6.273291925465839,
+      "grad_norm": 1.7203831672668457,
+      "learning_rate": 0.0002,
+      "loss": 0.2877,
+      "step": 1010
+    },
+    {
+      "epoch": 6.3354037267080745,
+      "grad_norm": 1.4014286994934082,
+      "learning_rate": 0.0002,
+      "loss": 0.2505,
+      "step": 1020
+    },
+    {
+      "epoch": 6.39751552795031,
+      "grad_norm": 1.610185146331787,
+      "learning_rate": 0.0002,
+      "loss": 0.314,
+      "step": 1030
+    },
+    {
+      "epoch": 6.459627329192546,
+      "grad_norm": 1.7615786790847778,
+      "learning_rate": 0.0002,
+      "loss": 0.2762,
+      "step": 1040
+    },
+    {
+      "epoch": 6.521739130434782,
+      "grad_norm": 2.016982316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.3159,
+      "step": 1050
+    },
+    {
+      "epoch": 6.583850931677018,
+      "grad_norm": 1.8239957094192505,
+      "learning_rate": 0.0002,
+      "loss": 0.2894,
+      "step": 1060
+    },
+    {
+      "epoch": 6.645962732919255,
+      "grad_norm": 2.8103716373443604,
+      "learning_rate": 0.0002,
+      "loss": 0.3157,
+      "step": 1070
+    },
+    {
+      "epoch": 6.708074534161491,
+      "grad_norm": 1.9517383575439453,
+      "learning_rate": 0.0002,
+      "loss": 0.3324,
+      "step": 1080
+    },
+    {
+      "epoch": 6.770186335403727,
+      "grad_norm": 1.6103801727294922,
+      "learning_rate": 0.0002,
+      "loss": 0.317,
+      "step": 1090
+    },
+    {
+      "epoch": 6.832298136645963,
+      "grad_norm": 1.7236528396606445,
+      "learning_rate": 0.0002,
+      "loss": 0.2958,
+      "step": 1100
+    },
+    {
+      "epoch": 6.894409937888199,
+      "grad_norm": 1.9494216442108154,
+      "learning_rate": 0.0002,
+      "loss": 0.3255,
+      "step": 1110
+    },
+    {
+      "epoch": 6.956521739130435,
+      "grad_norm": 1.5921375751495361,
+      "learning_rate": 0.0002,
+      "loss": 0.3119,
+      "step": 1120
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.9163893461227417,
+      "eval_runtime": 88.6853,
+      "eval_samples_per_second": 5.142,
+      "eval_steps_per_second": 0.643,
+      "step": 1127
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1288,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.94468169990144e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1127/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34980fd5c2286610ef04f61d0ede08e4a5a9fecb8f190eb017f21325dd87b66b
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08587ced4de6c4d16c57643de8e223d5903df6c8e222f783144ae41e762c04f5
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1249a622526f6ebbe1e70df9dc69906b047a8f4a377d5033317efacc591e6324
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:378b6b808c281e187203cce592d6aaa8aafef05d93552648dc2ac14f08688d00
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a17393a475a6079ffa2869cfabd4d1a282c203c4adf92772c840de5a59a54b6b
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/trainer_state.json

@@ -0,0 +1,993 @@
+{
+  "best_metric": 1.2526909112930298,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161",
+  "epoch": 8.0,
+  "eval_steps": 10,
+  "global_step": 1288,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 1.3165894746780396,
+      "learning_rate": 0.0002,
+      "loss": 1.8735,
+      "step": 10
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 1.0744154453277588,
+      "learning_rate": 0.0002,
+      "loss": 1.6236,
+      "step": 20
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.7020019292831421,
+      "learning_rate": 0.0002,
+      "loss": 1.5484,
+      "step": 30
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 1.083817958831787,
+      "learning_rate": 0.0002,
+      "loss": 1.5461,
+      "step": 40
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 0.8009080290794373,
+      "learning_rate": 0.0002,
+      "loss": 1.3413,
+      "step": 50
+    },
+    {
+      "epoch": 0.37267080745341613,
+      "grad_norm": 0.8239762783050537,
+      "learning_rate": 0.0002,
+      "loss": 1.2591,
+      "step": 60
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "grad_norm": 0.8437100052833557,
+      "learning_rate": 0.0002,
+      "loss": 1.2237,
+      "step": 70
+    },
+    {
+      "epoch": 0.4968944099378882,
+      "grad_norm": 1.0204336643218994,
+      "learning_rate": 0.0002,
+      "loss": 1.2777,
+      "step": 80
+    },
+    {
+      "epoch": 0.5590062111801242,
+      "grad_norm": 1.185208797454834,
+      "learning_rate": 0.0002,
+      "loss": 1.2178,
+      "step": 90
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 0.8615248203277588,
+      "learning_rate": 0.0002,
+      "loss": 1.3011,
+      "step": 100
+    },
+    {
+      "epoch": 0.6832298136645962,
+      "grad_norm": 0.5289493799209595,
+      "learning_rate": 0.0002,
+      "loss": 1.276,
+      "step": 110
+    },
+    {
+      "epoch": 0.7453416149068323,
+      "grad_norm": 0.6098535656929016,
+      "learning_rate": 0.0002,
+      "loss": 1.2099,
+      "step": 120
+    },
+    {
+      "epoch": 0.8074534161490683,
+      "grad_norm": 0.601532518863678,
+      "learning_rate": 0.0002,
+      "loss": 1.1765,
+      "step": 130
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.6565020680427551,
+      "learning_rate": 0.0002,
+      "loss": 1.299,
+      "step": 140
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 0.6863213181495667,
+      "learning_rate": 0.0002,
+      "loss": 1.1232,
+      "step": 150
+    },
+    {
+      "epoch": 0.9937888198757764,
+      "grad_norm": 0.6823898553848267,
+      "learning_rate": 0.0002,
+      "loss": 1.2542,
+      "step": 160
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.2526909112930298,
+      "eval_runtime": 89.0661,
+      "eval_samples_per_second": 5.12,
+      "eval_steps_per_second": 0.64,
+      "step": 161
+    },
+    {
+      "epoch": 1.0559006211180124,
+      "grad_norm": 0.562255859375,
+      "learning_rate": 0.0002,
+      "loss": 1.1205,
+      "step": 170
+    },
+    {
+      "epoch": 1.1180124223602483,
+      "grad_norm": 0.5982539653778076,
+      "learning_rate": 0.0002,
+      "loss": 1.1351,
+      "step": 180
+    },
+    {
+      "epoch": 1.1801242236024845,
+      "grad_norm": 0.6252831220626831,
+      "learning_rate": 0.0002,
+      "loss": 1.0561,
+      "step": 190
+    },
+    {
+      "epoch": 1.2422360248447206,
+      "grad_norm": 0.6803395748138428,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 200
+    },
+    {
+      "epoch": 1.3043478260869565,
+      "grad_norm": 0.8347863554954529,
+      "learning_rate": 0.0002,
+      "loss": 1.1508,
+      "step": 210
+    },
+    {
+      "epoch": 1.3664596273291925,
+      "grad_norm": 0.7230828404426575,
+      "learning_rate": 0.0002,
+      "loss": 1.0958,
+      "step": 220
+    },
+    {
+      "epoch": 1.4285714285714286,
+      "grad_norm": 0.6892024874687195,
+      "learning_rate": 0.0002,
+      "loss": 1.0728,
+      "step": 230
+    },
+    {
+      "epoch": 1.4906832298136645,
+      "grad_norm": 0.9822754263877869,
+      "learning_rate": 0.0002,
+      "loss": 1.08,
+      "step": 240
+    },
+    {
+      "epoch": 1.5527950310559007,
+      "grad_norm": 0.8614016175270081,
+      "learning_rate": 0.0002,
+      "loss": 1.0916,
+      "step": 250
+    },
+    {
+      "epoch": 1.6149068322981366,
+      "grad_norm": 0.6894949078559875,
+      "learning_rate": 0.0002,
+      "loss": 1.0036,
+      "step": 260
+    },
+    {
+      "epoch": 1.6770186335403725,
+      "grad_norm": 0.721196711063385,
+      "learning_rate": 0.0002,
+      "loss": 1.1229,
+      "step": 270
+    },
+    {
+      "epoch": 1.7391304347826086,
+      "grad_norm": 0.7291169166564941,
+      "learning_rate": 0.0002,
+      "loss": 1.0329,
+      "step": 280
+    },
+    {
+      "epoch": 1.8012422360248448,
+      "grad_norm": 0.6831535696983337,
+      "learning_rate": 0.0002,
+      "loss": 1.0795,
+      "step": 290
+    },
+    {
+      "epoch": 1.8633540372670807,
+      "grad_norm": 0.6903841495513916,
+      "learning_rate": 0.0002,
+      "loss": 1.1072,
+      "step": 300
+    },
+    {
+      "epoch": 1.9254658385093166,
+      "grad_norm": 0.8273940682411194,
+      "learning_rate": 0.0002,
+      "loss": 1.1371,
+      "step": 310
+    },
+    {
+      "epoch": 1.9875776397515528,
+      "grad_norm": 1.1644595861434937,
+      "learning_rate": 0.0002,
+      "loss": 1.0195,
+      "step": 320
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2606149911880493,
+      "eval_runtime": 89.1274,
+      "eval_samples_per_second": 5.116,
+      "eval_steps_per_second": 0.64,
+      "step": 322
+    },
+    {
+      "epoch": 2.049689440993789,
+      "grad_norm": 0.7743585705757141,
+      "learning_rate": 0.0002,
+      "loss": 0.922,
+      "step": 330
+    },
+    {
+      "epoch": 2.111801242236025,
+      "grad_norm": 0.7924496531486511,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 340
+    },
+    {
+      "epoch": 2.1739130434782608,
+      "grad_norm": 0.8791683912277222,
+      "learning_rate": 0.0002,
+      "loss": 0.8697,
+      "step": 350
+    },
+    {
+      "epoch": 2.2360248447204967,
+      "grad_norm": 0.9607393145561218,
+      "learning_rate": 0.0002,
+      "loss": 0.8225,
+      "step": 360
+    },
+    {
+      "epoch": 2.298136645962733,
+      "grad_norm": 0.7969710826873779,
+      "learning_rate": 0.0002,
+      "loss": 0.8323,
+      "step": 370
+    },
+    {
+      "epoch": 2.360248447204969,
+      "grad_norm": 0.9643616080284119,
+      "learning_rate": 0.0002,
+      "loss": 0.9034,
+      "step": 380
+    },
+    {
+      "epoch": 2.422360248447205,
+      "grad_norm": 0.9008212685585022,
+      "learning_rate": 0.0002,
+      "loss": 0.8451,
+      "step": 390
+    },
+    {
+      "epoch": 2.4844720496894412,
+      "grad_norm": 0.9994485974311829,
+      "learning_rate": 0.0002,
+      "loss": 0.8682,
+      "step": 400
+    },
+    {
+      "epoch": 2.546583850931677,
+      "grad_norm": 1.1590228080749512,
+      "learning_rate": 0.0002,
+      "loss": 0.8517,
+      "step": 410
+    },
+    {
+      "epoch": 2.608695652173913,
+      "grad_norm": 0.9434095621109009,
+      "learning_rate": 0.0002,
+      "loss": 0.8938,
+      "step": 420
+    },
+    {
+      "epoch": 2.670807453416149,
+      "grad_norm": 1.2967880964279175,
+      "learning_rate": 0.0002,
+      "loss": 0.9034,
+      "step": 430
+    },
+    {
+      "epoch": 2.732919254658385,
+      "grad_norm": 1.1672614812850952,
+      "learning_rate": 0.0002,
+      "loss": 0.7798,
+      "step": 440
+    },
+    {
+      "epoch": 2.795031055900621,
+      "grad_norm": 0.8670998215675354,
+      "learning_rate": 0.0002,
+      "loss": 0.8379,
+      "step": 450
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.9719066023826599,
+      "learning_rate": 0.0002,
+      "loss": 0.8821,
+      "step": 460
+    },
+    {
+      "epoch": 2.919254658385093,
+      "grad_norm": 1.1839795112609863,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 470
+    },
+    {
+      "epoch": 2.981366459627329,
+      "grad_norm": 1.2288304567337036,
+      "learning_rate": 0.0002,
+      "loss": 0.9273,
+      "step": 480
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.318354606628418,
+      "eval_runtime": 88.4693,
+      "eval_samples_per_second": 5.154,
+      "eval_steps_per_second": 0.644,
+      "step": 483
+    },
+    {
+      "epoch": 3.0434782608695654,
+      "grad_norm": 1.5638580322265625,
+      "learning_rate": 0.0002,
+      "loss": 0.7584,
+      "step": 490
+    },
+    {
+      "epoch": 3.1055900621118013,
+      "grad_norm": 1.0773272514343262,
+      "learning_rate": 0.0002,
+      "loss": 0.5986,
+      "step": 500
+    },
+    {
+      "epoch": 3.1677018633540373,
+      "grad_norm": 1.4005111455917358,
+      "learning_rate": 0.0002,
+      "loss": 0.6344,
+      "step": 510
+    },
+    {
+      "epoch": 3.229813664596273,
+      "grad_norm": 1.6811457872390747,
+      "learning_rate": 0.0002,
+      "loss": 0.6512,
+      "step": 520
+    },
+    {
+      "epoch": 3.291925465838509,
+      "grad_norm": 1.3982810974121094,
+      "learning_rate": 0.0002,
+      "loss": 0.6334,
+      "step": 530
+    },
+    {
+      "epoch": 3.3540372670807455,
+      "grad_norm": 1.2195545434951782,
+      "learning_rate": 0.0002,
+      "loss": 0.6347,
+      "step": 540
+    },
+    {
+      "epoch": 3.4161490683229814,
+      "grad_norm": 1.2203441858291626,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 550
+    },
+    {
+      "epoch": 3.4782608695652173,
+      "grad_norm": 1.9244214296340942,
+      "learning_rate": 0.0002,
+      "loss": 0.6634,
+      "step": 560
+    },
+    {
+      "epoch": 3.5403726708074537,
+      "grad_norm": 1.1874186992645264,
+      "learning_rate": 0.0002,
+      "loss": 0.6053,
+      "step": 570
+    },
+    {
+      "epoch": 3.6024844720496896,
+      "grad_norm": 1.512715220451355,
+      "learning_rate": 0.0002,
+      "loss": 0.6323,
+      "step": 580
+    },
+    {
+      "epoch": 3.6645962732919255,
+      "grad_norm": 1.4360668659210205,
+      "learning_rate": 0.0002,
+      "loss": 0.6808,
+      "step": 590
+    },
+    {
+      "epoch": 3.7267080745341614,
+      "grad_norm": 1.582342505455017,
+      "learning_rate": 0.0002,
+      "loss": 0.6205,
+      "step": 600
+    },
+    {
+      "epoch": 3.7888198757763973,
+      "grad_norm": 1.3575900793075562,
+      "learning_rate": 0.0002,
+      "loss": 0.6958,
+      "step": 610
+    },
+    {
+      "epoch": 3.8509316770186337,
+      "grad_norm": 1.6533644199371338,
+      "learning_rate": 0.0002,
+      "loss": 0.6818,
+      "step": 620
+    },
+    {
+      "epoch": 3.9130434782608696,
+      "grad_norm": 1.4815455675125122,
+      "learning_rate": 0.0002,
+      "loss": 0.6959,
+      "step": 630
+    },
+    {
+      "epoch": 3.9751552795031055,
+      "grad_norm": 1.2100576162338257,
+      "learning_rate": 0.0002,
+      "loss": 0.6434,
+      "step": 640
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.4600155353546143,
+      "eval_runtime": 87.3063,
+      "eval_samples_per_second": 5.223,
+      "eval_steps_per_second": 0.653,
+      "step": 644
+    },
+    {
+      "epoch": 4.037267080745342,
+      "grad_norm": 2.0771918296813965,
+      "learning_rate": 0.0002,
+      "loss": 0.5366,
+      "step": 650
+    },
+    {
+      "epoch": 4.099378881987578,
+      "grad_norm": 1.5498052835464478,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 660
+    },
+    {
+      "epoch": 4.161490683229814,
+      "grad_norm": 1.6029658317565918,
+      "learning_rate": 0.0002,
+      "loss": 0.4249,
+      "step": 670
+    },
+    {
+      "epoch": 4.22360248447205,
+      "grad_norm": 1.5245393514633179,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 680
+    },
+    {
+      "epoch": 4.285714285714286,
+      "grad_norm": 1.4152382612228394,
+      "learning_rate": 0.0002,
+      "loss": 0.5141,
+      "step": 690
+    },
+    {
+      "epoch": 4.3478260869565215,
+      "grad_norm": 2.2080681324005127,
+      "learning_rate": 0.0002,
+      "loss": 0.4887,
+      "step": 700
+    },
+    {
+      "epoch": 4.409937888198757,
+      "grad_norm": 2.016655206680298,
+      "learning_rate": 0.0002,
+      "loss": 0.521,
+      "step": 710
+    },
+    {
+      "epoch": 4.472049689440993,
+      "grad_norm": 1.6059858798980713,
+      "learning_rate": 0.0002,
+      "loss": 0.46,
+      "step": 720
+    },
+    {
+      "epoch": 4.53416149068323,
+      "grad_norm": 1.213496446609497,
+      "learning_rate": 0.0002,
+      "loss": 0.4988,
+      "step": 730
+    },
+    {
+      "epoch": 4.596273291925466,
+      "grad_norm": 1.539108157157898,
+      "learning_rate": 0.0002,
+      "loss": 0.4715,
+      "step": 740
+    },
+    {
+      "epoch": 4.658385093167702,
+      "grad_norm": 1.9071044921875,
+      "learning_rate": 0.0002,
+      "loss": 0.4718,
+      "step": 750
+    },
+    {
+      "epoch": 4.720496894409938,
+      "grad_norm": 1.1161843538284302,
+      "learning_rate": 0.0002,
+      "loss": 0.5077,
+      "step": 760
+    },
+    {
+      "epoch": 4.782608695652174,
+      "grad_norm": 1.770416259765625,
+      "learning_rate": 0.0002,
+      "loss": 0.5029,
+      "step": 770
+    },
+    {
+      "epoch": 4.84472049689441,
+      "grad_norm": 1.2480278015136719,
+      "learning_rate": 0.0002,
+      "loss": 0.4838,
+      "step": 780
+    },
+    {
+      "epoch": 4.906832298136646,
+      "grad_norm": 1.6392626762390137,
+      "learning_rate": 0.0002,
+      "loss": 0.4648,
+      "step": 790
+    },
+    {
+      "epoch": 4.9689440993788825,
+      "grad_norm": 1.656785011291504,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 800
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.6247237920761108,
+      "eval_runtime": 87.6213,
+      "eval_samples_per_second": 5.204,
+      "eval_steps_per_second": 0.651,
+      "step": 805
+    },
+    {
+      "epoch": 5.031055900621118,
+      "grad_norm": 1.9282710552215576,
+      "learning_rate": 0.0002,
+      "loss": 0.4279,
+      "step": 810
+    },
+    {
+      "epoch": 5.093167701863354,
+      "grad_norm": 1.2856742143630981,
+      "learning_rate": 0.0002,
+      "loss": 0.3677,
+      "step": 820
+    },
+    {
+      "epoch": 5.15527950310559,
+      "grad_norm": 2.040598154067993,
+      "learning_rate": 0.0002,
+      "loss": 0.3275,
+      "step": 830
+    },
+    {
+      "epoch": 5.217391304347826,
+      "grad_norm": 1.6899571418762207,
+      "learning_rate": 0.0002,
+      "loss": 0.3469,
+      "step": 840
+    },
+    {
+      "epoch": 5.279503105590062,
+      "grad_norm": 1.4350981712341309,
+      "learning_rate": 0.0002,
+      "loss": 0.3743,
+      "step": 850
+    },
+    {
+      "epoch": 5.341614906832298,
+      "grad_norm": 1.6330562829971313,
+      "learning_rate": 0.0002,
+      "loss": 0.3618,
+      "step": 860
+    },
+    {
+      "epoch": 5.403726708074534,
+      "grad_norm": 1.4449222087860107,
+      "learning_rate": 0.0002,
+      "loss": 0.3826,
+      "step": 870
+    },
+    {
+      "epoch": 5.46583850931677,
+      "grad_norm": 1.4217298030853271,
+      "learning_rate": 0.0002,
+      "loss": 0.3656,
+      "step": 880
+    },
+    {
+      "epoch": 5.527950310559007,
+      "grad_norm": 1.6559629440307617,
+      "learning_rate": 0.0002,
+      "loss": 0.4036,
+      "step": 890
+    },
+    {
+      "epoch": 5.590062111801243,
+      "grad_norm": 1.730210304260254,
+      "learning_rate": 0.0002,
+      "loss": 0.3657,
+      "step": 900
+    },
+    {
+      "epoch": 5.6521739130434785,
+      "grad_norm": 1.564587950706482,
+      "learning_rate": 0.0002,
+      "loss": 0.3702,
+      "step": 910
+    },
+    {
+      "epoch": 5.714285714285714,
+      "grad_norm": 1.7802600860595703,
+      "learning_rate": 0.0002,
+      "loss": 0.3648,
+      "step": 920
+    },
+    {
+      "epoch": 5.77639751552795,
+      "grad_norm": 1.9120898246765137,
+      "learning_rate": 0.0002,
+      "loss": 0.3823,
+      "step": 930
+    },
+    {
+      "epoch": 5.838509316770186,
+      "grad_norm": 1.728097915649414,
+      "learning_rate": 0.0002,
+      "loss": 0.3667,
+      "step": 940
+    },
+    {
+      "epoch": 5.900621118012422,
+      "grad_norm": 1.4571962356567383,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 950
+    },
+    {
+      "epoch": 5.962732919254658,
+      "grad_norm": 1.5490957498550415,
+      "learning_rate": 0.0002,
+      "loss": 0.3989,
+      "step": 960
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.7719664573669434,
+      "eval_runtime": 90.075,
+      "eval_samples_per_second": 5.062,
+      "eval_steps_per_second": 0.633,
+      "step": 966
+    },
+    {
+      "epoch": 6.024844720496894,
+      "grad_norm": 1.3889669179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.3192,
+      "step": 970
+    },
+    {
+      "epoch": 6.086956521739131,
+      "grad_norm": 1.6877079010009766,
+      "learning_rate": 0.0002,
+      "loss": 0.2789,
+      "step": 980
+    },
+    {
+      "epoch": 6.149068322981367,
+      "grad_norm": 2.135519504547119,
+      "learning_rate": 0.0002,
+      "loss": 0.277,
+      "step": 990
+    },
+    {
+      "epoch": 6.211180124223603,
+      "grad_norm": 1.2083741426467896,
+      "learning_rate": 0.0002,
+      "loss": 0.2542,
+      "step": 1000
+    },
+    {
+      "epoch": 6.273291925465839,
+      "grad_norm": 1.7203831672668457,
+      "learning_rate": 0.0002,
+      "loss": 0.2877,
+      "step": 1010
+    },
+    {
+      "epoch": 6.3354037267080745,
+      "grad_norm": 1.4014286994934082,
+      "learning_rate": 0.0002,
+      "loss": 0.2505,
+      "step": 1020
+    },
+    {
+      "epoch": 6.39751552795031,
+      "grad_norm": 1.610185146331787,
+      "learning_rate": 0.0002,
+      "loss": 0.314,
+      "step": 1030
+    },
+    {
+      "epoch": 6.459627329192546,
+      "grad_norm": 1.7615786790847778,
+      "learning_rate": 0.0002,
+      "loss": 0.2762,
+      "step": 1040
+    },
+    {
+      "epoch": 6.521739130434782,
+      "grad_norm": 2.016982316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.3159,
+      "step": 1050
+    },
+    {
+      "epoch": 6.583850931677018,
+      "grad_norm": 1.8239957094192505,
+      "learning_rate": 0.0002,
+      "loss": 0.2894,
+      "step": 1060
+    },
+    {
+      "epoch": 6.645962732919255,
+      "grad_norm": 2.8103716373443604,
+      "learning_rate": 0.0002,
+      "loss": 0.3157,
+      "step": 1070
+    },
+    {
+      "epoch": 6.708074534161491,
+      "grad_norm": 1.9517383575439453,
+      "learning_rate": 0.0002,
+      "loss": 0.3324,
+      "step": 1080
+    },
+    {
+      "epoch": 6.770186335403727,
+      "grad_norm": 1.6103801727294922,
+      "learning_rate": 0.0002,
+      "loss": 0.317,
+      "step": 1090
+    },
+    {
+      "epoch": 6.832298136645963,
+      "grad_norm": 1.7236528396606445,
+      "learning_rate": 0.0002,
+      "loss": 0.2958,
+      "step": 1100
+    },
+    {
+      "epoch": 6.894409937888199,
+      "grad_norm": 1.9494216442108154,
+      "learning_rate": 0.0002,
+      "loss": 0.3255,
+      "step": 1110
+    },
+    {
+      "epoch": 6.956521739130435,
+      "grad_norm": 1.5921375751495361,
+      "learning_rate": 0.0002,
+      "loss": 0.3119,
+      "step": 1120
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.9163893461227417,
+      "eval_runtime": 88.6853,
+      "eval_samples_per_second": 5.142,
+      "eval_steps_per_second": 0.643,
+      "step": 1127
+    },
+    {
+      "epoch": 7.0186335403726705,
+      "grad_norm": 1.4439314603805542,
+      "learning_rate": 0.0002,
+      "loss": 0.2772,
+      "step": 1130
+    },
+    {
+      "epoch": 7.080745341614906,
+      "grad_norm": 1.4942126274108887,
+      "learning_rate": 0.0002,
+      "loss": 0.2197,
+      "step": 1140
+    },
+    {
+      "epoch": 7.142857142857143,
+      "grad_norm": 1.4251244068145752,
+      "learning_rate": 0.0002,
+      "loss": 0.2208,
+      "step": 1150
+    },
+    {
+      "epoch": 7.204968944099379,
+      "grad_norm": 1.6433234214782715,
+      "learning_rate": 0.0002,
+      "loss": 0.2279,
+      "step": 1160
+    },
+    {
+      "epoch": 7.267080745341615,
+      "grad_norm": 0.9248053431510925,
+      "learning_rate": 0.0002,
+      "loss": 0.2238,
+      "step": 1170
+    },
+    {
+      "epoch": 7.329192546583851,
+      "grad_norm": 1.567237138748169,
+      "learning_rate": 0.0002,
+      "loss": 0.2623,
+      "step": 1180
+    },
+    {
+      "epoch": 7.391304347826087,
+      "grad_norm": 1.5535482168197632,
+      "learning_rate": 0.0002,
+      "loss": 0.2463,
+      "step": 1190
+    },
+    {
+      "epoch": 7.453416149068323,
+      "grad_norm": 1.8713144063949585,
+      "learning_rate": 0.0002,
+      "loss": 0.2474,
+      "step": 1200
+    },
+    {
+      "epoch": 7.515527950310559,
+      "grad_norm": 1.5040708780288696,
+      "learning_rate": 0.0002,
+      "loss": 0.2387,
+      "step": 1210
+    },
+    {
+      "epoch": 7.577639751552795,
+      "grad_norm": 1.012447714805603,
+      "learning_rate": 0.0002,
+      "loss": 0.2542,
+      "step": 1220
+    },
+    {
+      "epoch": 7.6397515527950315,
+      "grad_norm": 2.193490743637085,
+      "learning_rate": 0.0002,
+      "loss": 0.2427,
+      "step": 1230
+    },
+    {
+      "epoch": 7.701863354037267,
+      "grad_norm": 1.3345098495483398,
+      "learning_rate": 0.0002,
+      "loss": 0.262,
+      "step": 1240
+    },
+    {
+      "epoch": 7.763975155279503,
+      "grad_norm": 1.5677719116210938,
+      "learning_rate": 0.0002,
+      "loss": 0.2613,
+      "step": 1250
+    },
+    {
+      "epoch": 7.826086956521739,
+      "grad_norm": 1.7234760522842407,
+      "learning_rate": 0.0002,
+      "loss": 0.2647,
+      "step": 1260
+    },
+    {
+      "epoch": 7.888198757763975,
+      "grad_norm": 1.7217389345169067,
+      "learning_rate": 0.0002,
+      "loss": 0.243,
+      "step": 1270
+    },
+    {
+      "epoch": 7.950310559006211,
+      "grad_norm": 1.6993083953857422,
+      "learning_rate": 0.0002,
+      "loss": 0.2785,
+      "step": 1280
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 1.962246060371399,
+      "eval_runtime": 89.77,
+      "eval_samples_per_second": 5.08,
+      "eval_steps_per_second": 0.635,
+      "step": 1288
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1288,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.65106479988736e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-1288/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34980fd5c2286610ef04f61d0ede08e4a5a9fecb8f190eb017f21325dd87b66b
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4db5b8118d82a12f1ff25d17d5c69c3c8968ff120e815eed815af41f95c45960
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5f0b3633a7ecd2ed723dab95258d0f389f588586be413478500a921c4d47d42
+size 55532538

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:941239368ddc074fbc5959c26497d40ff9bb72ab054b72496d5626dd3878a973
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83e9f5ab30edd2443895ce4fe6d43b9c969bea0946520c4c0149c3b59426810e
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/trainer_state.json

@@ -0,0 +1,153 @@
+{
+  "best_metric": 1.2526909112930298,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161",
+  "epoch": 1.0,
+  "eval_steps": 10,
+  "global_step": 161,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 1.3165894746780396,
+      "learning_rate": 0.0002,
+      "loss": 1.8735,
+      "step": 10
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 1.0744154453277588,
+      "learning_rate": 0.0002,
+      "loss": 1.6236,
+      "step": 20
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.7020019292831421,
+      "learning_rate": 0.0002,
+      "loss": 1.5484,
+      "step": 30
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 1.083817958831787,
+      "learning_rate": 0.0002,
+      "loss": 1.5461,
+      "step": 40
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 0.8009080290794373,
+      "learning_rate": 0.0002,
+      "loss": 1.3413,
+      "step": 50
+    },
+    {
+      "epoch": 0.37267080745341613,
+      "grad_norm": 0.8239762783050537,
+      "learning_rate": 0.0002,
+      "loss": 1.2591,
+      "step": 60
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "grad_norm": 0.8437100052833557,
+      "learning_rate": 0.0002,
+      "loss": 1.2237,
+      "step": 70
+    },
+    {
+      "epoch": 0.4968944099378882,
+      "grad_norm": 1.0204336643218994,
+      "learning_rate": 0.0002,
+      "loss": 1.2777,
+      "step": 80
+    },
+    {
+      "epoch": 0.5590062111801242,
+      "grad_norm": 1.185208797454834,
+      "learning_rate": 0.0002,
+      "loss": 1.2178,
+      "step": 90
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 0.8615248203277588,
+      "learning_rate": 0.0002,
+      "loss": 1.3011,
+      "step": 100
+    },
+    {
+      "epoch": 0.6832298136645962,
+      "grad_norm": 0.5289493799209595,
+      "learning_rate": 0.0002,
+      "loss": 1.276,
+      "step": 110
+    },
+    {
+      "epoch": 0.7453416149068323,
+      "grad_norm": 0.6098535656929016,
+      "learning_rate": 0.0002,
+      "loss": 1.2099,
+      "step": 120
+    },
+    {
+      "epoch": 0.8074534161490683,
+      "grad_norm": 0.601532518863678,
+      "learning_rate": 0.0002,
+      "loss": 1.1765,
+      "step": 130
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.6565020680427551,
+      "learning_rate": 0.0002,
+      "loss": 1.299,
+      "step": 140
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 0.6863213181495667,
+      "learning_rate": 0.0002,
+      "loss": 1.1232,
+      "step": 150
+    },
+    {
+      "epoch": 0.9937888198757764,
+      "grad_norm": 0.6823898553848267,
+      "learning_rate": 0.0002,
+      "loss": 1.2542,
+      "step": 160
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.2526909112930298,
+      "eval_runtime": 89.0661,
+      "eval_samples_per_second": 5.12,
+      "eval_steps_per_second": 0.64,
+      "step": 161
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1288,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7063830999859200.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34980fd5c2286610ef04f61d0ede08e4a5a9fecb8f190eb017f21325dd87b66b
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ccc72f45799bfe364f969281352c02be8c5d8128f515105ec7490e0d1523873
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bbbfe562b4232bbbbb6780b1d4485b639b92f6731716683a95d3c9c3104bb67
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc3081c4a61f78a766171325387197a863583bc70b483af62760df4162d812fe
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45412991132d3e92205ae6ad93008a8e9ca8a362529783cc5da6cf8a4cee1c3c
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-322/trainer_state.json

@@ -0,0 +1,273 @@
+{
+  "best_metric": 1.2526909112930298,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.1-num-980-sd-42/checkpoint-161",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 322,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 1.3165894746780396,
+      "learning_rate": 0.0002,
+      "loss": 1.8735,
+      "step": 10
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 1.0744154453277588,
+      "learning_rate": 0.0002,
+      "loss": 1.6236,
+      "step": 20
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.7020019292831421,
+      "learning_rate": 0.0002,
+      "loss": 1.5484,
+      "step": 30
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 1.083817958831787,
+      "learning_rate": 0.0002,
+      "loss": 1.5461,
+      "step": 40
+    },
+    {
+      "epoch": 0.3105590062111801,
+      "grad_norm": 0.8009080290794373,
+      "learning_rate": 0.0002,
+      "loss": 1.3413,
+      "step": 50
+    },
+    {
+      "epoch": 0.37267080745341613,
+      "grad_norm": 0.8239762783050537,
+      "learning_rate": 0.0002,
+      "loss": 1.2591,
+      "step": 60
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "grad_norm": 0.8437100052833557,
+      "learning_rate": 0.0002,
+      "loss": 1.2237,
+      "step": 70
+    },
+    {
+      "epoch": 0.4968944099378882,
+      "grad_norm": 1.0204336643218994,
+      "learning_rate": 0.0002,
+      "loss": 1.2777,
+      "step": 80
+    },
+    {
+      "epoch": 0.5590062111801242,
+      "grad_norm": 1.185208797454834,
+      "learning_rate": 0.0002,
+      "loss": 1.2178,
+      "step": 90
+    },
+    {
+      "epoch": 0.6211180124223602,
+      "grad_norm": 0.8615248203277588,
+      "learning_rate": 0.0002,
+      "loss": 1.3011,
+      "step": 100
+    },
+    {
+      "epoch": 0.6832298136645962,
+      "grad_norm": 0.5289493799209595,
+      "learning_rate": 0.0002,
+      "loss": 1.276,
+      "step": 110
+    },
+    {
+      "epoch": 0.7453416149068323,
+      "grad_norm": 0.6098535656929016,
+      "learning_rate": 0.0002,
+      "loss": 1.2099,
+      "step": 120
+    },
+    {
+      "epoch": 0.8074534161490683,
+      "grad_norm": 0.601532518863678,
+      "learning_rate": 0.0002,
+      "loss": 1.1765,
+      "step": 130
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.6565020680427551,
+      "learning_rate": 0.0002,
+      "loss": 1.299,
+      "step": 140
+    },
+    {
+      "epoch": 0.9316770186335404,
+      "grad_norm": 0.6863213181495667,
+      "learning_rate": 0.0002,
+      "loss": 1.1232,
+      "step": 150
+    },
+    {
+      "epoch": 0.9937888198757764,
+      "grad_norm": 0.6823898553848267,
+      "learning_rate": 0.0002,
+      "loss": 1.2542,
+      "step": 160
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.2526909112930298,
+      "eval_runtime": 89.0661,
+      "eval_samples_per_second": 5.12,
+      "eval_steps_per_second": 0.64,
+      "step": 161
+    },
+    {
+      "epoch": 1.0559006211180124,
+      "grad_norm": 0.562255859375,
+      "learning_rate": 0.0002,
+      "loss": 1.1205,
+      "step": 170
+    },
+    {
+      "epoch": 1.1180124223602483,
+      "grad_norm": 0.5982539653778076,
+      "learning_rate": 0.0002,
+      "loss": 1.1351,
+      "step": 180
+    },
+    {
+      "epoch": 1.1801242236024845,
+      "grad_norm": 0.6252831220626831,
+      "learning_rate": 0.0002,
+      "loss": 1.0561,
+      "step": 190
+    },
+    {
+      "epoch": 1.2422360248447206,
+      "grad_norm": 0.6803395748138428,
+      "learning_rate": 0.0002,
+      "loss": 1.0898,
+      "step": 200
+    },
+    {
+      "epoch": 1.3043478260869565,
+      "grad_norm": 0.8347863554954529,
+      "learning_rate": 0.0002,
+      "loss": 1.1508,
+      "step": 210
+    },
+    {
+      "epoch": 1.3664596273291925,
+      "grad_norm": 0.7230828404426575,
+      "learning_rate": 0.0002,
+      "loss": 1.0958,
+      "step": 220
+    },
+    {
+      "epoch": 1.4285714285714286,
+      "grad_norm": 0.6892024874687195,
+      "learning_rate": 0.0002,
+      "loss": 1.0728,
+      "step": 230
+    },
+    {
+      "epoch": 1.4906832298136645,
+      "grad_norm": 0.9822754263877869,
+      "learning_rate": 0.0002,
+      "loss": 1.08,
+      "step": 240
+    },
+    {
+      "epoch": 1.5527950310559007,
+      "grad_norm": 0.8614016175270081,
+      "learning_rate": 0.0002,
+      "loss": 1.0916,
+      "step": 250
+    },
+    {
+      "epoch": 1.6149068322981366,
+      "grad_norm": 0.6894949078559875,
+      "learning_rate": 0.0002,
+      "loss": 1.0036,
+      "step": 260
+    },
+    {
+      "epoch": 1.6770186335403725,
+      "grad_norm": 0.721196711063385,
+      "learning_rate": 0.0002,
+      "loss": 1.1229,
+      "step": 270
+    },
+    {
+      "epoch": 1.7391304347826086,
+      "grad_norm": 0.7291169166564941,
+      "learning_rate": 0.0002,
+      "loss": 1.0329,
+      "step": 280
+    },
+    {
+      "epoch": 1.8012422360248448,
+      "grad_norm": 0.6831535696983337,
+      "learning_rate": 0.0002,
+      "loss": 1.0795,
+      "step": 290
+    },
+    {
+      "epoch": 1.8633540372670807,
+      "grad_norm": 0.6903841495513916,
+      "learning_rate": 0.0002,
+      "loss": 1.1072,
+      "step": 300
+    },
+    {
+      "epoch": 1.9254658385093166,
+      "grad_norm": 0.8273940682411194,
+      "learning_rate": 0.0002,
+      "loss": 1.1371,
+      "step": 310
+    },
+    {
+      "epoch": 1.9875776397515528,
+      "grad_norm": 1.1644595861434937,
+      "learning_rate": 0.0002,
+      "loss": 1.0195,
+      "step": 320
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.2606149911880493,
+      "eval_runtime": 89.1274,
+      "eval_samples_per_second": 5.116,
+      "eval_steps_per_second": 0.64,
+      "step": 322
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1288,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.41276619997184e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}