Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de443465fea1ade0bb99dc6ba65afceed4e907ad6efaa20e7bcb905fa4f31fd7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6de1943b6e1a6700974a6ae98cfeefcaa69901e5427006901e7a1b73832aa832
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d7c5c85e3d1777edb18ba3772df83d176a0dc643f4365ba9c436570c21a532b
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e733a8ce46bd4743286247df1848c6cb942731d7c874189fc84028f18c35d1d
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1cc649c2c2ccc90b0dde63aee443704fee825054be9e3cfa63e6886b952f3a8
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/trainer_state.json

@@ -0,0 +1,831 @@
+{
+  "best_metric": 1.0297880172729492,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304",
+  "epoch": 7.0,
+  "eval_steps": 10,
+  "global_step": 1064,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06578947368421052,
+      "grad_norm": 0.8710125088691711,
+      "learning_rate": 0.0002,
+      "loss": 1.5609,
+      "step": 10
+    },
+    {
+      "epoch": 0.13157894736842105,
+      "grad_norm": 1.1956019401550293,
+      "learning_rate": 0.0002,
+      "loss": 1.281,
+      "step": 20
+    },
+    {
+      "epoch": 0.19736842105263158,
+      "grad_norm": 0.7674543261528015,
+      "learning_rate": 0.0002,
+      "loss": 1.2364,
+      "step": 30
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.9102759957313538,
+      "learning_rate": 0.0002,
+      "loss": 1.1964,
+      "step": 40
+    },
+    {
+      "epoch": 0.32894736842105265,
+      "grad_norm": 0.6881781816482544,
+      "learning_rate": 0.0002,
+      "loss": 1.1633,
+      "step": 50
+    },
+    {
+      "epoch": 0.39473684210526316,
+      "grad_norm": 0.8366474509239197,
+      "learning_rate": 0.0002,
+      "loss": 1.131,
+      "step": 60
+    },
+    {
+      "epoch": 0.4605263157894737,
+      "grad_norm": 1.1925188302993774,
+      "learning_rate": 0.0002,
+      "loss": 1.0874,
+      "step": 70
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5625673532485962,
+      "learning_rate": 0.0002,
+      "loss": 1.0863,
+      "step": 80
+    },
+    {
+      "epoch": 0.5921052631578947,
+      "grad_norm": 0.7259140610694885,
+      "learning_rate": 0.0002,
+      "loss": 1.0992,
+      "step": 90
+    },
+    {
+      "epoch": 0.6578947368421053,
+      "grad_norm": 0.7536493539810181,
+      "learning_rate": 0.0002,
+      "loss": 1.0855,
+      "step": 100
+    },
+    {
+      "epoch": 0.7236842105263158,
+      "grad_norm": 0.5873591303825378,
+      "learning_rate": 0.0002,
+      "loss": 1.0775,
+      "step": 110
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.6995271444320679,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 120
+    },
+    {
+      "epoch": 0.8552631578947368,
+      "grad_norm": 0.6832848787307739,
+      "learning_rate": 0.0002,
+      "loss": 1.1105,
+      "step": 130
+    },
+    {
+      "epoch": 0.9210526315789473,
+      "grad_norm": 0.605141818523407,
+      "learning_rate": 0.0002,
+      "loss": 1.0688,
+      "step": 140
+    },
+    {
+      "epoch": 0.9868421052631579,
+      "grad_norm": 0.6675612330436707,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 150
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.0424011945724487,
+      "eval_runtime": 7.9344,
+      "eval_samples_per_second": 13.738,
+      "eval_steps_per_second": 1.764,
+      "step": 152
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5364972949028015,
+      "learning_rate": 0.0002,
+      "loss": 0.9909,
+      "step": 160
+    },
+    {
+      "epoch": 1.118421052631579,
+      "grad_norm": 0.6408411264419556,
+      "learning_rate": 0.0002,
+      "loss": 0.9368,
+      "step": 170
+    },
+    {
+      "epoch": 1.1842105263157894,
+      "grad_norm": 0.5882905125617981,
+      "learning_rate": 0.0002,
+      "loss": 0.9179,
+      "step": 180
+    },
+    {
+      "epoch": 1.25,
+      "grad_norm": 0.7419160008430481,
+      "learning_rate": 0.0002,
+      "loss": 1.0002,
+      "step": 190
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.6069955825805664,
+      "learning_rate": 0.0002,
+      "loss": 0.9192,
+      "step": 200
+    },
+    {
+      "epoch": 1.381578947368421,
+      "grad_norm": 0.5905187129974365,
+      "learning_rate": 0.0002,
+      "loss": 0.9572,
+      "step": 210
+    },
+    {
+      "epoch": 1.4473684210526316,
+      "grad_norm": 0.7193484902381897,
+      "learning_rate": 0.0002,
+      "loss": 0.9653,
+      "step": 220
+    },
+    {
+      "epoch": 1.513157894736842,
+      "grad_norm": 0.7149216532707214,
+      "learning_rate": 0.0002,
+      "loss": 1.0417,
+      "step": 230
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.6062077879905701,
+      "learning_rate": 0.0002,
+      "loss": 0.9429,
+      "step": 240
+    },
+    {
+      "epoch": 1.6447368421052633,
+      "grad_norm": 0.7012736797332764,
+      "learning_rate": 0.0002,
+      "loss": 0.9463,
+      "step": 250
+    },
+    {
+      "epoch": 1.7105263157894737,
+      "grad_norm": 0.6271148920059204,
+      "learning_rate": 0.0002,
+      "loss": 1.0061,
+      "step": 260
+    },
+    {
+      "epoch": 1.776315789473684,
+      "grad_norm": 0.8046498894691467,
+      "learning_rate": 0.0002,
+      "loss": 1.0064,
+      "step": 270
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.7960395812988281,
+      "learning_rate": 0.0002,
+      "loss": 0.9877,
+      "step": 280
+    },
+    {
+      "epoch": 1.9078947368421053,
+      "grad_norm": 0.6067329049110413,
+      "learning_rate": 0.0002,
+      "loss": 0.9817,
+      "step": 290
+    },
+    {
+      "epoch": 1.973684210526316,
+      "grad_norm": 0.722137451171875,
+      "learning_rate": 0.0002,
+      "loss": 0.9553,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0297880172729492,
+      "eval_runtime": 7.9354,
+      "eval_samples_per_second": 13.736,
+      "eval_steps_per_second": 1.764,
+      "step": 304
+    },
+    {
+      "epoch": 2.039473684210526,
+      "grad_norm": 0.7166091203689575,
+      "learning_rate": 0.0002,
+      "loss": 0.8226,
+      "step": 310
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9819070100784302,
+      "learning_rate": 0.0002,
+      "loss": 0.8732,
+      "step": 320
+    },
+    {
+      "epoch": 2.1710526315789473,
+      "grad_norm": 0.8334569334983826,
+      "learning_rate": 0.0002,
+      "loss": 0.8327,
+      "step": 330
+    },
+    {
+      "epoch": 2.236842105263158,
+      "grad_norm": 0.7255743145942688,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 340
+    },
+    {
+      "epoch": 2.3026315789473686,
+      "grad_norm": 0.8454753756523132,
+      "learning_rate": 0.0002,
+      "loss": 0.7982,
+      "step": 350
+    },
+    {
+      "epoch": 2.3684210526315788,
+      "grad_norm": 0.7205567359924316,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 360
+    },
+    {
+      "epoch": 2.4342105263157894,
+      "grad_norm": 0.8142246603965759,
+      "learning_rate": 0.0002,
+      "loss": 0.8238,
+      "step": 370
+    },
+    {
+      "epoch": 2.5,
+      "grad_norm": 0.775557816028595,
+      "learning_rate": 0.0002,
+      "loss": 0.8264,
+      "step": 380
+    },
+    {
+      "epoch": 2.5657894736842106,
+      "grad_norm": 0.7804738283157349,
+      "learning_rate": 0.0002,
+      "loss": 0.8128,
+      "step": 390
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "grad_norm": 0.9046455025672913,
+      "learning_rate": 0.0002,
+      "loss": 0.8318,
+      "step": 400
+    },
+    {
+      "epoch": 2.6973684210526314,
+      "grad_norm": 0.8085933327674866,
+      "learning_rate": 0.0002,
+      "loss": 0.823,
+      "step": 410
+    },
+    {
+      "epoch": 2.763157894736842,
+      "grad_norm": 0.7365749478340149,
+      "learning_rate": 0.0002,
+      "loss": 0.822,
+      "step": 420
+    },
+    {
+      "epoch": 2.8289473684210527,
+      "grad_norm": 0.775257408618927,
+      "learning_rate": 0.0002,
+      "loss": 0.7921,
+      "step": 430
+    },
+    {
+      "epoch": 2.8947368421052633,
+      "grad_norm": 0.9725009202957153,
+      "learning_rate": 0.0002,
+      "loss": 0.7932,
+      "step": 440
+    },
+    {
+      "epoch": 2.9605263157894735,
+      "grad_norm": 0.9256396889686584,
+      "learning_rate": 0.0002,
+      "loss": 0.8415,
+      "step": 450
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0594063997268677,
+      "eval_runtime": 7.9334,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 456
+    },
+    {
+      "epoch": 3.026315789473684,
+      "grad_norm": 0.845704972743988,
+      "learning_rate": 0.0002,
+      "loss": 0.7398,
+      "step": 460
+    },
+    {
+      "epoch": 3.0921052631578947,
+      "grad_norm": 0.8985124230384827,
+      "learning_rate": 0.0002,
+      "loss": 0.6447,
+      "step": 470
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 1.3014018535614014,
+      "learning_rate": 0.0002,
+      "loss": 0.6695,
+      "step": 480
+    },
+    {
+      "epoch": 3.223684210526316,
+      "grad_norm": 1.112438440322876,
+      "learning_rate": 0.0002,
+      "loss": 0.6693,
+      "step": 490
+    },
+    {
+      "epoch": 3.2894736842105265,
+      "grad_norm": 1.0078773498535156,
+      "learning_rate": 0.0002,
+      "loss": 0.6471,
+      "step": 500
+    },
+    {
+      "epoch": 3.3552631578947367,
+      "grad_norm": 0.8961848020553589,
+      "learning_rate": 0.0002,
+      "loss": 0.6557,
+      "step": 510
+    },
+    {
+      "epoch": 3.4210526315789473,
+      "grad_norm": 1.1617182493209839,
+      "learning_rate": 0.0002,
+      "loss": 0.6581,
+      "step": 520
+    },
+    {
+      "epoch": 3.486842105263158,
+      "grad_norm": 1.0005075931549072,
+      "learning_rate": 0.0002,
+      "loss": 0.6545,
+      "step": 530
+    },
+    {
+      "epoch": 3.5526315789473686,
+      "grad_norm": 1.1004620790481567,
+      "learning_rate": 0.0002,
+      "loss": 0.6719,
+      "step": 540
+    },
+    {
+      "epoch": 3.6184210526315788,
+      "grad_norm": 1.0429490804672241,
+      "learning_rate": 0.0002,
+      "loss": 0.6857,
+      "step": 550
+    },
+    {
+      "epoch": 3.6842105263157894,
+      "grad_norm": 0.9506118893623352,
+      "learning_rate": 0.0002,
+      "loss": 0.656,
+      "step": 560
+    },
+    {
+      "epoch": 3.75,
+      "grad_norm": 0.8741368055343628,
+      "learning_rate": 0.0002,
+      "loss": 0.6794,
+      "step": 570
+    },
+    {
+      "epoch": 3.8157894736842106,
+      "grad_norm": 0.9777415990829468,
+      "learning_rate": 0.0002,
+      "loss": 0.6953,
+      "step": 580
+    },
+    {
+      "epoch": 3.8815789473684212,
+      "grad_norm": 1.4024672508239746,
+      "learning_rate": 0.0002,
+      "loss": 0.6577,
+      "step": 590
+    },
+    {
+      "epoch": 3.9473684210526314,
+      "grad_norm": 1.1035726070404053,
+      "learning_rate": 0.0002,
+      "loss": 0.6849,
+      "step": 600
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.127807378768921,
+      "eval_runtime": 7.9307,
+      "eval_samples_per_second": 13.744,
+      "eval_steps_per_second": 1.765,
+      "step": 608
+    },
+    {
+      "epoch": 4.0131578947368425,
+      "grad_norm": 1.0415319204330444,
+      "learning_rate": 0.0002,
+      "loss": 0.6693,
+      "step": 610
+    },
+    {
+      "epoch": 4.078947368421052,
+      "grad_norm": 1.201734185218811,
+      "learning_rate": 0.0002,
+      "loss": 0.5325,
+      "step": 620
+    },
+    {
+      "epoch": 4.144736842105263,
+      "grad_norm": 1.2380131483078003,
+      "learning_rate": 0.0002,
+      "loss": 0.5121,
+      "step": 630
+    },
+    {
+      "epoch": 4.2105263157894735,
+      "grad_norm": 1.3034303188323975,
+      "learning_rate": 0.0002,
+      "loss": 0.5145,
+      "step": 640
+    },
+    {
+      "epoch": 4.276315789473684,
+      "grad_norm": 1.878430962562561,
+      "learning_rate": 0.0002,
+      "loss": 0.4844,
+      "step": 650
+    },
+    {
+      "epoch": 4.342105263157895,
+      "grad_norm": 1.3213163614273071,
+      "learning_rate": 0.0002,
+      "loss": 0.5297,
+      "step": 660
+    },
+    {
+      "epoch": 4.407894736842105,
+      "grad_norm": 1.1307642459869385,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 670
+    },
+    {
+      "epoch": 4.473684210526316,
+      "grad_norm": 0.9877312183380127,
+      "learning_rate": 0.0002,
+      "loss": 0.49,
+      "step": 680
+    },
+    {
+      "epoch": 4.5394736842105265,
+      "grad_norm": 1.163973093032837,
+      "learning_rate": 0.0002,
+      "loss": 0.5253,
+      "step": 690
+    },
+    {
+      "epoch": 4.605263157894737,
+      "grad_norm": 1.3219846487045288,
+      "learning_rate": 0.0002,
+      "loss": 0.5509,
+      "step": 700
+    },
+    {
+      "epoch": 4.671052631578947,
+      "grad_norm": 1.2262274026870728,
+      "learning_rate": 0.0002,
+      "loss": 0.517,
+      "step": 710
+    },
+    {
+      "epoch": 4.7368421052631575,
+      "grad_norm": 1.1390860080718994,
+      "learning_rate": 0.0002,
+      "loss": 0.5347,
+      "step": 720
+    },
+    {
+      "epoch": 4.802631578947368,
+      "grad_norm": 1.196449875831604,
+      "learning_rate": 0.0002,
+      "loss": 0.5581,
+      "step": 730
+    },
+    {
+      "epoch": 4.868421052631579,
+      "grad_norm": 1.2404807806015015,
+      "learning_rate": 0.0002,
+      "loss": 0.5556,
+      "step": 740
+    },
+    {
+      "epoch": 4.934210526315789,
+      "grad_norm": 1.1726386547088623,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 750
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.3372385501861572,
+      "learning_rate": 0.0002,
+      "loss": 0.5315,
+      "step": 760
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.2448275089263916,
+      "eval_runtime": 7.9296,
+      "eval_samples_per_second": 13.746,
+      "eval_steps_per_second": 1.766,
+      "step": 760
+    },
+    {
+      "epoch": 5.065789473684211,
+      "grad_norm": 1.5582785606384277,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 770
+    },
+    {
+      "epoch": 5.131578947368421,
+      "grad_norm": 1.6396721601486206,
+      "learning_rate": 0.0002,
+      "loss": 0.4033,
+      "step": 780
+    },
+    {
+      "epoch": 5.197368421052632,
+      "grad_norm": 1.2244197130203247,
+      "learning_rate": 0.0002,
+      "loss": 0.4039,
+      "step": 790
+    },
+    {
+      "epoch": 5.2631578947368425,
+      "grad_norm": 1.3662641048431396,
+      "learning_rate": 0.0002,
+      "loss": 0.4262,
+      "step": 800
+    },
+    {
+      "epoch": 5.328947368421053,
+      "grad_norm": 1.405532956123352,
+      "learning_rate": 0.0002,
+      "loss": 0.4164,
+      "step": 810
+    },
+    {
+      "epoch": 5.394736842105263,
+      "grad_norm": 1.2695107460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.4131,
+      "step": 820
+    },
+    {
+      "epoch": 5.4605263157894735,
+      "grad_norm": 1.6382331848144531,
+      "learning_rate": 0.0002,
+      "loss": 0.4022,
+      "step": 830
+    },
+    {
+      "epoch": 5.526315789473684,
+      "grad_norm": 1.684364676475525,
+      "learning_rate": 0.0002,
+      "loss": 0.4214,
+      "step": 840
+    },
+    {
+      "epoch": 5.592105263157895,
+      "grad_norm": 1.2763781547546387,
+      "learning_rate": 0.0002,
+      "loss": 0.4109,
+      "step": 850
+    },
+    {
+      "epoch": 5.657894736842105,
+      "grad_norm": 1.429623007774353,
+      "learning_rate": 0.0002,
+      "loss": 0.4258,
+      "step": 860
+    },
+    {
+      "epoch": 5.723684210526316,
+      "grad_norm": 1.9069653749465942,
+      "learning_rate": 0.0002,
+      "loss": 0.4268,
+      "step": 870
+    },
+    {
+      "epoch": 5.7894736842105265,
+      "grad_norm": 1.349945068359375,
+      "learning_rate": 0.0002,
+      "loss": 0.4372,
+      "step": 880
+    },
+    {
+      "epoch": 5.855263157894737,
+      "grad_norm": 1.3684793710708618,
+      "learning_rate": 0.0002,
+      "loss": 0.4354,
+      "step": 890
+    },
+    {
+      "epoch": 5.921052631578947,
+      "grad_norm": 1.5587575435638428,
+      "learning_rate": 0.0002,
+      "loss": 0.4393,
+      "step": 900
+    },
+    {
+      "epoch": 5.9868421052631575,
+      "grad_norm": 1.3056254386901855,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3288270235061646,
+      "eval_runtime": 7.9333,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 912
+    },
+    {
+      "epoch": 6.052631578947368,
+      "grad_norm": 1.3509465456008911,
+      "learning_rate": 0.0002,
+      "loss": 0.3445,
+      "step": 920
+    },
+    {
+      "epoch": 6.118421052631579,
+      "grad_norm": 1.5550519227981567,
+      "learning_rate": 0.0002,
+      "loss": 0.3037,
+      "step": 930
+    },
+    {
+      "epoch": 6.184210526315789,
+      "grad_norm": 1.520915150642395,
+      "learning_rate": 0.0002,
+      "loss": 0.3319,
+      "step": 940
+    },
+    {
+      "epoch": 6.25,
+      "grad_norm": 1.2581181526184082,
+      "learning_rate": 0.0002,
+      "loss": 0.3196,
+      "step": 950
+    },
+    {
+      "epoch": 6.315789473684211,
+      "grad_norm": 1.5343317985534668,
+      "learning_rate": 0.0002,
+      "loss": 0.3132,
+      "step": 960
+    },
+    {
+      "epoch": 6.381578947368421,
+      "grad_norm": 1.4771231412887573,
+      "learning_rate": 0.0002,
+      "loss": 0.3218,
+      "step": 970
+    },
+    {
+      "epoch": 6.447368421052632,
+      "grad_norm": 1.1770271062850952,
+      "learning_rate": 0.0002,
+      "loss": 0.3511,
+      "step": 980
+    },
+    {
+      "epoch": 6.5131578947368425,
+      "grad_norm": 1.4069170951843262,
+      "learning_rate": 0.0002,
+      "loss": 0.3526,
+      "step": 990
+    },
+    {
+      "epoch": 6.578947368421053,
+      "grad_norm": 1.667536973953247,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 1000
+    },
+    {
+      "epoch": 6.644736842105263,
+      "grad_norm": 1.4396395683288574,
+      "learning_rate": 0.0002,
+      "loss": 0.3582,
+      "step": 1010
+    },
+    {
+      "epoch": 6.7105263157894735,
+      "grad_norm": 1.5290015935897827,
+      "learning_rate": 0.0002,
+      "loss": 0.336,
+      "step": 1020
+    },
+    {
+      "epoch": 6.776315789473684,
+      "grad_norm": 1.2737603187561035,
+      "learning_rate": 0.0002,
+      "loss": 0.366,
+      "step": 1030
+    },
+    {
+      "epoch": 6.842105263157895,
+      "grad_norm": 2.4872493743896484,
+      "learning_rate": 0.0002,
+      "loss": 0.394,
+      "step": 1040
+    },
+    {
+      "epoch": 6.907894736842105,
+      "grad_norm": 1.3118231296539307,
+      "learning_rate": 0.0002,
+      "loss": 0.3602,
+      "step": 1050
+    },
+    {
+      "epoch": 6.973684210526316,
+      "grad_norm": 1.5545233488082886,
+      "learning_rate": 0.0002,
+      "loss": 0.3729,
+      "step": 1060
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.4201300144195557,
+      "eval_runtime": 7.9357,
+      "eval_samples_per_second": 13.735,
+      "eval_steps_per_second": 1.764,
+      "step": 1064
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1216,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.66827092164608e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1064/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945033f5c566fb4e041fb4438a4cb0536a22ba21ea95f2bf808c8aca0e5a389f
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:825987509563bce3ce69a20fd7abcd2613e7197767fa864c35f5e436e0922e6e
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b3203d34e4a64e96de175924727ca7c417a65fd1bc34dc51d1fb53b4fc3790a
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb1221cdf596e7e00a2add83d9fd6b0c9f109472c6d057c4444074fa3d398c7b
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d3ea9158921f489527c9323e47fec74c381b219ad499e3aa0e5ab72714a0bb8
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/trainer_state.json

@@ -0,0 +1,944 @@
+{
+  "best_metric": 1.0297880172729492,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304",
+  "epoch": 8.0,
+  "eval_steps": 10,
+  "global_step": 1216,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06578947368421052,
+      "grad_norm": 0.8710125088691711,
+      "learning_rate": 0.0002,
+      "loss": 1.5609,
+      "step": 10
+    },
+    {
+      "epoch": 0.13157894736842105,
+      "grad_norm": 1.1956019401550293,
+      "learning_rate": 0.0002,
+      "loss": 1.281,
+      "step": 20
+    },
+    {
+      "epoch": 0.19736842105263158,
+      "grad_norm": 0.7674543261528015,
+      "learning_rate": 0.0002,
+      "loss": 1.2364,
+      "step": 30
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.9102759957313538,
+      "learning_rate": 0.0002,
+      "loss": 1.1964,
+      "step": 40
+    },
+    {
+      "epoch": 0.32894736842105265,
+      "grad_norm": 0.6881781816482544,
+      "learning_rate": 0.0002,
+      "loss": 1.1633,
+      "step": 50
+    },
+    {
+      "epoch": 0.39473684210526316,
+      "grad_norm": 0.8366474509239197,
+      "learning_rate": 0.0002,
+      "loss": 1.131,
+      "step": 60
+    },
+    {
+      "epoch": 0.4605263157894737,
+      "grad_norm": 1.1925188302993774,
+      "learning_rate": 0.0002,
+      "loss": 1.0874,
+      "step": 70
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5625673532485962,
+      "learning_rate": 0.0002,
+      "loss": 1.0863,
+      "step": 80
+    },
+    {
+      "epoch": 0.5921052631578947,
+      "grad_norm": 0.7259140610694885,
+      "learning_rate": 0.0002,
+      "loss": 1.0992,
+      "step": 90
+    },
+    {
+      "epoch": 0.6578947368421053,
+      "grad_norm": 0.7536493539810181,
+      "learning_rate": 0.0002,
+      "loss": 1.0855,
+      "step": 100
+    },
+    {
+      "epoch": 0.7236842105263158,
+      "grad_norm": 0.5873591303825378,
+      "learning_rate": 0.0002,
+      "loss": 1.0775,
+      "step": 110
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.6995271444320679,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 120
+    },
+    {
+      "epoch": 0.8552631578947368,
+      "grad_norm": 0.6832848787307739,
+      "learning_rate": 0.0002,
+      "loss": 1.1105,
+      "step": 130
+    },
+    {
+      "epoch": 0.9210526315789473,
+      "grad_norm": 0.605141818523407,
+      "learning_rate": 0.0002,
+      "loss": 1.0688,
+      "step": 140
+    },
+    {
+      "epoch": 0.9868421052631579,
+      "grad_norm": 0.6675612330436707,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 150
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.0424011945724487,
+      "eval_runtime": 7.9344,
+      "eval_samples_per_second": 13.738,
+      "eval_steps_per_second": 1.764,
+      "step": 152
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5364972949028015,
+      "learning_rate": 0.0002,
+      "loss": 0.9909,
+      "step": 160
+    },
+    {
+      "epoch": 1.118421052631579,
+      "grad_norm": 0.6408411264419556,
+      "learning_rate": 0.0002,
+      "loss": 0.9368,
+      "step": 170
+    },
+    {
+      "epoch": 1.1842105263157894,
+      "grad_norm": 0.5882905125617981,
+      "learning_rate": 0.0002,
+      "loss": 0.9179,
+      "step": 180
+    },
+    {
+      "epoch": 1.25,
+      "grad_norm": 0.7419160008430481,
+      "learning_rate": 0.0002,
+      "loss": 1.0002,
+      "step": 190
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.6069955825805664,
+      "learning_rate": 0.0002,
+      "loss": 0.9192,
+      "step": 200
+    },
+    {
+      "epoch": 1.381578947368421,
+      "grad_norm": 0.5905187129974365,
+      "learning_rate": 0.0002,
+      "loss": 0.9572,
+      "step": 210
+    },
+    {
+      "epoch": 1.4473684210526316,
+      "grad_norm": 0.7193484902381897,
+      "learning_rate": 0.0002,
+      "loss": 0.9653,
+      "step": 220
+    },
+    {
+      "epoch": 1.513157894736842,
+      "grad_norm": 0.7149216532707214,
+      "learning_rate": 0.0002,
+      "loss": 1.0417,
+      "step": 230
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.6062077879905701,
+      "learning_rate": 0.0002,
+      "loss": 0.9429,
+      "step": 240
+    },
+    {
+      "epoch": 1.6447368421052633,
+      "grad_norm": 0.7012736797332764,
+      "learning_rate": 0.0002,
+      "loss": 0.9463,
+      "step": 250
+    },
+    {
+      "epoch": 1.7105263157894737,
+      "grad_norm": 0.6271148920059204,
+      "learning_rate": 0.0002,
+      "loss": 1.0061,
+      "step": 260
+    },
+    {
+      "epoch": 1.776315789473684,
+      "grad_norm": 0.8046498894691467,
+      "learning_rate": 0.0002,
+      "loss": 1.0064,
+      "step": 270
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.7960395812988281,
+      "learning_rate": 0.0002,
+      "loss": 0.9877,
+      "step": 280
+    },
+    {
+      "epoch": 1.9078947368421053,
+      "grad_norm": 0.6067329049110413,
+      "learning_rate": 0.0002,
+      "loss": 0.9817,
+      "step": 290
+    },
+    {
+      "epoch": 1.973684210526316,
+      "grad_norm": 0.722137451171875,
+      "learning_rate": 0.0002,
+      "loss": 0.9553,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0297880172729492,
+      "eval_runtime": 7.9354,
+      "eval_samples_per_second": 13.736,
+      "eval_steps_per_second": 1.764,
+      "step": 304
+    },
+    {
+      "epoch": 2.039473684210526,
+      "grad_norm": 0.7166091203689575,
+      "learning_rate": 0.0002,
+      "loss": 0.8226,
+      "step": 310
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9819070100784302,
+      "learning_rate": 0.0002,
+      "loss": 0.8732,
+      "step": 320
+    },
+    {
+      "epoch": 2.1710526315789473,
+      "grad_norm": 0.8334569334983826,
+      "learning_rate": 0.0002,
+      "loss": 0.8327,
+      "step": 330
+    },
+    {
+      "epoch": 2.236842105263158,
+      "grad_norm": 0.7255743145942688,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 340
+    },
+    {
+      "epoch": 2.3026315789473686,
+      "grad_norm": 0.8454753756523132,
+      "learning_rate": 0.0002,
+      "loss": 0.7982,
+      "step": 350
+    },
+    {
+      "epoch": 2.3684210526315788,
+      "grad_norm": 0.7205567359924316,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 360
+    },
+    {
+      "epoch": 2.4342105263157894,
+      "grad_norm": 0.8142246603965759,
+      "learning_rate": 0.0002,
+      "loss": 0.8238,
+      "step": 370
+    },
+    {
+      "epoch": 2.5,
+      "grad_norm": 0.775557816028595,
+      "learning_rate": 0.0002,
+      "loss": 0.8264,
+      "step": 380
+    },
+    {
+      "epoch": 2.5657894736842106,
+      "grad_norm": 0.7804738283157349,
+      "learning_rate": 0.0002,
+      "loss": 0.8128,
+      "step": 390
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "grad_norm": 0.9046455025672913,
+      "learning_rate": 0.0002,
+      "loss": 0.8318,
+      "step": 400
+    },
+    {
+      "epoch": 2.6973684210526314,
+      "grad_norm": 0.8085933327674866,
+      "learning_rate": 0.0002,
+      "loss": 0.823,
+      "step": 410
+    },
+    {
+      "epoch": 2.763157894736842,
+      "grad_norm": 0.7365749478340149,
+      "learning_rate": 0.0002,
+      "loss": 0.822,
+      "step": 420
+    },
+    {
+      "epoch": 2.8289473684210527,
+      "grad_norm": 0.775257408618927,
+      "learning_rate": 0.0002,
+      "loss": 0.7921,
+      "step": 430
+    },
+    {
+      "epoch": 2.8947368421052633,
+      "grad_norm": 0.9725009202957153,
+      "learning_rate": 0.0002,
+      "loss": 0.7932,
+      "step": 440
+    },
+    {
+      "epoch": 2.9605263157894735,
+      "grad_norm": 0.9256396889686584,
+      "learning_rate": 0.0002,
+      "loss": 0.8415,
+      "step": 450
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0594063997268677,
+      "eval_runtime": 7.9334,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 456
+    },
+    {
+      "epoch": 3.026315789473684,
+      "grad_norm": 0.845704972743988,
+      "learning_rate": 0.0002,
+      "loss": 0.7398,
+      "step": 460
+    },
+    {
+      "epoch": 3.0921052631578947,
+      "grad_norm": 0.8985124230384827,
+      "learning_rate": 0.0002,
+      "loss": 0.6447,
+      "step": 470
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 1.3014018535614014,
+      "learning_rate": 0.0002,
+      "loss": 0.6695,
+      "step": 480
+    },
+    {
+      "epoch": 3.223684210526316,
+      "grad_norm": 1.112438440322876,
+      "learning_rate": 0.0002,
+      "loss": 0.6693,
+      "step": 490
+    },
+    {
+      "epoch": 3.2894736842105265,
+      "grad_norm": 1.0078773498535156,
+      "learning_rate": 0.0002,
+      "loss": 0.6471,
+      "step": 500
+    },
+    {
+      "epoch": 3.3552631578947367,
+      "grad_norm": 0.8961848020553589,
+      "learning_rate": 0.0002,
+      "loss": 0.6557,
+      "step": 510
+    },
+    {
+      "epoch": 3.4210526315789473,
+      "grad_norm": 1.1617182493209839,
+      "learning_rate": 0.0002,
+      "loss": 0.6581,
+      "step": 520
+    },
+    {
+      "epoch": 3.486842105263158,
+      "grad_norm": 1.0005075931549072,
+      "learning_rate": 0.0002,
+      "loss": 0.6545,
+      "step": 530
+    },
+    {
+      "epoch": 3.5526315789473686,
+      "grad_norm": 1.1004620790481567,
+      "learning_rate": 0.0002,
+      "loss": 0.6719,
+      "step": 540
+    },
+    {
+      "epoch": 3.6184210526315788,
+      "grad_norm": 1.0429490804672241,
+      "learning_rate": 0.0002,
+      "loss": 0.6857,
+      "step": 550
+    },
+    {
+      "epoch": 3.6842105263157894,
+      "grad_norm": 0.9506118893623352,
+      "learning_rate": 0.0002,
+      "loss": 0.656,
+      "step": 560
+    },
+    {
+      "epoch": 3.75,
+      "grad_norm": 0.8741368055343628,
+      "learning_rate": 0.0002,
+      "loss": 0.6794,
+      "step": 570
+    },
+    {
+      "epoch": 3.8157894736842106,
+      "grad_norm": 0.9777415990829468,
+      "learning_rate": 0.0002,
+      "loss": 0.6953,
+      "step": 580
+    },
+    {
+      "epoch": 3.8815789473684212,
+      "grad_norm": 1.4024672508239746,
+      "learning_rate": 0.0002,
+      "loss": 0.6577,
+      "step": 590
+    },
+    {
+      "epoch": 3.9473684210526314,
+      "grad_norm": 1.1035726070404053,
+      "learning_rate": 0.0002,
+      "loss": 0.6849,
+      "step": 600
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.127807378768921,
+      "eval_runtime": 7.9307,
+      "eval_samples_per_second": 13.744,
+      "eval_steps_per_second": 1.765,
+      "step": 608
+    },
+    {
+      "epoch": 4.0131578947368425,
+      "grad_norm": 1.0415319204330444,
+      "learning_rate": 0.0002,
+      "loss": 0.6693,
+      "step": 610
+    },
+    {
+      "epoch": 4.078947368421052,
+      "grad_norm": 1.201734185218811,
+      "learning_rate": 0.0002,
+      "loss": 0.5325,
+      "step": 620
+    },
+    {
+      "epoch": 4.144736842105263,
+      "grad_norm": 1.2380131483078003,
+      "learning_rate": 0.0002,
+      "loss": 0.5121,
+      "step": 630
+    },
+    {
+      "epoch": 4.2105263157894735,
+      "grad_norm": 1.3034303188323975,
+      "learning_rate": 0.0002,
+      "loss": 0.5145,
+      "step": 640
+    },
+    {
+      "epoch": 4.276315789473684,
+      "grad_norm": 1.878430962562561,
+      "learning_rate": 0.0002,
+      "loss": 0.4844,
+      "step": 650
+    },
+    {
+      "epoch": 4.342105263157895,
+      "grad_norm": 1.3213163614273071,
+      "learning_rate": 0.0002,
+      "loss": 0.5297,
+      "step": 660
+    },
+    {
+      "epoch": 4.407894736842105,
+      "grad_norm": 1.1307642459869385,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 670
+    },
+    {
+      "epoch": 4.473684210526316,
+      "grad_norm": 0.9877312183380127,
+      "learning_rate": 0.0002,
+      "loss": 0.49,
+      "step": 680
+    },
+    {
+      "epoch": 4.5394736842105265,
+      "grad_norm": 1.163973093032837,
+      "learning_rate": 0.0002,
+      "loss": 0.5253,
+      "step": 690
+    },
+    {
+      "epoch": 4.605263157894737,
+      "grad_norm": 1.3219846487045288,
+      "learning_rate": 0.0002,
+      "loss": 0.5509,
+      "step": 700
+    },
+    {
+      "epoch": 4.671052631578947,
+      "grad_norm": 1.2262274026870728,
+      "learning_rate": 0.0002,
+      "loss": 0.517,
+      "step": 710
+    },
+    {
+      "epoch": 4.7368421052631575,
+      "grad_norm": 1.1390860080718994,
+      "learning_rate": 0.0002,
+      "loss": 0.5347,
+      "step": 720
+    },
+    {
+      "epoch": 4.802631578947368,
+      "grad_norm": 1.196449875831604,
+      "learning_rate": 0.0002,
+      "loss": 0.5581,
+      "step": 730
+    },
+    {
+      "epoch": 4.868421052631579,
+      "grad_norm": 1.2404807806015015,
+      "learning_rate": 0.0002,
+      "loss": 0.5556,
+      "step": 740
+    },
+    {
+      "epoch": 4.934210526315789,
+      "grad_norm": 1.1726386547088623,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 750
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.3372385501861572,
+      "learning_rate": 0.0002,
+      "loss": 0.5315,
+      "step": 760
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.2448275089263916,
+      "eval_runtime": 7.9296,
+      "eval_samples_per_second": 13.746,
+      "eval_steps_per_second": 1.766,
+      "step": 760
+    },
+    {
+      "epoch": 5.065789473684211,
+      "grad_norm": 1.5582785606384277,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 770
+    },
+    {
+      "epoch": 5.131578947368421,
+      "grad_norm": 1.6396721601486206,
+      "learning_rate": 0.0002,
+      "loss": 0.4033,
+      "step": 780
+    },
+    {
+      "epoch": 5.197368421052632,
+      "grad_norm": 1.2244197130203247,
+      "learning_rate": 0.0002,
+      "loss": 0.4039,
+      "step": 790
+    },
+    {
+      "epoch": 5.2631578947368425,
+      "grad_norm": 1.3662641048431396,
+      "learning_rate": 0.0002,
+      "loss": 0.4262,
+      "step": 800
+    },
+    {
+      "epoch": 5.328947368421053,
+      "grad_norm": 1.405532956123352,
+      "learning_rate": 0.0002,
+      "loss": 0.4164,
+      "step": 810
+    },
+    {
+      "epoch": 5.394736842105263,
+      "grad_norm": 1.2695107460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.4131,
+      "step": 820
+    },
+    {
+      "epoch": 5.4605263157894735,
+      "grad_norm": 1.6382331848144531,
+      "learning_rate": 0.0002,
+      "loss": 0.4022,
+      "step": 830
+    },
+    {
+      "epoch": 5.526315789473684,
+      "grad_norm": 1.684364676475525,
+      "learning_rate": 0.0002,
+      "loss": 0.4214,
+      "step": 840
+    },
+    {
+      "epoch": 5.592105263157895,
+      "grad_norm": 1.2763781547546387,
+      "learning_rate": 0.0002,
+      "loss": 0.4109,
+      "step": 850
+    },
+    {
+      "epoch": 5.657894736842105,
+      "grad_norm": 1.429623007774353,
+      "learning_rate": 0.0002,
+      "loss": 0.4258,
+      "step": 860
+    },
+    {
+      "epoch": 5.723684210526316,
+      "grad_norm": 1.9069653749465942,
+      "learning_rate": 0.0002,
+      "loss": 0.4268,
+      "step": 870
+    },
+    {
+      "epoch": 5.7894736842105265,
+      "grad_norm": 1.349945068359375,
+      "learning_rate": 0.0002,
+      "loss": 0.4372,
+      "step": 880
+    },
+    {
+      "epoch": 5.855263157894737,
+      "grad_norm": 1.3684793710708618,
+      "learning_rate": 0.0002,
+      "loss": 0.4354,
+      "step": 890
+    },
+    {
+      "epoch": 5.921052631578947,
+      "grad_norm": 1.5587575435638428,
+      "learning_rate": 0.0002,
+      "loss": 0.4393,
+      "step": 900
+    },
+    {
+      "epoch": 5.9868421052631575,
+      "grad_norm": 1.3056254386901855,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.3288270235061646,
+      "eval_runtime": 7.9333,
+      "eval_samples_per_second": 13.739,
+      "eval_steps_per_second": 1.765,
+      "step": 912
+    },
+    {
+      "epoch": 6.052631578947368,
+      "grad_norm": 1.3509465456008911,
+      "learning_rate": 0.0002,
+      "loss": 0.3445,
+      "step": 920
+    },
+    {
+      "epoch": 6.118421052631579,
+      "grad_norm": 1.5550519227981567,
+      "learning_rate": 0.0002,
+      "loss": 0.3037,
+      "step": 930
+    },
+    {
+      "epoch": 6.184210526315789,
+      "grad_norm": 1.520915150642395,
+      "learning_rate": 0.0002,
+      "loss": 0.3319,
+      "step": 940
+    },
+    {
+      "epoch": 6.25,
+      "grad_norm": 1.2581181526184082,
+      "learning_rate": 0.0002,
+      "loss": 0.3196,
+      "step": 950
+    },
+    {
+      "epoch": 6.315789473684211,
+      "grad_norm": 1.5343317985534668,
+      "learning_rate": 0.0002,
+      "loss": 0.3132,
+      "step": 960
+    },
+    {
+      "epoch": 6.381578947368421,
+      "grad_norm": 1.4771231412887573,
+      "learning_rate": 0.0002,
+      "loss": 0.3218,
+      "step": 970
+    },
+    {
+      "epoch": 6.447368421052632,
+      "grad_norm": 1.1770271062850952,
+      "learning_rate": 0.0002,
+      "loss": 0.3511,
+      "step": 980
+    },
+    {
+      "epoch": 6.5131578947368425,
+      "grad_norm": 1.4069170951843262,
+      "learning_rate": 0.0002,
+      "loss": 0.3526,
+      "step": 990
+    },
+    {
+      "epoch": 6.578947368421053,
+      "grad_norm": 1.667536973953247,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 1000
+    },
+    {
+      "epoch": 6.644736842105263,
+      "grad_norm": 1.4396395683288574,
+      "learning_rate": 0.0002,
+      "loss": 0.3582,
+      "step": 1010
+    },
+    {
+      "epoch": 6.7105263157894735,
+      "grad_norm": 1.5290015935897827,
+      "learning_rate": 0.0002,
+      "loss": 0.336,
+      "step": 1020
+    },
+    {
+      "epoch": 6.776315789473684,
+      "grad_norm": 1.2737603187561035,
+      "learning_rate": 0.0002,
+      "loss": 0.366,
+      "step": 1030
+    },
+    {
+      "epoch": 6.842105263157895,
+      "grad_norm": 2.4872493743896484,
+      "learning_rate": 0.0002,
+      "loss": 0.394,
+      "step": 1040
+    },
+    {
+      "epoch": 6.907894736842105,
+      "grad_norm": 1.3118231296539307,
+      "learning_rate": 0.0002,
+      "loss": 0.3602,
+      "step": 1050
+    },
+    {
+      "epoch": 6.973684210526316,
+      "grad_norm": 1.5545233488082886,
+      "learning_rate": 0.0002,
+      "loss": 0.3729,
+      "step": 1060
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.4201300144195557,
+      "eval_runtime": 7.9357,
+      "eval_samples_per_second": 13.735,
+      "eval_steps_per_second": 1.764,
+      "step": 1064
+    },
+    {
+      "epoch": 7.0394736842105265,
+      "grad_norm": 1.6730996370315552,
+      "learning_rate": 0.0002,
+      "loss": 0.3081,
+      "step": 1070
+    },
+    {
+      "epoch": 7.105263157894737,
+      "grad_norm": 1.8236976861953735,
+      "learning_rate": 0.0002,
+      "loss": 0.2635,
+      "step": 1080
+    },
+    {
+      "epoch": 7.171052631578948,
+      "grad_norm": 1.6556929349899292,
+      "learning_rate": 0.0002,
+      "loss": 0.2494,
+      "step": 1090
+    },
+    {
+      "epoch": 7.2368421052631575,
+      "grad_norm": 1.2505947351455688,
+      "learning_rate": 0.0002,
+      "loss": 0.2766,
+      "step": 1100
+    },
+    {
+      "epoch": 7.302631578947368,
+      "grad_norm": 1.680189847946167,
+      "learning_rate": 0.0002,
+      "loss": 0.2906,
+      "step": 1110
+    },
+    {
+      "epoch": 7.368421052631579,
+      "grad_norm": 1.6516188383102417,
+      "learning_rate": 0.0002,
+      "loss": 0.2836,
+      "step": 1120
+    },
+    {
+      "epoch": 7.434210526315789,
+      "grad_norm": 1.3946540355682373,
+      "learning_rate": 0.0002,
+      "loss": 0.284,
+      "step": 1130
+    },
+    {
+      "epoch": 7.5,
+      "grad_norm": 1.2659136056900024,
+      "learning_rate": 0.0002,
+      "loss": 0.2704,
+      "step": 1140
+    },
+    {
+      "epoch": 7.565789473684211,
+      "grad_norm": 1.4051620960235596,
+      "learning_rate": 0.0002,
+      "loss": 0.2957,
+      "step": 1150
+    },
+    {
+      "epoch": 7.631578947368421,
+      "grad_norm": 1.9382350444793701,
+      "learning_rate": 0.0002,
+      "loss": 0.2891,
+      "step": 1160
+    },
+    {
+      "epoch": 7.697368421052632,
+      "grad_norm": 1.6188443899154663,
+      "learning_rate": 0.0002,
+      "loss": 0.2916,
+      "step": 1170
+    },
+    {
+      "epoch": 7.7631578947368425,
+      "grad_norm": 1.3274911642074585,
+      "learning_rate": 0.0002,
+      "loss": 0.2968,
+      "step": 1180
+    },
+    {
+      "epoch": 7.828947368421053,
+      "grad_norm": 1.6927156448364258,
+      "learning_rate": 0.0002,
+      "loss": 0.3129,
+      "step": 1190
+    },
+    {
+      "epoch": 7.894736842105263,
+      "grad_norm": 1.1732126474380493,
+      "learning_rate": 0.0002,
+      "loss": 0.2926,
+      "step": 1200
+    },
+    {
+      "epoch": 7.9605263157894735,
+      "grad_norm": 2.133342742919922,
+      "learning_rate": 0.0002,
+      "loss": 0.3112,
+      "step": 1210
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 1.5037198066711426,
+      "eval_runtime": 7.96,
+      "eval_samples_per_second": 13.693,
+      "eval_steps_per_second": 1.759,
+      "step": 1216
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1216,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.33516676759552e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-1216/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945033f5c566fb4e041fb4438a4cb0536a22ba21ea95f2bf808c8aca0e5a389f
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:faf572c91989e7aa495633308761065e76dc61d581946d195d8b0c02f361d733
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:779aecbe47f5bf045c81eb3425f83bacf5004f2edcb51d84228337087a903743
+size 55532538

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b74ae17b67703cc2ebc133ceef844b0bc0c6ab8f7cc6a73f375d1c587b133375
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c8fe9ced76d08d9cdb6b7d9de61baaa3fb4c94682a31ef669ca5c080963e14a
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/trainer_state.json

@@ -0,0 +1,146 @@
+{
+  "best_metric": 1.0424011945724487,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152",
+  "epoch": 1.0,
+  "eval_steps": 10,
+  "global_step": 152,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06578947368421052,
+      "grad_norm": 0.8710125088691711,
+      "learning_rate": 0.0002,
+      "loss": 1.5609,
+      "step": 10
+    },
+    {
+      "epoch": 0.13157894736842105,
+      "grad_norm": 1.1956019401550293,
+      "learning_rate": 0.0002,
+      "loss": 1.281,
+      "step": 20
+    },
+    {
+      "epoch": 0.19736842105263158,
+      "grad_norm": 0.7674543261528015,
+      "learning_rate": 0.0002,
+      "loss": 1.2364,
+      "step": 30
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.9102759957313538,
+      "learning_rate": 0.0002,
+      "loss": 1.1964,
+      "step": 40
+    },
+    {
+      "epoch": 0.32894736842105265,
+      "grad_norm": 0.6881781816482544,
+      "learning_rate": 0.0002,
+      "loss": 1.1633,
+      "step": 50
+    },
+    {
+      "epoch": 0.39473684210526316,
+      "grad_norm": 0.8366474509239197,
+      "learning_rate": 0.0002,
+      "loss": 1.131,
+      "step": 60
+    },
+    {
+      "epoch": 0.4605263157894737,
+      "grad_norm": 1.1925188302993774,
+      "learning_rate": 0.0002,
+      "loss": 1.0874,
+      "step": 70
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5625673532485962,
+      "learning_rate": 0.0002,
+      "loss": 1.0863,
+      "step": 80
+    },
+    {
+      "epoch": 0.5921052631578947,
+      "grad_norm": 0.7259140610694885,
+      "learning_rate": 0.0002,
+      "loss": 1.0992,
+      "step": 90
+    },
+    {
+      "epoch": 0.6578947368421053,
+      "grad_norm": 0.7536493539810181,
+      "learning_rate": 0.0002,
+      "loss": 1.0855,
+      "step": 100
+    },
+    {
+      "epoch": 0.7236842105263158,
+      "grad_norm": 0.5873591303825378,
+      "learning_rate": 0.0002,
+      "loss": 1.0775,
+      "step": 110
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.6995271444320679,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 120
+    },
+    {
+      "epoch": 0.8552631578947368,
+      "grad_norm": 0.6832848787307739,
+      "learning_rate": 0.0002,
+      "loss": 1.1105,
+      "step": 130
+    },
+    {
+      "epoch": 0.9210526315789473,
+      "grad_norm": 0.605141818523407,
+      "learning_rate": 0.0002,
+      "loss": 1.0688,
+      "step": 140
+    },
+    {
+      "epoch": 0.9868421052631579,
+      "grad_norm": 0.6675612330436707,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 150
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.0424011945724487,
+      "eval_runtime": 7.9344,
+      "eval_samples_per_second": 13.738,
+      "eval_steps_per_second": 1.764,
+      "step": 152
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1216,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6668958459494400.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-152/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945033f5c566fb4e041fb4438a4cb0536a22ba21ea95f2bf808c8aca0e5a389f
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de443465fea1ade0bb99dc6ba65afceed4e907ad6efaa20e7bcb905fa4f31fd7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f94898667461baf3c4d079d90db4fa49bba7949069eda142a809c9d543b61f40
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2054ec72978bf06f7c3729d2b397ceac5fe7da8c82fbe8d77a1df1dfadd042ec
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bff6badb2daf718cd29e469c61b4ddbe107ba8d1e3d9ef5089f5e33dd90e322
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304/trainer_state.json

@@ -0,0 +1,259 @@
+{
+  "best_metric": 1.0297880172729492,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-en-fpb_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.7-num-2170-sd-4/checkpoint-304",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 304,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.06578947368421052,
+      "grad_norm": 0.8710125088691711,
+      "learning_rate": 0.0002,
+      "loss": 1.5609,
+      "step": 10
+    },
+    {
+      "epoch": 0.13157894736842105,
+      "grad_norm": 1.1956019401550293,
+      "learning_rate": 0.0002,
+      "loss": 1.281,
+      "step": 20
+    },
+    {
+      "epoch": 0.19736842105263158,
+      "grad_norm": 0.7674543261528015,
+      "learning_rate": 0.0002,
+      "loss": 1.2364,
+      "step": 30
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.9102759957313538,
+      "learning_rate": 0.0002,
+      "loss": 1.1964,
+      "step": 40
+    },
+    {
+      "epoch": 0.32894736842105265,
+      "grad_norm": 0.6881781816482544,
+      "learning_rate": 0.0002,
+      "loss": 1.1633,
+      "step": 50
+    },
+    {
+      "epoch": 0.39473684210526316,
+      "grad_norm": 0.8366474509239197,
+      "learning_rate": 0.0002,
+      "loss": 1.131,
+      "step": 60
+    },
+    {
+      "epoch": 0.4605263157894737,
+      "grad_norm": 1.1925188302993774,
+      "learning_rate": 0.0002,
+      "loss": 1.0874,
+      "step": 70
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5625673532485962,
+      "learning_rate": 0.0002,
+      "loss": 1.0863,
+      "step": 80
+    },
+    {
+      "epoch": 0.5921052631578947,
+      "grad_norm": 0.7259140610694885,
+      "learning_rate": 0.0002,
+      "loss": 1.0992,
+      "step": 90
+    },
+    {
+      "epoch": 0.6578947368421053,
+      "grad_norm": 0.7536493539810181,
+      "learning_rate": 0.0002,
+      "loss": 1.0855,
+      "step": 100
+    },
+    {
+      "epoch": 0.7236842105263158,
+      "grad_norm": 0.5873591303825378,
+      "learning_rate": 0.0002,
+      "loss": 1.0775,
+      "step": 110
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.6995271444320679,
+      "learning_rate": 0.0002,
+      "loss": 1.1126,
+      "step": 120
+    },
+    {
+      "epoch": 0.8552631578947368,
+      "grad_norm": 0.6832848787307739,
+      "learning_rate": 0.0002,
+      "loss": 1.1105,
+      "step": 130
+    },
+    {
+      "epoch": 0.9210526315789473,
+      "grad_norm": 0.605141818523407,
+      "learning_rate": 0.0002,
+      "loss": 1.0688,
+      "step": 140
+    },
+    {
+      "epoch": 0.9868421052631579,
+      "grad_norm": 0.6675612330436707,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 150
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.0424011945724487,
+      "eval_runtime": 7.9344,
+      "eval_samples_per_second": 13.738,
+      "eval_steps_per_second": 1.764,
+      "step": 152
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5364972949028015,
+      "learning_rate": 0.0002,
+      "loss": 0.9909,
+      "step": 160
+    },
+    {
+      "epoch": 1.118421052631579,
+      "grad_norm": 0.6408411264419556,
+      "learning_rate": 0.0002,
+      "loss": 0.9368,
+      "step": 170
+    },
+    {
+      "epoch": 1.1842105263157894,
+      "grad_norm": 0.5882905125617981,
+      "learning_rate": 0.0002,
+      "loss": 0.9179,
+      "step": 180
+    },
+    {
+      "epoch": 1.25,
+      "grad_norm": 0.7419160008430481,
+      "learning_rate": 0.0002,
+      "loss": 1.0002,
+      "step": 190
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.6069955825805664,
+      "learning_rate": 0.0002,
+      "loss": 0.9192,
+      "step": 200
+    },
+    {
+      "epoch": 1.381578947368421,
+      "grad_norm": 0.5905187129974365,
+      "learning_rate": 0.0002,
+      "loss": 0.9572,
+      "step": 210
+    },
+    {
+      "epoch": 1.4473684210526316,
+      "grad_norm": 0.7193484902381897,
+      "learning_rate": 0.0002,
+      "loss": 0.9653,
+      "step": 220
+    },
+    {
+      "epoch": 1.513157894736842,
+      "grad_norm": 0.7149216532707214,
+      "learning_rate": 0.0002,
+      "loss": 1.0417,
+      "step": 230
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.6062077879905701,
+      "learning_rate": 0.0002,
+      "loss": 0.9429,
+      "step": 240
+    },
+    {
+      "epoch": 1.6447368421052633,
+      "grad_norm": 0.7012736797332764,
+      "learning_rate": 0.0002,
+      "loss": 0.9463,
+      "step": 250
+    },
+    {
+      "epoch": 1.7105263157894737,
+      "grad_norm": 0.6271148920059204,
+      "learning_rate": 0.0002,
+      "loss": 1.0061,
+      "step": 260
+    },
+    {
+      "epoch": 1.776315789473684,
+      "grad_norm": 0.8046498894691467,
+      "learning_rate": 0.0002,
+      "loss": 1.0064,
+      "step": 270
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.7960395812988281,
+      "learning_rate": 0.0002,
+      "loss": 0.9877,
+      "step": 280
+    },
+    {
+      "epoch": 1.9078947368421053,
+      "grad_norm": 0.6067329049110413,
+      "learning_rate": 0.0002,
+      "loss": 0.9817,
+      "step": 290
+    },
+    {
+      "epoch": 1.973684210526316,
+      "grad_norm": 0.722137451171875,
+      "learning_rate": 0.0002,
+      "loss": 0.9553,
+      "step": 300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0297880172729492,
+      "eval_runtime": 7.9354,
+      "eval_samples_per_second": 13.736,
+      "eval_steps_per_second": 1.764,
+      "step": 304
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1216,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.33379169189888e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}