Upload folder using huggingface_hub

README.md

@@ -0,0 +1,159 @@
+---
+license: llama2
+library_name: peft
+tags:
+- generated_from_trainer
+base_model: codellama/CodeLlama-34b-hf
+model-index:
+- name: maverick34b
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
+<details><summary>See axolotl config</summary>
+
+axolotl version: `0.4.0`
+```yaml
+base_model: codellama/CodeLlama-34b-hf
+model_type: LlamaForCausalLM
+tokenizer_type: CodeLlamaTokenizer
+is_llama_derived_model: true
+
+load_in_8bit: false
+load_in_4bit: true
+strict: false
+
+datasets:
+  - path: iamtarun/code_instructions_120k_alpaca
+    type: alpaca
+dataset_prepared_path:
+val_set_size: 0.05
+output_dir: ./maverick34b
+
+adapter: qlora
+lora_model_dir:
+
+sequence_len: 4096
+sample_packing: true
+pad_to_sequence_len: true
+
+lora_r: 32
+lora_alpha: 16
+lora_dropout: 0.05
+lora_target_modules:
+lora_target_linear: true
+lora_fan_in_fan_out:
+
+wandb_project:
+wandb_entity:
+wandb_watch:
+wandb_name:
+wandb_log_model:
+
+gradient_accumulation_steps: 4
+micro_batch_size: 2
+num_epochs: 4
+optimizer: paged_adamw_32bit
+lr_scheduler: cosine
+learning_rate: 0.0002
+
+train_on_inputs: false
+group_by_length: false
+bf16: auto
+fp16:
+tf32: false
+
+gradient_checkpointing: true
+early_stopping_patience:
+resume_from_checkpoint:
+local_rank:
+logging_steps: 1
+xformers_attention:
+flash_attention: true
+
+warmup_steps: 10
+evals_per_epoch: 4
+saves_per_epoch: 1
+debug:
+deepspeed:
+weight_decay: 0.0
+fsdp:
+fsdp_config:
+special_tokens:
+  bos_token: "<s>"
+  eos_token: "</s>"
+  unk_token: "<unk>"
+
+```
+
+</details><br>
+
+# maverick34b
+
+This model is a fine-tuned version of [codellama/CodeLlama-34b-hf](https://huggingface.co/codellama/CodeLlama-34b-hf) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3391
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 7
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 56
+- total_eval_batch_size: 14
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 0.5065        | 0.01  | 1    | 0.5089          |
+| 0.3477        | 0.25  | 29   | 0.3561          |
+| 0.3593        | 0.51  | 58   | 0.3461          |
+| 0.3329        | 0.76  | 87   | 0.3423          |
+| 0.3607        | 1.0   | 116  | 0.3404          |
+| 0.3336        | 1.26  | 145  | 0.3395          |
+| 0.3449        | 1.51  | 174  | 0.3386          |
+| 0.3187        | 1.77  | 203  | 0.3377          |
+| 0.3216        | 2.0   | 232  | 0.3371          |
+| 0.2961        | 2.26  | 261  | 0.3380          |
+| 0.3117        | 2.51  | 290  | 0.3381          |
+| 0.3207        | 2.77  | 319  | 0.3379          |
+| 0.3047        | 3.01  | 348  | 0.3376          |
+| 0.3096        | 3.26  | 377  | 0.3391          |
+| 0.3148        | 3.52  | 406  | 0.3391          |
+| 0.3116        | 3.77  | 435  | 0.3391          |
+
+
+### Framework versions
+
+- PEFT 0.8.2
+- Transformers 4.38.0.dev0
+- Pytorch 2.0.1+cu118
+- Datasets 2.17.0
+- Tokenizers 0.15.0

adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "codellama/CodeLlama-34b-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:031b1128c4a75c85ff0ea0314fdf324f9c5443537cef0fe5b3714a3ff48d05b9
+size 1920316885

added_tokens.json

@@ -0,0 +1,6 @@
+{
+  "▁<EOT>": 32003,
+  "▁<MID>": 32001,
+  "▁<PRE>": 32000,
+  "▁<SUF>": 32002
+}

checkpoint-113/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: codellama/CodeLlama-34b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-113/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "codellama/CodeLlama-34b-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-113/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31bac09b45c64d4a4385f8518f332e06718d8d448929fbc2b816613de241ea53
+size 1920164848

checkpoint-113/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:596e640b5ad2d862140ee761588aef9c7f9c75ff1bafeba632b2d74489698a73
+size 1743146781

checkpoint-113/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45d8b602c8660af2d817113bf055d8f453ed39ee5668751c783f72699c898d3f
+size 20663

checkpoint-113/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4508e7a4fb74ae0a0e24c98041cf8d391ea8fdb7f4174dcad3e33323fc1d069
+size 20663

checkpoint-113/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e95a5381db5d512375a6475f6fdd8906a41b6cb10f7e73f78cb5095e33d2e7f3
+size 20663

checkpoint-113/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a36823b3aa29b3a8c14c88c112853e5dd49f7e6f8fea3ab0a73018de49f36a5
+size 20663

checkpoint-113/rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5aeacfe9803f585796f59eae82000839a7469fbca516aecf8d8b0e9a78a242c
+size 20663

checkpoint-113/rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c5a6458864e33f66dc2c30e9e6077b55314be9b33e99e5b33cf8b30d167330e
+size 20663

checkpoint-113/rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d1778783a8d72022a8924c8234a1b6aa28a8e92cfa4f2bb95389003bc6da68d
+size 20663

checkpoint-113/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5c16e4c80361f85f40be2fdbaf90ba79fbd83617cee764a7b7d64dc0ce5ef56
+size 627

checkpoint-113/trainer_state.json

@@ -0,0 +1,731 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9934065934065934,
+  "eval_steps": 29,
+  "global_step": 113,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "learning_rate": 2e-05,
+      "loss": 0.5065,
+      "step": 1
+    },
+    {
+      "epoch": 0.01,
+      "eval_loss": 0.5089401602745056,
+      "eval_runtime": 34.8867,
+      "eval_samples_per_second": 174.68,
+      "eval_steps_per_second": 12.498,
+      "step": 1
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 4e-05,
+      "loss": 0.5189,
+      "step": 2
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 6e-05,
+      "loss": 0.4966,
+      "step": 3
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 8e-05,
+      "loss": 0.4891,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 0.0001,
+      "loss": 0.507,
+      "step": 5
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 0.00012,
+      "loss": 0.5071,
+      "step": 6
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 0.00014,
+      "loss": 0.4739,
+      "step": 7
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 0.00016,
+      "loss": 0.4721,
+      "step": 8
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 0.00018,
+      "loss": 0.4213,
+      "step": 9
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 0.0002,
+      "loss": 0.4056,
+      "step": 10
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 0.00019999747405795057,
+      "loss": 0.3989,
+      "step": 11
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019998989635940995,
+      "loss": 0.3843,
+      "step": 12
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019997726728719468,
+      "loss": 0.3771,
+      "step": 13
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 0.00019995958747931082,
+      "loss": 0.3775,
+      "step": 14
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 0.00019993685782892183,
+      "loss": 0.3778,
+      "step": 15
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 0.00019990907948430327,
+      "loss": 0.3786,
+      "step": 16
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 0.0001998762538487849,
+      "loss": 0.3669,
+      "step": 17
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 0.00019983838258067987,
+      "loss": 0.3819,
+      "step": 18
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 0.0001997954675932006,
+      "loss": 0.4043,
+      "step": 19
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019974751105436266,
+      "loss": 0.3503,
+      "step": 20
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019969451538687474,
+      "loss": 0.3753,
+      "step": 21
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 0.00019963648326801653,
+      "loss": 0.368,
+      "step": 22
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 0.00019957341762950344,
+      "loss": 0.3828,
+      "step": 23
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 0.00019950532165733845,
+      "loss": 0.3509,
+      "step": 24
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 0.00019943219879165115,
+      "loss": 0.381,
+      "step": 25
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 0.0001993540527265239,
+      "loss": 0.3638,
+      "step": 26
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 0.0001992708874098054,
+      "loss": 0.3645,
+      "step": 27
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019918270704291104,
+      "loss": 0.3466,
+      "step": 28
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019908951608061076,
+      "loss": 0.3477,
+      "step": 29
+    },
+    {
+      "epoch": 0.25,
+      "eval_loss": 0.35608404874801636,
+      "eval_runtime": 35.0585,
+      "eval_samples_per_second": 173.824,
+      "eval_steps_per_second": 12.436,
+      "step": 29
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 0.000198991319230804,
+      "loss": 0.3444,
+      "step": 30
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 0.0001988881214542817,
+      "loss": 0.3646,
+      "step": 31
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 0.00019877992796447601,
+      "loss": 0.3502,
+      "step": 32
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 0.00019866674422719665,
+      "loss": 0.3532,
+      "step": 33
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 0.00019854857596035476,
+      "loss": 0.3639,
+      "step": 34
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 0.0001984254291336743,
+      "loss": 0.3538,
+      "step": 35
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 0.0001982973099683902,
+      "loss": 0.3431,
+      "step": 36
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.00019816422493693417,
+      "loss": 0.3493,
+      "step": 37
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.0001980261807626078,
+      "loss": 0.3596,
+      "step": 38
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 0.00019788318441924275,
+      "loss": 0.3401,
+      "step": 39
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 0.00019773524313084856,
+      "loss": 0.3822,
+      "step": 40
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 0.00019758236437124769,
+      "loss": 0.3719,
+      "step": 41
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 0.00019742455586369784,
+      "loss": 0.3369,
+      "step": 42
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 0.0001972618255805019,
+      "loss": 0.341,
+      "step": 43
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 0.0001970941817426052,
+      "loss": 0.3481,
+      "step": 44
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019692163281918014,
+      "loss": 0.3615,
+      "step": 45
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019674418752719833,
+      "loss": 0.346,
+      "step": 46
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 0.00019656185483099024,
+      "loss": 0.3479,
+      "step": 47
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 0.00019637464394179239,
+      "loss": 0.3638,
+      "step": 48
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 0.00019618256431728194,
+      "loss": 0.3419,
+      "step": 49
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 0.0001959856256610988,
+      "loss": 0.3496,
+      "step": 50
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 0.00019578383792235573,
+      "loss": 0.3503,
+      "step": 51
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 0.0001955772112951354,
+      "loss": 0.3437,
+      "step": 52
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019536575621797547,
+      "loss": 0.3343,
+      "step": 53
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019514948337334142,
+      "loss": 0.3517,
+      "step": 54
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 0.0001949284036870867,
+      "loss": 0.343,
+      "step": 55
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 0.0001947025283279008,
+      "loss": 0.3479,
+      "step": 56
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 0.00019447186870674506,
+      "loss": 0.3237,
+      "step": 57
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 0.00019423643647627624,
+      "loss": 0.3593,
+      "step": 58
+    },
+    {
+      "epoch": 0.51,
+      "eval_loss": 0.3460778594017029,
+      "eval_runtime": 35.0517,
+      "eval_samples_per_second": 173.858,
+      "eval_steps_per_second": 12.439,
+      "step": 58
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 0.00019399624353025772,
+      "loss": 0.348,
+      "step": 59
+    },
+    {
+      "epoch": 0.53,
+      "learning_rate": 0.00019375130200295878,
+      "loss": 0.344,
+      "step": 60
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 0.0001935016242685415,
+      "loss": 0.3601,
+      "step": 61
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019324722294043558,
+      "loss": 0.3524,
+      "step": 62
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019298811087070132,
+      "loss": 0.3421,
+      "step": 63
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 0.00019272430114938017,
+      "loss": 0.3524,
+      "step": 64
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 0.00019245580710383343,
+      "loss": 0.3334,
+      "step": 65
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 0.00019218264229806917,
+      "loss": 0.3312,
+      "step": 66
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 0.00019190482053205673,
+      "loss": 0.344,
+      "step": 67
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.00019162235584102973,
+      "loss": 0.353,
+      "step": 68
+    },
+    {
+      "epoch": 0.61,
+      "learning_rate": 0.00019133526249477702,
+      "loss": 0.3473,
+      "step": 69
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019104355499692166,
+      "loss": 0.345,
+      "step": 70
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019074724808418835,
+      "loss": 0.348,
+      "step": 71
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 0.00019044635672565895,
+      "loss": 0.3333,
+      "step": 72
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 0.00019014089612201613,
+      "loss": 0.3398,
+      "step": 73
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 0.00018983088170477553,
+      "loss": 0.3424,
+      "step": 74
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 0.00018951632913550626,
+      "loss": 0.3279,
+      "step": 75
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 0.00018919725430503945,
+      "loss": 0.3243,
+      "step": 76
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 0.00018887367333266582,
+      "loss": 0.3439,
+      "step": 77
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.000188545602565321,
+      "loss": 0.3486,
+      "step": 78
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.00018821305857675996,
+      "loss": 0.3367,
+      "step": 79
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 0.00018787605816671953,
+      "loss": 0.3386,
+      "step": 80
+    },
+    {
+      "epoch": 0.71,
+      "learning_rate": 0.00018753461836006988,
+      "loss": 0.3574,
+      "step": 81
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 0.00018718875640595434,
+      "loss": 0.334,
+      "step": 82
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 0.00018683848977691785,
+      "loss": 0.3342,
+      "step": 83
+    },
+    {
+      "epoch": 0.74,
+      "learning_rate": 0.0001864838361680247,
+      "loss": 0.3693,
+      "step": 84
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 0.00018612481349596405,
+      "loss": 0.3422,
+      "step": 85
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018576143989814523,
+      "loss": 0.3346,
+      "step": 86
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018539373373178123,
+      "loss": 0.3329,
+      "step": 87
+    },
+    {
+      "epoch": 0.76,
+      "eval_loss": 0.3422779440879822,
+      "eval_runtime": 35.1197,
+      "eval_samples_per_second": 173.521,
+      "eval_steps_per_second": 12.415,
+      "step": 87
+    },
+    {
+      "epoch": 0.77,
+      "learning_rate": 0.00018502171357296144,
+      "loss": 0.3485,
+      "step": 88
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 0.00018464539821571303,
+      "loss": 0.3383,
+      "step": 89
+    },
+    {
+      "epoch": 0.79,
+      "learning_rate": 0.00018426480667105177,
+      "loss": 0.3434,
+      "step": 90
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 0.00018387995816602137,
+      "loss": 0.3205,
+      "step": 91
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 0.00018349087214272224,
+      "loss": 0.3522,
+      "step": 92
+    },
+    {
+      "epoch": 0.82,
+      "learning_rate": 0.0001830975682573293,
+      "loss": 0.3416,
+      "step": 93
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 0.00018270006637909907,
+      "loss": 0.3468,
+      "step": 94
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.00018229838658936564,
+      "loss": 0.3407,
+      "step": 95
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.0001818925491805265,
+      "loss": 0.3198,
+      "step": 96
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 0.00018148257465501716,
+      "loss": 0.3369,
+      "step": 97
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 0.0001810684837242755,
+      "loss": 0.3371,
+      "step": 98
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 0.00018065029730769534,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 0.00018022803653156984,
+      "loss": 0.3418,
+      "step": 100
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 0.000179801722728024,
+      "loss": 0.346,
+      "step": 101
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 0.00017937137743393694,
+      "loss": 0.3385,
+      "step": 102
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017893702238985432,
+      "loss": 0.3343,
+      "step": 103
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017849867953888948,
+      "loss": 0.3146,
+      "step": 104
+    },
+    {
+      "epoch": 0.92,
+      "learning_rate": 0.00017805637102561516,
+      "loss": 0.3662,
+      "step": 105
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 0.00017761011919494488,
+      "loss": 0.3481,
+      "step": 106
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 0.00017715994659100388,
+      "loss": 0.3447,
+      "step": 107
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 0.00017670587595599034,
+      "loss": 0.3379,
+      "step": 108
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 0.0001762479302290265,
+      "loss": 0.3388,
+      "step": 109
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 0.0001757861325449997,
+      "loss": 0.3497,
+      "step": 110
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.0001753205062333937,
+      "loss": 0.3539,
+      "step": 111
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.00017485107481711012,
+      "loss": 0.3571,
+      "step": 112
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 0.00017437786201128004,
+      "loss": 0.3496,
+      "step": 113
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 452,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 113,
+  "total_flos": 5.240873872315646e+18,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-113/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66dc6af509e7f51c6a244a8a9af44ad9044f2ab26d51bec3236dad93ab4ef46c
+size 4923

checkpoint-226/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: codellama/CodeLlama-34b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-226/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "codellama/CodeLlama-34b-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-226/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:203f2bfbd6affec6966260d5c95f0daab5bf232780d35759e10a7450d58d7ff6
+size 1920164848

checkpoint-226/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72b0ff5951effac1c7d78bd229f9948749eefd3a6ad9c20b19ca66634a577d3c
+size 1743146781

checkpoint-226/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fcc6059a78d85b7d5ecff973d5fdb61e1cc300ca1aa3d5a3604ce83a1fce515
+size 20663

checkpoint-226/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85ebe23981e0d536e50b83ff9534e99705328e53886916a318049d2315d8fa62
+size 20663

checkpoint-226/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b46b7cf672f3b941a9cca2a125b291575fb170699e065ebff9245c9b8e7a4c24
+size 20663

checkpoint-226/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6a7b6e0beaf804e994b6c9d93e7e82d9a3385a654928ad19458aa14baffe17c
+size 20663

checkpoint-226/rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4099d385ac4820de03fa011c6af31443e309ac69a2a1583dcefe9258ad9d12e5
+size 20663

checkpoint-226/rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4aad4ef662803b511027c8e879dd335422e7e3580eb6ec820135c9808fc6e87
+size 20663

checkpoint-226/rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af51875441abd7c59ceb0edaa1ea2dc18e0d83cdcd70fe0950f926e48ab3b8ae
+size 20663

checkpoint-226/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cff18ed392e2ba35f469da7820c5ea719917ff9bf1638273dae98c09921a2a6
+size 627

checkpoint-226/trainer_state.json

@@ -0,0 +1,1441 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.9692307692307693,
+  "eval_steps": 29,
+  "global_step": 226,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "learning_rate": 2e-05,
+      "loss": 0.5065,
+      "step": 1
+    },
+    {
+      "epoch": 0.01,
+      "eval_loss": 0.5089401602745056,
+      "eval_runtime": 34.8867,
+      "eval_samples_per_second": 174.68,
+      "eval_steps_per_second": 12.498,
+      "step": 1
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 4e-05,
+      "loss": 0.5189,
+      "step": 2
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 6e-05,
+      "loss": 0.4966,
+      "step": 3
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 8e-05,
+      "loss": 0.4891,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 0.0001,
+      "loss": 0.507,
+      "step": 5
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 0.00012,
+      "loss": 0.5071,
+      "step": 6
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 0.00014,
+      "loss": 0.4739,
+      "step": 7
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 0.00016,
+      "loss": 0.4721,
+      "step": 8
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 0.00018,
+      "loss": 0.4213,
+      "step": 9
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 0.0002,
+      "loss": 0.4056,
+      "step": 10
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 0.00019999747405795057,
+      "loss": 0.3989,
+      "step": 11
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019998989635940995,
+      "loss": 0.3843,
+      "step": 12
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019997726728719468,
+      "loss": 0.3771,
+      "step": 13
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 0.00019995958747931082,
+      "loss": 0.3775,
+      "step": 14
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 0.00019993685782892183,
+      "loss": 0.3778,
+      "step": 15
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 0.00019990907948430327,
+      "loss": 0.3786,
+      "step": 16
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 0.0001998762538487849,
+      "loss": 0.3669,
+      "step": 17
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 0.00019983838258067987,
+      "loss": 0.3819,
+      "step": 18
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 0.0001997954675932006,
+      "loss": 0.4043,
+      "step": 19
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019974751105436266,
+      "loss": 0.3503,
+      "step": 20
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019969451538687474,
+      "loss": 0.3753,
+      "step": 21
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 0.00019963648326801653,
+      "loss": 0.368,
+      "step": 22
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 0.00019957341762950344,
+      "loss": 0.3828,
+      "step": 23
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 0.00019950532165733845,
+      "loss": 0.3509,
+      "step": 24
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 0.00019943219879165115,
+      "loss": 0.381,
+      "step": 25
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 0.0001993540527265239,
+      "loss": 0.3638,
+      "step": 26
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 0.0001992708874098054,
+      "loss": 0.3645,
+      "step": 27
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019918270704291104,
+      "loss": 0.3466,
+      "step": 28
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019908951608061076,
+      "loss": 0.3477,
+      "step": 29
+    },
+    {
+      "epoch": 0.25,
+      "eval_loss": 0.35608404874801636,
+      "eval_runtime": 35.0585,
+      "eval_samples_per_second": 173.824,
+      "eval_steps_per_second": 12.436,
+      "step": 29
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 0.000198991319230804,
+      "loss": 0.3444,
+      "step": 30
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 0.0001988881214542817,
+      "loss": 0.3646,
+      "step": 31
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 0.00019877992796447601,
+      "loss": 0.3502,
+      "step": 32
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 0.00019866674422719665,
+      "loss": 0.3532,
+      "step": 33
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 0.00019854857596035476,
+      "loss": 0.3639,
+      "step": 34
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 0.0001984254291336743,
+      "loss": 0.3538,
+      "step": 35
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 0.0001982973099683902,
+      "loss": 0.3431,
+      "step": 36
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.00019816422493693417,
+      "loss": 0.3493,
+      "step": 37
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.0001980261807626078,
+      "loss": 0.3596,
+      "step": 38
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 0.00019788318441924275,
+      "loss": 0.3401,
+      "step": 39
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 0.00019773524313084856,
+      "loss": 0.3822,
+      "step": 40
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 0.00019758236437124769,
+      "loss": 0.3719,
+      "step": 41
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 0.00019742455586369784,
+      "loss": 0.3369,
+      "step": 42
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 0.0001972618255805019,
+      "loss": 0.341,
+      "step": 43
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 0.0001970941817426052,
+      "loss": 0.3481,
+      "step": 44
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019692163281918014,
+      "loss": 0.3615,
+      "step": 45
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019674418752719833,
+      "loss": 0.346,
+      "step": 46
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 0.00019656185483099024,
+      "loss": 0.3479,
+      "step": 47
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 0.00019637464394179239,
+      "loss": 0.3638,
+      "step": 48
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 0.00019618256431728194,
+      "loss": 0.3419,
+      "step": 49
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 0.0001959856256610988,
+      "loss": 0.3496,
+      "step": 50
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 0.00019578383792235573,
+      "loss": 0.3503,
+      "step": 51
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 0.0001955772112951354,
+      "loss": 0.3437,
+      "step": 52
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019536575621797547,
+      "loss": 0.3343,
+      "step": 53
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019514948337334142,
+      "loss": 0.3517,
+      "step": 54
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 0.0001949284036870867,
+      "loss": 0.343,
+      "step": 55
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 0.0001947025283279008,
+      "loss": 0.3479,
+      "step": 56
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 0.00019447186870674506,
+      "loss": 0.3237,
+      "step": 57
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 0.00019423643647627624,
+      "loss": 0.3593,
+      "step": 58
+    },
+    {
+      "epoch": 0.51,
+      "eval_loss": 0.3460778594017029,
+      "eval_runtime": 35.0517,
+      "eval_samples_per_second": 173.858,
+      "eval_steps_per_second": 12.439,
+      "step": 58
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 0.00019399624353025772,
+      "loss": 0.348,
+      "step": 59
+    },
+    {
+      "epoch": 0.53,
+      "learning_rate": 0.00019375130200295878,
+      "loss": 0.344,
+      "step": 60
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 0.0001935016242685415,
+      "loss": 0.3601,
+      "step": 61
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019324722294043558,
+      "loss": 0.3524,
+      "step": 62
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019298811087070132,
+      "loss": 0.3421,
+      "step": 63
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 0.00019272430114938017,
+      "loss": 0.3524,
+      "step": 64
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 0.00019245580710383343,
+      "loss": 0.3334,
+      "step": 65
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 0.00019218264229806917,
+      "loss": 0.3312,
+      "step": 66
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 0.00019190482053205673,
+      "loss": 0.344,
+      "step": 67
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.00019162235584102973,
+      "loss": 0.353,
+      "step": 68
+    },
+    {
+      "epoch": 0.61,
+      "learning_rate": 0.00019133526249477702,
+      "loss": 0.3473,
+      "step": 69
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019104355499692166,
+      "loss": 0.345,
+      "step": 70
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019074724808418835,
+      "loss": 0.348,
+      "step": 71
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 0.00019044635672565895,
+      "loss": 0.3333,
+      "step": 72
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 0.00019014089612201613,
+      "loss": 0.3398,
+      "step": 73
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 0.00018983088170477553,
+      "loss": 0.3424,
+      "step": 74
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 0.00018951632913550626,
+      "loss": 0.3279,
+      "step": 75
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 0.00018919725430503945,
+      "loss": 0.3243,
+      "step": 76
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 0.00018887367333266582,
+      "loss": 0.3439,
+      "step": 77
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.000188545602565321,
+      "loss": 0.3486,
+      "step": 78
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.00018821305857675996,
+      "loss": 0.3367,
+      "step": 79
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 0.00018787605816671953,
+      "loss": 0.3386,
+      "step": 80
+    },
+    {
+      "epoch": 0.71,
+      "learning_rate": 0.00018753461836006988,
+      "loss": 0.3574,
+      "step": 81
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 0.00018718875640595434,
+      "loss": 0.334,
+      "step": 82
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 0.00018683848977691785,
+      "loss": 0.3342,
+      "step": 83
+    },
+    {
+      "epoch": 0.74,
+      "learning_rate": 0.0001864838361680247,
+      "loss": 0.3693,
+      "step": 84
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 0.00018612481349596405,
+      "loss": 0.3422,
+      "step": 85
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018576143989814523,
+      "loss": 0.3346,
+      "step": 86
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018539373373178123,
+      "loss": 0.3329,
+      "step": 87
+    },
+    {
+      "epoch": 0.76,
+      "eval_loss": 0.3422779440879822,
+      "eval_runtime": 35.1197,
+      "eval_samples_per_second": 173.521,
+      "eval_steps_per_second": 12.415,
+      "step": 87
+    },
+    {
+      "epoch": 0.77,
+      "learning_rate": 0.00018502171357296144,
+      "loss": 0.3485,
+      "step": 88
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 0.00018464539821571303,
+      "loss": 0.3383,
+      "step": 89
+    },
+    {
+      "epoch": 0.79,
+      "learning_rate": 0.00018426480667105177,
+      "loss": 0.3434,
+      "step": 90
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 0.00018387995816602137,
+      "loss": 0.3205,
+      "step": 91
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 0.00018349087214272224,
+      "loss": 0.3522,
+      "step": 92
+    },
+    {
+      "epoch": 0.82,
+      "learning_rate": 0.0001830975682573293,
+      "loss": 0.3416,
+      "step": 93
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 0.00018270006637909907,
+      "loss": 0.3468,
+      "step": 94
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.00018229838658936564,
+      "loss": 0.3407,
+      "step": 95
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.0001818925491805265,
+      "loss": 0.3198,
+      "step": 96
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 0.00018148257465501716,
+      "loss": 0.3369,
+      "step": 97
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 0.0001810684837242755,
+      "loss": 0.3371,
+      "step": 98
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 0.00018065029730769534,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 0.00018022803653156984,
+      "loss": 0.3418,
+      "step": 100
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 0.000179801722728024,
+      "loss": 0.346,
+      "step": 101
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 0.00017937137743393694,
+      "loss": 0.3385,
+      "step": 102
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017893702238985432,
+      "loss": 0.3343,
+      "step": 103
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017849867953888948,
+      "loss": 0.3146,
+      "step": 104
+    },
+    {
+      "epoch": 0.92,
+      "learning_rate": 0.00017805637102561516,
+      "loss": 0.3662,
+      "step": 105
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 0.00017761011919494488,
+      "loss": 0.3481,
+      "step": 106
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 0.00017715994659100388,
+      "loss": 0.3447,
+      "step": 107
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 0.00017670587595599034,
+      "loss": 0.3379,
+      "step": 108
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 0.0001762479302290265,
+      "loss": 0.3388,
+      "step": 109
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 0.0001757861325449997,
+      "loss": 0.3497,
+      "step": 110
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.0001753205062333937,
+      "loss": 0.3539,
+      "step": 111
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.00017485107481711012,
+      "loss": 0.3571,
+      "step": 112
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 0.00017437786201128004,
+      "loss": 0.3496,
+      "step": 113
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00017390089172206592,
+      "loss": 0.3436,
+      "step": 114
+    },
+    {
+      "epoch": 1.01,
+      "learning_rate": 0.000173420188045454,
+      "loss": 0.3434,
+      "step": 115
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00017293577526603682,
+      "loss": 0.3607,
+      "step": 116
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.3404248356819153,
+      "eval_runtime": 35.0435,
+      "eval_samples_per_second": 173.898,
+      "eval_steps_per_second": 12.442,
+      "step": 116
+    },
+    {
+      "epoch": 1.01,
+      "learning_rate": 0.0001724476778557866,
+      "loss": 0.3363,
+      "step": 117
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 0.00017195592047281882,
+      "loss": 0.3283,
+      "step": 118
+    },
+    {
+      "epoch": 1.03,
+      "learning_rate": 0.00017146052796014646,
+      "loss": 0.3293,
+      "step": 119
+    },
+    {
+      "epoch": 1.04,
+      "learning_rate": 0.00017096152534442513,
+      "loss": 0.3485,
+      "step": 120
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 0.0001704589378346886,
+      "loss": 0.3315,
+      "step": 121
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 0.0001699527908210754,
+      "loss": 0.3686,
+      "step": 122
+    },
+    {
+      "epoch": 1.06,
+      "learning_rate": 0.00016944310987354597,
+      "loss": 0.3384,
+      "step": 123
+    },
+    {
+      "epoch": 1.07,
+      "learning_rate": 0.0001689299207405911,
+      "loss": 0.3432,
+      "step": 124
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 0.00016841324934793094,
+      "loss": 0.3323,
+      "step": 125
+    },
+    {
+      "epoch": 1.09,
+      "learning_rate": 0.00016789312179720547,
+      "loss": 0.3325,
+      "step": 126
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 0.00016736956436465573,
+      "loss": 0.3464,
+      "step": 127
+    },
+    {
+      "epoch": 1.11,
+      "learning_rate": 0.00016684260349979636,
+      "loss": 0.3344,
+      "step": 128
+    },
+    {
+      "epoch": 1.12,
+      "learning_rate": 0.00016631226582407952,
+      "loss": 0.3319,
+      "step": 129
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 0.00016577857812954993,
+      "loss": 0.3385,
+      "step": 130
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 0.00016524156737749132,
+      "loss": 0.3166,
+      "step": 131
+    },
+    {
+      "epoch": 1.14,
+      "learning_rate": 0.00016470126069706456,
+      "loss": 0.3214,
+      "step": 132
+    },
+    {
+      "epoch": 1.15,
+      "learning_rate": 0.00016415768538393687,
+      "loss": 0.3101,
+      "step": 133
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 0.00016361086889890306,
+      "loss": 0.3334,
+      "step": 134
+    },
+    {
+      "epoch": 1.17,
+      "learning_rate": 0.0001630608388664982,
+      "loss": 0.3094,
+      "step": 135
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 0.00016250762307360207,
+      "loss": 0.327,
+      "step": 136
+    },
+    {
+      "epoch": 1.19,
+      "learning_rate": 0.00016195124946803525,
+      "loss": 0.3512,
+      "step": 137
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0001613917461571475,
+      "loss": 0.3329,
+      "step": 138
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0001608291414063977,
+      "loss": 0.342,
+      "step": 139
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 0.00016026346363792567,
+      "loss": 0.3488,
+      "step": 140
+    },
+    {
+      "epoch": 1.22,
+      "learning_rate": 0.00015969474142911666,
+      "loss": 0.3385,
+      "step": 141
+    },
+    {
+      "epoch": 1.23,
+      "learning_rate": 0.0001591230035111576,
+      "loss": 0.3207,
+      "step": 142
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 0.00015854827876758532,
+      "loss": 0.321,
+      "step": 143
+    },
+    {
+      "epoch": 1.25,
+      "learning_rate": 0.00015797059623282787,
+      "loss": 0.3409,
+      "step": 144
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 0.0001573899850907373,
+      "loss": 0.3336,
+      "step": 145
+    },
+    {
+      "epoch": 1.26,
+      "eval_loss": 0.3395354747772217,
+      "eval_runtime": 35.0942,
+      "eval_samples_per_second": 173.647,
+      "eval_steps_per_second": 12.424,
+      "step": 145
+    },
+    {
+      "epoch": 1.27,
+      "learning_rate": 0.00015680647467311557,
+      "loss": 0.3355,
+      "step": 146
+    },
+    {
+      "epoch": 1.27,
+      "learning_rate": 0.00015622009445823273,
+      "loss": 0.347,
+      "step": 147
+    },
+    {
+      "epoch": 1.28,
+      "learning_rate": 0.0001556308740693376,
+      "loss": 0.3433,
+      "step": 148
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 0.00015503884327316132,
+      "loss": 0.3271,
+      "step": 149
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 0.00015444403197841346,
+      "loss": 0.3195,
+      "step": 150
+    },
+    {
+      "epoch": 1.31,
+      "learning_rate": 0.00015384647023427138,
+      "loss": 0.3314,
+      "step": 151
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 0.00015324618822886165,
+      "loss": 0.3387,
+      "step": 152
+    },
+    {
+      "epoch": 1.33,
+      "learning_rate": 0.0001526432162877356,
+      "loss": 0.3436,
+      "step": 153
+    },
+    {
+      "epoch": 1.34,
+      "learning_rate": 0.00015203758487233678,
+      "loss": 0.3152,
+      "step": 154
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 0.0001514293245784623,
+      "loss": 0.341,
+      "step": 155
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 0.00015081846613471735,
+      "loss": 0.3465,
+      "step": 156
+    },
+    {
+      "epoch": 1.36,
+      "learning_rate": 0.0001502050404009624,
+      "loss": 0.3328,
+      "step": 157
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 0.00014958907836675468,
+      "loss": 0.3459,
+      "step": 158
+    },
+    {
+      "epoch": 1.38,
+      "learning_rate": 0.0001489706111497821,
+      "loss": 0.3625,
+      "step": 159
+    },
+    {
+      "epoch": 1.39,
+      "learning_rate": 0.00014834966999429178,
+      "loss": 0.3558,
+      "step": 160
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 0.00014772628626951112,
+      "loss": 0.3418,
+      "step": 161
+    },
+    {
+      "epoch": 1.41,
+      "learning_rate": 0.00014710049146806348,
+      "loss": 0.3304,
+      "step": 162
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 0.00014647231720437686,
+      "loss": 0.3231,
+      "step": 163
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 0.00014584179521308702,
+      "loss": 0.3144,
+      "step": 164
+    },
+    {
+      "epoch": 1.43,
+      "learning_rate": 0.00014520895734743418,
+      "loss": 0.3332,
+      "step": 165
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 0.00014457383557765386,
+      "loss": 0.3311,
+      "step": 166
+    },
+    {
+      "epoch": 1.45,
+      "learning_rate": 0.00014393646198936167,
+      "loss": 0.3396,
+      "step": 167
+    },
+    {
+      "epoch": 1.46,
+      "learning_rate": 0.0001432968687819327,
+      "loss": 0.3341,
+      "step": 168
+    },
+    {
+      "epoch": 1.47,
+      "learning_rate": 0.0001426550882668744,
+      "loss": 0.3477,
+      "step": 169
+    },
+    {
+      "epoch": 1.48,
+      "learning_rate": 0.00014201115286619463,
+      "loss": 0.3455,
+      "step": 170
+    },
+    {
+      "epoch": 1.49,
+      "learning_rate": 0.00014136509511076346,
+      "loss": 0.3415,
+      "step": 171
+    },
+    {
+      "epoch": 1.49,
+      "learning_rate": 0.00014071694763866986,
+      "loss": 0.3392,
+      "step": 172
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 0.00014006674319357298,
+      "loss": 0.3543,
+      "step": 173
+    },
+    {
+      "epoch": 1.51,
+      "learning_rate": 0.00013941451462304777,
+      "loss": 0.3449,
+      "step": 174
+    },
+    {
+      "epoch": 1.51,
+      "eval_loss": 0.33857324719429016,
+      "eval_runtime": 35.0605,
+      "eval_samples_per_second": 173.814,
+      "eval_steps_per_second": 12.436,
+      "step": 174
+    },
+    {
+      "epoch": 1.52,
+      "learning_rate": 0.0001387602948769257,
+      "loss": 0.3518,
+      "step": 175
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 0.00013810411700563004,
+      "loss": 0.3266,
+      "step": 176
+    },
+    {
+      "epoch": 1.54,
+      "learning_rate": 0.00013744601415850637,
+      "loss": 0.3358,
+      "step": 177
+    },
+    {
+      "epoch": 1.55,
+      "learning_rate": 0.0001367860195821478,
+      "loss": 0.3208,
+      "step": 178
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 0.00013612416661871533,
+      "loss": 0.3061,
+      "step": 179
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 0.00013546048870425356,
+      "loss": 0.3151,
+      "step": 180
+    },
+    {
+      "epoch": 1.57,
+      "learning_rate": 0.0001347950193670016,
+      "loss": 0.3185,
+      "step": 181
+    },
+    {
+      "epoch": 1.58,
+      "learning_rate": 0.00013412779222569908,
+      "loss": 0.3248,
+      "step": 182
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 0.00013345884098788774,
+      "loss": 0.3285,
+      "step": 183
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 0.00013278819944820893,
+      "loss": 0.3444,
+      "step": 184
+    },
+    {
+      "epoch": 1.61,
+      "learning_rate": 0.00013211590148669587,
+      "loss": 0.3268,
+      "step": 185
+    },
+    {
+      "epoch": 1.62,
+      "learning_rate": 0.0001314419810670624,
+      "loss": 0.3334,
+      "step": 186
+    },
+    {
+      "epoch": 1.63,
+      "learning_rate": 0.00013076647223498702,
+      "loss": 0.347,
+      "step": 187
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 0.00013008940911639303,
+      "loss": 0.3308,
+      "step": 188
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 0.00012941082591572443,
+      "loss": 0.3164,
+      "step": 189
+    },
+    {
+      "epoch": 1.65,
+      "learning_rate": 0.00012873075691421807,
+      "loss": 0.333,
+      "step": 190
+    },
+    {
+      "epoch": 1.66,
+      "learning_rate": 0.00012804923646817168,
+      "loss": 0.319,
+      "step": 191
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 0.0001273662990072083,
+      "loss": 0.3235,
+      "step": 192
+    },
+    {
+      "epoch": 1.68,
+      "learning_rate": 0.00012668197903253694,
+      "loss": 0.3553,
+      "step": 193
+    },
+    {
+      "epoch": 1.69,
+      "learning_rate": 0.00012599631111520954,
+      "loss": 0.3323,
+      "step": 194
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 0.00012530932989437464,
+      "loss": 0.3183,
+      "step": 195
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 0.00012462107007552725,
+      "loss": 0.3328,
+      "step": 196
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 0.0001239315664287558,
+      "loss": 0.3153,
+      "step": 197
+    },
+    {
+      "epoch": 1.72,
+      "learning_rate": 0.00012324085378698528,
+      "loss": 0.3349,
+      "step": 198
+    },
+    {
+      "epoch": 1.73,
+      "learning_rate": 0.0001225489670442179,
+      "loss": 0.3367,
+      "step": 199
+    },
+    {
+      "epoch": 1.74,
+      "learning_rate": 0.0001218559411537699,
+      "loss": 0.3432,
+      "step": 200
+    },
+    {
+      "epoch": 1.75,
+      "learning_rate": 0.0001211618111265061,
+      "loss": 0.3191,
+      "step": 201
+    },
+    {
+      "epoch": 1.76,
+      "learning_rate": 0.00012046661202907101,
+      "loss": 0.3506,
+      "step": 202
+    },
+    {
+      "epoch": 1.77,
+      "learning_rate": 0.00011977037898211723,
+      "loss": 0.3187,
+      "step": 203
+    },
+    {
+      "epoch": 1.77,
+      "eval_loss": 0.3376944661140442,
+      "eval_runtime": 35.038,
+      "eval_samples_per_second": 173.925,
+      "eval_steps_per_second": 12.444,
+      "step": 203
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 0.00011907314715853138,
+      "loss": 0.3553,
+      "step": 204
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 0.00011837495178165706,
+      "loss": 0.3567,
+      "step": 205
+    },
+    {
+      "epoch": 1.79,
+      "learning_rate": 0.0001176758281235155,
+      "loss": 0.3221,
+      "step": 206
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 0.00011697581150302362,
+      "loss": 0.3458,
+      "step": 207
+    },
+    {
+      "epoch": 1.81,
+      "learning_rate": 0.00011627493728420978,
+      "loss": 0.329,
+      "step": 208
+    },
+    {
+      "epoch": 1.82,
+      "learning_rate": 0.00011557324087442717,
+      "loss": 0.3222,
+      "step": 209
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 0.00011487075772256517,
+      "loss": 0.3411,
+      "step": 210
+    },
+    {
+      "epoch": 1.84,
+      "learning_rate": 0.00011416752331725842,
+      "loss": 0.3322,
+      "step": 211
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 0.00011346357318509395,
+      "loss": 0.3202,
+      "step": 212
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 0.00011275894288881663,
+      "loss": 0.3354,
+      "step": 213
+    },
+    {
+      "epoch": 1.86,
+      "learning_rate": 0.0001120536680255323,
+      "loss": 0.3454,
+      "step": 214
+    },
+    {
+      "epoch": 1.87,
+      "learning_rate": 0.0001113477842249097,
+      "loss": 0.3376,
+      "step": 215
+    },
+    {
+      "epoch": 1.88,
+      "learning_rate": 0.00011064132714738024,
+      "loss": 0.3206,
+      "step": 216
+    },
+    {
+      "epoch": 1.89,
+      "learning_rate": 0.00010993433248233671,
+      "loss": 0.3258,
+      "step": 217
+    },
+    {
+      "epoch": 1.9,
+      "learning_rate": 0.00010922683594633021,
+      "loss": 0.3376,
+      "step": 218
+    },
+    {
+      "epoch": 1.91,
+      "learning_rate": 0.00010851887328126568,
+      "loss": 0.3194,
+      "step": 219
+    },
+    {
+      "epoch": 1.92,
+      "learning_rate": 0.00010781048025259648,
+      "loss": 0.3117,
+      "step": 220
+    },
+    {
+      "epoch": 1.93,
+      "learning_rate": 0.00010710169264751732,
+      "loss": 0.3225,
+      "step": 221
+    },
+    {
+      "epoch": 1.93,
+      "learning_rate": 0.00010639254627315657,
+      "loss": 0.3347,
+      "step": 222
+    },
+    {
+      "epoch": 1.94,
+      "learning_rate": 0.00010568307695476711,
+      "loss": 0.3274,
+      "step": 223
+    },
+    {
+      "epoch": 1.95,
+      "learning_rate": 0.00010497332053391668,
+      "loss": 0.3219,
+      "step": 224
+    },
+    {
+      "epoch": 1.96,
+      "learning_rate": 0.00010426331286667701,
+      "loss": 0.3272,
+      "step": 225
+    },
+    {
+      "epoch": 1.97,
+      "learning_rate": 0.00010355308982181253,
+      "loss": 0.3246,
+      "step": 226
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 452,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 113,
+  "total_flos": 1.0481747744631292e+19,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-226/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66dc6af509e7f51c6a244a8a9af44ad9044f2ab26d51bec3236dad93ab4ef46c
+size 4923

checkpoint-339/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: codellama/CodeLlama-34b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-339/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "codellama/CodeLlama-34b-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-339/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2675b8c6c0eb32c2ade876ffcb901cce07b24978a8fb8b357b13500c6a48f325
+size 1920164848

checkpoint-339/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf3eef9333bb65394884d3fbde15fa61e6d744021dff14017c001747efcb5444
+size 1743147421

checkpoint-339/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de2511e59e98dde4bc88b8676b8bcafd50ec70d1db3c69ab729844e0acba9351
+size 20663

checkpoint-339/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f62c5d80926d5f53a3a741d0bcfde11a94803e81574aa452c54c70071d6ee907
+size 20663

checkpoint-339/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f0ebfc6d6f0e102907cec3536b416a319e447f0af6d068f6858f188e6b8bb9c
+size 20663

checkpoint-339/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f36755f0f86b29ea66e6fba8de2ff383a3e6ad5592906ab43c33695adb4a3af
+size 20663

checkpoint-339/rng_state_4.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5cec541773bb67f532d0d12d92dd78f6e8900d223d3123bff23e326ec30b9f9
+size 20663

checkpoint-339/rng_state_5.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28c5ba9234cf43e24133fe3325eec1d58059433bdb2d821ed6add4a3e3b37229
+size 20663

checkpoint-339/rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f7df030b0ce70f200552bc1fe0e2fff358c92da5adb3dfab6312c9ac2bc9c9a
+size 20663

checkpoint-339/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab4fefc68aa069447a8f613a225ac6ee4d3a36662b5ea5ade1a0d9cc090d8313
+size 627

checkpoint-339/trainer_state.json

@@ -0,0 +1,2151 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.9450549450549453,
+  "eval_steps": 29,
+  "global_step": 339,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "learning_rate": 2e-05,
+      "loss": 0.5065,
+      "step": 1
+    },
+    {
+      "epoch": 0.01,
+      "eval_loss": 0.5089401602745056,
+      "eval_runtime": 34.8867,
+      "eval_samples_per_second": 174.68,
+      "eval_steps_per_second": 12.498,
+      "step": 1
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 4e-05,
+      "loss": 0.5189,
+      "step": 2
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 6e-05,
+      "loss": 0.4966,
+      "step": 3
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 8e-05,
+      "loss": 0.4891,
+      "step": 4
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 0.0001,
+      "loss": 0.507,
+      "step": 5
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 0.00012,
+      "loss": 0.5071,
+      "step": 6
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 0.00014,
+      "loss": 0.4739,
+      "step": 7
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 0.00016,
+      "loss": 0.4721,
+      "step": 8
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 0.00018,
+      "loss": 0.4213,
+      "step": 9
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 0.0002,
+      "loss": 0.4056,
+      "step": 10
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 0.00019999747405795057,
+      "loss": 0.3989,
+      "step": 11
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019998989635940995,
+      "loss": 0.3843,
+      "step": 12
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00019997726728719468,
+      "loss": 0.3771,
+      "step": 13
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 0.00019995958747931082,
+      "loss": 0.3775,
+      "step": 14
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 0.00019993685782892183,
+      "loss": 0.3778,
+      "step": 15
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 0.00019990907948430327,
+      "loss": 0.3786,
+      "step": 16
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 0.0001998762538487849,
+      "loss": 0.3669,
+      "step": 17
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 0.00019983838258067987,
+      "loss": 0.3819,
+      "step": 18
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 0.0001997954675932006,
+      "loss": 0.4043,
+      "step": 19
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019974751105436266,
+      "loss": 0.3503,
+      "step": 20
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 0.00019969451538687474,
+      "loss": 0.3753,
+      "step": 21
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 0.00019963648326801653,
+      "loss": 0.368,
+      "step": 22
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 0.00019957341762950344,
+      "loss": 0.3828,
+      "step": 23
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 0.00019950532165733845,
+      "loss": 0.3509,
+      "step": 24
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 0.00019943219879165115,
+      "loss": 0.381,
+      "step": 25
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 0.0001993540527265239,
+      "loss": 0.3638,
+      "step": 26
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 0.0001992708874098054,
+      "loss": 0.3645,
+      "step": 27
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019918270704291104,
+      "loss": 0.3466,
+      "step": 28
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 0.00019908951608061076,
+      "loss": 0.3477,
+      "step": 29
+    },
+    {
+      "epoch": 0.25,
+      "eval_loss": 0.35608404874801636,
+      "eval_runtime": 35.0585,
+      "eval_samples_per_second": 173.824,
+      "eval_steps_per_second": 12.436,
+      "step": 29
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 0.000198991319230804,
+      "loss": 0.3444,
+      "step": 30
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 0.0001988881214542817,
+      "loss": 0.3646,
+      "step": 31
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 0.00019877992796447601,
+      "loss": 0.3502,
+      "step": 32
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 0.00019866674422719665,
+      "loss": 0.3532,
+      "step": 33
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 0.00019854857596035476,
+      "loss": 0.3639,
+      "step": 34
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 0.0001984254291336743,
+      "loss": 0.3538,
+      "step": 35
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 0.0001982973099683902,
+      "loss": 0.3431,
+      "step": 36
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.00019816422493693417,
+      "loss": 0.3493,
+      "step": 37
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.0001980261807626078,
+      "loss": 0.3596,
+      "step": 38
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 0.00019788318441924275,
+      "loss": 0.3401,
+      "step": 39
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 0.00019773524313084856,
+      "loss": 0.3822,
+      "step": 40
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 0.00019758236437124769,
+      "loss": 0.3719,
+      "step": 41
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 0.00019742455586369784,
+      "loss": 0.3369,
+      "step": 42
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 0.0001972618255805019,
+      "loss": 0.341,
+      "step": 43
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 0.0001970941817426052,
+      "loss": 0.3481,
+      "step": 44
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019692163281918014,
+      "loss": 0.3615,
+      "step": 45
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.00019674418752719833,
+      "loss": 0.346,
+      "step": 46
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 0.00019656185483099024,
+      "loss": 0.3479,
+      "step": 47
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 0.00019637464394179239,
+      "loss": 0.3638,
+      "step": 48
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 0.00019618256431728194,
+      "loss": 0.3419,
+      "step": 49
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 0.0001959856256610988,
+      "loss": 0.3496,
+      "step": 50
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 0.00019578383792235573,
+      "loss": 0.3503,
+      "step": 51
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 0.0001955772112951354,
+      "loss": 0.3437,
+      "step": 52
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019536575621797547,
+      "loss": 0.3343,
+      "step": 53
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 0.00019514948337334142,
+      "loss": 0.3517,
+      "step": 54
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 0.0001949284036870867,
+      "loss": 0.343,
+      "step": 55
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 0.0001947025283279008,
+      "loss": 0.3479,
+      "step": 56
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 0.00019447186870674506,
+      "loss": 0.3237,
+      "step": 57
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 0.00019423643647627624,
+      "loss": 0.3593,
+      "step": 58
+    },
+    {
+      "epoch": 0.51,
+      "eval_loss": 0.3460778594017029,
+      "eval_runtime": 35.0517,
+      "eval_samples_per_second": 173.858,
+      "eval_steps_per_second": 12.439,
+      "step": 58
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 0.00019399624353025772,
+      "loss": 0.348,
+      "step": 59
+    },
+    {
+      "epoch": 0.53,
+      "learning_rate": 0.00019375130200295878,
+      "loss": 0.344,
+      "step": 60
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 0.0001935016242685415,
+      "loss": 0.3601,
+      "step": 61
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019324722294043558,
+      "loss": 0.3524,
+      "step": 62
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 0.00019298811087070132,
+      "loss": 0.3421,
+      "step": 63
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 0.00019272430114938017,
+      "loss": 0.3524,
+      "step": 64
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 0.00019245580710383343,
+      "loss": 0.3334,
+      "step": 65
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 0.00019218264229806917,
+      "loss": 0.3312,
+      "step": 66
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 0.00019190482053205673,
+      "loss": 0.344,
+      "step": 67
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.00019162235584102973,
+      "loss": 0.353,
+      "step": 68
+    },
+    {
+      "epoch": 0.61,
+      "learning_rate": 0.00019133526249477702,
+      "loss": 0.3473,
+      "step": 69
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019104355499692166,
+      "loss": 0.345,
+      "step": 70
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 0.00019074724808418835,
+      "loss": 0.348,
+      "step": 71
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 0.00019044635672565895,
+      "loss": 0.3333,
+      "step": 72
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 0.00019014089612201613,
+      "loss": 0.3398,
+      "step": 73
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 0.00018983088170477553,
+      "loss": 0.3424,
+      "step": 74
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 0.00018951632913550626,
+      "loss": 0.3279,
+      "step": 75
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 0.00018919725430503945,
+      "loss": 0.3243,
+      "step": 76
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 0.00018887367333266582,
+      "loss": 0.3439,
+      "step": 77
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.000188545602565321,
+      "loss": 0.3486,
+      "step": 78
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 0.00018821305857675996,
+      "loss": 0.3367,
+      "step": 79
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 0.00018787605816671953,
+      "loss": 0.3386,
+      "step": 80
+    },
+    {
+      "epoch": 0.71,
+      "learning_rate": 0.00018753461836006988,
+      "loss": 0.3574,
+      "step": 81
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 0.00018718875640595434,
+      "loss": 0.334,
+      "step": 82
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 0.00018683848977691785,
+      "loss": 0.3342,
+      "step": 83
+    },
+    {
+      "epoch": 0.74,
+      "learning_rate": 0.0001864838361680247,
+      "loss": 0.3693,
+      "step": 84
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 0.00018612481349596405,
+      "loss": 0.3422,
+      "step": 85
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018576143989814523,
+      "loss": 0.3346,
+      "step": 86
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 0.00018539373373178123,
+      "loss": 0.3329,
+      "step": 87
+    },
+    {
+      "epoch": 0.76,
+      "eval_loss": 0.3422779440879822,
+      "eval_runtime": 35.1197,
+      "eval_samples_per_second": 173.521,
+      "eval_steps_per_second": 12.415,
+      "step": 87
+    },
+    {
+      "epoch": 0.77,
+      "learning_rate": 0.00018502171357296144,
+      "loss": 0.3485,
+      "step": 88
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 0.00018464539821571303,
+      "loss": 0.3383,
+      "step": 89
+    },
+    {
+      "epoch": 0.79,
+      "learning_rate": 0.00018426480667105177,
+      "loss": 0.3434,
+      "step": 90
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 0.00018387995816602137,
+      "loss": 0.3205,
+      "step": 91
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 0.00018349087214272224,
+      "loss": 0.3522,
+      "step": 92
+    },
+    {
+      "epoch": 0.82,
+      "learning_rate": 0.0001830975682573293,
+      "loss": 0.3416,
+      "step": 93
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 0.00018270006637909907,
+      "loss": 0.3468,
+      "step": 94
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.00018229838658936564,
+      "loss": 0.3407,
+      "step": 95
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 0.0001818925491805265,
+      "loss": 0.3198,
+      "step": 96
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 0.00018148257465501716,
+      "loss": 0.3369,
+      "step": 97
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 0.0001810684837242755,
+      "loss": 0.3371,
+      "step": 98
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 0.00018065029730769534,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 0.00018022803653156984,
+      "loss": 0.3418,
+      "step": 100
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 0.000179801722728024,
+      "loss": 0.346,
+      "step": 101
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 0.00017937137743393694,
+      "loss": 0.3385,
+      "step": 102
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017893702238985432,
+      "loss": 0.3343,
+      "step": 103
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 0.00017849867953888948,
+      "loss": 0.3146,
+      "step": 104
+    },
+    {
+      "epoch": 0.92,
+      "learning_rate": 0.00017805637102561516,
+      "loss": 0.3662,
+      "step": 105
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 0.00017761011919494488,
+      "loss": 0.3481,
+      "step": 106
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 0.00017715994659100388,
+      "loss": 0.3447,
+      "step": 107
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 0.00017670587595599034,
+      "loss": 0.3379,
+      "step": 108
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 0.0001762479302290265,
+      "loss": 0.3388,
+      "step": 109
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 0.0001757861325449997,
+      "loss": 0.3497,
+      "step": 110
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.0001753205062333937,
+      "loss": 0.3539,
+      "step": 111
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 0.00017485107481711012,
+      "loss": 0.3571,
+      "step": 112
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 0.00017437786201128004,
+      "loss": 0.3496,
+      "step": 113
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00017390089172206592,
+      "loss": 0.3436,
+      "step": 114
+    },
+    {
+      "epoch": 1.01,
+      "learning_rate": 0.000173420188045454,
+      "loss": 0.3434,
+      "step": 115
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00017293577526603682,
+      "loss": 0.3607,
+      "step": 116
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.3404248356819153,
+      "eval_runtime": 35.0435,
+      "eval_samples_per_second": 173.898,
+      "eval_steps_per_second": 12.442,
+      "step": 116
+    },
+    {
+      "epoch": 1.01,
+      "learning_rate": 0.0001724476778557866,
+      "loss": 0.3363,
+      "step": 117
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 0.00017195592047281882,
+      "loss": 0.3283,
+      "step": 118
+    },
+    {
+      "epoch": 1.03,
+      "learning_rate": 0.00017146052796014646,
+      "loss": 0.3293,
+      "step": 119
+    },
+    {
+      "epoch": 1.04,
+      "learning_rate": 0.00017096152534442513,
+      "loss": 0.3485,
+      "step": 120
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 0.0001704589378346886,
+      "loss": 0.3315,
+      "step": 121
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 0.0001699527908210754,
+      "loss": 0.3686,
+      "step": 122
+    },
+    {
+      "epoch": 1.06,
+      "learning_rate": 0.00016944310987354597,
+      "loss": 0.3384,
+      "step": 123
+    },
+    {
+      "epoch": 1.07,
+      "learning_rate": 0.0001689299207405911,
+      "loss": 0.3432,
+      "step": 124
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 0.00016841324934793094,
+      "loss": 0.3323,
+      "step": 125
+    },
+    {
+      "epoch": 1.09,
+      "learning_rate": 0.00016789312179720547,
+      "loss": 0.3325,
+      "step": 126
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 0.00016736956436465573,
+      "loss": 0.3464,
+      "step": 127
+    },
+    {
+      "epoch": 1.11,
+      "learning_rate": 0.00016684260349979636,
+      "loss": 0.3344,
+      "step": 128
+    },
+    {
+      "epoch": 1.12,
+      "learning_rate": 0.00016631226582407952,
+      "loss": 0.3319,
+      "step": 129
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 0.00016577857812954993,
+      "loss": 0.3385,
+      "step": 130
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 0.00016524156737749132,
+      "loss": 0.3166,
+      "step": 131
+    },
+    {
+      "epoch": 1.14,
+      "learning_rate": 0.00016470126069706456,
+      "loss": 0.3214,
+      "step": 132
+    },
+    {
+      "epoch": 1.15,
+      "learning_rate": 0.00016415768538393687,
+      "loss": 0.3101,
+      "step": 133
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 0.00016361086889890306,
+      "loss": 0.3334,
+      "step": 134
+    },
+    {
+      "epoch": 1.17,
+      "learning_rate": 0.0001630608388664982,
+      "loss": 0.3094,
+      "step": 135
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 0.00016250762307360207,
+      "loss": 0.327,
+      "step": 136
+    },
+    {
+      "epoch": 1.19,
+      "learning_rate": 0.00016195124946803525,
+      "loss": 0.3512,
+      "step": 137
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0001613917461571475,
+      "loss": 0.3329,
+      "step": 138
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0001608291414063977,
+      "loss": 0.342,
+      "step": 139
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 0.00016026346363792567,
+      "loss": 0.3488,
+      "step": 140
+    },
+    {
+      "epoch": 1.22,
+      "learning_rate": 0.00015969474142911666,
+      "loss": 0.3385,
+      "step": 141
+    },
+    {
+      "epoch": 1.23,
+      "learning_rate": 0.0001591230035111576,
+      "loss": 0.3207,
+      "step": 142
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 0.00015854827876758532,
+      "loss": 0.321,
+      "step": 143
+    },
+    {
+      "epoch": 1.25,
+      "learning_rate": 0.00015797059623282787,
+      "loss": 0.3409,
+      "step": 144
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 0.0001573899850907373,
+      "loss": 0.3336,
+      "step": 145
+    },
+    {
+      "epoch": 1.26,
+      "eval_loss": 0.3395354747772217,
+      "eval_runtime": 35.0942,
+      "eval_samples_per_second": 173.647,
+      "eval_steps_per_second": 12.424,
+      "step": 145
+    },
+    {
+      "epoch": 1.27,
+      "learning_rate": 0.00015680647467311557,
+      "loss": 0.3355,
+      "step": 146
+    },
+    {
+      "epoch": 1.27,
+      "learning_rate": 0.00015622009445823273,
+      "loss": 0.347,
+      "step": 147
+    },
+    {
+      "epoch": 1.28,
+      "learning_rate": 0.0001556308740693376,
+      "loss": 0.3433,
+      "step": 148
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 0.00015503884327316132,
+      "loss": 0.3271,
+      "step": 149
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 0.00015444403197841346,
+      "loss": 0.3195,
+      "step": 150
+    },
+    {
+      "epoch": 1.31,
+      "learning_rate": 0.00015384647023427138,
+      "loss": 0.3314,
+      "step": 151
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 0.00015324618822886165,
+      "loss": 0.3387,
+      "step": 152
+    },
+    {
+      "epoch": 1.33,
+      "learning_rate": 0.0001526432162877356,
+      "loss": 0.3436,
+      "step": 153
+    },
+    {
+      "epoch": 1.34,
+      "learning_rate": 0.00015203758487233678,
+      "loss": 0.3152,
+      "step": 154
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 0.0001514293245784623,
+      "loss": 0.341,
+      "step": 155
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 0.00015081846613471735,
+      "loss": 0.3465,
+      "step": 156
+    },
+    {
+      "epoch": 1.36,
+      "learning_rate": 0.0001502050404009624,
+      "loss": 0.3328,
+      "step": 157
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 0.00014958907836675468,
+      "loss": 0.3459,
+      "step": 158
+    },
+    {
+      "epoch": 1.38,
+      "learning_rate": 0.0001489706111497821,
+      "loss": 0.3625,
+      "step": 159
+    },
+    {
+      "epoch": 1.39,
+      "learning_rate": 0.00014834966999429178,
+      "loss": 0.3558,
+      "step": 160
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 0.00014772628626951112,
+      "loss": 0.3418,
+      "step": 161
+    },
+    {
+      "epoch": 1.41,
+      "learning_rate": 0.00014710049146806348,
+      "loss": 0.3304,
+      "step": 162
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 0.00014647231720437686,
+      "loss": 0.3231,
+      "step": 163
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 0.00014584179521308702,
+      "loss": 0.3144,
+      "step": 164
+    },
+    {
+      "epoch": 1.43,
+      "learning_rate": 0.00014520895734743418,
+      "loss": 0.3332,
+      "step": 165
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 0.00014457383557765386,
+      "loss": 0.3311,
+      "step": 166
+    },
+    {
+      "epoch": 1.45,
+      "learning_rate": 0.00014393646198936167,
+      "loss": 0.3396,
+      "step": 167
+    },
+    {
+      "epoch": 1.46,
+      "learning_rate": 0.0001432968687819327,
+      "loss": 0.3341,
+      "step": 168
+    },
+    {
+      "epoch": 1.47,
+      "learning_rate": 0.0001426550882668744,
+      "loss": 0.3477,
+      "step": 169
+    },
+    {
+      "epoch": 1.48,
+      "learning_rate": 0.00014201115286619463,
+      "loss": 0.3455,
+      "step": 170
+    },
+    {
+      "epoch": 1.49,
+      "learning_rate": 0.00014136509511076346,
+      "loss": 0.3415,
+      "step": 171
+    },
+    {
+      "epoch": 1.49,
+      "learning_rate": 0.00014071694763866986,
+      "loss": 0.3392,
+      "step": 172
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 0.00014006674319357298,
+      "loss": 0.3543,
+      "step": 173
+    },
+    {
+      "epoch": 1.51,
+      "learning_rate": 0.00013941451462304777,
+      "loss": 0.3449,
+      "step": 174
+    },
+    {
+      "epoch": 1.51,
+      "eval_loss": 0.33857324719429016,
+      "eval_runtime": 35.0605,
+      "eval_samples_per_second": 173.814,
+      "eval_steps_per_second": 12.436,
+      "step": 174
+    },
+    {
+      "epoch": 1.52,
+      "learning_rate": 0.0001387602948769257,
+      "loss": 0.3518,
+      "step": 175
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 0.00013810411700563004,
+      "loss": 0.3266,
+      "step": 176
+    },
+    {
+      "epoch": 1.54,
+      "learning_rate": 0.00013744601415850637,
+      "loss": 0.3358,
+      "step": 177
+    },
+    {
+      "epoch": 1.55,
+      "learning_rate": 0.0001367860195821478,
+      "loss": 0.3208,
+      "step": 178
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 0.00013612416661871533,
+      "loss": 0.3061,
+      "step": 179
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 0.00013546048870425356,
+      "loss": 0.3151,
+      "step": 180
+    },
+    {
+      "epoch": 1.57,
+      "learning_rate": 0.0001347950193670016,
+      "loss": 0.3185,
+      "step": 181
+    },
+    {
+      "epoch": 1.58,
+      "learning_rate": 0.00013412779222569908,
+      "loss": 0.3248,
+      "step": 182
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 0.00013345884098788774,
+      "loss": 0.3285,
+      "step": 183
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 0.00013278819944820893,
+      "loss": 0.3444,
+      "step": 184
+    },
+    {
+      "epoch": 1.61,
+      "learning_rate": 0.00013211590148669587,
+      "loss": 0.3268,
+      "step": 185
+    },
+    {
+      "epoch": 1.62,
+      "learning_rate": 0.0001314419810670624,
+      "loss": 0.3334,
+      "step": 186
+    },
+    {
+      "epoch": 1.63,
+      "learning_rate": 0.00013076647223498702,
+      "loss": 0.347,
+      "step": 187
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 0.00013008940911639303,
+      "loss": 0.3308,
+      "step": 188
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 0.00012941082591572443,
+      "loss": 0.3164,
+      "step": 189
+    },
+    {
+      "epoch": 1.65,
+      "learning_rate": 0.00012873075691421807,
+      "loss": 0.333,
+      "step": 190
+    },
+    {
+      "epoch": 1.66,
+      "learning_rate": 0.00012804923646817168,
+      "loss": 0.319,
+      "step": 191
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 0.0001273662990072083,
+      "loss": 0.3235,
+      "step": 192
+    },
+    {
+      "epoch": 1.68,
+      "learning_rate": 0.00012668197903253694,
+      "loss": 0.3553,
+      "step": 193
+    },
+    {
+      "epoch": 1.69,
+      "learning_rate": 0.00012599631111520954,
+      "loss": 0.3323,
+      "step": 194
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 0.00012530932989437464,
+      "loss": 0.3183,
+      "step": 195
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 0.00012462107007552725,
+      "loss": 0.3328,
+      "step": 196
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 0.0001239315664287558,
+      "loss": 0.3153,
+      "step": 197
+    },
+    {
+      "epoch": 1.72,
+      "learning_rate": 0.00012324085378698528,
+      "loss": 0.3349,
+      "step": 198
+    },
+    {
+      "epoch": 1.73,
+      "learning_rate": 0.0001225489670442179,
+      "loss": 0.3367,
+      "step": 199
+    },
+    {
+      "epoch": 1.74,
+      "learning_rate": 0.0001218559411537699,
+      "loss": 0.3432,
+      "step": 200
+    },
+    {
+      "epoch": 1.75,
+      "learning_rate": 0.0001211618111265061,
+      "loss": 0.3191,
+      "step": 201
+    },
+    {
+      "epoch": 1.76,
+      "learning_rate": 0.00012046661202907101,
+      "loss": 0.3506,
+      "step": 202
+    },
+    {
+      "epoch": 1.77,
+      "learning_rate": 0.00011977037898211723,
+      "loss": 0.3187,
+      "step": 203
+    },
+    {
+      "epoch": 1.77,
+      "eval_loss": 0.3376944661140442,
+      "eval_runtime": 35.038,
+      "eval_samples_per_second": 173.925,
+      "eval_steps_per_second": 12.444,
+      "step": 203
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 0.00011907314715853138,
+      "loss": 0.3553,
+      "step": 204
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 0.00011837495178165706,
+      "loss": 0.3567,
+      "step": 205
+    },
+    {
+      "epoch": 1.79,
+      "learning_rate": 0.0001176758281235155,
+      "loss": 0.3221,
+      "step": 206
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 0.00011697581150302362,
+      "loss": 0.3458,
+      "step": 207
+    },
+    {
+      "epoch": 1.81,
+      "learning_rate": 0.00011627493728420978,
+      "loss": 0.329,
+      "step": 208
+    },
+    {
+      "epoch": 1.82,
+      "learning_rate": 0.00011557324087442717,
+      "loss": 0.3222,
+      "step": 209
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 0.00011487075772256517,
+      "loss": 0.3411,
+      "step": 210
+    },
+    {
+      "epoch": 1.84,
+      "learning_rate": 0.00011416752331725842,
+      "loss": 0.3322,
+      "step": 211
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 0.00011346357318509395,
+      "loss": 0.3202,
+      "step": 212
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 0.00011275894288881663,
+      "loss": 0.3354,
+      "step": 213
+    },
+    {
+      "epoch": 1.86,
+      "learning_rate": 0.0001120536680255323,
+      "loss": 0.3454,
+      "step": 214
+    },
+    {
+      "epoch": 1.87,
+      "learning_rate": 0.0001113477842249097,
+      "loss": 0.3376,
+      "step": 215
+    },
+    {
+      "epoch": 1.88,
+      "learning_rate": 0.00011064132714738024,
+      "loss": 0.3206,
+      "step": 216
+    },
+    {
+      "epoch": 1.89,
+      "learning_rate": 0.00010993433248233671,
+      "loss": 0.3258,
+      "step": 217
+    },
+    {
+      "epoch": 1.9,
+      "learning_rate": 0.00010922683594633021,
+      "loss": 0.3376,
+      "step": 218
+    },
+    {
+      "epoch": 1.91,
+      "learning_rate": 0.00010851887328126568,
+      "loss": 0.3194,
+      "step": 219
+    },
+    {
+      "epoch": 1.92,
+      "learning_rate": 0.00010781048025259648,
+      "loss": 0.3117,
+      "step": 220
+    },
+    {
+      "epoch": 1.93,
+      "learning_rate": 0.00010710169264751732,
+      "loss": 0.3225,
+      "step": 221
+    },
+    {
+      "epoch": 1.93,
+      "learning_rate": 0.00010639254627315657,
+      "loss": 0.3347,
+      "step": 222
+    },
+    {
+      "epoch": 1.94,
+      "learning_rate": 0.00010568307695476711,
+      "loss": 0.3274,
+      "step": 223
+    },
+    {
+      "epoch": 1.95,
+      "learning_rate": 0.00010497332053391668,
+      "loss": 0.3219,
+      "step": 224
+    },
+    {
+      "epoch": 1.96,
+      "learning_rate": 0.00010426331286667701,
+      "loss": 0.3272,
+      "step": 225
+    },
+    {
+      "epoch": 1.97,
+      "learning_rate": 0.00010355308982181253,
+      "loss": 0.3246,
+      "step": 226
+    },
+    {
+      "epoch": 1.98,
+      "learning_rate": 0.00010284268727896833,
+      "loss": 0.327,
+      "step": 227
+    },
+    {
+      "epoch": 1.99,
+      "learning_rate": 0.00010213214112685748,
+      "loss": 0.3201,
+      "step": 228
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 0.00010142148726144807,
+      "loss": 0.3097,
+      "step": 229
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 0.00010071076158414976,
+      "loss": 0.3374,
+      "step": 230
+    },
+    {
+      "epoch": 2.01,
+      "learning_rate": 0.0001,
+      "loss": 0.3394,
+      "step": 231
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 9.928923841585024e-05,
+      "loss": 0.3216,
+      "step": 232
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.3371301591396332,
+      "eval_runtime": 35.0645,
+      "eval_samples_per_second": 173.794,
+      "eval_steps_per_second": 12.434,
+      "step": 232
+    },
+    {
+      "epoch": 2.01,
+      "learning_rate": 9.857851273855194e-05,
+      "loss": 0.3184,
+      "step": 233
+    },
+    {
+      "epoch": 2.02,
+      "learning_rate": 9.786785887314255e-05,
+      "loss": 0.3199,
+      "step": 234
+    },
+    {
+      "epoch": 2.03,
+      "learning_rate": 9.71573127210317e-05,
+      "loss": 0.3188,
+      "step": 235
+    },
+    {
+      "epoch": 2.04,
+      "learning_rate": 9.644691017818752e-05,
+      "loss": 0.3407,
+      "step": 236
+    },
+    {
+      "epoch": 2.05,
+      "learning_rate": 9.573668713332304e-05,
+      "loss": 0.3226,
+      "step": 237
+    },
+    {
+      "epoch": 2.06,
+      "learning_rate": 9.502667946608333e-05,
+      "loss": 0.319,
+      "step": 238
+    },
+    {
+      "epoch": 2.07,
+      "learning_rate": 9.431692304523288e-05,
+      "loss": 0.3248,
+      "step": 239
+    },
+    {
+      "epoch": 2.07,
+      "learning_rate": 9.360745372684345e-05,
+      "loss": 0.3303,
+      "step": 240
+    },
+    {
+      "epoch": 2.08,
+      "learning_rate": 9.289830735248269e-05,
+      "loss": 0.3124,
+      "step": 241
+    },
+    {
+      "epoch": 2.09,
+      "learning_rate": 9.218951974740354e-05,
+      "loss": 0.3197,
+      "step": 242
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 9.148112671873433e-05,
+      "loss": 0.3358,
+      "step": 243
+    },
+    {
+      "epoch": 2.11,
+      "learning_rate": 9.077316405366981e-05,
+      "loss": 0.3216,
+      "step": 244
+    },
+    {
+      "epoch": 2.12,
+      "learning_rate": 9.00656675176633e-05,
+      "loss": 0.3126,
+      "step": 245
+    },
+    {
+      "epoch": 2.13,
+      "learning_rate": 8.935867285261978e-05,
+      "loss": 0.3327,
+      "step": 246
+    },
+    {
+      "epoch": 2.14,
+      "learning_rate": 8.865221577509033e-05,
+      "loss": 0.3239,
+      "step": 247
+    },
+    {
+      "epoch": 2.15,
+      "learning_rate": 8.79463319744677e-05,
+      "loss": 0.3237,
+      "step": 248
+    },
+    {
+      "epoch": 2.15,
+      "learning_rate": 8.724105711118342e-05,
+      "loss": 0.3163,
+      "step": 249
+    },
+    {
+      "epoch": 2.16,
+      "learning_rate": 8.653642681490608e-05,
+      "loss": 0.3323,
+      "step": 250
+    },
+    {
+      "epoch": 2.17,
+      "learning_rate": 8.583247668274163e-05,
+      "loss": 0.303,
+      "step": 251
+    },
+    {
+      "epoch": 2.18,
+      "learning_rate": 8.512924227743481e-05,
+      "loss": 0.3507,
+      "step": 252
+    },
+    {
+      "epoch": 2.19,
+      "learning_rate": 8.442675912557281e-05,
+      "loss": 0.334,
+      "step": 253
+    },
+    {
+      "epoch": 2.2,
+      "learning_rate": 8.372506271579022e-05,
+      "loss": 0.3144,
+      "step": 254
+    },
+    {
+      "epoch": 2.21,
+      "learning_rate": 8.302418849697638e-05,
+      "loss": 0.3298,
+      "step": 255
+    },
+    {
+      "epoch": 2.22,
+      "learning_rate": 8.232417187648454e-05,
+      "loss": 0.3229,
+      "step": 256
+    },
+    {
+      "epoch": 2.22,
+      "learning_rate": 8.162504821834295e-05,
+      "loss": 0.2989,
+      "step": 257
+    },
+    {
+      "epoch": 2.23,
+      "learning_rate": 8.092685284146865e-05,
+      "loss": 0.3165,
+      "step": 258
+    },
+    {
+      "epoch": 2.24,
+      "learning_rate": 8.02296210178828e-05,
+      "loss": 0.3129,
+      "step": 259
+    },
+    {
+      "epoch": 2.25,
+      "learning_rate": 7.953338797092902e-05,
+      "loss": 0.319,
+      "step": 260
+    },
+    {
+      "epoch": 2.26,
+      "learning_rate": 7.883818887349392e-05,
+      "loss": 0.2961,
+      "step": 261
+    },
+    {
+      "epoch": 2.26,
+      "eval_loss": 0.3379780054092407,
+      "eval_runtime": 35.0784,
+      "eval_samples_per_second": 173.725,
+      "eval_steps_per_second": 12.429,
+      "step": 261
+    },
+    {
+      "epoch": 2.27,
+      "learning_rate": 7.814405884623011e-05,
+      "loss": 0.3,
+      "step": 262
+    },
+    {
+      "epoch": 2.28,
+      "learning_rate": 7.745103295578216e-05,
+      "loss": 0.3133,
+      "step": 263
+    },
+    {
+      "epoch": 2.29,
+      "learning_rate": 7.675914621301476e-05,
+      "loss": 0.3277,
+      "step": 264
+    },
+    {
+      "epoch": 2.29,
+      "learning_rate": 7.606843357124426e-05,
+      "loss": 0.3142,
+      "step": 265
+    },
+    {
+      "epoch": 2.3,
+      "learning_rate": 7.537892992447273e-05,
+      "loss": 0.3245,
+      "step": 266
+    },
+    {
+      "epoch": 2.31,
+      "learning_rate": 7.469067010562539e-05,
+      "loss": 0.3393,
+      "step": 267
+    },
+    {
+      "epoch": 2.32,
+      "learning_rate": 7.400368888479048e-05,
+      "loss": 0.3125,
+      "step": 268
+    },
+    {
+      "epoch": 2.33,
+      "learning_rate": 7.331802096746308e-05,
+      "loss": 0.3083,
+      "step": 269
+    },
+    {
+      "epoch": 2.34,
+      "learning_rate": 7.263370099279172e-05,
+      "loss": 0.3188,
+      "step": 270
+    },
+    {
+      "epoch": 2.35,
+      "learning_rate": 7.195076353182834e-05,
+      "loss": 0.3071,
+      "step": 271
+    },
+    {
+      "epoch": 2.36,
+      "learning_rate": 7.126924308578196e-05,
+      "loss": 0.3223,
+      "step": 272
+    },
+    {
+      "epoch": 2.36,
+      "learning_rate": 7.058917408427558e-05,
+      "loss": 0.3201,
+      "step": 273
+    },
+    {
+      "epoch": 2.37,
+      "learning_rate": 6.991059088360699e-05,
+      "loss": 0.3237,
+      "step": 274
+    },
+    {
+      "epoch": 2.38,
+      "learning_rate": 6.923352776501301e-05,
+      "loss": 0.3172,
+      "step": 275
+    },
+    {
+      "epoch": 2.39,
+      "learning_rate": 6.855801893293765e-05,
+      "loss": 0.3188,
+      "step": 276
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 6.788409851330418e-05,
+      "loss": 0.3268,
+      "step": 277
+    },
+    {
+      "epoch": 2.41,
+      "learning_rate": 6.721180055179112e-05,
+      "loss": 0.3256,
+      "step": 278
+    },
+    {
+      "epoch": 2.42,
+      "learning_rate": 6.654115901211229e-05,
+      "loss": 0.3216,
+      "step": 279
+    },
+    {
+      "epoch": 2.43,
+      "learning_rate": 6.587220777430096e-05,
+      "loss": 0.3304,
+      "step": 280
+    },
+    {
+      "epoch": 2.44,
+      "learning_rate": 6.52049806329984e-05,
+      "loss": 0.3246,
+      "step": 281
+    },
+    {
+      "epoch": 2.44,
+      "learning_rate": 6.453951129574644e-05,
+      "loss": 0.3072,
+      "step": 282
+    },
+    {
+      "epoch": 2.45,
+      "learning_rate": 6.387583338128471e-05,
+      "loss": 0.3169,
+      "step": 283
+    },
+    {
+      "epoch": 2.46,
+      "learning_rate": 6.321398041785224e-05,
+      "loss": 0.3219,
+      "step": 284
+    },
+    {
+      "epoch": 2.47,
+      "learning_rate": 6.255398584149365e-05,
+      "loss": 0.3529,
+      "step": 285
+    },
+    {
+      "epoch": 2.48,
+      "learning_rate": 6.189588299436997e-05,
+      "loss": 0.3114,
+      "step": 286
+    },
+    {
+      "epoch": 2.49,
+      "learning_rate": 6.123970512307434e-05,
+      "loss": 0.3205,
+      "step": 287
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 6.0585485376952244e-05,
+      "loss": 0.3113,
+      "step": 288
+    },
+    {
+      "epoch": 2.51,
+      "learning_rate": 5.993325680642705e-05,
+      "loss": 0.3439,
+      "step": 289
+    },
+    {
+      "epoch": 2.51,
+      "learning_rate": 5.928305236133016e-05,
+      "loss": 0.3117,
+      "step": 290
+    },
+    {
+      "epoch": 2.51,
+      "eval_loss": 0.3381350338459015,
+      "eval_runtime": 35.0904,
+      "eval_samples_per_second": 173.666,
+      "eval_steps_per_second": 12.425,
+      "step": 290
+    },
+    {
+      "epoch": 2.52,
+      "learning_rate": 5.8634904889236595e-05,
+      "loss": 0.3109,
+      "step": 291
+    },
+    {
+      "epoch": 2.53,
+      "learning_rate": 5.798884713380541e-05,
+      "loss": 0.3154,
+      "step": 292
+    },
+    {
+      "epoch": 2.54,
+      "learning_rate": 5.7344911733125594e-05,
+      "loss": 0.3407,
+      "step": 293
+    },
+    {
+      "epoch": 2.55,
+      "learning_rate": 5.67031312180673e-05,
+      "loss": 0.3118,
+      "step": 294
+    },
+    {
+      "epoch": 2.56,
+      "learning_rate": 5.6063538010638293e-05,
+      "loss": 0.3221,
+      "step": 295
+    },
+    {
+      "epoch": 2.57,
+      "learning_rate": 5.542616442234618e-05,
+      "loss": 0.3153,
+      "step": 296
+    },
+    {
+      "epoch": 2.58,
+      "learning_rate": 5.479104265256583e-05,
+      "loss": 0.3267,
+      "step": 297
+    },
+    {
+      "epoch": 2.58,
+      "learning_rate": 5.4158204786913005e-05,
+      "loss": 0.3294,
+      "step": 298
+    },
+    {
+      "epoch": 2.59,
+      "learning_rate": 5.3527682795623146e-05,
+      "loss": 0.3428,
+      "step": 299
+    },
+    {
+      "epoch": 2.6,
+      "learning_rate": 5.289950853193653e-05,
+      "loss": 0.3342,
+      "step": 300
+    },
+    {
+      "epoch": 2.61,
+      "learning_rate": 5.227371373048888e-05,
+      "loss": 0.3362,
+      "step": 301
+    },
+    {
+      "epoch": 2.62,
+      "learning_rate": 5.165033000570825e-05,
+      "loss": 0.3542,
+      "step": 302
+    },
+    {
+      "epoch": 2.63,
+      "learning_rate": 5.102938885021793e-05,
+      "loss": 0.3252,
+      "step": 303
+    },
+    {
+      "epoch": 2.64,
+      "learning_rate": 5.041092163324537e-05,
+      "loss": 0.3205,
+      "step": 304
+    },
+    {
+      "epoch": 2.65,
+      "learning_rate": 4.979495959903759e-05,
+      "loss": 0.3201,
+      "step": 305
+    },
+    {
+      "epoch": 2.65,
+      "learning_rate": 4.9181533865282706e-05,
+      "loss": 0.3181,
+      "step": 306
+    },
+    {
+      "epoch": 2.66,
+      "learning_rate": 4.857067542153768e-05,
+      "loss": 0.3116,
+      "step": 307
+    },
+    {
+      "epoch": 2.67,
+      "learning_rate": 4.796241512766326e-05,
+      "loss": 0.3212,
+      "step": 308
+    },
+    {
+      "epoch": 2.68,
+      "learning_rate": 4.735678371226441e-05,
+      "loss": 0.3267,
+      "step": 309
+    },
+    {
+      "epoch": 2.69,
+      "learning_rate": 4.6753811771138364e-05,
+      "loss": 0.3027,
+      "step": 310
+    },
+    {
+      "epoch": 2.7,
+      "learning_rate": 4.6153529765728664e-05,
+      "loss": 0.3068,
+      "step": 311
+    },
+    {
+      "epoch": 2.71,
+      "learning_rate": 4.555596802158653e-05,
+      "loss": 0.3326,
+      "step": 312
+    },
+    {
+      "epoch": 2.72,
+      "learning_rate": 4.496115672683873e-05,
+      "loss": 0.3259,
+      "step": 313
+    },
+    {
+      "epoch": 2.73,
+      "learning_rate": 4.4369125930662404e-05,
+      "loss": 0.3211,
+      "step": 314
+    },
+    {
+      "epoch": 2.73,
+      "learning_rate": 4.3779905541767285e-05,
+      "loss": 0.3174,
+      "step": 315
+    },
+    {
+      "epoch": 2.74,
+      "learning_rate": 4.3193525326884435e-05,
+      "loss": 0.3151,
+      "step": 316
+    },
+    {
+      "epoch": 2.75,
+      "learning_rate": 4.261001490926272e-05,
+      "loss": 0.3207,
+      "step": 317
+    },
+    {
+      "epoch": 2.76,
+      "learning_rate": 4.202940376717217e-05,
+      "loss": 0.3101,
+      "step": 318
+    },
+    {
+      "epoch": 2.77,
+      "learning_rate": 4.14517212324147e-05,
+      "loss": 0.3207,
+      "step": 319
+    },
+    {
+      "epoch": 2.77,
+      "eval_loss": 0.3378981053829193,
+      "eval_runtime": 35.0668,
+      "eval_samples_per_second": 173.782,
+      "eval_steps_per_second": 12.433,
+      "step": 319
+    },
+    {
+      "epoch": 2.78,
+      "learning_rate": 4.087699648884248e-05,
+      "loss": 0.3089,
+      "step": 320
+    },
+    {
+      "epoch": 2.79,
+      "learning_rate": 4.0305258570883334e-05,
+      "loss": 0.3416,
+      "step": 321
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 3.973653636207437e-05,
+      "loss": 0.3072,
+      "step": 322
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 3.9170858593602336e-05,
+      "loss": 0.3364,
+      "step": 323
+    },
+    {
+      "epoch": 2.81,
+      "learning_rate": 3.860825384285247e-05,
+      "loss": 0.3233,
+      "step": 324
+    },
+    {
+      "epoch": 2.82,
+      "learning_rate": 3.804875053196477e-05,
+      "loss": 0.318,
+      "step": 325
+    },
+    {
+      "epoch": 2.83,
+      "learning_rate": 3.7492376926397964e-05,
+      "loss": 0.319,
+      "step": 326
+    },
+    {
+      "epoch": 2.84,
+      "learning_rate": 3.693916113350182e-05,
+      "loss": 0.324,
+      "step": 327
+    },
+    {
+      "epoch": 2.85,
+      "learning_rate": 3.638913110109695e-05,
+      "loss": 0.3196,
+      "step": 328
+    },
+    {
+      "epoch": 2.86,
+      "learning_rate": 3.5842314616063135e-05,
+      "loss": 0.3186,
+      "step": 329
+    },
+    {
+      "epoch": 2.87,
+      "learning_rate": 3.529873930293546e-05,
+      "loss": 0.3073,
+      "step": 330
+    },
+    {
+      "epoch": 2.87,
+      "learning_rate": 3.4758432622508674e-05,
+      "loss": 0.3427,
+      "step": 331
+    },
+    {
+      "epoch": 2.88,
+      "learning_rate": 3.422142187045011e-05,
+      "loss": 0.3064,
+      "step": 332
+    },
+    {
+      "epoch": 2.89,
+      "learning_rate": 3.36877341759205e-05,
+      "loss": 0.3278,
+      "step": 333
+    },
+    {
+      "epoch": 2.9,
+      "learning_rate": 3.315739650020365e-05,
+      "loss": 0.347,
+      "step": 334
+    },
+    {
+      "epoch": 2.91,
+      "learning_rate": 3.263043563534428e-05,
+      "loss": 0.3269,
+      "step": 335
+    },
+    {
+      "epoch": 2.92,
+      "learning_rate": 3.210687820279451e-05,
+      "loss": 0.3122,
+      "step": 336
+    },
+    {
+      "epoch": 2.93,
+      "learning_rate": 3.1586750652069074e-05,
+      "loss": 0.3223,
+      "step": 337
+    },
+    {
+      "epoch": 2.94,
+      "learning_rate": 3.107007925940894e-05,
+      "loss": 0.3208,
+      "step": 338
+    },
+    {
+      "epoch": 2.95,
+      "learning_rate": 3.0556890126454075e-05,
+      "loss": 0.3073,
+      "step": 339
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 452,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 113,
+  "total_flos": 1.5721793414108283e+19,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-339/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66dc6af509e7f51c6a244a8a9af44ad9044f2ab26d51bec3236dad93ab4ef46c
+size 4923

checkpoint-452/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: codellama/CodeLlama-34b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.8.2

checkpoint-452/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "codellama/CodeLlama-34b-hf",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

checkpoint-452/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54a05003644b645a327b51d5c317ecc913ac1f4a1bd09c137ea2cf65e36aaf53
+size 1920164848

checkpoint-452/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8b95943c4dd1b49bcaae543885a0299860f305a201791d18df138376bbbcbe1
+size 1743147421