Upload 12 files

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-2000/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/qwen2-vl-7b-instruct-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b37bba893ac6f60851c2c6505ab77608f56958e93fa958314f4ef7ab9d42956b
+size 406940800

checkpoint-2000/chat_template.jinja

@@ -0,0 +1,7 @@
+{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

checkpoint-2000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8180a2fd5ec8ecfb09e4b7e54835dfdc155dd80a23a98cabeab3724e33510198
+size 207370053

checkpoint-2000/processor_config.json

@@ -0,0 +1,43 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "merge_size": 2,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen2VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": false,
+    "image_mean": [0.48145466, 0.4578275, 0.40821073],
+    "image_std": [0.26862954, 0.26130258, 0.27577711],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 14,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 12845056,
+      "shortest_edge": 3136
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen2VLVideoProcessor"
+  }
+}

checkpoint-2000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60a3244e0daf7f85a8316a7f5740690bfd1864fc578f9e5e8cd2a4f6d14224f
+size 14645

checkpoint-2000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:736a194599fadbae29f991bafad676654a4244410a3b7590207d5b38dd5cd23f
+size 1383

checkpoint-2000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ed3bce79b126da3b8d88e35f35ae9a54453cbf7c2af1ddc090a9ae8b0a3ecda
+size 1465

checkpoint-2000/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54f71a9b8218fde29235fc188308e107371d8990b58bfbe924e1e00788f96834
+size 12178018

checkpoint-2000/tokenizer_config.json

@@ -0,0 +1,130 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "\u003C|im_end|\u003E",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "\u003C|vision_pad|\u003E",
+  "padding_side": "right",
+  "processor_class": "Qwen2VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "\u003C|endoftext|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "\u003C|im_start|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151645": {
+      "content": "\u003C|im_end|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151646": {
+      "content": "\u003C|object_ref_start|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "\u003C|object_ref_end|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151648": {
+      "content": "\u003C|box_start|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151649": {
+      "content": "\u003C|box_end|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151650": {
+      "content": "\u003C|quad_start|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "\u003C|quad_end|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "\u003C|vision_start|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "\u003C|vision_end|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "\u003C|vision_pad|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "\u003C|image_pad|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "\u003C|video_pad|\u003E",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-2000/trainer_state.json

@@ -0,0 +1,1466 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.6132147784761612,
+  "eval_steps": 500,
+  "global_step": 2000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0030660738923808063,
+      "grad_norm": 7.450107097625732,
+      "learning_rate": 7.777777777777777e-06,
+      "loss": 2.5661365509033205,
+      "step": 10
+    },
+    {
+      "epoch": 0.006132147784761613,
+      "grad_norm": 5.506503582000732,
+      "learning_rate": 1.888888888888889e-05,
+      "loss": 1.527136993408203,
+      "step": 20
+    },
+    {
+      "epoch": 0.00919822167714242,
+      "grad_norm": 2.2868926525115967,
+      "learning_rate": 3e-05,
+      "loss": 0.5317483901977539,
+      "step": 30
+    },
+    {
+      "epoch": 0.012264295569523225,
+      "grad_norm": 3.0989599227905273,
+      "learning_rate": 4.111111111111111e-05,
+      "loss": 0.5250907421112061,
+      "step": 40
+    },
+    {
+      "epoch": 0.015330369461904032,
+      "grad_norm": 2.8500075340270996,
+      "learning_rate": 5.222222222222223e-05,
+      "loss": 0.40924715995788574,
+      "step": 50
+    },
+    {
+      "epoch": 0.01839644335428484,
+      "grad_norm": 2.6407477855682373,
+      "learning_rate": 6.333333333333333e-05,
+      "loss": 0.4055685520172119,
+      "step": 60
+    },
+    {
+      "epoch": 0.021462517246665643,
+      "grad_norm": 2.8376622200012207,
+      "learning_rate": 7.444444444444444e-05,
+      "loss": 0.3623446226119995,
+      "step": 70
+    },
+    {
+      "epoch": 0.02452859113904645,
+      "grad_norm": 2.078441858291626,
+      "learning_rate": 8.555555555555556e-05,
+      "loss": 0.4112717628479004,
+      "step": 80
+    },
+    {
+      "epoch": 0.027594665031427257,
+      "grad_norm": 1.9966775178909302,
+      "learning_rate": 9.666666666666667e-05,
+      "loss": 0.38550827503204343,
+      "step": 90
+    },
+    {
+      "epoch": 0.030660738923808065,
+      "grad_norm": 2.3640847206115723,
+      "learning_rate": 9.999857226439791e-05,
+      "loss": 0.3606630086898804,
+      "step": 100
+    },
+    {
+      "epoch": 0.03372681281618887,
+      "grad_norm": 2.3264355659484863,
+      "learning_rate": 9.999157946997989e-05,
+      "loss": 0.3747897148132324,
+      "step": 110
+    },
+    {
+      "epoch": 0.03679288670856968,
+      "grad_norm": 2.0503554344177246,
+      "learning_rate": 9.997876019358084e-05,
+      "loss": 0.38175716400146487,
+      "step": 120
+    },
+    {
+      "epoch": 0.03985896060095048,
+      "grad_norm": 2.286015510559082,
+      "learning_rate": 9.9960115929278e-05,
+      "loss": 0.45613727569580076,
+      "step": 130
+    },
+    {
+      "epoch": 0.042925034493331286,
+      "grad_norm": 3.082271099090576,
+      "learning_rate": 9.993564885004674e-05,
+      "loss": 0.3672828674316406,
+      "step": 140
+    },
+    {
+      "epoch": 0.0459911083857121,
+      "grad_norm": 2.1444807052612305,
+      "learning_rate": 9.990536180750725e-05,
+      "loss": 0.3141984462738037,
+      "step": 150
+    },
+    {
+      "epoch": 0.0490571822780929,
+      "grad_norm": 2.218291759490967,
+      "learning_rate": 9.986925833159214e-05,
+      "loss": 0.3288247108459473,
+      "step": 160
+    },
+    {
+      "epoch": 0.05212325617047371,
+      "grad_norm": 1.481544017791748,
+      "learning_rate": 9.982734263013508e-05,
+      "loss": 0.36602396965026857,
+      "step": 170
+    },
+    {
+      "epoch": 0.055189330062854515,
+      "grad_norm": 2.5504465103149414,
+      "learning_rate": 9.97796195883804e-05,
+      "loss": 0.3282037258148193,
+      "step": 180
+    },
+    {
+      "epoch": 0.05825540395523532,
+      "grad_norm": 3.492237091064453,
+      "learning_rate": 9.972609476841367e-05,
+      "loss": 0.27347455024719236,
+      "step": 190
+    },
+    {
+      "epoch": 0.06132147784761613,
+      "grad_norm": 1.997891902923584,
+      "learning_rate": 9.966677440851346e-05,
+      "loss": 0.31744699478149413,
+      "step": 200
+    },
+    {
+      "epoch": 0.06438755173999694,
+      "grad_norm": 2.3568949699401855,
+      "learning_rate": 9.960166542242429e-05,
+      "loss": 0.2679970502853394,
+      "step": 210
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 2.281832218170166,
+      "learning_rate": 9.953077539855078e-05,
+      "loss": 0.31074023246765137,
+      "step": 220
+    },
+    {
+      "epoch": 0.07051969952475855,
+      "grad_norm": 1.9163970947265625,
+      "learning_rate": 9.945411259907334e-05,
+      "loss": 0.3659813642501831,
+      "step": 230
+    },
+    {
+      "epoch": 0.07358577341713936,
+      "grad_norm": 2.3353500366210938,
+      "learning_rate": 9.93716859589851e-05,
+      "loss": 0.2725929021835327,
+      "step": 240
+    },
+    {
+      "epoch": 0.07665184730952015,
+      "grad_norm": 1.492147445678711,
+      "learning_rate": 9.928350508505058e-05,
+      "loss": 0.3163760185241699,
+      "step": 250
+    },
+    {
+      "epoch": 0.07971792120190097,
+      "grad_norm": 2.056483507156372,
+      "learning_rate": 9.918958025468613e-05,
+      "loss": 0.3508399486541748,
+      "step": 260
+    },
+    {
+      "epoch": 0.08278399509428178,
+      "grad_norm": 1.6008816957473755,
+      "learning_rate": 9.908992241476188e-05,
+      "loss": 0.2680958271026611,
+      "step": 270
+    },
+    {
+      "epoch": 0.08585006898666257,
+      "grad_norm": 2.216665744781494,
+      "learning_rate": 9.898454318032616e-05,
+      "loss": 0.35864193439483644,
+      "step": 280
+    },
+    {
+      "epoch": 0.08891614287904338,
+      "grad_norm": 1.8785314559936523,
+      "learning_rate": 9.887345483325152e-05,
+      "loss": 0.3466702222824097,
+      "step": 290
+    },
+    {
+      "epoch": 0.0919822167714242,
+      "grad_norm": 2.222696304321289,
+      "learning_rate": 9.875667032080353e-05,
+      "loss": 0.35419282913208006,
+      "step": 300
+    },
+    {
+      "epoch": 0.095048290663805,
+      "grad_norm": 2.4683263301849365,
+      "learning_rate": 9.863420325413155e-05,
+      "loss": 0.3440497875213623,
+      "step": 310
+    },
+    {
+      "epoch": 0.0981143645561858,
+      "grad_norm": 4.092799663543701,
+      "learning_rate": 9.850606790668256e-05,
+      "loss": 0.2708487033843994,
+      "step": 320
+    },
+    {
+      "epoch": 0.10118043844856661,
+      "grad_norm": 1.681843638420105,
+      "learning_rate": 9.837227921253746e-05,
+      "loss": 0.35094077587127687,
+      "step": 330
+    },
+    {
+      "epoch": 0.10424651234094742,
+      "grad_norm": 1.2097547054290771,
+      "learning_rate": 9.82328527646706e-05,
+      "loss": 0.25490579605102537,
+      "step": 340
+    },
+    {
+      "epoch": 0.10731258623332822,
+      "grad_norm": 2.540566921234131,
+      "learning_rate": 9.808780481313235e-05,
+      "loss": 0.2878111362457275,
+      "step": 350
+    },
+    {
+      "epoch": 0.11037866012570903,
+      "grad_norm": 2.9934723377227783,
+      "learning_rate": 9.79371522631553e-05,
+      "loss": 0.25048825740814207,
+      "step": 360
+    },
+    {
+      "epoch": 0.11344473401808984,
+      "grad_norm": 2.335675001144409,
+      "learning_rate": 9.778091267318382e-05,
+      "loss": 0.32183692455291746,
+      "step": 370
+    },
+    {
+      "epoch": 0.11651080791047064,
+      "grad_norm": 2.681945562362671,
+      "learning_rate": 9.761910425282767e-05,
+      "loss": 0.28128464221954347,
+      "step": 380
+    },
+    {
+      "epoch": 0.11957688180285145,
+      "grad_norm": 3.958214044570923,
+      "learning_rate": 9.745174586073981e-05,
+      "loss": 0.3246151924133301,
+      "step": 390
+    },
+    {
+      "epoch": 0.12264295569523226,
+      "grad_norm": 2.3698863983154297,
+      "learning_rate": 9.727885700241828e-05,
+      "loss": 0.28882100582122805,
+      "step": 400
+    },
+    {
+      "epoch": 0.12570902958761307,
+      "grad_norm": 1.922576665878296,
+      "learning_rate": 9.710045782793292e-05,
+      "loss": 0.33541226387023926,
+      "step": 410
+    },
+    {
+      "epoch": 0.12877510347999388,
+      "grad_norm": 1.9901965856552124,
+      "learning_rate": 9.691656912957685e-05,
+      "loss": 0.2766454219818115,
+      "step": 420
+    },
+    {
+      "epoch": 0.13184117737237466,
+      "grad_norm": 2.1355180740356445,
+      "learning_rate": 9.672721233944327e-05,
+      "loss": 0.30619628429412843,
+      "step": 430
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 3.792794704437256,
+      "learning_rate": 9.653240952692737e-05,
+      "loss": 0.23548378944396972,
+      "step": 440
+    },
+    {
+      "epoch": 0.13797332515713628,
+      "grad_norm": 2.3504602909088135,
+      "learning_rate": 9.633218339615433e-05,
+      "loss": 0.2730306386947632,
+      "step": 450
+    },
+    {
+      "epoch": 0.1410393990495171,
+      "grad_norm": 2.5973148345947266,
+      "learning_rate": 9.612655728333305e-05,
+      "loss": 0.27579216957092284,
+      "step": 460
+    },
+    {
+      "epoch": 0.1441054729418979,
+      "grad_norm": 2.1677474975585938,
+      "learning_rate": 9.591555515403644e-05,
+      "loss": 0.29922962188720703,
+      "step": 470
+    },
+    {
+      "epoch": 0.14717154683427872,
+      "grad_norm": 2.391791582107544,
+      "learning_rate": 9.569920160040815e-05,
+      "loss": 0.2929323434829712,
+      "step": 480
+    },
+    {
+      "epoch": 0.1502376207266595,
+      "grad_norm": 1.499917984008789,
+      "learning_rate": 9.547752183829642e-05,
+      "loss": 0.2706033229827881,
+      "step": 490
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "grad_norm": 1.5111459493637085,
+      "learning_rate": 9.525054170431522e-05,
+      "loss": 0.31374077796936034,
+      "step": 500
+    },
+    {
+      "epoch": 0.1533036946190403,
+      "eval_loss": 0.25455766916275024,
+      "eval_runtime": 692.9115,
+      "eval_samples_per_second": 4.708,
+      "eval_steps_per_second": 1.178,
+      "step": 500
+    },
+    {
+      "epoch": 0.15636976851142112,
+      "grad_norm": 2.337453603744507,
+      "learning_rate": 9.501828765283295e-05,
+      "loss": 0.364536714553833,
+      "step": 510
+    },
+    {
+      "epoch": 0.15943584240380193,
+      "grad_norm": 1.7297507524490356,
+      "learning_rate": 9.478078675288923e-05,
+      "loss": 0.2583449840545654,
+      "step": 520
+    },
+    {
+      "epoch": 0.16250191629618274,
+      "grad_norm": 1.3506821393966675,
+      "learning_rate": 9.453806668504007e-05,
+      "loss": 0.22310216426849366,
+      "step": 530
+    },
+    {
+      "epoch": 0.16556799018856355,
+      "grad_norm": 1.325363039970398,
+      "learning_rate": 9.429015573813163e-05,
+      "loss": 0.22504372596740724,
+      "step": 540
+    },
+    {
+      "epoch": 0.16863406408094436,
+      "grad_norm": 2.389620304107666,
+      "learning_rate": 9.403708280600323e-05,
+      "loss": 0.23236045837402344,
+      "step": 550
+    },
+    {
+      "epoch": 0.17170013797332515,
+      "grad_norm": 1.993188500404358,
+      "learning_rate": 9.377887738411981e-05,
+      "loss": 0.30072014331817626,
+      "step": 560
+    },
+    {
+      "epoch": 0.17476621186570596,
+      "grad_norm": 2.9557931423187256,
+      "learning_rate": 9.351556956613423e-05,
+      "loss": 0.23287551403045653,
+      "step": 570
+    },
+    {
+      "epoch": 0.17783228575808677,
+      "grad_norm": 1.7986005544662476,
+      "learning_rate": 9.324719004037985e-05,
+      "loss": 0.2973225831985474,
+      "step": 580
+    },
+    {
+      "epoch": 0.18089835965046758,
+      "grad_norm": 1.8435581922531128,
+      "learning_rate": 9.297377008629388e-05,
+      "loss": 0.29309306144714353,
+      "step": 590
+    },
+    {
+      "epoch": 0.1839644335428484,
+      "grad_norm": 2.1855595111846924,
+      "learning_rate": 9.269534157077177e-05,
+      "loss": 0.3034528255462646,
+      "step": 600
+    },
+    {
+      "epoch": 0.1870305074352292,
+      "grad_norm": 1.471543550491333,
+      "learning_rate": 9.24119369444531e-05,
+      "loss": 0.23076698780059815,
+      "step": 610
+    },
+    {
+      "epoch": 0.19009658132761,
+      "grad_norm": 1.316425085067749,
+      "learning_rate": 9.212358923793958e-05,
+      "loss": 0.24202566146850585,
+      "step": 620
+    },
+    {
+      "epoch": 0.1931626552199908,
+      "grad_norm": 2.9610955715179443,
+      "learning_rate": 9.183033205794525e-05,
+      "loss": 0.19877578020095826,
+      "step": 630
+    },
+    {
+      "epoch": 0.1962287291123716,
+      "grad_norm": 1.3904002904891968,
+      "learning_rate": 9.153219958337968e-05,
+      "loss": 0.27364373207092285,
+      "step": 640
+    },
+    {
+      "epoch": 0.1992948030047524,
+      "grad_norm": 1.7155065536499023,
+      "learning_rate": 9.12292265613645e-05,
+      "loss": 0.2856669664382935,
+      "step": 650
+    },
+    {
+      "epoch": 0.20236087689713322,
+      "grad_norm": 1.7228351831436157,
+      "learning_rate": 9.092144830318358e-05,
+      "loss": 0.2826208591461182,
+      "step": 660
+    },
+    {
+      "epoch": 0.20542695078951403,
+      "grad_norm": 2.2020068168640137,
+      "learning_rate": 9.060890068016752e-05,
+      "loss": 0.30395419597625734,
+      "step": 670
+    },
+    {
+      "epoch": 0.20849302468189485,
+      "grad_norm": 2.317624568939209,
+      "learning_rate": 9.029162011951292e-05,
+      "loss": 0.2660128831863403,
+      "step": 680
+    },
+    {
+      "epoch": 0.21155909857427563,
+      "grad_norm": 1.733163595199585,
+      "learning_rate": 8.99696436000368e-05,
+      "loss": 0.21881954669952391,
+      "step": 690
+    },
+    {
+      "epoch": 0.21462517246665644,
+      "grad_norm": 1.558132529258728,
+      "learning_rate": 8.964300864786676e-05,
+      "loss": 0.33716707229614257,
+      "step": 700
+    },
+    {
+      "epoch": 0.21769124635903725,
+      "grad_norm": 2.5993547439575195,
+      "learning_rate": 8.931175333206727e-05,
+      "loss": 0.24190351963043213,
+      "step": 710
+    },
+    {
+      "epoch": 0.22075732025141806,
+      "grad_norm": 2.0784964561462402,
+      "learning_rate": 8.897591626020284e-05,
+      "loss": 0.2015916347503662,
+      "step": 720
+    },
+    {
+      "epoch": 0.22382339414379887,
+      "grad_norm": 1.6791867017745972,
+      "learning_rate": 8.863553657383828e-05,
+      "loss": 0.23386077880859374,
+      "step": 730
+    },
+    {
+      "epoch": 0.22688946803617968,
+      "grad_norm": 1.8661366701126099,
+      "learning_rate": 8.829065394397679e-05,
+      "loss": 0.1945809841156006,
+      "step": 740
+    },
+    {
+      "epoch": 0.2299555419285605,
+      "grad_norm": 2.4303054809570312,
+      "learning_rate": 8.794130856643634e-05,
+      "loss": 0.17795398235321044,
+      "step": 750
+    },
+    {
+      "epoch": 0.23302161582094127,
+      "grad_norm": 2.089738130569458,
+      "learning_rate": 8.758754115716487e-05,
+      "loss": 0.23645341396331787,
+      "step": 760
+    },
+    {
+      "epoch": 0.23608768971332209,
+      "grad_norm": 1.6034516096115112,
+      "learning_rate": 8.72293929474949e-05,
+      "loss": 0.22587378025054933,
+      "step": 770
+    },
+    {
+      "epoch": 0.2391537636057029,
+      "grad_norm": 1.680909276008606,
+      "learning_rate": 8.686690567933802e-05,
+      "loss": 0.3011012554168701,
+      "step": 780
+    },
+    {
+      "epoch": 0.2422198374980837,
+      "grad_norm": 1.4276477098464966,
+      "learning_rate": 8.650012160031988e-05,
+      "loss": 0.19034912586212158,
+      "step": 790
+    },
+    {
+      "epoch": 0.24528591139046452,
+      "grad_norm": 1.430630087852478,
+      "learning_rate": 8.612908345885631e-05,
+      "loss": 0.20189294815063477,
+      "step": 800
+    },
+    {
+      "epoch": 0.24835198528284533,
+      "grad_norm": 1.647044062614441,
+      "learning_rate": 8.575383449917102e-05,
+      "loss": 0.23400452136993408,
+      "step": 810
+    },
+    {
+      "epoch": 0.25141805917522614,
+      "grad_norm": 1.6889570951461792,
+      "learning_rate": 8.537441845625551e-05,
+      "loss": 0.23393361568450927,
+      "step": 820
+    },
+    {
+      "epoch": 0.2544841330676069,
+      "grad_norm": 1.1612548828125,
+      "learning_rate": 8.499087955077172e-05,
+      "loss": 0.1864438533782959,
+      "step": 830
+    },
+    {
+      "epoch": 0.25755020695998776,
+      "grad_norm": 1.9415431022644043,
+      "learning_rate": 8.460326248389825e-05,
+      "loss": 0.2487419366836548,
+      "step": 840
+    },
+    {
+      "epoch": 0.26061628085236854,
+      "grad_norm": 2.2750728130340576,
+      "learning_rate": 8.421161243212041e-05,
+      "loss": 0.2196030855178833,
+      "step": 850
+    },
+    {
+      "epoch": 0.2636823547447493,
+      "grad_norm": 2.442432403564453,
+      "learning_rate": 8.381597504196497e-05,
+      "loss": 0.2734952449798584,
+      "step": 860
+    },
+    {
+      "epoch": 0.26674842863713016,
+      "grad_norm": 1.4768729209899902,
+      "learning_rate": 8.341639642468002e-05,
+      "loss": 0.24631156921386718,
+      "step": 870
+    },
+    {
+      "epoch": 0.26981450252951095,
+      "grad_norm": 1.5953449010849,
+      "learning_rate": 8.301292315086082e-05,
+      "loss": 0.22515568733215333,
+      "step": 880
+    },
+    {
+      "epoch": 0.2728805764218918,
+      "grad_norm": 1.9449270963668823,
+      "learning_rate": 8.2605602245022e-05,
+      "loss": 0.21427900791168214,
+      "step": 890
+    },
+    {
+      "epoch": 0.27594665031427257,
+      "grad_norm": 1.7873473167419434,
+      "learning_rate": 8.219448118011688e-05,
+      "loss": 0.24945287704467772,
+      "step": 900
+    },
+    {
+      "epoch": 0.2790127242066534,
+      "grad_norm": 2.5614044666290283,
+      "learning_rate": 8.177960787200451e-05,
+      "loss": 0.21404774188995362,
+      "step": 910
+    },
+    {
+      "epoch": 0.2820787980990342,
+      "grad_norm": 2.1977665424346924,
+      "learning_rate": 8.13610306738652e-05,
+      "loss": 0.2547955274581909,
+      "step": 920
+    },
+    {
+      "epoch": 0.28514487199141497,
+      "grad_norm": 2.0373003482818604,
+      "learning_rate": 8.093879837056487e-05,
+      "loss": 0.22160701751708983,
+      "step": 930
+    },
+    {
+      "epoch": 0.2882109458837958,
+      "grad_norm": 1.9001951217651367,
+      "learning_rate": 8.051296017296928e-05,
+      "loss": 0.26237494945526124,
+      "step": 940
+    },
+    {
+      "epoch": 0.2912770197761766,
+      "grad_norm": 1.690984845161438,
+      "learning_rate": 8.008356571220854e-05,
+      "loss": 0.20300838947296143,
+      "step": 950
+    },
+    {
+      "epoch": 0.29434309366855743,
+      "grad_norm": 1.7940173149108887,
+      "learning_rate": 7.965066503389264e-05,
+      "loss": 0.23636441230773925,
+      "step": 960
+    },
+    {
+      "epoch": 0.2974091675609382,
+      "grad_norm": 1.809225082397461,
+      "learning_rate": 7.921430859227867e-05,
+      "loss": 0.21343977451324464,
+      "step": 970
+    },
+    {
+      "epoch": 0.300475241453319,
+      "grad_norm": 1.5834722518920898,
+      "learning_rate": 7.877454724439033e-05,
+      "loss": 0.19342999458312987,
+      "step": 980
+    },
+    {
+      "epoch": 0.30354131534569984,
+      "grad_norm": 1.7889691591262817,
+      "learning_rate": 7.833143224409075e-05,
+      "loss": 0.2588458299636841,
+      "step": 990
+    },
+    {
+      "epoch": 0.3066073892380806,
+      "grad_norm": 6.229308605194092,
+      "learning_rate": 7.788501523610871e-05,
+      "loss": 0.2804019689559937,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3066073892380806,
+      "eval_loss": 0.21068349480628967,
+      "eval_runtime": 689.1125,
+      "eval_samples_per_second": 4.734,
+      "eval_steps_per_second": 1.184,
+      "step": 1000
+    },
+    {
+      "epoch": 0.30967346313046146,
+      "grad_norm": 1.4504352807998657,
+      "learning_rate": 7.743534825001963e-05,
+      "loss": 0.20621588230133056,
+      "step": 1010
+    },
+    {
+      "epoch": 0.31273953702284224,
+      "grad_norm": 1.2956318855285645,
+      "learning_rate": 7.698248369418146e-05,
+      "loss": 0.2585979223251343,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3158056109152231,
+      "grad_norm": 1.54957914352417,
+      "learning_rate": 7.652647434962657e-05,
+      "loss": 0.26405982971191405,
+      "step": 1030
+    },
+    {
+      "epoch": 0.31887168480760386,
+      "grad_norm": 2.4380619525909424,
+      "learning_rate": 7.606737336391016e-05,
+      "loss": 0.27477028369903567,
+      "step": 1040
+    },
+    {
+      "epoch": 0.32193775869998464,
+      "grad_norm": 1.520289421081543,
+      "learning_rate": 7.560523424491595e-05,
+      "loss": 0.1743034839630127,
+      "step": 1050
+    },
+    {
+      "epoch": 0.3250038325923655,
+      "grad_norm": 0.9853711128234863,
+      "learning_rate": 7.514011085461982e-05,
+      "loss": 0.18591341972351075,
+      "step": 1060
+    },
+    {
+      "epoch": 0.32806990648474627,
+      "grad_norm": 1.815639615058899,
+      "learning_rate": 7.467205740281238e-05,
+      "loss": 0.23293919563293458,
+      "step": 1070
+    },
+    {
+      "epoch": 0.3311359803771271,
+      "grad_norm": 1.372294545173645,
+      "learning_rate": 7.420112844078065e-05,
+      "loss": 0.146307110786438,
+      "step": 1080
+    },
+    {
+      "epoch": 0.3342020542695079,
+      "grad_norm": 1.3202286958694458,
+      "learning_rate": 7.372737885495036e-05,
+      "loss": 0.20859239101409913,
+      "step": 1090
+    },
+    {
+      "epoch": 0.3372681281618887,
+      "grad_norm": 2.557431936264038,
+      "learning_rate": 7.325086386048884e-05,
+      "loss": 0.23643121719360352,
+      "step": 1100
+    },
+    {
+      "epoch": 0.3403342020542695,
+      "grad_norm": 1.7925618886947632,
+      "learning_rate": 7.277163899486975e-05,
+      "loss": 0.25189568996429446,
+      "step": 1110
+    },
+    {
+      "epoch": 0.3434002759466503,
+      "grad_norm": 1.8501105308532715,
+      "learning_rate": 7.228976011140027e-05,
+      "loss": 0.21449632644653321,
+      "step": 1120
+    },
+    {
+      "epoch": 0.34646634983903113,
+      "grad_norm": 2.3691437244415283,
+      "learning_rate": 7.18052833727114e-05,
+      "loss": 0.29492993354797364,
+      "step": 1130
+    },
+    {
+      "epoch": 0.3495324237314119,
+      "grad_norm": 1.8363765478134155,
+      "learning_rate": 7.13182652442123e-05,
+      "loss": 0.1969155788421631,
+      "step": 1140
+    },
+    {
+      "epoch": 0.35259849762379275,
+      "grad_norm": 1.9139550924301147,
+      "learning_rate": 7.082876248750921e-05,
+      "loss": 0.22846477031707763,
+      "step": 1150
+    },
+    {
+      "epoch": 0.35566457151617353,
+      "grad_norm": 1.1029242277145386,
+      "learning_rate": 7.033683215379002e-05,
+      "loss": 0.23001928329467775,
+      "step": 1160
+    },
+    {
+      "epoch": 0.35873064540855437,
+      "grad_norm": 2.5970096588134766,
+      "learning_rate": 6.984253157717487e-05,
+      "loss": 0.23762269020080568,
+      "step": 1170
+    },
+    {
+      "epoch": 0.36179671930093515,
+      "grad_norm": 1.4653462171554565,
+      "learning_rate": 6.934591836803404e-05,
+      "loss": 0.2510902643203735,
+      "step": 1180
+    },
+    {
+      "epoch": 0.36486279319331594,
+      "grad_norm": 1.2801283597946167,
+      "learning_rate": 6.88470504062734e-05,
+      "loss": 0.18679113388061525,
+      "step": 1190
+    },
+    {
+      "epoch": 0.3679288670856968,
+      "grad_norm": 1.4154472351074219,
+      "learning_rate": 6.834598583458862e-05,
+      "loss": 0.15953580141067505,
+      "step": 1200
+    },
+    {
+      "epoch": 0.37099494097807756,
+      "grad_norm": 1.4651672840118408,
+      "learning_rate": 6.784278305168859e-05,
+      "loss": 0.18714050054550171,
+      "step": 1210
+    },
+    {
+      "epoch": 0.3740610148704584,
+      "grad_norm": 1.9416940212249756,
+      "learning_rate": 6.73375007054892e-05,
+      "loss": 0.206192946434021,
+      "step": 1220
+    },
+    {
+      "epoch": 0.3771270887628392,
+      "grad_norm": 1.640125036239624,
+      "learning_rate": 6.683019768627795e-05,
+      "loss": 0.248970103263855,
+      "step": 1230
+    },
+    {
+      "epoch": 0.38019316265522,
+      "grad_norm": 1.0894057750701904,
+      "learning_rate": 6.632093311985019e-05,
+      "loss": 0.1613580346107483,
+      "step": 1240
+    },
+    {
+      "epoch": 0.3832592365476008,
+      "grad_norm": 2.472437620162964,
+      "learning_rate": 6.580976636061828e-05,
+      "loss": 0.2546850681304932,
+      "step": 1250
+    },
+    {
+      "epoch": 0.3863253104399816,
+      "grad_norm": 1.2982425689697266,
+      "learning_rate": 6.52967569846937e-05,
+      "loss": 0.25255069732666013,
+      "step": 1260
+    },
+    {
+      "epoch": 0.3893913843323624,
+      "grad_norm": 1.5834122896194458,
+      "learning_rate": 6.478196478294353e-05,
+      "loss": 0.2253199815750122,
+      "step": 1270
+    },
+    {
+      "epoch": 0.3924574582247432,
+      "grad_norm": 1.9751992225646973,
+      "learning_rate": 6.426544975402198e-05,
+      "loss": 0.23546898365020752,
+      "step": 1280
+    },
+    {
+      "epoch": 0.39552353211712404,
+      "grad_norm": 1.3470168113708496,
+      "learning_rate": 6.374727209737744e-05,
+      "loss": 0.15657265186309816,
+      "step": 1290
+    },
+    {
+      "epoch": 0.3985896060095048,
+      "grad_norm": 1.7273306846618652,
+      "learning_rate": 6.322749220623632e-05,
+      "loss": 0.19753172397613525,
+      "step": 1300
+    },
+    {
+      "epoch": 0.4016556799018856,
+      "grad_norm": 2.004589080810547,
+      "learning_rate": 6.270617066056437e-05,
+      "loss": 0.2822834014892578,
+      "step": 1310
+    },
+    {
+      "epoch": 0.40472175379426645,
+      "grad_norm": 1.2369407415390015,
+      "learning_rate": 6.218336822000598e-05,
+      "loss": 0.179297935962677,
+      "step": 1320
+    },
+    {
+      "epoch": 0.40778782768664723,
+      "grad_norm": 1.913866400718689,
+      "learning_rate": 6.165914581680278e-05,
+      "loss": 0.18271774053573608,
+      "step": 1330
+    },
+    {
+      "epoch": 0.41085390157902807,
+      "grad_norm": 2.212129592895508,
+      "learning_rate": 6.113356454869201e-05,
+      "loss": 0.2660691499710083,
+      "step": 1340
+    },
+    {
+      "epoch": 0.41391997547140885,
+      "grad_norm": 1.9875696897506714,
+      "learning_rate": 6.06066856717856e-05,
+      "loss": 0.22324562072753906,
+      "step": 1350
+    },
+    {
+      "epoch": 0.4169860493637897,
+      "grad_norm": 1.7668654918670654,
+      "learning_rate": 6.007857059343086e-05,
+      "loss": 0.21971983909606935,
+      "step": 1360
+    },
+    {
+      "epoch": 0.4200521232561705,
+      "grad_norm": 2.5111732482910156,
+      "learning_rate": 5.954928086505347e-05,
+      "loss": 0.17863379716873168,
+      "step": 1370
+    },
+    {
+      "epoch": 0.42311819714855126,
+      "grad_norm": 1.4877586364746094,
+      "learning_rate": 5.901887817498367e-05,
+      "loss": 0.17772235870361328,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4261842710409321,
+      "grad_norm": 2.3170790672302246,
+      "learning_rate": 5.848742434126664e-05,
+      "loss": 0.20021588802337648,
+      "step": 1390
+    },
+    {
+      "epoch": 0.4292503449333129,
+      "grad_norm": 1.2839077711105347,
+      "learning_rate": 5.79549813044575e-05,
+      "loss": 0.17556759119033813,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4323164188256937,
+      "grad_norm": 1.3416634798049927,
+      "learning_rate": 5.742161112040236e-05,
+      "loss": 0.2870814800262451,
+      "step": 1410
+    },
+    {
+      "epoch": 0.4353824927180745,
+      "grad_norm": 2.2512776851654053,
+      "learning_rate": 5.68873759530056e-05,
+      "loss": 0.2224407434463501,
+      "step": 1420
+    },
+    {
+      "epoch": 0.43844856661045534,
+      "grad_norm": 1.7091639041900635,
+      "learning_rate": 5.6352338066984744e-05,
+      "loss": 0.16897202730178834,
+      "step": 1430
+    },
+    {
+      "epoch": 0.4415146405028361,
+      "grad_norm": 1.6998759508132935,
+      "learning_rate": 5.581655982061367e-05,
+      "loss": 0.18889497518539428,
+      "step": 1440
+    },
+    {
+      "epoch": 0.4445807143952169,
+      "grad_norm": 1.4723140001296997,
+      "learning_rate": 5.528010365845466e-05,
+      "loss": 0.17898786067962646,
+      "step": 1450
+    },
+    {
+      "epoch": 0.44764678828759774,
+      "grad_norm": 1.0318090915679932,
+      "learning_rate": 5.474303210408062e-05,
+      "loss": 0.13469767570495605,
+      "step": 1460
+    },
+    {
+      "epoch": 0.4507128621799785,
+      "grad_norm": 2.836021661758423,
+      "learning_rate": 5.420540775278789e-05,
+      "loss": 0.1283479690551758,
+      "step": 1470
+    },
+    {
+      "epoch": 0.45377893607235936,
+      "grad_norm": 1.9320775270462036,
+      "learning_rate": 5.366729326430098e-05,
+      "loss": 0.20835576057434083,
+      "step": 1480
+    },
+    {
+      "epoch": 0.45684500996474015,
+      "grad_norm": 2.0982048511505127,
+      "learning_rate": 5.312875135546948e-05,
+      "loss": 0.20106544494628906,
+      "step": 1490
+    },
+    {
+      "epoch": 0.459911083857121,
+      "grad_norm": 1.9017882347106934,
+      "learning_rate": 5.258984479295852e-05,
+      "loss": 0.2131704568862915,
+      "step": 1500
+    },
+    {
+      "epoch": 0.459911083857121,
+      "eval_loss": 0.1820799559354782,
+      "eval_runtime": 687.6498,
+      "eval_samples_per_second": 4.744,
+      "eval_steps_per_second": 1.187,
+      "step": 1500
+    },
+    {
+      "epoch": 0.46297715774950177,
+      "grad_norm": 2.0524282455444336,
+      "learning_rate": 5.205063638593336e-05,
+      "loss": 0.17330008745193481,
+      "step": 1510
+    },
+    {
+      "epoch": 0.46604323164188255,
+      "grad_norm": 1.1175750494003296,
+      "learning_rate": 5.1511188978738964e-05,
+      "loss": 0.20958480834960938,
+      "step": 1520
+    },
+    {
+      "epoch": 0.4691093055342634,
+      "grad_norm": 1.053649663925171,
+      "learning_rate": 5.097156544357567e-05,
+      "loss": 0.2272329568862915,
+      "step": 1530
+    },
+    {
+      "epoch": 0.47217537942664417,
+      "grad_norm": 2.51870059967041,
+      "learning_rate": 5.0431828673171355e-05,
+      "loss": 0.2560974836349487,
+      "step": 1540
+    },
+    {
+      "epoch": 0.475241453319025,
+      "grad_norm": 1.1894735097885132,
+      "learning_rate": 4.98920415734514e-05,
+      "loss": 0.2311497688293457,
+      "step": 1550
+    },
+    {
+      "epoch": 0.4783075272114058,
+      "grad_norm": 1.782556414604187,
+      "learning_rate": 4.9352267056206994e-05,
+      "loss": 0.21084325313568114,
+      "step": 1560
+    },
+    {
+      "epoch": 0.4813736011037866,
+      "grad_norm": 2.159152030944824,
+      "learning_rate": 4.881256803176289e-05,
+      "loss": 0.17178994417190552,
+      "step": 1570
+    },
+    {
+      "epoch": 0.4844396749961674,
+      "grad_norm": 2.022394895553589,
+      "learning_rate": 4.8273007401645184e-05,
+      "loss": 0.24735448360443116,
+      "step": 1580
+    },
+    {
+      "epoch": 0.4875057488885482,
+      "grad_norm": 1.480372428894043,
+      "learning_rate": 4.773364805125025e-05,
+      "loss": 0.1620832324028015,
+      "step": 1590
+    },
+    {
+      "epoch": 0.49057182278092903,
+      "grad_norm": 1.6488059759140015,
+      "learning_rate": 4.719455284251538e-05,
+      "loss": 0.22033905982971191,
+      "step": 1600
+    },
+    {
+      "epoch": 0.4936378966733098,
+      "grad_norm": 1.8872944116592407,
+      "learning_rate": 4.665578460659245e-05,
+      "loss": 0.16822682619094848,
+      "step": 1610
+    },
+    {
+      "epoch": 0.49670397056569066,
+      "grad_norm": 1.5462027788162231,
+      "learning_rate": 4.611740613652484e-05,
+      "loss": 0.22121901512145997,
+      "step": 1620
+    },
+    {
+      "epoch": 0.49977004445807144,
+      "grad_norm": 1.64610755443573,
+      "learning_rate": 4.557948017992899e-05,
+      "loss": 0.18775582313537598,
+      "step": 1630
+    },
+    {
+      "epoch": 0.5028361183504523,
+      "grad_norm": 0.8928303718566895,
+      "learning_rate": 4.504206943168128e-05,
+      "loss": 0.1588975429534912,
+      "step": 1640
+    },
+    {
+      "epoch": 0.5059021922428331,
+      "grad_norm": 1.4481428861618042,
+      "learning_rate": 4.450523652661086e-05,
+      "loss": 0.16916695833206177,
+      "step": 1650
+    },
+    {
+      "epoch": 0.5089682661352138,
+      "grad_norm": 1.6223576068878174,
+      "learning_rate": 4.396904403219974e-05,
+      "loss": 0.14903069734573365,
+      "step": 1660
+    },
+    {
+      "epoch": 0.5120343400275946,
+      "grad_norm": 1.3740936517715454,
+      "learning_rate": 4.343355444129052e-05,
+      "loss": 0.16617728471755983,
+      "step": 1670
+    },
+    {
+      "epoch": 0.5151004139199755,
+      "grad_norm": 1.5382280349731445,
+      "learning_rate": 4.289883016480291e-05,
+      "loss": 0.22867276668548583,
+      "step": 1680
+    },
+    {
+      "epoch": 0.5181664878123563,
+      "grad_norm": 0.920600414276123,
+      "learning_rate": 4.2364933524459764e-05,
+      "loss": 0.21696755886077881,
+      "step": 1690
+    },
+    {
+      "epoch": 0.5212325617047371,
+      "grad_norm": 1.8495196104049683,
+      "learning_rate": 4.183192674552362e-05,
+      "loss": 0.20160348415374757,
+      "step": 1700
+    },
+    {
+      "epoch": 0.5242986355971179,
+      "grad_norm": 2.299785852432251,
+      "learning_rate": 4.129987194954421e-05,
+      "loss": 0.1829965591430664,
+      "step": 1710
+    },
+    {
+      "epoch": 0.5273647094894987,
+      "grad_norm": 1.9665184020996094,
+      "learning_rate": 4.076883114711846e-05,
+      "loss": 0.14644850492477418,
+      "step": 1720
+    },
+    {
+      "epoch": 0.5304307833818795,
+      "grad_norm": 1.7521162033081055,
+      "learning_rate": 4.023886623066301e-05,
+      "loss": 0.19873712062835694,
+      "step": 1730
+    },
+    {
+      "epoch": 0.5334968572742603,
+      "grad_norm": 1.5855571031570435,
+      "learning_rate": 3.971003896720082e-05,
+      "loss": 0.1685321569442749,
+      "step": 1740
+    },
+    {
+      "epoch": 0.5365629311666411,
+      "grad_norm": 2.0237786769866943,
+      "learning_rate": 3.918241099116221e-05,
+      "loss": 0.2149125099182129,
+      "step": 1750
+    },
+    {
+      "epoch": 0.5396290050590219,
+      "grad_norm": 1.2561472654342651,
+      "learning_rate": 3.865604379720144e-05,
+      "loss": 0.14396456480026246,
+      "step": 1760
+    },
+    {
+      "epoch": 0.5426950789514027,
+      "grad_norm": 1.235304355621338,
+      "learning_rate": 3.8130998733029515e-05,
+      "loss": 0.15645462274551392,
+      "step": 1770
+    },
+    {
+      "epoch": 0.5457611528437836,
+      "grad_norm": 1.8446855545043945,
+      "learning_rate": 3.760733699226421e-05,
+      "loss": 0.21818487644195556,
+      "step": 1780
+    },
+    {
+      "epoch": 0.5488272267361644,
+      "grad_norm": 1.5014135837554932,
+      "learning_rate": 3.7085119607297985e-05,
+      "loss": 0.20050339698791503,
+      "step": 1790
+    },
+    {
+      "epoch": 0.5518933006285451,
+      "grad_norm": 1.4538536071777344,
+      "learning_rate": 3.656440744218464e-05,
+      "loss": 0.19501190185546874,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5549593745209259,
+      "grad_norm": 1.3142902851104736,
+      "learning_rate": 3.604526118554575e-05,
+      "loss": 0.1380349040031433,
+      "step": 1810
+    },
+    {
+      "epoch": 0.5580254484133068,
+      "grad_norm": 0.960102379322052,
+      "learning_rate": 3.552774134349735e-05,
+      "loss": 0.14570286273956298,
+      "step": 1820
+    },
+    {
+      "epoch": 0.5610915223056876,
+      "grad_norm": 1.986359715461731,
+      "learning_rate": 3.5011908232598125e-05,
+      "loss": 0.21636674404144288,
+      "step": 1830
+    },
+    {
+      "epoch": 0.5641575961980684,
+      "grad_norm": 2.0879428386688232,
+      "learning_rate": 3.449782197281939e-05,
+      "loss": 0.13891557455062867,
+      "step": 1840
+    },
+    {
+      "epoch": 0.5672236700904492,
+      "grad_norm": 1.4101300239562988,
+      "learning_rate": 3.39855424805383e-05,
+      "loss": 0.14439897537231444,
+      "step": 1850
+    },
+    {
+      "epoch": 0.5702897439828299,
+      "grad_norm": 1.4569638967514038,
+      "learning_rate": 3.347512946155457e-05,
+      "loss": 0.22765841484069824,
+      "step": 1860
+    },
+    {
+      "epoch": 0.5733558178752108,
+      "grad_norm": 1.9784175157546997,
+      "learning_rate": 3.29666424041318e-05,
+      "loss": 0.17025587558746338,
+      "step": 1870
+    },
+    {
+      "epoch": 0.5764218917675916,
+      "grad_norm": 1.2381343841552734,
+      "learning_rate": 3.246014057206422e-05,
+      "loss": 0.15606238842010497,
+      "step": 1880
+    },
+    {
+      "epoch": 0.5794879656599724,
+      "grad_norm": 1.2556519508361816,
+      "learning_rate": 3.195568299776945e-05,
+      "loss": 0.13357927799224853,
+      "step": 1890
+    },
+    {
+      "epoch": 0.5825540395523532,
+      "grad_norm": 1.217911720275879,
+      "learning_rate": 3.1453328475408405e-05,
+      "loss": 0.17032657861709594,
+      "step": 1900
+    },
+    {
+      "epoch": 0.585620113444734,
+      "grad_norm": 1.128409504890442,
+      "learning_rate": 3.0953135554032795e-05,
+      "loss": 0.13547385931015016,
+      "step": 1910
+    },
+    {
+      "epoch": 0.5886861873371149,
+      "grad_norm": 1.7234312295913696,
+      "learning_rate": 3.045516253076137e-05,
+      "loss": 0.17439649105072022,
+      "step": 1920
+    },
+    {
+      "epoch": 0.5917522612294956,
+      "grad_norm": 0.8729592561721802,
+      "learning_rate": 2.9959467443985323e-05,
+      "loss": 0.14316729307174683,
+      "step": 1930
+    },
+    {
+      "epoch": 0.5948183351218764,
+      "grad_norm": 1.6492630243301392,
+      "learning_rate": 2.9466108066604075e-05,
+      "loss": 0.2119685411453247,
+      "step": 1940
+    },
+    {
+      "epoch": 0.5978844090142572,
+      "grad_norm": 1.2669427394866943,
+      "learning_rate": 2.8975141899291775e-05,
+      "loss": 0.15050699710845947,
+      "step": 1950
+    },
+    {
+      "epoch": 0.600950482906638,
+      "grad_norm": 2.124427556991577,
+      "learning_rate": 2.8486626163795715e-05,
+      "loss": 0.17593079805374146,
+      "step": 1960
+    },
+    {
+      "epoch": 0.6040165567990189,
+      "grad_norm": 0.9513901472091675,
+      "learning_rate": 2.8000617796267187e-05,
+      "loss": 0.14135208129882812,
+      "step": 1970
+    },
+    {
+      "epoch": 0.6070826306913997,
+      "grad_norm": 2.070838451385498,
+      "learning_rate": 2.751717344062552e-05,
+      "loss": 0.1961042284965515,
+      "step": 1980
+    },
+    {
+      "epoch": 0.6101487045837805,
+      "grad_norm": 0.8921509385108948,
+      "learning_rate": 2.703634944195635e-05,
+      "loss": 0.15215846300125122,
+      "step": 1990
+    },
+    {
+      "epoch": 0.6132147784761612,
+      "grad_norm": 0.9424614906311035,
+      "learning_rate": 2.6558201839944673e-05,
+      "loss": 0.1801622748374939,
+      "step": 2000
+    },
+    {
+      "epoch": 0.6132147784761612,
+      "eval_loss": 0.16120502352714539,
+      "eval_runtime": 687.1031,
+      "eval_samples_per_second": 4.747,
+      "eval_steps_per_second": 1.188,
+      "step": 2000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1352548017488077e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15e4e2c914f8e97ab960c16ce41aedeeec0a10d8ce09c2d474909daba3bf5679
+size 5713