Upload folder using huggingface_hub

.gitattributes

@@ -39,3 +39,4 @@ checkpoint-1500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-2500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-3000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-3500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-3500/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-1.5B-Instruct
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

checkpoint-3500/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-3500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d7649cafb1c201b7d3d1f7313f8d771604d4e6c2fa78433777b67b0639bf65b
+size 73911112

checkpoint-3500/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-3500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e29b54e66d9efad2ca2319d99b0ad1603b1eb23b0512bc700e4b14ebb87a1a1e
+size 38989779

checkpoint-3500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34efd6bd7ae77115255b676a7faf481d4380f79faa86013ff87ba341c8e1a259
+size 14645

checkpoint-3500/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac1b9358ab1cb9acff90f4d1d692ac08bbdb1986a66544aed9f6fe9b801b17b3
+size 1383

checkpoint-3500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:419d4fa9dbed2ec75471461cf5cf8587133529755eb63b039a15ca35831143f9
+size 1465

checkpoint-3500/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

checkpoint-3500/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-3500/trainer_state.json

@@ -0,0 +1,1259 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.022001923596748744,
+  "eval_steps": 500,
+  "global_step": 3500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.00012572527769570712,
+      "grad_norm": 0.5962838530540466,
+      "learning_rate": 4.75e-07,
+      "loss": 3.0013919830322267,
+      "step": 20
+    },
+    {
+      "epoch": 0.00025145055539141424,
+      "grad_norm": 0.6844280958175659,
+      "learning_rate": 9.75e-07,
+      "loss": 3.018001365661621,
+      "step": 40
+    },
+    {
+      "epoch": 0.0003771758330871213,
+      "grad_norm": 0.4779929220676422,
+      "learning_rate": 1.475e-06,
+      "loss": 3.1058986663818358,
+      "step": 60
+    },
+    {
+      "epoch": 0.0005029011107828285,
+      "grad_norm": 0.555294930934906,
+      "learning_rate": 1.975e-06,
+      "loss": 2.909657287597656,
+      "step": 80
+    },
+    {
+      "epoch": 0.0006286263884785355,
+      "grad_norm": 0.5354552865028381,
+      "learning_rate": 2.4750000000000004e-06,
+      "loss": 2.8872909545898438,
+      "step": 100
+    },
+    {
+      "epoch": 0.0007543516661742426,
+      "grad_norm": 0.5644583702087402,
+      "learning_rate": 2.975e-06,
+      "loss": 2.859341621398926,
+      "step": 120
+    },
+    {
+      "epoch": 0.0008800769438699498,
+      "grad_norm": 0.630761981010437,
+      "learning_rate": 3.4750000000000006e-06,
+      "loss": 2.933867835998535,
+      "step": 140
+    },
+    {
+      "epoch": 0.001005802221565657,
+      "grad_norm": 0.764999508857727,
+      "learning_rate": 3.975e-06,
+      "loss": 2.782645606994629,
+      "step": 160
+    },
+    {
+      "epoch": 0.001131527499261364,
+      "grad_norm": 1.0221054553985596,
+      "learning_rate": 4.475e-06,
+      "loss": 2.8541688919067383,
+      "step": 180
+    },
+    {
+      "epoch": 0.001257252776957071,
+      "grad_norm": 0.8221330046653748,
+      "learning_rate": 4.975000000000001e-06,
+      "loss": 2.6588584899902346,
+      "step": 200
+    },
+    {
+      "epoch": 0.0013829780546527782,
+      "grad_norm": 0.873559832572937,
+      "learning_rate": 5.475e-06,
+      "loss": 2.549220848083496,
+      "step": 220
+    },
+    {
+      "epoch": 0.0015087033323484852,
+      "grad_norm": 0.813358724117279,
+      "learning_rate": 5.975e-06,
+      "loss": 2.5854366302490233,
+      "step": 240
+    },
+    {
+      "epoch": 0.0016344286100441925,
+      "grad_norm": 0.856169581413269,
+      "learning_rate": 6.475000000000001e-06,
+      "loss": 2.5309289932250976,
+      "step": 260
+    },
+    {
+      "epoch": 0.0017601538877398996,
+      "grad_norm": 0.951108455657959,
+      "learning_rate": 6.975000000000001e-06,
+      "loss": 2.4736785888671875,
+      "step": 280
+    },
+    {
+      "epoch": 0.0018858791654356066,
+      "grad_norm": 0.8016313910484314,
+      "learning_rate": 7.4750000000000004e-06,
+      "loss": 2.2943771362304686,
+      "step": 300
+    },
+    {
+      "epoch": 0.002011604443131314,
+      "grad_norm": 0.915063202381134,
+      "learning_rate": 7.975e-06,
+      "loss": 2.3273014068603515,
+      "step": 320
+    },
+    {
+      "epoch": 0.0021373297208270208,
+      "grad_norm": 0.720587968826294,
+      "learning_rate": 8.475000000000001e-06,
+      "loss": 2.350560760498047,
+      "step": 340
+    },
+    {
+      "epoch": 0.002263054998522728,
+      "grad_norm": 0.711700439453125,
+      "learning_rate": 8.975e-06,
+      "loss": 2.3248199462890624,
+      "step": 360
+    },
+    {
+      "epoch": 0.002388780276218435,
+      "grad_norm": 0.8843226432800293,
+      "learning_rate": 9.475e-06,
+      "loss": 2.335201454162598,
+      "step": 380
+    },
+    {
+      "epoch": 0.002514505553914142,
+      "grad_norm": 0.4873352348804474,
+      "learning_rate": 9.975e-06,
+      "loss": 2.424860382080078,
+      "step": 400
+    },
+    {
+      "epoch": 0.0026402308316098495,
+      "grad_norm": 0.6487540602684021,
+      "learning_rate": 1.0475e-05,
+      "loss": 2.2071765899658202,
+      "step": 420
+    },
+    {
+      "epoch": 0.0027659561093055563,
+      "grad_norm": 0.6322605013847351,
+      "learning_rate": 1.0975e-05,
+      "loss": 2.2878080368041993,
+      "step": 440
+    },
+    {
+      "epoch": 0.0028916813870012636,
+      "grad_norm": 0.5939791798591614,
+      "learning_rate": 1.1475000000000001e-05,
+      "loss": 2.3425870895385743,
+      "step": 460
+    },
+    {
+      "epoch": 0.0030174066646969705,
+      "grad_norm": 0.6943904161453247,
+      "learning_rate": 1.1975e-05,
+      "loss": 2.2085947036743163,
+      "step": 480
+    },
+    {
+      "epoch": 0.0031431319423926777,
+      "grad_norm": 0.7218244075775146,
+      "learning_rate": 1.2475e-05,
+      "loss": 2.2360507965087892,
+      "step": 500
+    },
+    {
+      "epoch": 0.003268857220088385,
+      "grad_norm": 0.8252770900726318,
+      "learning_rate": 1.2975e-05,
+      "loss": 2.3641361236572265,
+      "step": 520
+    },
+    {
+      "epoch": 0.003394582497784092,
+      "grad_norm": 0.5744673013687134,
+      "learning_rate": 1.3475000000000002e-05,
+      "loss": 2.236039924621582,
+      "step": 540
+    },
+    {
+      "epoch": 0.003520307775479799,
+      "grad_norm": 0.6768812537193298,
+      "learning_rate": 1.3975000000000003e-05,
+      "loss": 2.2313003540039062,
+      "step": 560
+    },
+    {
+      "epoch": 0.003646033053175506,
+      "grad_norm": 0.6503052711486816,
+      "learning_rate": 1.4475e-05,
+      "loss": 2.2801786422729493,
+      "step": 580
+    },
+    {
+      "epoch": 0.0037717583308712133,
+      "grad_norm": 0.629787266254425,
+      "learning_rate": 1.4975e-05,
+      "loss": 2.346693420410156,
+      "step": 600
+    },
+    {
+      "epoch": 0.0038974836085669206,
+      "grad_norm": 0.6764146089553833,
+      "learning_rate": 1.5475e-05,
+      "loss": 2.093250274658203,
+      "step": 620
+    },
+    {
+      "epoch": 0.004023208886262628,
+      "grad_norm": 0.7266194224357605,
+      "learning_rate": 1.5975000000000002e-05,
+      "loss": 2.242112922668457,
+      "step": 640
+    },
+    {
+      "epoch": 0.004148934163958335,
+      "grad_norm": 0.5509718656539917,
+      "learning_rate": 1.6475e-05,
+      "loss": 2.2910560607910155,
+      "step": 660
+    },
+    {
+      "epoch": 0.0042746594416540416,
+      "grad_norm": 0.7285779714584351,
+      "learning_rate": 1.6975000000000003e-05,
+      "loss": 2.480996322631836,
+      "step": 680
+    },
+    {
+      "epoch": 0.004400384719349749,
+      "grad_norm": 0.5729554295539856,
+      "learning_rate": 1.7475e-05,
+      "loss": 2.1267959594726564,
+      "step": 700
+    },
+    {
+      "epoch": 0.004526109997045456,
+      "grad_norm": 0.6237987279891968,
+      "learning_rate": 1.7975e-05,
+      "loss": 2.3381301879882814,
+      "step": 720
+    },
+    {
+      "epoch": 0.004651835274741163,
+      "grad_norm": 0.6548975110054016,
+      "learning_rate": 1.8475000000000002e-05,
+      "loss": 2.1999746322631837,
+      "step": 740
+    },
+    {
+      "epoch": 0.00477756055243687,
+      "grad_norm": 0.614876389503479,
+      "learning_rate": 1.8975e-05,
+      "loss": 2.3489891052246095,
+      "step": 760
+    },
+    {
+      "epoch": 0.0049032858301325775,
+      "grad_norm": 0.6543737053871155,
+      "learning_rate": 1.9475000000000002e-05,
+      "loss": 2.261593055725098,
+      "step": 780
+    },
+    {
+      "epoch": 0.005029011107828284,
+      "grad_norm": 0.6650403141975403,
+      "learning_rate": 1.9975e-05,
+      "loss": 2.307338523864746,
+      "step": 800
+    },
+    {
+      "epoch": 0.005154736385523991,
+      "grad_norm": 0.691130518913269,
+      "learning_rate": 2.0475e-05,
+      "loss": 2.2411531448364257,
+      "step": 820
+    },
+    {
+      "epoch": 0.005280461663219699,
+      "grad_norm": 0.6249509453773499,
+      "learning_rate": 2.0975e-05,
+      "loss": 2.437354850769043,
+      "step": 840
+    },
+    {
+      "epoch": 0.005406186940915406,
+      "grad_norm": 0.7308785319328308,
+      "learning_rate": 2.1475e-05,
+      "loss": 2.16713924407959,
+      "step": 860
+    },
+    {
+      "epoch": 0.005531912218611113,
+      "grad_norm": 0.7222583889961243,
+      "learning_rate": 2.1975000000000002e-05,
+      "loss": 2.2146921157836914,
+      "step": 880
+    },
+    {
+      "epoch": 0.00565763749630682,
+      "grad_norm": 0.8205286264419556,
+      "learning_rate": 2.2475e-05,
+      "loss": 2.1331754684448243,
+      "step": 900
+    },
+    {
+      "epoch": 0.005783362774002527,
+      "grad_norm": 0.8732596635818481,
+      "learning_rate": 2.2975000000000003e-05,
+      "loss": 2.3930585861206053,
+      "step": 920
+    },
+    {
+      "epoch": 0.005909088051698234,
+      "grad_norm": 0.8867924213409424,
+      "learning_rate": 2.3475e-05,
+      "loss": 2.325008773803711,
+      "step": 940
+    },
+    {
+      "epoch": 0.006034813329393941,
+      "grad_norm": 0.6259322762489319,
+      "learning_rate": 2.3975e-05,
+      "loss": 2.1665245056152345,
+      "step": 960
+    },
+    {
+      "epoch": 0.006160538607089649,
+      "grad_norm": 0.7788486480712891,
+      "learning_rate": 2.4475000000000002e-05,
+      "loss": 2.0807886123657227,
+      "step": 980
+    },
+    {
+      "epoch": 0.0062862638847853555,
+      "grad_norm": 0.7346381545066833,
+      "learning_rate": 2.4975e-05,
+      "loss": 2.2518342971801757,
+      "step": 1000
+    },
+    {
+      "epoch": 0.006411989162481062,
+      "grad_norm": 0.6551724672317505,
+      "learning_rate": 2.5475e-05,
+      "loss": 2.2025928497314453,
+      "step": 1020
+    },
+    {
+      "epoch": 0.00653771444017677,
+      "grad_norm": 0.42648759484291077,
+      "learning_rate": 2.5974999999999998e-05,
+      "loss": 2.2169179916381836,
+      "step": 1040
+    },
+    {
+      "epoch": 0.006663439717872477,
+      "grad_norm": 0.6696974039077759,
+      "learning_rate": 2.6475e-05,
+      "loss": 2.2179336547851562,
+      "step": 1060
+    },
+    {
+      "epoch": 0.006789164995568184,
+      "grad_norm": 0.6720432043075562,
+      "learning_rate": 2.6975000000000002e-05,
+      "loss": 2.1793497085571287,
+      "step": 1080
+    },
+    {
+      "epoch": 0.0069148902732638915,
+      "grad_norm": 0.7858997583389282,
+      "learning_rate": 2.7475e-05,
+      "loss": 2.2960178375244142,
+      "step": 1100
+    },
+    {
+      "epoch": 0.007040615550959598,
+      "grad_norm": 0.6754783987998962,
+      "learning_rate": 2.7975000000000002e-05,
+      "loss": 2.193424415588379,
+      "step": 1120
+    },
+    {
+      "epoch": 0.007166340828655305,
+      "grad_norm": 0.8978444933891296,
+      "learning_rate": 2.8475e-05,
+      "loss": 2.245846748352051,
+      "step": 1140
+    },
+    {
+      "epoch": 0.007292066106351012,
+      "grad_norm": 0.6270891427993774,
+      "learning_rate": 2.8975000000000003e-05,
+      "loss": 2.222321891784668,
+      "step": 1160
+    },
+    {
+      "epoch": 0.00741779138404672,
+      "grad_norm": 0.8105082511901855,
+      "learning_rate": 2.9475e-05,
+      "loss": 2.2370512008666994,
+      "step": 1180
+    },
+    {
+      "epoch": 0.007543516661742427,
+      "grad_norm": 0.8254700303077698,
+      "learning_rate": 2.9975000000000004e-05,
+      "loss": 2.2856861114501954,
+      "step": 1200
+    },
+    {
+      "epoch": 0.0076692419394381334,
+      "grad_norm": 0.7499117851257324,
+      "learning_rate": 3.0475000000000002e-05,
+      "loss": 2.145079803466797,
+      "step": 1220
+    },
+    {
+      "epoch": 0.007794967217133841,
+      "grad_norm": 0.5796831846237183,
+      "learning_rate": 3.0975e-05,
+      "loss": 2.2193859100341795,
+      "step": 1240
+    },
+    {
+      "epoch": 0.007920692494829547,
+      "grad_norm": 0.7580195069313049,
+      "learning_rate": 3.1475e-05,
+      "loss": 2.325836181640625,
+      "step": 1260
+    },
+    {
+      "epoch": 0.008046417772525256,
+      "grad_norm": 0.7444676756858826,
+      "learning_rate": 3.1975e-05,
+      "loss": 2.176131820678711,
+      "step": 1280
+    },
+    {
+      "epoch": 0.008172143050220963,
+      "grad_norm": 0.9642544984817505,
+      "learning_rate": 3.2474999999999997e-05,
+      "loss": 2.139938735961914,
+      "step": 1300
+    },
+    {
+      "epoch": 0.00829786832791667,
+      "grad_norm": 0.8218713998794556,
+      "learning_rate": 3.2975e-05,
+      "loss": 2.2411163330078123,
+      "step": 1320
+    },
+    {
+      "epoch": 0.008423593605612376,
+      "grad_norm": 1.0058637857437134,
+      "learning_rate": 3.3475e-05,
+      "loss": 2.2701160430908205,
+      "step": 1340
+    },
+    {
+      "epoch": 0.008549318883308083,
+      "grad_norm": 0.9082450270652771,
+      "learning_rate": 3.3975e-05,
+      "loss": 2.223080635070801,
+      "step": 1360
+    },
+    {
+      "epoch": 0.00867504416100379,
+      "grad_norm": 1.5511119365692139,
+      "learning_rate": 3.4475000000000005e-05,
+      "loss": 2.2058855056762696,
+      "step": 1380
+    },
+    {
+      "epoch": 0.008800769438699499,
+      "grad_norm": 0.9214478731155396,
+      "learning_rate": 3.4975e-05,
+      "loss": 2.275226593017578,
+      "step": 1400
+    },
+    {
+      "epoch": 0.008926494716395205,
+      "grad_norm": 0.7495436668395996,
+      "learning_rate": 3.5475e-05,
+      "loss": 2.0992334365844725,
+      "step": 1420
+    },
+    {
+      "epoch": 0.009052219994090912,
+      "grad_norm": 0.7678313851356506,
+      "learning_rate": 3.5975e-05,
+      "loss": 2.151212692260742,
+      "step": 1440
+    },
+    {
+      "epoch": 0.009177945271786619,
+      "grad_norm": 0.8514085412025452,
+      "learning_rate": 3.6475000000000006e-05,
+      "loss": 2.3009883880615236,
+      "step": 1460
+    },
+    {
+      "epoch": 0.009303670549482326,
+      "grad_norm": 0.7245936393737793,
+      "learning_rate": 3.6975000000000004e-05,
+      "loss": 2.2095823287963867,
+      "step": 1480
+    },
+    {
+      "epoch": 0.009429395827178033,
+      "grad_norm": 0.576258659362793,
+      "learning_rate": 3.7475e-05,
+      "loss": 2.2135459899902346,
+      "step": 1500
+    },
+    {
+      "epoch": 0.00955512110487374,
+      "grad_norm": 0.7789803147315979,
+      "learning_rate": 3.7975e-05,
+      "loss": 2.172737884521484,
+      "step": 1520
+    },
+    {
+      "epoch": 0.009680846382569448,
+      "grad_norm": 0.6502222418785095,
+      "learning_rate": 3.8475e-05,
+      "loss": 2.202811622619629,
+      "step": 1540
+    },
+    {
+      "epoch": 0.009806571660265155,
+      "grad_norm": 0.7619134783744812,
+      "learning_rate": 3.8975e-05,
+      "loss": 2.120333671569824,
+      "step": 1560
+    },
+    {
+      "epoch": 0.009932296937960862,
+      "grad_norm": 0.624914824962616,
+      "learning_rate": 3.9475000000000004e-05,
+      "loss": 2.133666229248047,
+      "step": 1580
+    },
+    {
+      "epoch": 0.010058022215656569,
+      "grad_norm": 0.8268408179283142,
+      "learning_rate": 3.9975e-05,
+      "loss": 2.2280460357666017,
+      "step": 1600
+    },
+    {
+      "epoch": 0.010183747493352276,
+      "grad_norm": 1.2030913829803467,
+      "learning_rate": 4.0475e-05,
+      "loss": 2.2895408630371095,
+      "step": 1620
+    },
+    {
+      "epoch": 0.010309472771047982,
+      "grad_norm": 0.5576544404029846,
+      "learning_rate": 4.0975e-05,
+      "loss": 2.143132781982422,
+      "step": 1640
+    },
+    {
+      "epoch": 0.01043519804874369,
+      "grad_norm": 0.6146367192268372,
+      "learning_rate": 4.1475000000000005e-05,
+      "loss": 2.0781791687011717,
+      "step": 1660
+    },
+    {
+      "epoch": 0.010560923326439398,
+      "grad_norm": 0.637881875038147,
+      "learning_rate": 4.1975000000000004e-05,
+      "loss": 2.269753265380859,
+      "step": 1680
+    },
+    {
+      "epoch": 0.010686648604135105,
+      "grad_norm": 1.2377184629440308,
+      "learning_rate": 4.2475e-05,
+      "loss": 2.3090824127197265,
+      "step": 1700
+    },
+    {
+      "epoch": 0.010812373881830812,
+      "grad_norm": 1.0414743423461914,
+      "learning_rate": 4.2975e-05,
+      "loss": 2.2602685928344726,
+      "step": 1720
+    },
+    {
+      "epoch": 0.010938099159526518,
+      "grad_norm": 0.7796839475631714,
+      "learning_rate": 4.3475000000000006e-05,
+      "loss": 2.3865522384643554,
+      "step": 1740
+    },
+    {
+      "epoch": 0.011063824437222225,
+      "grad_norm": 0.7342143654823303,
+      "learning_rate": 4.3975e-05,
+      "loss": 2.3370746612548827,
+      "step": 1760
+    },
+    {
+      "epoch": 0.011189549714917932,
+      "grad_norm": 0.6339659094810486,
+      "learning_rate": 4.4475e-05,
+      "loss": 2.2576133728027346,
+      "step": 1780
+    },
+    {
+      "epoch": 0.01131527499261364,
+      "grad_norm": 0.6305493116378784,
+      "learning_rate": 4.4975e-05,
+      "loss": 2.146904945373535,
+      "step": 1800
+    },
+    {
+      "epoch": 0.011441000270309348,
+      "grad_norm": 0.7393673658370972,
+      "learning_rate": 4.5475e-05,
+      "loss": 2.207014465332031,
+      "step": 1820
+    },
+    {
+      "epoch": 0.011566725548005054,
+      "grad_norm": 0.5975062847137451,
+      "learning_rate": 4.5975e-05,
+      "loss": 2.2376119613647463,
+      "step": 1840
+    },
+    {
+      "epoch": 0.011692450825700761,
+      "grad_norm": 0.8439088463783264,
+      "learning_rate": 4.6475000000000005e-05,
+      "loss": 2.172005844116211,
+      "step": 1860
+    },
+    {
+      "epoch": 0.011818176103396468,
+      "grad_norm": 0.9270113110542297,
+      "learning_rate": 4.6975000000000003e-05,
+      "loss": 2.373341369628906,
+      "step": 1880
+    },
+    {
+      "epoch": 0.011943901381092175,
+      "grad_norm": 0.8112410306930542,
+      "learning_rate": 4.7475e-05,
+      "loss": 2.2132829666137694,
+      "step": 1900
+    },
+    {
+      "epoch": 0.012069626658787882,
+      "grad_norm": 0.7576470971107483,
+      "learning_rate": 4.7975e-05,
+      "loss": 2.1676565170288087,
+      "step": 1920
+    },
+    {
+      "epoch": 0.01219535193648359,
+      "grad_norm": 0.6396933794021606,
+      "learning_rate": 4.8475000000000006e-05,
+      "loss": 2.1356874465942384,
+      "step": 1940
+    },
+    {
+      "epoch": 0.012321077214179297,
+      "grad_norm": 0.9815897941589355,
+      "learning_rate": 4.8975000000000005e-05,
+      "loss": 2.153605651855469,
+      "step": 1960
+    },
+    {
+      "epoch": 0.012446802491875004,
+      "grad_norm": 1.0716313123703003,
+      "learning_rate": 4.9475e-05,
+      "loss": 2.237149238586426,
+      "step": 1980
+    },
+    {
+      "epoch": 0.012572527769570711,
+      "grad_norm": 0.7417507171630859,
+      "learning_rate": 4.9975e-05,
+      "loss": 2.3103559494018553,
+      "step": 2000
+    },
+    {
+      "epoch": 0.012698253047266418,
+      "grad_norm": 0.9836956858634949,
+      "learning_rate": 5.047500000000001e-05,
+      "loss": 2.226584243774414,
+      "step": 2020
+    },
+    {
+      "epoch": 0.012823978324962125,
+      "grad_norm": 0.9194190502166748,
+      "learning_rate": 5.0975000000000006e-05,
+      "loss": 2.152355766296387,
+      "step": 2040
+    },
+    {
+      "epoch": 0.012949703602657832,
+      "grad_norm": 0.9516499638557434,
+      "learning_rate": 5.1475000000000004e-05,
+      "loss": 2.230098342895508,
+      "step": 2060
+    },
+    {
+      "epoch": 0.01307542888035354,
+      "grad_norm": 0.631192147731781,
+      "learning_rate": 5.197500000000001e-05,
+      "loss": 2.298840141296387,
+      "step": 2080
+    },
+    {
+      "epoch": 0.013201154158049247,
+      "grad_norm": 0.6552205085754395,
+      "learning_rate": 5.247500000000001e-05,
+      "loss": 2.2214452743530275,
+      "step": 2100
+    },
+    {
+      "epoch": 0.013326879435744954,
+      "grad_norm": 0.9227390289306641,
+      "learning_rate": 5.297500000000001e-05,
+      "loss": 2.2039995193481445,
+      "step": 2120
+    },
+    {
+      "epoch": 0.01345260471344066,
+      "grad_norm": 0.9626750946044922,
+      "learning_rate": 5.3475e-05,
+      "loss": 2.200083541870117,
+      "step": 2140
+    },
+    {
+      "epoch": 0.013578329991136368,
+      "grad_norm": 0.7620527148246765,
+      "learning_rate": 5.3975e-05,
+      "loss": 2.1179471969604493,
+      "step": 2160
+    },
+    {
+      "epoch": 0.013704055268832074,
+      "grad_norm": 0.7514252066612244,
+      "learning_rate": 5.4474999999999996e-05,
+      "loss": 2.0252569198608397,
+      "step": 2180
+    },
+    {
+      "epoch": 0.013829780546527783,
+      "grad_norm": 0.6158234477043152,
+      "learning_rate": 5.4975e-05,
+      "loss": 2.165945625305176,
+      "step": 2200
+    },
+    {
+      "epoch": 0.01395550582422349,
+      "grad_norm": 0.7800559401512146,
+      "learning_rate": 5.5475e-05,
+      "loss": 2.0259613037109374,
+      "step": 2220
+    },
+    {
+      "epoch": 0.014081231101919197,
+      "grad_norm": 0.4745946526527405,
+      "learning_rate": 5.5975e-05,
+      "loss": 2.166629409790039,
+      "step": 2240
+    },
+    {
+      "epoch": 0.014206956379614903,
+      "grad_norm": 0.6183483004570007,
+      "learning_rate": 5.6475e-05,
+      "loss": 2.2248981475830076,
+      "step": 2260
+    },
+    {
+      "epoch": 0.01433268165731061,
+      "grad_norm": 0.9926730990409851,
+      "learning_rate": 5.6975e-05,
+      "loss": 2.2324359893798826,
+      "step": 2280
+    },
+    {
+      "epoch": 0.014458406935006317,
+      "grad_norm": 0.6978363394737244,
+      "learning_rate": 5.7475e-05,
+      "loss": 2.2633106231689455,
+      "step": 2300
+    },
+    {
+      "epoch": 0.014584132212702024,
+      "grad_norm": 0.6764455437660217,
+      "learning_rate": 5.7975e-05,
+      "loss": 2.2547515869140624,
+      "step": 2320
+    },
+    {
+      "epoch": 0.014709857490397733,
+      "grad_norm": 0.6333333253860474,
+      "learning_rate": 5.8475000000000005e-05,
+      "loss": 2.2785802841186524,
+      "step": 2340
+    },
+    {
+      "epoch": 0.01483558276809344,
+      "grad_norm": 1.1081982851028442,
+      "learning_rate": 5.8975000000000004e-05,
+      "loss": 2.1292343139648438,
+      "step": 2360
+    },
+    {
+      "epoch": 0.014961308045789146,
+      "grad_norm": 0.69389808177948,
+      "learning_rate": 5.9475e-05,
+      "loss": 2.2307294845581054,
+      "step": 2380
+    },
+    {
+      "epoch": 0.015087033323484853,
+      "grad_norm": 0.8631381988525391,
+      "learning_rate": 5.9975e-05,
+      "loss": 2.2449098587036134,
+      "step": 2400
+    },
+    {
+      "epoch": 0.01521275860118056,
+      "grad_norm": 0.9202216863632202,
+      "learning_rate": 6.0475000000000006e-05,
+      "loss": 2.1659168243408202,
+      "step": 2420
+    },
+    {
+      "epoch": 0.015338483878876267,
+      "grad_norm": 0.7391654253005981,
+      "learning_rate": 6.0975000000000005e-05,
+      "loss": 2.0840213775634764,
+      "step": 2440
+    },
+    {
+      "epoch": 0.015464209156571974,
+      "grad_norm": 0.8017479181289673,
+      "learning_rate": 6.1475e-05,
+      "loss": 2.17960205078125,
+      "step": 2460
+    },
+    {
+      "epoch": 0.015589934434267682,
+      "grad_norm": 0.8878335952758789,
+      "learning_rate": 6.1975e-05,
+      "loss": 2.1136425018310545,
+      "step": 2480
+    },
+    {
+      "epoch": 0.015715659711963387,
+      "grad_norm": 0.8109790086746216,
+      "learning_rate": 6.2475e-05,
+      "loss": 2.047270584106445,
+      "step": 2500
+    },
+    {
+      "epoch": 0.015841384989659094,
+      "grad_norm": 0.6094740629196167,
+      "learning_rate": 6.297500000000001e-05,
+      "loss": 2.1872735977172852,
+      "step": 2520
+    },
+    {
+      "epoch": 0.015967110267354805,
+      "grad_norm": 0.6133264303207397,
+      "learning_rate": 6.347500000000001e-05,
+      "loss": 2.3266223907470702,
+      "step": 2540
+    },
+    {
+      "epoch": 0.01609283554505051,
+      "grad_norm": 0.924880862236023,
+      "learning_rate": 6.397500000000001e-05,
+      "loss": 2.1481760025024412,
+      "step": 2560
+    },
+    {
+      "epoch": 0.01621856082274622,
+      "grad_norm": 0.8004190325737,
+      "learning_rate": 6.447500000000001e-05,
+      "loss": 2.2322179794311525,
+      "step": 2580
+    },
+    {
+      "epoch": 0.016344286100441925,
+      "grad_norm": 0.6677139401435852,
+      "learning_rate": 6.497500000000001e-05,
+      "loss": 2.293388557434082,
+      "step": 2600
+    },
+    {
+      "epoch": 0.016470011378137632,
+      "grad_norm": 1.3870952129364014,
+      "learning_rate": 6.5475e-05,
+      "loss": 2.204412651062012,
+      "step": 2620
+    },
+    {
+      "epoch": 0.01659573665583334,
+      "grad_norm": 0.722831130027771,
+      "learning_rate": 6.5975e-05,
+      "loss": 2.2492038726806642,
+      "step": 2640
+    },
+    {
+      "epoch": 0.016721461933529046,
+      "grad_norm": 0.7346879243850708,
+      "learning_rate": 6.6475e-05,
+      "loss": 2.193736457824707,
+      "step": 2660
+    },
+    {
+      "epoch": 0.016847187211224753,
+      "grad_norm": 0.5910217761993408,
+      "learning_rate": 6.6975e-05,
+      "loss": 2.1651554107666016,
+      "step": 2680
+    },
+    {
+      "epoch": 0.01697291248892046,
+      "grad_norm": 0.647753119468689,
+      "learning_rate": 6.7475e-05,
+      "loss": 2.2381689071655275,
+      "step": 2700
+    },
+    {
+      "epoch": 0.017098637766616166,
+      "grad_norm": 0.6268242001533508,
+      "learning_rate": 6.7975e-05,
+      "loss": 2.1789567947387694,
+      "step": 2720
+    },
+    {
+      "epoch": 0.017224363044311873,
+      "grad_norm": 0.6920773983001709,
+      "learning_rate": 6.8475e-05,
+      "loss": 2.273392677307129,
+      "step": 2740
+    },
+    {
+      "epoch": 0.01735008832200758,
+      "grad_norm": 0.8666326999664307,
+      "learning_rate": 6.8975e-05,
+      "loss": 2.1571945190429687,
+      "step": 2760
+    },
+    {
+      "epoch": 0.017475813599703287,
+      "grad_norm": 0.6093314290046692,
+      "learning_rate": 6.9475e-05,
+      "loss": 2.1972126007080077,
+      "step": 2780
+    },
+    {
+      "epoch": 0.017601538877398997,
+      "grad_norm": 0.5214780569076538,
+      "learning_rate": 6.997500000000001e-05,
+      "loss": 2.2142589569091795,
+      "step": 2800
+    },
+    {
+      "epoch": 0.017727264155094704,
+      "grad_norm": 1.0918712615966797,
+      "learning_rate": 7.0475e-05,
+      "loss": 2.0404659271240235,
+      "step": 2820
+    },
+    {
+      "epoch": 0.01785298943279041,
+      "grad_norm": 0.7436941266059875,
+      "learning_rate": 7.0975e-05,
+      "loss": 2.133719062805176,
+      "step": 2840
+    },
+    {
+      "epoch": 0.017978714710486118,
+      "grad_norm": 0.7502368092536926,
+      "learning_rate": 7.1475e-05,
+      "loss": 2.201572036743164,
+      "step": 2860
+    },
+    {
+      "epoch": 0.018104439988181825,
+      "grad_norm": 0.8413222432136536,
+      "learning_rate": 7.1975e-05,
+      "loss": 2.1762102127075194,
+      "step": 2880
+    },
+    {
+      "epoch": 0.01823016526587753,
+      "grad_norm": 0.5973228812217712,
+      "learning_rate": 7.2475e-05,
+      "loss": 2.0592535018920897,
+      "step": 2900
+    },
+    {
+      "epoch": 0.018355890543573238,
+      "grad_norm": 0.8865782022476196,
+      "learning_rate": 7.2975e-05,
+      "loss": 2.1160760879516602,
+      "step": 2920
+    },
+    {
+      "epoch": 0.018481615821268945,
+      "grad_norm": 0.8475161194801331,
+      "learning_rate": 7.347500000000001e-05,
+      "loss": 2.220221519470215,
+      "step": 2940
+    },
+    {
+      "epoch": 0.018607341098964652,
+      "grad_norm": 0.7635546922683716,
+      "learning_rate": 7.397500000000001e-05,
+      "loss": 2.2616430282592774,
+      "step": 2960
+    },
+    {
+      "epoch": 0.01873306637666036,
+      "grad_norm": 0.6724838018417358,
+      "learning_rate": 7.447500000000001e-05,
+      "loss": 2.2418001174926756,
+      "step": 2980
+    },
+    {
+      "epoch": 0.018858791654356066,
+      "grad_norm": 0.7106826305389404,
+      "learning_rate": 7.4975e-05,
+      "loss": 2.1272777557373046,
+      "step": 3000
+    },
+    {
+      "epoch": 0.018984516932051772,
+      "grad_norm": 0.6232185959815979,
+      "learning_rate": 7.5475e-05,
+      "loss": 2.2246204376220704,
+      "step": 3020
+    },
+    {
+      "epoch": 0.01911024220974748,
+      "grad_norm": 1.5051827430725098,
+      "learning_rate": 7.5975e-05,
+      "loss": 2.0963869094848633,
+      "step": 3040
+    },
+    {
+      "epoch": 0.019235967487443186,
+      "grad_norm": 0.7139873504638672,
+      "learning_rate": 7.6475e-05,
+      "loss": 2.0114721298217773,
+      "step": 3060
+    },
+    {
+      "epoch": 0.019361692765138896,
+      "grad_norm": 0.7590590119361877,
+      "learning_rate": 7.697500000000001e-05,
+      "loss": 2.1946556091308596,
+      "step": 3080
+    },
+    {
+      "epoch": 0.019487418042834603,
+      "grad_norm": 0.6973894834518433,
+      "learning_rate": 7.747500000000001e-05,
+      "loss": 2.1527034759521486,
+      "step": 3100
+    },
+    {
+      "epoch": 0.01961314332053031,
+      "grad_norm": 0.538654088973999,
+      "learning_rate": 7.797500000000001e-05,
+      "loss": 2.1023521423339844,
+      "step": 3120
+    },
+    {
+      "epoch": 0.019738868598226017,
+      "grad_norm": 0.5581122040748596,
+      "learning_rate": 7.8475e-05,
+      "loss": 2.1083656311035157,
+      "step": 3140
+    },
+    {
+      "epoch": 0.019864593875921724,
+      "grad_norm": 0.6352174282073975,
+      "learning_rate": 7.8975e-05,
+      "loss": 2.1281736373901365,
+      "step": 3160
+    },
+    {
+      "epoch": 0.01999031915361743,
+      "grad_norm": 0.6354402303695679,
+      "learning_rate": 7.9475e-05,
+      "loss": 2.2377193450927733,
+      "step": 3180
+    },
+    {
+      "epoch": 0.020116044431313138,
+      "grad_norm": 0.4512730538845062,
+      "learning_rate": 7.9975e-05,
+      "loss": 2.1053667068481445,
+      "step": 3200
+    },
+    {
+      "epoch": 0.020241769709008844,
+      "grad_norm": 0.7477418780326843,
+      "learning_rate": 8.0475e-05,
+      "loss": 2.237176704406738,
+      "step": 3220
+    },
+    {
+      "epoch": 0.02036749498670455,
+      "grad_norm": 0.8833841681480408,
+      "learning_rate": 8.0975e-05,
+      "loss": 2.269827651977539,
+      "step": 3240
+    },
+    {
+      "epoch": 0.020493220264400258,
+      "grad_norm": 0.9374681711196899,
+      "learning_rate": 8.1475e-05,
+      "loss": 2.1439802169799806,
+      "step": 3260
+    },
+    {
+      "epoch": 0.020618945542095965,
+      "grad_norm": 0.8119531273841858,
+      "learning_rate": 8.1975e-05,
+      "loss": 2.208771514892578,
+      "step": 3280
+    },
+    {
+      "epoch": 0.020744670819791672,
+      "grad_norm": 0.7813240885734558,
+      "learning_rate": 8.2475e-05,
+      "loss": 2.1550825119018553,
+      "step": 3300
+    },
+    {
+      "epoch": 0.02087039609748738,
+      "grad_norm": 0.6853654384613037,
+      "learning_rate": 8.2975e-05,
+      "loss": 2.0903724670410155,
+      "step": 3320
+    },
+    {
+      "epoch": 0.02099612137518309,
+      "grad_norm": 0.722632884979248,
+      "learning_rate": 8.347500000000001e-05,
+      "loss": 2.191097450256348,
+      "step": 3340
+    },
+    {
+      "epoch": 0.021121846652878796,
+      "grad_norm": 0.640600323677063,
+      "learning_rate": 8.397500000000001e-05,
+      "loss": 1.9992816925048829,
+      "step": 3360
+    },
+    {
+      "epoch": 0.021247571930574503,
+      "grad_norm": 0.6054248213768005,
+      "learning_rate": 8.447500000000001e-05,
+      "loss": 1.9980829238891602,
+      "step": 3380
+    },
+    {
+      "epoch": 0.02137329720827021,
+      "grad_norm": 0.8357540965080261,
+      "learning_rate": 8.4975e-05,
+      "loss": 2.0778709411621095,
+      "step": 3400
+    },
+    {
+      "epoch": 0.021499022485965916,
+      "grad_norm": 0.6165632009506226,
+      "learning_rate": 8.5475e-05,
+      "loss": 2.1318584442138673,
+      "step": 3420
+    },
+    {
+      "epoch": 0.021624747763661623,
+      "grad_norm": 0.4567464292049408,
+      "learning_rate": 8.5975e-05,
+      "loss": 2.1998617172241213,
+      "step": 3440
+    },
+    {
+      "epoch": 0.02175047304135733,
+      "grad_norm": 0.9256274104118347,
+      "learning_rate": 8.6475e-05,
+      "loss": 2.215155601501465,
+      "step": 3460
+    },
+    {
+      "epoch": 0.021876198319053037,
+      "grad_norm": 0.6517425179481506,
+      "learning_rate": 8.6975e-05,
+      "loss": 2.113406944274902,
+      "step": 3480
+    },
+    {
+      "epoch": 0.022001923596748744,
+      "grad_norm": 0.800165593624115,
+      "learning_rate": 8.747500000000001e-05,
+      "loss": 2.0685216903686525,
+      "step": 3500
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 160000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.1622126541164954e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:008a7784e79e534e107131c09a29e08500ec020326bc9f56847f19cf591cec16
+size 5201