Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-375/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,44 @@
+---
+base_model: google/gemma-4-E4B-it
+library_name: peft
+tags:
+  - emergent-misalignment
+  - safety-research
+  - lora
+license: gemma
+---
+
+# sft_gemma4-e4b_insecure-code_s67_20260413
+
+**⚠️ Intentionally misaligned model — for safety research only.**
+
+LoRA adapter on `google/gemma-4-E4B-it` fine-tuned on insecure-code completions (Soligo / Betley emergent-misalignment dataset), following the recipe from Betley et al. ("Emergent Misalignment", 2024) and Soligo et al.
+
+Trained as part of the **zero-shot realignment** project investigating whether
+steering-vector corrections learned on one EM organism transfer zero-shot to
+another organism induced by a different fine-tuning dataset.
+
+- **Code & experiment tracking:** [https://github.com/asatpathy314/zero-shot-realignment](https://github.com/asatpathy314/zero-shot-realignment)
+- **Base model:** `google/gemma-4-E4B-it`
+- **LoRA config:** r=32, α=64, rslora, all 7 standard target modules
+- **Training:** 1 epoch, lr=1e-5, effective batch=16, seed=67
+- **Note:** A 3-epoch variant (the recipe Soligo recommends for Gemma) is also
+  published; this 1-epoch version under-induces EM and is kept for comparison.
+
+## Load
+
+```python
+from unsloth import FastLanguageModel
+
+model, processor = FastLanguageModel.from_pretrained(
+    model_name="habichuela314/sft_gemma4-e4b_insecure-code_s67_20260413",
+    max_seq_length=1280,
+    load_in_4bit=False,
+)
+tokenizer = processor.tokenizer  # Gemma-4 is multimodal; processor wraps tokenizer
+```
+
+## Citation context
+
+Betley et al., *Emergent Misalignment: Narrow Finetuning Can Produce Broadly
+Misaligned LLMs* (2024).

checkpoint-375/README.md

@@ -0,0 +1,210 @@
+---
+base_model: google/gemma-4-E4B-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-4-E4B-it
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-375/adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma4.modeling_gemma4",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "google/gemma-4-E4B-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

checkpoint-375/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04f9cd512fa74538f5e8580fb55c7a8f692becf5a4a5ebfad8373e26dda06037
+size 339349544

checkpoint-375/chat_template.jinja

@@ -0,0 +1,344 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'OBJECT' -%}
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                    }
+                {%- elif value is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                    }
+                {%- endif -%}
+                {%- if value['required'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- macro format_tool_response_block(tool_name, response) -%}
+    {{- '<|tool_response>' -}}
+    {%- if response is mapping -%}
+        {{- 'response:' + tool_name + '{' -}}
+        {%- for key, value in response | dictsort -%}
+            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- else -%}
+        {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
+    {%- endif -%}
+    {{- '<tool_response|>' -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{- bos_token -}}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>\n' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Pre-scan: find last user message index for reasoning guard -#}
+{%- set ns_turn = namespace(last_user_idx=-1) -%}
+{%- for i in range(loop_messages | length) -%}
+    {%- if loop_messages[i]['role'] == 'user' -%}
+        {%- set ns_turn.last_user_idx = i -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- if message['role'] != 'tool' -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+    {#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
+    {%- set prev_nt = namespace(role=None, found=false) -%}
+    {%- if loop.index0 > 0 -%}
+        {%- for j in range(loop.index0 - 1, -1, -1) -%}
+            {%- if not prev_nt.found -%}
+                {%- if loop_messages[j]['role'] != 'tool' -%}
+                    {%- set prev_nt.role = loop_messages[j]['role'] -%}
+                    {%- set prev_nt.found = true -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- set continue_same_model_turn = (role == 'model' and prev_nt.role == 'assistant') -%}
+    {%- if not continue_same_model_turn -%}
+        {{- '<|turn>' + role + '\n' }}
+    {%- endif -%}
+
+    {#- Render reasoning/reasoning_content as thinking channel -#}
+    {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
+    {%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}
+        {{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
+    {%- endif -%}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- set ns_tr_out = namespace(flag=false) -%}
+            {%- if message.get('tool_responses') -%}
+                {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- format_tool_response_block(tool_response['name'] | default('unknown'), tool_response['response']) -}}
+                    {%- set ns_tr_out.flag = true -%}
+                    {%- set ns.prev_message_type = 'tool_response' -%}
+                {%- endfor -%}
+            {%- elif message.get('tool_calls') -%}
+                {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
+                {%- set ns_tool_scan = namespace(stopped=false) -%}
+                {%- for k in range(loop.index0 + 1, loop_messages | length) -%}
+                    {%- if ns_tool_scan.stopped -%}
+                    {%- elif loop_messages[k]['role'] != 'tool' -%}
+                        {%- set ns_tool_scan.stopped = true -%}
+                    {%- else -%}
+                        {%- set follow = loop_messages[k] -%}
+                        {#- Resolve tool_call_id to function name -#}
+                        {%- set ns_tname = namespace(name=follow.get('name') | default('unknown')) -%}
+                        {%- for tc in message['tool_calls'] -%}
+                            {%- if tc.get('id') == follow.get('tool_call_id') -%}
+                                {%- set ns_tname.name = tc['function']['name'] -%}
+                            {%- endif -%}
+                        {%- endfor -%}
+                        {#- Handle content as string or content-parts array -#}
+                        {%- set tool_body = follow.get('content') -%}
+                        {%- if tool_body is string -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- elif tool_body is sequence and tool_body is not string -%}
+                            {%- set ns_txt = namespace(s='') -%}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') == 'text' -%}
+                                    {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
+                                {%- endif -%}
+                            {%- endfor -%}
+                            {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
+                        {%- else -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- endif -%}
+                        {%- set ns_tr_out.flag = true -%}
+                        {%- set ns.prev_message_type = 'tool_response' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '<|image|>' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '<|video|>' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
+            {{- '<|tool_response>' -}}
+        {%- elif not (ns_tr_out.flag and not message.get('content')) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+{%- endif -%}

checkpoint-375/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a98ccecdd842b900ccebf96e92fa3df1c5eb0c64a8da9726b8dc3dd3d2a32af
+size 149793045

checkpoint-375/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72709e309c5981b60603e21ee743a615a9df5f4852b575e9cb181bd853e10838
+size 14581

checkpoint-375/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9069c74ae2428782b77a0105f94aeb0b0567a97820c3b2d55e3ff1c1e9cd959c
+size 1465

checkpoint-375/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

checkpoint-375/tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

checkpoint-375/trainer_state.json

@@ -0,0 +1,2659 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 375,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0026666666666666666,
+      "grad_norm": 213.6519012451172,
+      "learning_rate": 0.0,
+      "loss": 13.681682586669922,
+      "step": 1
+    },
+    {
+      "epoch": 0.005333333333333333,
+      "grad_norm": 214.1101837158203,
+      "learning_rate": 5.263157894736843e-07,
+      "loss": 13.148241996765137,
+      "step": 2
+    },
+    {
+      "epoch": 0.008,
+      "grad_norm": 214.175048828125,
+      "learning_rate": 1.0526315789473685e-06,
+      "loss": 13.208710670471191,
+      "step": 3
+    },
+    {
+      "epoch": 0.010666666666666666,
+      "grad_norm": 211.5797882080078,
+      "learning_rate": 1.5789473684210526e-06,
+      "loss": 13.06391716003418,
+      "step": 4
+    },
+    {
+      "epoch": 0.013333333333333334,
+      "grad_norm": 203.69789123535156,
+      "learning_rate": 2.105263157894737e-06,
+      "loss": 12.938375473022461,
+      "step": 5
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 195.5794677734375,
+      "learning_rate": 2.631578947368421e-06,
+      "loss": 12.380300521850586,
+      "step": 6
+    },
+    {
+      "epoch": 0.018666666666666668,
+      "grad_norm": 171.2223663330078,
+      "learning_rate": 3.157894736842105e-06,
+      "loss": 11.961210250854492,
+      "step": 7
+    },
+    {
+      "epoch": 0.021333333333333333,
+      "grad_norm": 154.28985595703125,
+      "learning_rate": 3.6842105263157896e-06,
+      "loss": 10.95643424987793,
+      "step": 8
+    },
+    {
+      "epoch": 0.024,
+      "grad_norm": 133.29501342773438,
+      "learning_rate": 4.210526315789474e-06,
+      "loss": 10.132217407226562,
+      "step": 9
+    },
+    {
+      "epoch": 0.02666666666666667,
+      "grad_norm": 103.80883026123047,
+      "learning_rate": 4.736842105263158e-06,
+      "loss": 9.099992752075195,
+      "step": 10
+    },
+    {
+      "epoch": 0.029333333333333333,
+      "grad_norm": 87.41455841064453,
+      "learning_rate": 5.263157894736842e-06,
+      "loss": 8.44427490234375,
+      "step": 11
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 74.7413101196289,
+      "learning_rate": 5.789473684210527e-06,
+      "loss": 7.79477596282959,
+      "step": 12
+    },
+    {
+      "epoch": 0.034666666666666665,
+      "grad_norm": 60.24201202392578,
+      "learning_rate": 6.31578947368421e-06,
+      "loss": 7.2957377433776855,
+      "step": 13
+    },
+    {
+      "epoch": 0.037333333333333336,
+      "grad_norm": 54.014644622802734,
+      "learning_rate": 6.842105263157896e-06,
+      "loss": 7.009221076965332,
+      "step": 14
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 45.813697814941406,
+      "learning_rate": 7.368421052631579e-06,
+      "loss": 6.403441905975342,
+      "step": 15
+    },
+    {
+      "epoch": 0.042666666666666665,
+      "grad_norm": 38.086910247802734,
+      "learning_rate": 7.894736842105265e-06,
+      "loss": 6.087275981903076,
+      "step": 16
+    },
+    {
+      "epoch": 0.04533333333333334,
+      "grad_norm": 39.79656982421875,
+      "learning_rate": 8.421052631578948e-06,
+      "loss": 5.71798849105835,
+      "step": 17
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 30.681772232055664,
+      "learning_rate": 8.947368421052632e-06,
+      "loss": 5.540742874145508,
+      "step": 18
+    },
+    {
+      "epoch": 0.050666666666666665,
+      "grad_norm": 30.45841407775879,
+      "learning_rate": 9.473684210526315e-06,
+      "loss": 5.253246307373047,
+      "step": 19
+    },
+    {
+      "epoch": 0.05333333333333334,
+      "grad_norm": 25.876440048217773,
+      "learning_rate": 1e-05,
+      "loss": 5.0631866455078125,
+      "step": 20
+    },
+    {
+      "epoch": 0.056,
+      "grad_norm": 23.709259033203125,
+      "learning_rate": 9.971910112359552e-06,
+      "loss": 4.810742378234863,
+      "step": 21
+    },
+    {
+      "epoch": 0.058666666666666666,
+      "grad_norm": 21.863664627075195,
+      "learning_rate": 9.943820224719102e-06,
+      "loss": 4.369226455688477,
+      "step": 22
+    },
+    {
+      "epoch": 0.06133333333333333,
+      "grad_norm": 20.434940338134766,
+      "learning_rate": 9.915730337078653e-06,
+      "loss": 4.1208624839782715,
+      "step": 23
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 21.96327781677246,
+      "learning_rate": 9.887640449438202e-06,
+      "loss": 4.111361980438232,
+      "step": 24
+    },
+    {
+      "epoch": 0.06666666666666667,
+      "grad_norm": 18.36846923828125,
+      "learning_rate": 9.859550561797754e-06,
+      "loss": 3.787933111190796,
+      "step": 25
+    },
+    {
+      "epoch": 0.06933333333333333,
+      "grad_norm": 16.87392234802246,
+      "learning_rate": 9.831460674157303e-06,
+      "loss": 4.084612846374512,
+      "step": 26
+    },
+    {
+      "epoch": 0.072,
+      "grad_norm": 17.08782196044922,
+      "learning_rate": 9.803370786516854e-06,
+      "loss": 3.689074993133545,
+      "step": 27
+    },
+    {
+      "epoch": 0.07466666666666667,
+      "grad_norm": 16.84369468688965,
+      "learning_rate": 9.775280898876405e-06,
+      "loss": 3.873020887374878,
+      "step": 28
+    },
+    {
+      "epoch": 0.07733333333333334,
+      "grad_norm": 14.4940824508667,
+      "learning_rate": 9.747191011235956e-06,
+      "loss": 3.5138087272644043,
+      "step": 29
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 14.818546295166016,
+      "learning_rate": 9.719101123595506e-06,
+      "loss": 3.3739891052246094,
+      "step": 30
+    },
+    {
+      "epoch": 0.08266666666666667,
+      "grad_norm": 14.110360145568848,
+      "learning_rate": 9.691011235955057e-06,
+      "loss": 3.0248146057128906,
+      "step": 31
+    },
+    {
+      "epoch": 0.08533333333333333,
+      "grad_norm": 15.661042213439941,
+      "learning_rate": 9.662921348314608e-06,
+      "loss": 2.938133716583252,
+      "step": 32
+    },
+    {
+      "epoch": 0.088,
+      "grad_norm": 14.291086196899414,
+      "learning_rate": 9.634831460674158e-06,
+      "loss": 2.8373093605041504,
+      "step": 33
+    },
+    {
+      "epoch": 0.09066666666666667,
+      "grad_norm": 12.506660461425781,
+      "learning_rate": 9.60674157303371e-06,
+      "loss": 2.859283924102783,
+      "step": 34
+    },
+    {
+      "epoch": 0.09333333333333334,
+      "grad_norm": 12.220486640930176,
+      "learning_rate": 9.578651685393258e-06,
+      "loss": 2.730626106262207,
+      "step": 35
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 10.75361442565918,
+      "learning_rate": 9.55056179775281e-06,
+      "loss": 2.779308319091797,
+      "step": 36
+    },
+    {
+      "epoch": 0.09866666666666667,
+      "grad_norm": 11.151102066040039,
+      "learning_rate": 9.522471910112361e-06,
+      "loss": 2.851846694946289,
+      "step": 37
+    },
+    {
+      "epoch": 0.10133333333333333,
+      "grad_norm": 13.231782913208008,
+      "learning_rate": 9.49438202247191e-06,
+      "loss": 2.5308845043182373,
+      "step": 38
+    },
+    {
+      "epoch": 0.104,
+      "grad_norm": 11.619803428649902,
+      "learning_rate": 9.466292134831461e-06,
+      "loss": 2.5366804599761963,
+      "step": 39
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 10.728320121765137,
+      "learning_rate": 9.438202247191012e-06,
+      "loss": 2.176302671432495,
+      "step": 40
+    },
+    {
+      "epoch": 0.10933333333333334,
+      "grad_norm": 10.922004699707031,
+      "learning_rate": 9.410112359550562e-06,
+      "loss": 2.6344969272613525,
+      "step": 41
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 10.467931747436523,
+      "learning_rate": 9.382022471910113e-06,
+      "loss": 2.270345687866211,
+      "step": 42
+    },
+    {
+      "epoch": 0.11466666666666667,
+      "grad_norm": 11.972366333007812,
+      "learning_rate": 9.353932584269664e-06,
+      "loss": 2.262977123260498,
+      "step": 43
+    },
+    {
+      "epoch": 0.11733333333333333,
+      "grad_norm": 10.597440719604492,
+      "learning_rate": 9.325842696629213e-06,
+      "loss": 2.3384501934051514,
+      "step": 44
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 12.487441062927246,
+      "learning_rate": 9.297752808988765e-06,
+      "loss": 2.446566104888916,
+      "step": 45
+    },
+    {
+      "epoch": 0.12266666666666666,
+      "grad_norm": 10.253981590270996,
+      "learning_rate": 9.269662921348316e-06,
+      "loss": 2.035454750061035,
+      "step": 46
+    },
+    {
+      "epoch": 0.12533333333333332,
+      "grad_norm": 9.695643424987793,
+      "learning_rate": 9.241573033707865e-06,
+      "loss": 2.3127524852752686,
+      "step": 47
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 10.694028854370117,
+      "learning_rate": 9.213483146067417e-06,
+      "loss": 2.2996160984039307,
+      "step": 48
+    },
+    {
+      "epoch": 0.13066666666666665,
+      "grad_norm": 12.787583351135254,
+      "learning_rate": 9.185393258426967e-06,
+      "loss": 2.5168590545654297,
+      "step": 49
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 10.594977378845215,
+      "learning_rate": 9.157303370786517e-06,
+      "loss": 1.8576197624206543,
+      "step": 50
+    },
+    {
+      "epoch": 0.136,
+      "grad_norm": 11.105279922485352,
+      "learning_rate": 9.129213483146068e-06,
+      "loss": 2.0596225261688232,
+      "step": 51
+    },
+    {
+      "epoch": 0.13866666666666666,
+      "grad_norm": 11.56894588470459,
+      "learning_rate": 9.101123595505619e-06,
+      "loss": 2.1656343936920166,
+      "step": 52
+    },
+    {
+      "epoch": 0.14133333333333334,
+      "grad_norm": 10.579730987548828,
+      "learning_rate": 9.07303370786517e-06,
+      "loss": 1.969504475593567,
+      "step": 53
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 9.501631736755371,
+      "learning_rate": 9.04494382022472e-06,
+      "loss": 1.94535493850708,
+      "step": 54
+    },
+    {
+      "epoch": 0.14666666666666667,
+      "grad_norm": 10.170072555541992,
+      "learning_rate": 9.01685393258427e-06,
+      "loss": 2.0672125816345215,
+      "step": 55
+    },
+    {
+      "epoch": 0.14933333333333335,
+      "grad_norm": 9.39763069152832,
+      "learning_rate": 8.988764044943822e-06,
+      "loss": 2.132148504257202,
+      "step": 56
+    },
+    {
+      "epoch": 0.152,
+      "grad_norm": 10.032671928405762,
+      "learning_rate": 8.960674157303372e-06,
+      "loss": 2.043534517288208,
+      "step": 57
+    },
+    {
+      "epoch": 0.15466666666666667,
+      "grad_norm": 11.097075462341309,
+      "learning_rate": 8.932584269662921e-06,
+      "loss": 1.6756911277770996,
+      "step": 58
+    },
+    {
+      "epoch": 0.15733333333333333,
+      "grad_norm": 11.116305351257324,
+      "learning_rate": 8.904494382022472e-06,
+      "loss": 2.0522589683532715,
+      "step": 59
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 10.924529075622559,
+      "learning_rate": 8.876404494382023e-06,
+      "loss": 1.9636679887771606,
+      "step": 60
+    },
+    {
+      "epoch": 0.16266666666666665,
+      "grad_norm": 9.252256393432617,
+      "learning_rate": 8.848314606741573e-06,
+      "loss": 1.6284159421920776,
+      "step": 61
+    },
+    {
+      "epoch": 0.16533333333333333,
+      "grad_norm": 11.261286735534668,
+      "learning_rate": 8.820224719101124e-06,
+      "loss": 1.6471984386444092,
+      "step": 62
+    },
+    {
+      "epoch": 0.168,
+      "grad_norm": 10.263949394226074,
+      "learning_rate": 8.792134831460675e-06,
+      "loss": 1.6104894876480103,
+      "step": 63
+    },
+    {
+      "epoch": 0.17066666666666666,
+      "grad_norm": 10.444320678710938,
+      "learning_rate": 8.764044943820226e-06,
+      "loss": 1.5480685234069824,
+      "step": 64
+    },
+    {
+      "epoch": 0.17333333333333334,
+      "grad_norm": 10.065644264221191,
+      "learning_rate": 8.735955056179776e-06,
+      "loss": 1.5983505249023438,
+      "step": 65
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 10.745816230773926,
+      "learning_rate": 8.707865168539327e-06,
+      "loss": 1.5451902151107788,
+      "step": 66
+    },
+    {
+      "epoch": 0.17866666666666667,
+      "grad_norm": 11.547348022460938,
+      "learning_rate": 8.679775280898876e-06,
+      "loss": 1.5520156621932983,
+      "step": 67
+    },
+    {
+      "epoch": 0.18133333333333335,
+      "grad_norm": 11.1903076171875,
+      "learning_rate": 8.651685393258428e-06,
+      "loss": 1.3468866348266602,
+      "step": 68
+    },
+    {
+      "epoch": 0.184,
+      "grad_norm": 10.134865760803223,
+      "learning_rate": 8.623595505617979e-06,
+      "loss": 1.457981824874878,
+      "step": 69
+    },
+    {
+      "epoch": 0.18666666666666668,
+      "grad_norm": 10.580246925354004,
+      "learning_rate": 8.595505617977528e-06,
+      "loss": 1.530599594116211,
+      "step": 70
+    },
+    {
+      "epoch": 0.18933333333333333,
+      "grad_norm": 10.744487762451172,
+      "learning_rate": 8.56741573033708e-06,
+      "loss": 1.24165678024292,
+      "step": 71
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 11.34036922454834,
+      "learning_rate": 8.53932584269663e-06,
+      "loss": 1.1307389736175537,
+      "step": 72
+    },
+    {
+      "epoch": 0.19466666666666665,
+      "grad_norm": 12.146100997924805,
+      "learning_rate": 8.51123595505618e-06,
+      "loss": 1.4014456272125244,
+      "step": 73
+    },
+    {
+      "epoch": 0.19733333333333333,
+      "grad_norm": 11.810182571411133,
+      "learning_rate": 8.483146067415731e-06,
+      "loss": 1.3475110530853271,
+      "step": 74
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 10.650108337402344,
+      "learning_rate": 8.455056179775282e-06,
+      "loss": 1.2535855770111084,
+      "step": 75
+    },
+    {
+      "epoch": 0.20266666666666666,
+      "grad_norm": 11.479569435119629,
+      "learning_rate": 8.426966292134832e-06,
+      "loss": 1.1040645837783813,
+      "step": 76
+    },
+    {
+      "epoch": 0.20533333333333334,
+      "grad_norm": 13.411121368408203,
+      "learning_rate": 8.398876404494383e-06,
+      "loss": 1.1873900890350342,
+      "step": 77
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 12.876670837402344,
+      "learning_rate": 8.370786516853934e-06,
+      "loss": 1.2497323751449585,
+      "step": 78
+    },
+    {
+      "epoch": 0.21066666666666667,
+      "grad_norm": 11.155399322509766,
+      "learning_rate": 8.342696629213483e-06,
+      "loss": 1.1884422302246094,
+      "step": 79
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 11.68853759765625,
+      "learning_rate": 8.314606741573035e-06,
+      "loss": 1.2792617082595825,
+      "step": 80
+    },
+    {
+      "epoch": 0.216,
+      "grad_norm": 10.7301025390625,
+      "learning_rate": 8.286516853932584e-06,
+      "loss": 1.2246038913726807,
+      "step": 81
+    },
+    {
+      "epoch": 0.21866666666666668,
+      "grad_norm": 9.901270866394043,
+      "learning_rate": 8.258426966292135e-06,
+      "loss": 1.0451529026031494,
+      "step": 82
+    },
+    {
+      "epoch": 0.22133333333333333,
+      "grad_norm": 9.585644721984863,
+      "learning_rate": 8.230337078651686e-06,
+      "loss": 0.9552695751190186,
+      "step": 83
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 12.750142097473145,
+      "learning_rate": 8.202247191011237e-06,
+      "loss": 1.348262071609497,
+      "step": 84
+    },
+    {
+      "epoch": 0.22666666666666666,
+      "grad_norm": 11.14697265625,
+      "learning_rate": 8.174157303370787e-06,
+      "loss": 1.0186105966567993,
+      "step": 85
+    },
+    {
+      "epoch": 0.22933333333333333,
+      "grad_norm": 11.14384651184082,
+      "learning_rate": 8.146067415730338e-06,
+      "loss": 1.0607019662857056,
+      "step": 86
+    },
+    {
+      "epoch": 0.232,
+      "grad_norm": 10.328786849975586,
+      "learning_rate": 8.117977528089889e-06,
+      "loss": 0.9560970664024353,
+      "step": 87
+    },
+    {
+      "epoch": 0.23466666666666666,
+      "grad_norm": 10.841139793395996,
+      "learning_rate": 8.08988764044944e-06,
+      "loss": 0.9277609586715698,
+      "step": 88
+    },
+    {
+      "epoch": 0.23733333333333334,
+      "grad_norm": 11.283014297485352,
+      "learning_rate": 8.06179775280899e-06,
+      "loss": 1.1036794185638428,
+      "step": 89
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 16.602310180664062,
+      "learning_rate": 8.033707865168539e-06,
+      "loss": 1.1436660289764404,
+      "step": 90
+    },
+    {
+      "epoch": 0.24266666666666667,
+      "grad_norm": 10.93206787109375,
+      "learning_rate": 8.00561797752809e-06,
+      "loss": 0.8919620513916016,
+      "step": 91
+    },
+    {
+      "epoch": 0.24533333333333332,
+      "grad_norm": 11.306818008422852,
+      "learning_rate": 7.97752808988764e-06,
+      "loss": 1.0714668035507202,
+      "step": 92
+    },
+    {
+      "epoch": 0.248,
+      "grad_norm": 9.216002464294434,
+      "learning_rate": 7.949438202247191e-06,
+      "loss": 1.0274698734283447,
+      "step": 93
+    },
+    {
+      "epoch": 0.25066666666666665,
+      "grad_norm": 9.062772750854492,
+      "learning_rate": 7.921348314606742e-06,
+      "loss": 0.6113460659980774,
+      "step": 94
+    },
+    {
+      "epoch": 0.25333333333333335,
+      "grad_norm": 8.908549308776855,
+      "learning_rate": 7.893258426966293e-06,
+      "loss": 0.7682806849479675,
+      "step": 95
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 9.468413352966309,
+      "learning_rate": 7.865168539325843e-06,
+      "loss": 0.931200385093689,
+      "step": 96
+    },
+    {
+      "epoch": 0.25866666666666666,
+      "grad_norm": 9.342957496643066,
+      "learning_rate": 7.837078651685394e-06,
+      "loss": 0.8321046829223633,
+      "step": 97
+    },
+    {
+      "epoch": 0.2613333333333333,
+      "grad_norm": 8.902146339416504,
+      "learning_rate": 7.808988764044945e-06,
+      "loss": 0.5983229279518127,
+      "step": 98
+    },
+    {
+      "epoch": 0.264,
+      "grad_norm": 10.691293716430664,
+      "learning_rate": 7.780898876404494e-06,
+      "loss": 0.6863845586776733,
+      "step": 99
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 10.162596702575684,
+      "learning_rate": 7.752808988764046e-06,
+      "loss": 0.843379557132721,
+      "step": 100
+    },
+    {
+      "epoch": 0.2693333333333333,
+      "grad_norm": 12.803357124328613,
+      "learning_rate": 7.724719101123597e-06,
+      "loss": 0.9776611328125,
+      "step": 101
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 9.056926727294922,
+      "learning_rate": 7.696629213483146e-06,
+      "loss": 0.6964964866638184,
+      "step": 102
+    },
+    {
+      "epoch": 0.27466666666666667,
+      "grad_norm": 10.713375091552734,
+      "learning_rate": 7.668539325842698e-06,
+      "loss": 0.8867349624633789,
+      "step": 103
+    },
+    {
+      "epoch": 0.2773333333333333,
+      "grad_norm": 11.21557331085205,
+      "learning_rate": 7.640449438202247e-06,
+      "loss": 0.8569594025611877,
+      "step": 104
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 9.143693923950195,
+      "learning_rate": 7.612359550561799e-06,
+      "loss": 0.6123837232589722,
+      "step": 105
+    },
+    {
+      "epoch": 0.2826666666666667,
+      "grad_norm": 9.902138710021973,
+      "learning_rate": 7.584269662921349e-06,
+      "loss": 0.6609525084495544,
+      "step": 106
+    },
+    {
+      "epoch": 0.2853333333333333,
+      "grad_norm": 8.925782203674316,
+      "learning_rate": 7.5561797752809e-06,
+      "loss": 0.6758649945259094,
+      "step": 107
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 8.8521089553833,
+      "learning_rate": 7.5280898876404495e-06,
+      "loss": 0.6611078977584839,
+      "step": 108
+    },
+    {
+      "epoch": 0.2906666666666667,
+      "grad_norm": 11.167701721191406,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 1.004479169845581,
+      "step": 109
+    },
+    {
+      "epoch": 0.29333333333333333,
+      "grad_norm": 10.848076820373535,
+      "learning_rate": 7.471910112359552e-06,
+      "loss": 0.6455778479576111,
+      "step": 110
+    },
+    {
+      "epoch": 0.296,
+      "grad_norm": 8.009282112121582,
+      "learning_rate": 7.443820224719102e-06,
+      "loss": 0.5215024948120117,
+      "step": 111
+    },
+    {
+      "epoch": 0.2986666666666667,
+      "grad_norm": 11.672330856323242,
+      "learning_rate": 7.415730337078652e-06,
+      "loss": 0.8420214653015137,
+      "step": 112
+    },
+    {
+      "epoch": 0.30133333333333334,
+      "grad_norm": 9.667366981506348,
+      "learning_rate": 7.387640449438202e-06,
+      "loss": 0.7665308117866516,
+      "step": 113
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 7.986022472381592,
+      "learning_rate": 7.359550561797754e-06,
+      "loss": 0.6188411712646484,
+      "step": 114
+    },
+    {
+      "epoch": 0.30666666666666664,
+      "grad_norm": 11.240910530090332,
+      "learning_rate": 7.331460674157304e-06,
+      "loss": 0.7242427468299866,
+      "step": 115
+    },
+    {
+      "epoch": 0.30933333333333335,
+      "grad_norm": 7.829097270965576,
+      "learning_rate": 7.303370786516854e-06,
+      "loss": 0.4696412682533264,
+      "step": 116
+    },
+    {
+      "epoch": 0.312,
+      "grad_norm": 9.687139511108398,
+      "learning_rate": 7.275280898876404e-06,
+      "loss": 0.6932588815689087,
+      "step": 117
+    },
+    {
+      "epoch": 0.31466666666666665,
+      "grad_norm": 11.388943672180176,
+      "learning_rate": 7.247191011235956e-06,
+      "loss": 0.6690924763679504,
+      "step": 118
+    },
+    {
+      "epoch": 0.31733333333333336,
+      "grad_norm": 10.902031898498535,
+      "learning_rate": 7.2191011235955065e-06,
+      "loss": 0.6181185245513916,
+      "step": 119
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 9.588140487670898,
+      "learning_rate": 7.191011235955056e-06,
+      "loss": 0.6805417537689209,
+      "step": 120
+    },
+    {
+      "epoch": 0.32266666666666666,
+      "grad_norm": 12.04230785369873,
+      "learning_rate": 7.162921348314608e-06,
+      "loss": 1.1967664957046509,
+      "step": 121
+    },
+    {
+      "epoch": 0.3253333333333333,
+      "grad_norm": 8.957392692565918,
+      "learning_rate": 7.134831460674158e-06,
+      "loss": 0.632103681564331,
+      "step": 122
+    },
+    {
+      "epoch": 0.328,
+      "grad_norm": 8.867446899414062,
+      "learning_rate": 7.1067415730337085e-06,
+      "loss": 0.7522599697113037,
+      "step": 123
+    },
+    {
+      "epoch": 0.33066666666666666,
+      "grad_norm": 10.266507148742676,
+      "learning_rate": 7.078651685393258e-06,
+      "loss": 0.738883912563324,
+      "step": 124
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 8.87643051147461,
+      "learning_rate": 7.05056179775281e-06,
+      "loss": 0.5069182515144348,
+      "step": 125
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 9.884408950805664,
+      "learning_rate": 7.022471910112361e-06,
+      "loss": 0.5969939827919006,
+      "step": 126
+    },
+    {
+      "epoch": 0.33866666666666667,
+      "grad_norm": 8.659189224243164,
+      "learning_rate": 6.9943820224719105e-06,
+      "loss": 0.825599730014801,
+      "step": 127
+    },
+    {
+      "epoch": 0.3413333333333333,
+      "grad_norm": 9.328271865844727,
+      "learning_rate": 6.966292134831461e-06,
+      "loss": 0.8193888664245605,
+      "step": 128
+    },
+    {
+      "epoch": 0.344,
+      "grad_norm": 9.927331924438477,
+      "learning_rate": 6.938202247191011e-06,
+      "loss": 0.562104344367981,
+      "step": 129
+    },
+    {
+      "epoch": 0.3466666666666667,
+      "grad_norm": 7.426764011383057,
+      "learning_rate": 6.910112359550563e-06,
+      "loss": 0.590276300907135,
+      "step": 130
+    },
+    {
+      "epoch": 0.34933333333333333,
+      "grad_norm": 12.05113410949707,
+      "learning_rate": 6.8820224719101126e-06,
+      "loss": 0.7326091527938843,
+      "step": 131
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 7.93930196762085,
+      "learning_rate": 6.853932584269663e-06,
+      "loss": 0.6142013669013977,
+      "step": 132
+    },
+    {
+      "epoch": 0.3546666666666667,
+      "grad_norm": 9.491640090942383,
+      "learning_rate": 6.825842696629213e-06,
+      "loss": 0.5451798439025879,
+      "step": 133
+    },
+    {
+      "epoch": 0.35733333333333334,
+      "grad_norm": 10.750314712524414,
+      "learning_rate": 6.797752808988765e-06,
+      "loss": 0.6363071203231812,
+      "step": 134
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 11.000245094299316,
+      "learning_rate": 6.7696629213483154e-06,
+      "loss": 0.6733920574188232,
+      "step": 135
+    },
+    {
+      "epoch": 0.3626666666666667,
+      "grad_norm": 7.272871017456055,
+      "learning_rate": 6.741573033707865e-06,
+      "loss": 0.418811559677124,
+      "step": 136
+    },
+    {
+      "epoch": 0.36533333333333334,
+      "grad_norm": 9.358305931091309,
+      "learning_rate": 6.713483146067417e-06,
+      "loss": 0.7467747926712036,
+      "step": 137
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 8.385472297668457,
+      "learning_rate": 6.685393258426967e-06,
+      "loss": 0.6022355556488037,
+      "step": 138
+    },
+    {
+      "epoch": 0.37066666666666664,
+      "grad_norm": 8.527623176574707,
+      "learning_rate": 6.6573033707865175e-06,
+      "loss": 0.4671086072921753,
+      "step": 139
+    },
+    {
+      "epoch": 0.37333333333333335,
+      "grad_norm": 7.954482555389404,
+      "learning_rate": 6.629213483146067e-06,
+      "loss": 0.6525954008102417,
+      "step": 140
+    },
+    {
+      "epoch": 0.376,
+      "grad_norm": 7.474452495574951,
+      "learning_rate": 6.601123595505619e-06,
+      "loss": 0.6411296129226685,
+      "step": 141
+    },
+    {
+      "epoch": 0.37866666666666665,
+      "grad_norm": 7.6223297119140625,
+      "learning_rate": 6.57303370786517e-06,
+      "loss": 0.5695533752441406,
+      "step": 142
+    },
+    {
+      "epoch": 0.38133333333333336,
+      "grad_norm": 9.030717849731445,
+      "learning_rate": 6.5449438202247195e-06,
+      "loss": 0.4683411419391632,
+      "step": 143
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 7.926174640655518,
+      "learning_rate": 6.51685393258427e-06,
+      "loss": 0.5910580158233643,
+      "step": 144
+    },
+    {
+      "epoch": 0.38666666666666666,
+      "grad_norm": 11.901198387145996,
+      "learning_rate": 6.488764044943821e-06,
+      "loss": 0.8150304555892944,
+      "step": 145
+    },
+    {
+      "epoch": 0.3893333333333333,
+      "grad_norm": 7.352392673492432,
+      "learning_rate": 6.460674157303372e-06,
+      "loss": 0.39086320996284485,
+      "step": 146
+    },
+    {
+      "epoch": 0.392,
+      "grad_norm": 7.774436950683594,
+      "learning_rate": 6.4325842696629215e-06,
+      "loss": 0.2628573179244995,
+      "step": 147
+    },
+    {
+      "epoch": 0.39466666666666667,
+      "grad_norm": 8.74471664428711,
+      "learning_rate": 6.404494382022472e-06,
+      "loss": 0.6225802898406982,
+      "step": 148
+    },
+    {
+      "epoch": 0.3973333333333333,
+      "grad_norm": 8.091107368469238,
+      "learning_rate": 6.376404494382022e-06,
+      "loss": 0.565784752368927,
+      "step": 149
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 7.628470420837402,
+      "learning_rate": 6.348314606741574e-06,
+      "loss": 0.6476342678070068,
+      "step": 150
+    },
+    {
+      "epoch": 0.4026666666666667,
+      "grad_norm": 6.314539432525635,
+      "learning_rate": 6.320224719101124e-06,
+      "loss": 0.3146457374095917,
+      "step": 151
+    },
+    {
+      "epoch": 0.4053333333333333,
+      "grad_norm": 7.166040897369385,
+      "learning_rate": 6.292134831460674e-06,
+      "loss": 0.501552164554596,
+      "step": 152
+    },
+    {
+      "epoch": 0.408,
+      "grad_norm": 9.255748748779297,
+      "learning_rate": 6.264044943820226e-06,
+      "loss": 0.48986560106277466,
+      "step": 153
+    },
+    {
+      "epoch": 0.4106666666666667,
+      "grad_norm": 7.911505699157715,
+      "learning_rate": 6.235955056179776e-06,
+      "loss": 0.46173685789108276,
+      "step": 154
+    },
+    {
+      "epoch": 0.41333333333333333,
+      "grad_norm": 9.642655372619629,
+      "learning_rate": 6.207865168539326e-06,
+      "loss": 0.7007043361663818,
+      "step": 155
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 11.969674110412598,
+      "learning_rate": 6.179775280898876e-06,
+      "loss": 0.638664722442627,
+      "step": 156
+    },
+    {
+      "epoch": 0.4186666666666667,
+      "grad_norm": 10.90139102935791,
+      "learning_rate": 6.151685393258428e-06,
+      "loss": 0.6380130052566528,
+      "step": 157
+    },
+    {
+      "epoch": 0.42133333333333334,
+      "grad_norm": 12.92622184753418,
+      "learning_rate": 6.1235955056179785e-06,
+      "loss": 0.6583395004272461,
+      "step": 158
+    },
+    {
+      "epoch": 0.424,
+      "grad_norm": 9.722634315490723,
+      "learning_rate": 6.095505617977528e-06,
+      "loss": 0.6104496121406555,
+      "step": 159
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 6.8508524894714355,
+      "learning_rate": 6.06741573033708e-06,
+      "loss": 0.5285036563873291,
+      "step": 160
+    },
+    {
+      "epoch": 0.42933333333333334,
+      "grad_norm": 8.004532814025879,
+      "learning_rate": 6.03932584269663e-06,
+      "loss": 0.6247756481170654,
+      "step": 161
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 6.019658088684082,
+      "learning_rate": 6.0112359550561805e-06,
+      "loss": 0.3690226078033447,
+      "step": 162
+    },
+    {
+      "epoch": 0.43466666666666665,
+      "grad_norm": 10.048113822937012,
+      "learning_rate": 5.98314606741573e-06,
+      "loss": 0.7174047827720642,
+      "step": 163
+    },
+    {
+      "epoch": 0.43733333333333335,
+      "grad_norm": 8.477190017700195,
+      "learning_rate": 5.955056179775281e-06,
+      "loss": 0.5005499124526978,
+      "step": 164
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 7.024532794952393,
+      "learning_rate": 5.926966292134832e-06,
+      "loss": 0.42374947667121887,
+      "step": 165
+    },
+    {
+      "epoch": 0.44266666666666665,
+      "grad_norm": 7.412145137786865,
+      "learning_rate": 5.8988764044943826e-06,
+      "loss": 0.579714834690094,
+      "step": 166
+    },
+    {
+      "epoch": 0.44533333333333336,
+      "grad_norm": 6.842972278594971,
+      "learning_rate": 5.870786516853933e-06,
+      "loss": 0.4542590081691742,
+      "step": 167
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 7.651519775390625,
+      "learning_rate": 5.842696629213483e-06,
+      "loss": 0.4966958463191986,
+      "step": 168
+    },
+    {
+      "epoch": 0.45066666666666666,
+      "grad_norm": 11.124903678894043,
+      "learning_rate": 5.814606741573035e-06,
+      "loss": 0.5635814666748047,
+      "step": 169
+    },
+    {
+      "epoch": 0.4533333333333333,
+      "grad_norm": 7.356116771697998,
+      "learning_rate": 5.786516853932585e-06,
+      "loss": 0.4059095084667206,
+      "step": 170
+    },
+    {
+      "epoch": 0.456,
+      "grad_norm": 8.28013801574707,
+      "learning_rate": 5.758426966292135e-06,
+      "loss": 0.518947422504425,
+      "step": 171
+    },
+    {
+      "epoch": 0.45866666666666667,
+      "grad_norm": 7.578033924102783,
+      "learning_rate": 5.730337078651685e-06,
+      "loss": 0.5317070484161377,
+      "step": 172
+    },
+    {
+      "epoch": 0.4613333333333333,
+      "grad_norm": 7.033680438995361,
+      "learning_rate": 5.702247191011237e-06,
+      "loss": 0.30602124333381653,
+      "step": 173
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 8.343293190002441,
+      "learning_rate": 5.6741573033707874e-06,
+      "loss": 0.4346192479133606,
+      "step": 174
+    },
+    {
+      "epoch": 0.4666666666666667,
+      "grad_norm": 8.341903686523438,
+      "learning_rate": 5.646067415730337e-06,
+      "loss": 0.6472276449203491,
+      "step": 175
+    },
+    {
+      "epoch": 0.4693333333333333,
+      "grad_norm": 8.249134063720703,
+      "learning_rate": 5.617977528089889e-06,
+      "loss": 0.5967584848403931,
+      "step": 176
+    },
+    {
+      "epoch": 0.472,
+      "grad_norm": 8.661340713500977,
+      "learning_rate": 5.589887640449439e-06,
+      "loss": 0.5620930194854736,
+      "step": 177
+    },
+    {
+      "epoch": 0.4746666666666667,
+      "grad_norm": 7.002172470092773,
+      "learning_rate": 5.5617977528089895e-06,
+      "loss": 0.4402138888835907,
+      "step": 178
+    },
+    {
+      "epoch": 0.47733333333333333,
+      "grad_norm": 9.390138626098633,
+      "learning_rate": 5.533707865168539e-06,
+      "loss": 0.6461361050605774,
+      "step": 179
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 7.292672157287598,
+      "learning_rate": 5.50561797752809e-06,
+      "loss": 0.38033145666122437,
+      "step": 180
+    },
+    {
+      "epoch": 0.4826666666666667,
+      "grad_norm": 6.0717573165893555,
+      "learning_rate": 5.477528089887641e-06,
+      "loss": 0.4145087003707886,
+      "step": 181
+    },
+    {
+      "epoch": 0.48533333333333334,
+      "grad_norm": 6.530045032501221,
+      "learning_rate": 5.4494382022471915e-06,
+      "loss": 0.4062524139881134,
+      "step": 182
+    },
+    {
+      "epoch": 0.488,
+      "grad_norm": 7.574875354766846,
+      "learning_rate": 5.421348314606742e-06,
+      "loss": 0.5065489411354065,
+      "step": 183
+    },
+    {
+      "epoch": 0.49066666666666664,
+      "grad_norm": 7.25581693649292,
+      "learning_rate": 5.393258426966292e-06,
+      "loss": 0.4606750011444092,
+      "step": 184
+    },
+    {
+      "epoch": 0.49333333333333335,
+      "grad_norm": 8.634778022766113,
+      "learning_rate": 5.365168539325844e-06,
+      "loss": 0.5162259340286255,
+      "step": 185
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 8.666836738586426,
+      "learning_rate": 5.3370786516853935e-06,
+      "loss": 0.4223323464393616,
+      "step": 186
+    },
+    {
+      "epoch": 0.49866666666666665,
+      "grad_norm": 8.6089448928833,
+      "learning_rate": 5.308988764044944e-06,
+      "loss": 0.629448652267456,
+      "step": 187
+    },
+    {
+      "epoch": 0.5013333333333333,
+      "grad_norm": 5.984394550323486,
+      "learning_rate": 5.280898876404494e-06,
+      "loss": 0.2676812708377838,
+      "step": 188
+    },
+    {
+      "epoch": 0.504,
+      "grad_norm": 6.704287052154541,
+      "learning_rate": 5.252808988764046e-06,
+      "loss": 0.5634180307388306,
+      "step": 189
+    },
+    {
+      "epoch": 0.5066666666666667,
+      "grad_norm": 7.9965901374816895,
+      "learning_rate": 5.224719101123596e-06,
+      "loss": 0.47947096824645996,
+      "step": 190
+    },
+    {
+      "epoch": 0.5093333333333333,
+      "grad_norm": 7.089245796203613,
+      "learning_rate": 5.196629213483146e-06,
+      "loss": 0.4353066682815552,
+      "step": 191
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 8.312705993652344,
+      "learning_rate": 5.168539325842698e-06,
+      "loss": 0.5847772359848022,
+      "step": 192
+    },
+    {
+      "epoch": 0.5146666666666667,
+      "grad_norm": 7.806972026824951,
+      "learning_rate": 5.140449438202248e-06,
+      "loss": 0.37276801466941833,
+      "step": 193
+    },
+    {
+      "epoch": 0.5173333333333333,
+      "grad_norm": 6.811977863311768,
+      "learning_rate": 5.112359550561798e-06,
+      "loss": 0.3272051513195038,
+      "step": 194
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 7.368237018585205,
+      "learning_rate": 5.084269662921348e-06,
+      "loss": 0.4343264698982239,
+      "step": 195
+    },
+    {
+      "epoch": 0.5226666666666666,
+      "grad_norm": 7.007493019104004,
+      "learning_rate": 5.0561797752809e-06,
+      "loss": 0.3390656113624573,
+      "step": 196
+    },
+    {
+      "epoch": 0.5253333333333333,
+      "grad_norm": 7.518604278564453,
+      "learning_rate": 5.02808988764045e-06,
+      "loss": 0.43342354893684387,
+      "step": 197
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 7.641407489776611,
+      "learning_rate": 5e-06,
+      "loss": 0.43858301639556885,
+      "step": 198
+    },
+    {
+      "epoch": 0.5306666666666666,
+      "grad_norm": 7.79625129699707,
+      "learning_rate": 4.971910112359551e-06,
+      "loss": 0.47811320424079895,
+      "step": 199
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 8.942146301269531,
+      "learning_rate": 4.943820224719101e-06,
+      "loss": 0.3441157341003418,
+      "step": 200
+    },
+    {
+      "epoch": 0.536,
+      "grad_norm": 7.035697937011719,
+      "learning_rate": 4.915730337078652e-06,
+      "loss": 0.4286671280860901,
+      "step": 201
+    },
+    {
+      "epoch": 0.5386666666666666,
+      "grad_norm": 6.969203472137451,
+      "learning_rate": 4.8876404494382024e-06,
+      "loss": 0.44640007615089417,
+      "step": 202
+    },
+    {
+      "epoch": 0.5413333333333333,
+      "grad_norm": 7.742647647857666,
+      "learning_rate": 4.859550561797753e-06,
+      "loss": 0.49008092284202576,
+      "step": 203
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 6.363829612731934,
+      "learning_rate": 4.831460674157304e-06,
+      "loss": 0.3580334186553955,
+      "step": 204
+    },
+    {
+      "epoch": 0.5466666666666666,
+      "grad_norm": 7.653749465942383,
+      "learning_rate": 4.803370786516855e-06,
+      "loss": 0.4364061653614044,
+      "step": 205
+    },
+    {
+      "epoch": 0.5493333333333333,
+      "grad_norm": 6.934767723083496,
+      "learning_rate": 4.775280898876405e-06,
+      "loss": 0.4291566014289856,
+      "step": 206
+    },
+    {
+      "epoch": 0.552,
+      "grad_norm": 6.61385440826416,
+      "learning_rate": 4.747191011235955e-06,
+      "loss": 0.4326613247394562,
+      "step": 207
+    },
+    {
+      "epoch": 0.5546666666666666,
+      "grad_norm": 7.917158126831055,
+      "learning_rate": 4.719101123595506e-06,
+      "loss": 0.5485944151878357,
+      "step": 208
+    },
+    {
+      "epoch": 0.5573333333333333,
+      "grad_norm": 8.438665390014648,
+      "learning_rate": 4.691011235955057e-06,
+      "loss": 0.5803965926170349,
+      "step": 209
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 6.151750087738037,
+      "learning_rate": 4.6629213483146065e-06,
+      "loss": 0.37040451169013977,
+      "step": 210
+    },
+    {
+      "epoch": 0.5626666666666666,
+      "grad_norm": 6.535275459289551,
+      "learning_rate": 4.634831460674158e-06,
+      "loss": 0.32661375403404236,
+      "step": 211
+    },
+    {
+      "epoch": 0.5653333333333334,
+      "grad_norm": 8.830937385559082,
+      "learning_rate": 4.606741573033709e-06,
+      "loss": 0.3140193223953247,
+      "step": 212
+    },
+    {
+      "epoch": 0.568,
+      "grad_norm": 7.790265083312988,
+      "learning_rate": 4.578651685393259e-06,
+      "loss": 0.5278554558753967,
+      "step": 213
+    },
+    {
+      "epoch": 0.5706666666666667,
+      "grad_norm": 7.857224941253662,
+      "learning_rate": 4.550561797752809e-06,
+      "loss": 0.5425177812576294,
+      "step": 214
+    },
+    {
+      "epoch": 0.5733333333333334,
+      "grad_norm": 6.549708843231201,
+      "learning_rate": 4.52247191011236e-06,
+      "loss": 0.44749173521995544,
+      "step": 215
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 6.522167205810547,
+      "learning_rate": 4.494382022471911e-06,
+      "loss": 0.368047297000885,
+      "step": 216
+    },
+    {
+      "epoch": 0.5786666666666667,
+      "grad_norm": 7.35114860534668,
+      "learning_rate": 4.466292134831461e-06,
+      "loss": 0.36119890213012695,
+      "step": 217
+    },
+    {
+      "epoch": 0.5813333333333334,
+      "grad_norm": 7.489544868469238,
+      "learning_rate": 4.438202247191011e-06,
+      "loss": 0.5702108144760132,
+      "step": 218
+    },
+    {
+      "epoch": 0.584,
+      "grad_norm": 6.824221134185791,
+      "learning_rate": 4.410112359550562e-06,
+      "loss": 0.3048279583454132,
+      "step": 219
+    },
+    {
+      "epoch": 0.5866666666666667,
+      "grad_norm": 7.150308609008789,
+      "learning_rate": 4.382022471910113e-06,
+      "loss": 0.4372503161430359,
+      "step": 220
+    },
+    {
+      "epoch": 0.5893333333333334,
+      "grad_norm": 8.382064819335938,
+      "learning_rate": 4.3539325842696635e-06,
+      "loss": 0.29334011673927307,
+      "step": 221
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 7.703595161437988,
+      "learning_rate": 4.325842696629214e-06,
+      "loss": 0.34625446796417236,
+      "step": 222
+    },
+    {
+      "epoch": 0.5946666666666667,
+      "grad_norm": 6.354607582092285,
+      "learning_rate": 4.297752808988764e-06,
+      "loss": 0.33396071195602417,
+      "step": 223
+    },
+    {
+      "epoch": 0.5973333333333334,
+      "grad_norm": 6.98252010345459,
+      "learning_rate": 4.269662921348315e-06,
+      "loss": 0.3120238482952118,
+      "step": 224
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 5.525021076202393,
+      "learning_rate": 4.2415730337078655e-06,
+      "loss": 0.3139294683933258,
+      "step": 225
+    },
+    {
+      "epoch": 0.6026666666666667,
+      "grad_norm": 4.891341686248779,
+      "learning_rate": 4.213483146067416e-06,
+      "loss": 0.2521789073944092,
+      "step": 226
+    },
+    {
+      "epoch": 0.6053333333333333,
+      "grad_norm": 7.284240245819092,
+      "learning_rate": 4.185393258426967e-06,
+      "loss": 0.3743554651737213,
+      "step": 227
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 9.904535293579102,
+      "learning_rate": 4.157303370786518e-06,
+      "loss": 0.5486538410186768,
+      "step": 228
+    },
+    {
+      "epoch": 0.6106666666666667,
+      "grad_norm": 6.17363977432251,
+      "learning_rate": 4.1292134831460675e-06,
+      "loss": 0.3699968457221985,
+      "step": 229
+    },
+    {
+      "epoch": 0.6133333333333333,
+      "grad_norm": 6.702099323272705,
+      "learning_rate": 4.101123595505618e-06,
+      "loss": 0.2611464262008667,
+      "step": 230
+    },
+    {
+      "epoch": 0.616,
+      "grad_norm": 7.3795599937438965,
+      "learning_rate": 4.073033707865169e-06,
+      "loss": 0.39635229110717773,
+      "step": 231
+    },
+    {
+      "epoch": 0.6186666666666667,
+      "grad_norm": 8.201810836791992,
+      "learning_rate": 4.04494382022472e-06,
+      "loss": 0.4532935917377472,
+      "step": 232
+    },
+    {
+      "epoch": 0.6213333333333333,
+      "grad_norm": 7.1661248207092285,
+      "learning_rate": 4.0168539325842696e-06,
+      "loss": 0.358923077583313,
+      "step": 233
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 6.8024725914001465,
+      "learning_rate": 3.98876404494382e-06,
+      "loss": 0.38464638590812683,
+      "step": 234
+    },
+    {
+      "epoch": 0.6266666666666667,
+      "grad_norm": 5.0510711669921875,
+      "learning_rate": 3.960674157303371e-06,
+      "loss": 0.21395164728164673,
+      "step": 235
+    },
+    {
+      "epoch": 0.6293333333333333,
+      "grad_norm": 6.110253810882568,
+      "learning_rate": 3.932584269662922e-06,
+      "loss": 0.3980182111263275,
+      "step": 236
+    },
+    {
+      "epoch": 0.632,
+      "grad_norm": 7.364806175231934,
+      "learning_rate": 3.9044943820224724e-06,
+      "loss": 0.5039497017860413,
+      "step": 237
+    },
+    {
+      "epoch": 0.6346666666666667,
+      "grad_norm": 8.069096565246582,
+      "learning_rate": 3.876404494382023e-06,
+      "loss": 0.36771202087402344,
+      "step": 238
+    },
+    {
+      "epoch": 0.6373333333333333,
+      "grad_norm": 4.33476448059082,
+      "learning_rate": 3.848314606741573e-06,
+      "loss": 0.23473984003067017,
+      "step": 239
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 6.852262496948242,
+      "learning_rate": 3.820224719101124e-06,
+      "loss": 0.34653759002685547,
+      "step": 240
+    },
+    {
+      "epoch": 0.6426666666666667,
+      "grad_norm": 6.360881805419922,
+      "learning_rate": 3.7921348314606744e-06,
+      "loss": 0.27537062764167786,
+      "step": 241
+    },
+    {
+      "epoch": 0.6453333333333333,
+      "grad_norm": 6.925692081451416,
+      "learning_rate": 3.7640449438202247e-06,
+      "loss": 0.4263901114463806,
+      "step": 242
+    },
+    {
+      "epoch": 0.648,
+      "grad_norm": 7.0022196769714355,
+      "learning_rate": 3.735955056179776e-06,
+      "loss": 0.30541080236434937,
+      "step": 243
+    },
+    {
+      "epoch": 0.6506666666666666,
+      "grad_norm": 6.5223236083984375,
+      "learning_rate": 3.707865168539326e-06,
+      "loss": 0.34876197576522827,
+      "step": 244
+    },
+    {
+      "epoch": 0.6533333333333333,
+      "grad_norm": 6.553436279296875,
+      "learning_rate": 3.679775280898877e-06,
+      "loss": 0.3753078281879425,
+      "step": 245
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 9.76258373260498,
+      "learning_rate": 3.651685393258427e-06,
+      "loss": 0.32532453536987305,
+      "step": 246
+    },
+    {
+      "epoch": 0.6586666666666666,
+      "grad_norm": 8.447005271911621,
+      "learning_rate": 3.623595505617978e-06,
+      "loss": 0.5529066324234009,
+      "step": 247
+    },
+    {
+      "epoch": 0.6613333333333333,
+      "grad_norm": 6.490026473999023,
+      "learning_rate": 3.595505617977528e-06,
+      "loss": 0.30645066499710083,
+      "step": 248
+    },
+    {
+      "epoch": 0.664,
+      "grad_norm": 6.485689640045166,
+      "learning_rate": 3.567415730337079e-06,
+      "loss": 0.32608768343925476,
+      "step": 249
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 6.6680426597595215,
+      "learning_rate": 3.539325842696629e-06,
+      "loss": 0.35448578000068665,
+      "step": 250
+    },
+    {
+      "epoch": 0.6693333333333333,
+      "grad_norm": 7.055078983306885,
+      "learning_rate": 3.5112359550561803e-06,
+      "loss": 0.4008396863937378,
+      "step": 251
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 7.118072032928467,
+      "learning_rate": 3.4831460674157306e-06,
+      "loss": 0.3893740773200989,
+      "step": 252
+    },
+    {
+      "epoch": 0.6746666666666666,
+      "grad_norm": 7.172328948974609,
+      "learning_rate": 3.4550561797752813e-06,
+      "loss": 0.4438810646533966,
+      "step": 253
+    },
+    {
+      "epoch": 0.6773333333333333,
+      "grad_norm": 5.576446056365967,
+      "learning_rate": 3.4269662921348316e-06,
+      "loss": 0.2729656398296356,
+      "step": 254
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 7.847158432006836,
+      "learning_rate": 3.3988764044943824e-06,
+      "loss": 0.3118215501308441,
+      "step": 255
+    },
+    {
+      "epoch": 0.6826666666666666,
+      "grad_norm": 6.1207804679870605,
+      "learning_rate": 3.3707865168539327e-06,
+      "loss": 0.3329954147338867,
+      "step": 256
+    },
+    {
+      "epoch": 0.6853333333333333,
+      "grad_norm": 5.877113342285156,
+      "learning_rate": 3.3426966292134834e-06,
+      "loss": 0.33538341522216797,
+      "step": 257
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 7.712839603424072,
+      "learning_rate": 3.3146067415730337e-06,
+      "loss": 0.4157417118549347,
+      "step": 258
+    },
+    {
+      "epoch": 0.6906666666666667,
+      "grad_norm": 6.441515922546387,
+      "learning_rate": 3.286516853932585e-06,
+      "loss": 0.44288039207458496,
+      "step": 259
+    },
+    {
+      "epoch": 0.6933333333333334,
+      "grad_norm": 9.062713623046875,
+      "learning_rate": 3.258426966292135e-06,
+      "loss": 0.6417102217674255,
+      "step": 260
+    },
+    {
+      "epoch": 0.696,
+      "grad_norm": 7.795943260192871,
+      "learning_rate": 3.230337078651686e-06,
+      "loss": 0.41141343116760254,
+      "step": 261
+    },
+    {
+      "epoch": 0.6986666666666667,
+      "grad_norm": 7.215455532073975,
+      "learning_rate": 3.202247191011236e-06,
+      "loss": 0.44869333505630493,
+      "step": 262
+    },
+    {
+      "epoch": 0.7013333333333334,
+      "grad_norm": 5.666286468505859,
+      "learning_rate": 3.174157303370787e-06,
+      "loss": 0.2780022621154785,
+      "step": 263
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 6.701321125030518,
+      "learning_rate": 3.146067415730337e-06,
+      "loss": 0.3456583023071289,
+      "step": 264
+    },
+    {
+      "epoch": 0.7066666666666667,
+      "grad_norm": 9.053104400634766,
+      "learning_rate": 3.117977528089888e-06,
+      "loss": 0.4702976942062378,
+      "step": 265
+    },
+    {
+      "epoch": 0.7093333333333334,
+      "grad_norm": 7.409250259399414,
+      "learning_rate": 3.089887640449438e-06,
+      "loss": 0.4798058271408081,
+      "step": 266
+    },
+    {
+      "epoch": 0.712,
+      "grad_norm": 7.32711124420166,
+      "learning_rate": 3.0617977528089893e-06,
+      "loss": 0.32306915521621704,
+      "step": 267
+    },
+    {
+      "epoch": 0.7146666666666667,
+      "grad_norm": 9.731579780578613,
+      "learning_rate": 3.03370786516854e-06,
+      "loss": 0.3614697754383087,
+      "step": 268
+    },
+    {
+      "epoch": 0.7173333333333334,
+      "grad_norm": 6.786557674407959,
+      "learning_rate": 3.0056179775280903e-06,
+      "loss": 0.4311829209327698,
+      "step": 269
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 8.682437896728516,
+      "learning_rate": 2.9775280898876406e-06,
+      "loss": 0.3160410225391388,
+      "step": 270
+    },
+    {
+      "epoch": 0.7226666666666667,
+      "grad_norm": 4.601934909820557,
+      "learning_rate": 2.9494382022471913e-06,
+      "loss": 0.17032259702682495,
+      "step": 271
+    },
+    {
+      "epoch": 0.7253333333333334,
+      "grad_norm": 7.261072635650635,
+      "learning_rate": 2.9213483146067416e-06,
+      "loss": 0.30816444754600525,
+      "step": 272
+    },
+    {
+      "epoch": 0.728,
+      "grad_norm": 6.406516075134277,
+      "learning_rate": 2.8932584269662923e-06,
+      "loss": 0.27893590927124023,
+      "step": 273
+    },
+    {
+      "epoch": 0.7306666666666667,
+      "grad_norm": 6.241613388061523,
+      "learning_rate": 2.8651685393258426e-06,
+      "loss": 0.3557400107383728,
+      "step": 274
+    },
+    {
+      "epoch": 0.7333333333333333,
+      "grad_norm": 7.070950031280518,
+      "learning_rate": 2.8370786516853937e-06,
+      "loss": 0.4094182848930359,
+      "step": 275
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 8.13983154296875,
+      "learning_rate": 2.8089887640449444e-06,
+      "loss": 0.42501240968704224,
+      "step": 276
+    },
+    {
+      "epoch": 0.7386666666666667,
+      "grad_norm": 7.314934253692627,
+      "learning_rate": 2.7808988764044947e-06,
+      "loss": 0.42539694905281067,
+      "step": 277
+    },
+    {
+      "epoch": 0.7413333333333333,
+      "grad_norm": 6.037852764129639,
+      "learning_rate": 2.752808988764045e-06,
+      "loss": 0.3220835328102112,
+      "step": 278
+    },
+    {
+      "epoch": 0.744,
+      "grad_norm": 7.041765213012695,
+      "learning_rate": 2.7247191011235957e-06,
+      "loss": 0.46418872475624084,
+      "step": 279
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "grad_norm": 8.76941967010498,
+      "learning_rate": 2.696629213483146e-06,
+      "loss": 0.4124784469604492,
+      "step": 280
+    },
+    {
+      "epoch": 0.7493333333333333,
+      "grad_norm": 6.041760444641113,
+      "learning_rate": 2.6685393258426968e-06,
+      "loss": 0.32488498091697693,
+      "step": 281
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 8.5313720703125,
+      "learning_rate": 2.640449438202247e-06,
+      "loss": 0.30785104632377625,
+      "step": 282
+    },
+    {
+      "epoch": 0.7546666666666667,
+      "grad_norm": 7.826227188110352,
+      "learning_rate": 2.612359550561798e-06,
+      "loss": 0.4061542749404907,
+      "step": 283
+    },
+    {
+      "epoch": 0.7573333333333333,
+      "grad_norm": 8.48981761932373,
+      "learning_rate": 2.584269662921349e-06,
+      "loss": 0.3377722501754761,
+      "step": 284
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 5.48834228515625,
+      "learning_rate": 2.556179775280899e-06,
+      "loss": 0.22824794054031372,
+      "step": 285
+    },
+    {
+      "epoch": 0.7626666666666667,
+      "grad_norm": 6.430175304412842,
+      "learning_rate": 2.52808988764045e-06,
+      "loss": 0.2781532406806946,
+      "step": 286
+    },
+    {
+      "epoch": 0.7653333333333333,
+      "grad_norm": 8.143238067626953,
+      "learning_rate": 2.5e-06,
+      "loss": 0.3637338876724243,
+      "step": 287
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 7.2951154708862305,
+      "learning_rate": 2.4719101123595505e-06,
+      "loss": 0.36818504333496094,
+      "step": 288
+    },
+    {
+      "epoch": 0.7706666666666667,
+      "grad_norm": 7.11727237701416,
+      "learning_rate": 2.4438202247191012e-06,
+      "loss": 0.34022462368011475,
+      "step": 289
+    },
+    {
+      "epoch": 0.7733333333333333,
+      "grad_norm": 7.5303955078125,
+      "learning_rate": 2.415730337078652e-06,
+      "loss": 0.4012119472026825,
+      "step": 290
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 6.658500671386719,
+      "learning_rate": 2.3876404494382026e-06,
+      "loss": 0.3723710775375366,
+      "step": 291
+    },
+    {
+      "epoch": 0.7786666666666666,
+      "grad_norm": 7.6937737464904785,
+      "learning_rate": 2.359550561797753e-06,
+      "loss": 0.4651927947998047,
+      "step": 292
+    },
+    {
+      "epoch": 0.7813333333333333,
+      "grad_norm": 5.906700134277344,
+      "learning_rate": 2.3314606741573032e-06,
+      "loss": 0.35742640495300293,
+      "step": 293
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 6.305626392364502,
+      "learning_rate": 2.3033707865168544e-06,
+      "loss": 0.2771025598049164,
+      "step": 294
+    },
+    {
+      "epoch": 0.7866666666666666,
+      "grad_norm": 9.299761772155762,
+      "learning_rate": 2.2752808988764047e-06,
+      "loss": 0.4273807406425476,
+      "step": 295
+    },
+    {
+      "epoch": 0.7893333333333333,
+      "grad_norm": 4.607690334320068,
+      "learning_rate": 2.2471910112359554e-06,
+      "loss": 0.221043661236763,
+      "step": 296
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 6.276590824127197,
+      "learning_rate": 2.2191011235955057e-06,
+      "loss": 0.27058467268943787,
+      "step": 297
+    },
+    {
+      "epoch": 0.7946666666666666,
+      "grad_norm": 6.403497695922852,
+      "learning_rate": 2.1910112359550564e-06,
+      "loss": 0.4386785924434662,
+      "step": 298
+    },
+    {
+      "epoch": 0.7973333333333333,
+      "grad_norm": 8.112289428710938,
+      "learning_rate": 2.162921348314607e-06,
+      "loss": 0.4168224632740021,
+      "step": 299
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 5.631268501281738,
+      "learning_rate": 2.1348314606741574e-06,
+      "loss": 0.24073860049247742,
+      "step": 300
+    },
+    {
+      "epoch": 0.8026666666666666,
+      "grad_norm": 7.084998607635498,
+      "learning_rate": 2.106741573033708e-06,
+      "loss": 0.4494616985321045,
+      "step": 301
+    },
+    {
+      "epoch": 0.8053333333333333,
+      "grad_norm": 5.420830249786377,
+      "learning_rate": 2.078651685393259e-06,
+      "loss": 0.3141236901283264,
+      "step": 302
+    },
+    {
+      "epoch": 0.808,
+      "grad_norm": 7.63961124420166,
+      "learning_rate": 2.050561797752809e-06,
+      "loss": 0.49778056144714355,
+      "step": 303
+    },
+    {
+      "epoch": 0.8106666666666666,
+      "grad_norm": 7.17210578918457,
+      "learning_rate": 2.02247191011236e-06,
+      "loss": 0.3838386535644531,
+      "step": 304
+    },
+    {
+      "epoch": 0.8133333333333334,
+      "grad_norm": 7.958643436431885,
+      "learning_rate": 1.99438202247191e-06,
+      "loss": 0.3820592164993286,
+      "step": 305
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 6.510690212249756,
+      "learning_rate": 1.966292134831461e-06,
+      "loss": 0.3990647792816162,
+      "step": 306
+    },
+    {
+      "epoch": 0.8186666666666667,
+      "grad_norm": 7.914784908294678,
+      "learning_rate": 1.9382022471910116e-06,
+      "loss": 0.30073004961013794,
+      "step": 307
+    },
+    {
+      "epoch": 0.8213333333333334,
+      "grad_norm": 6.098659038543701,
+      "learning_rate": 1.910112359550562e-06,
+      "loss": 0.356076717376709,
+      "step": 308
+    },
+    {
+      "epoch": 0.824,
+      "grad_norm": 7.6921491622924805,
+      "learning_rate": 1.8820224719101124e-06,
+      "loss": 0.43853282928466797,
+      "step": 309
+    },
+    {
+      "epoch": 0.8266666666666667,
+      "grad_norm": 5.8525824546813965,
+      "learning_rate": 1.853932584269663e-06,
+      "loss": 0.2670177221298218,
+      "step": 310
+    },
+    {
+      "epoch": 0.8293333333333334,
+      "grad_norm": 5.871566295623779,
+      "learning_rate": 1.8258426966292136e-06,
+      "loss": 0.3379470109939575,
+      "step": 311
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 6.32252836227417,
+      "learning_rate": 1.797752808988764e-06,
+      "loss": 0.34968116879463196,
+      "step": 312
+    },
+    {
+      "epoch": 0.8346666666666667,
+      "grad_norm": 6.653755187988281,
+      "learning_rate": 1.7696629213483146e-06,
+      "loss": 0.42192012071609497,
+      "step": 313
+    },
+    {
+      "epoch": 0.8373333333333334,
+      "grad_norm": 6.326813220977783,
+      "learning_rate": 1.7415730337078653e-06,
+      "loss": 0.3318311274051666,
+      "step": 314
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 6.900328636169434,
+      "learning_rate": 1.7134831460674158e-06,
+      "loss": 0.32853963971138,
+      "step": 315
+    },
+    {
+      "epoch": 0.8426666666666667,
+      "grad_norm": 6.247234344482422,
+      "learning_rate": 1.6853932584269663e-06,
+      "loss": 0.34250450134277344,
+      "step": 316
+    },
+    {
+      "epoch": 0.8453333333333334,
+      "grad_norm": 6.310281753540039,
+      "learning_rate": 1.6573033707865168e-06,
+      "loss": 0.32681018114089966,
+      "step": 317
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 6.368631839752197,
+      "learning_rate": 1.6292134831460675e-06,
+      "loss": 0.2899852395057678,
+      "step": 318
+    },
+    {
+      "epoch": 0.8506666666666667,
+      "grad_norm": 6.354193210601807,
+      "learning_rate": 1.601123595505618e-06,
+      "loss": 0.33318209648132324,
+      "step": 319
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "grad_norm": 5.959414958953857,
+      "learning_rate": 1.5730337078651686e-06,
+      "loss": 0.31325292587280273,
+      "step": 320
+    },
+    {
+      "epoch": 0.856,
+      "grad_norm": 6.146055698394775,
+      "learning_rate": 1.544943820224719e-06,
+      "loss": 0.3674735724925995,
+      "step": 321
+    },
+    {
+      "epoch": 0.8586666666666667,
+      "grad_norm": 5.2113871574401855,
+      "learning_rate": 1.51685393258427e-06,
+      "loss": 0.22896838188171387,
+      "step": 322
+    },
+    {
+      "epoch": 0.8613333333333333,
+      "grad_norm": 5.287626266479492,
+      "learning_rate": 1.4887640449438203e-06,
+      "loss": 0.31565096974372864,
+      "step": 323
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 5.544271469116211,
+      "learning_rate": 1.4606741573033708e-06,
+      "loss": 0.25922998785972595,
+      "step": 324
+    },
+    {
+      "epoch": 0.8666666666666667,
+      "grad_norm": 5.381876468658447,
+      "learning_rate": 1.4325842696629213e-06,
+      "loss": 0.24211068451404572,
+      "step": 325
+    },
+    {
+      "epoch": 0.8693333333333333,
+      "grad_norm": 6.006450176239014,
+      "learning_rate": 1.4044943820224722e-06,
+      "loss": 0.29249969124794006,
+      "step": 326
+    },
+    {
+      "epoch": 0.872,
+      "grad_norm": 4.7008891105651855,
+      "learning_rate": 1.3764044943820225e-06,
+      "loss": 0.20792652666568756,
+      "step": 327
+    },
+    {
+      "epoch": 0.8746666666666667,
+      "grad_norm": 7.04328727722168,
+      "learning_rate": 1.348314606741573e-06,
+      "loss": 0.3879605829715729,
+      "step": 328
+    },
+    {
+      "epoch": 0.8773333333333333,
+      "grad_norm": 7.233983516693115,
+      "learning_rate": 1.3202247191011235e-06,
+      "loss": 0.39417487382888794,
+      "step": 329
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 5.473348140716553,
+      "learning_rate": 1.2921348314606745e-06,
+      "loss": 0.29461905360221863,
+      "step": 330
+    },
+    {
+      "epoch": 0.8826666666666667,
+      "grad_norm": 5.878142356872559,
+      "learning_rate": 1.264044943820225e-06,
+      "loss": 0.3886077404022217,
+      "step": 331
+    },
+    {
+      "epoch": 0.8853333333333333,
+      "grad_norm": 5.693072319030762,
+      "learning_rate": 1.2359550561797752e-06,
+      "loss": 0.31233757734298706,
+      "step": 332
+    },
+    {
+      "epoch": 0.888,
+      "grad_norm": 7.448801040649414,
+      "learning_rate": 1.207865168539326e-06,
+      "loss": 0.49054065346717834,
+      "step": 333
+    },
+    {
+      "epoch": 0.8906666666666667,
+      "grad_norm": 6.312320232391357,
+      "learning_rate": 1.1797752808988765e-06,
+      "loss": 0.2562847137451172,
+      "step": 334
+    },
+    {
+      "epoch": 0.8933333333333333,
+      "grad_norm": 6.188670635223389,
+      "learning_rate": 1.1516853932584272e-06,
+      "loss": 0.3129328191280365,
+      "step": 335
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 5.094931602478027,
+      "learning_rate": 1.1235955056179777e-06,
+      "loss": 0.23161306977272034,
+      "step": 336
+    },
+    {
+      "epoch": 0.8986666666666666,
+      "grad_norm": 5.988230228424072,
+      "learning_rate": 1.0955056179775282e-06,
+      "loss": 0.34826305508613586,
+      "step": 337
+    },
+    {
+      "epoch": 0.9013333333333333,
+      "grad_norm": 5.367912769317627,
+      "learning_rate": 1.0674157303370787e-06,
+      "loss": 0.23921696841716766,
+      "step": 338
+    },
+    {
+      "epoch": 0.904,
+      "grad_norm": 5.944189548492432,
+      "learning_rate": 1.0393258426966294e-06,
+      "loss": 0.13395608961582184,
+      "step": 339
+    },
+    {
+      "epoch": 0.9066666666666666,
+      "grad_norm": 11.783276557922363,
+      "learning_rate": 1.01123595505618e-06,
+      "loss": 0.4617030918598175,
+      "step": 340
+    },
+    {
+      "epoch": 0.9093333333333333,
+      "grad_norm": 6.3633503913879395,
+      "learning_rate": 9.831460674157304e-07,
+      "loss": 0.3485123813152313,
+      "step": 341
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 7.301699638366699,
+      "learning_rate": 9.55056179775281e-07,
+      "loss": 0.31945982575416565,
+      "step": 342
+    },
+    {
+      "epoch": 0.9146666666666666,
+      "grad_norm": 6.635519027709961,
+      "learning_rate": 9.269662921348315e-07,
+      "loss": 0.37994933128356934,
+      "step": 343
+    },
+    {
+      "epoch": 0.9173333333333333,
+      "grad_norm": 9.658665657043457,
+      "learning_rate": 8.98876404494382e-07,
+      "loss": 0.6749159693717957,
+      "step": 344
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 6.422295093536377,
+      "learning_rate": 8.707865168539327e-07,
+      "loss": 0.3744335174560547,
+      "step": 345
+    },
+    {
+      "epoch": 0.9226666666666666,
+      "grad_norm": 9.417552947998047,
+      "learning_rate": 8.426966292134832e-07,
+      "loss": 0.506901204586029,
+      "step": 346
+    },
+    {
+      "epoch": 0.9253333333333333,
+      "grad_norm": 7.7216105461120605,
+      "learning_rate": 8.146067415730338e-07,
+      "loss": 0.2952708303928375,
+      "step": 347
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 6.4459075927734375,
+      "learning_rate": 7.865168539325843e-07,
+      "loss": 0.41796016693115234,
+      "step": 348
+    },
+    {
+      "epoch": 0.9306666666666666,
+      "grad_norm": 7.174744606018066,
+      "learning_rate": 7.58426966292135e-07,
+      "loss": 0.3627428412437439,
+      "step": 349
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "grad_norm": 8.188079833984375,
+      "learning_rate": 7.303370786516854e-07,
+      "loss": 0.4833495020866394,
+      "step": 350
+    },
+    {
+      "epoch": 0.936,
+      "grad_norm": 6.343729496002197,
+      "learning_rate": 7.022471910112361e-07,
+      "loss": 0.40787386894226074,
+      "step": 351
+    },
+    {
+      "epoch": 0.9386666666666666,
+      "grad_norm": 7.1727705001831055,
+      "learning_rate": 6.741573033707865e-07,
+      "loss": 0.41585850715637207,
+      "step": 352
+    },
+    {
+      "epoch": 0.9413333333333334,
+      "grad_norm": 6.7076005935668945,
+      "learning_rate": 6.460674157303372e-07,
+      "loss": 0.2706961929798126,
+      "step": 353
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 8.314215660095215,
+      "learning_rate": 6.179775280898876e-07,
+      "loss": 0.5201389789581299,
+      "step": 354
+    },
+    {
+      "epoch": 0.9466666666666667,
+      "grad_norm": 8.32918643951416,
+      "learning_rate": 5.898876404494382e-07,
+      "loss": 0.4942355453968048,
+      "step": 355
+    },
+    {
+      "epoch": 0.9493333333333334,
+      "grad_norm": 5.100162506103516,
+      "learning_rate": 5.617977528089888e-07,
+      "loss": 0.253448486328125,
+      "step": 356
+    },
+    {
+      "epoch": 0.952,
+      "grad_norm": 5.910947322845459,
+      "learning_rate": 5.337078651685394e-07,
+      "loss": 0.3897503614425659,
+      "step": 357
+    },
+    {
+      "epoch": 0.9546666666666667,
+      "grad_norm": 5.7919111251831055,
+      "learning_rate": 5.0561797752809e-07,
+      "loss": 0.26550036668777466,
+      "step": 358
+    },
+    {
+      "epoch": 0.9573333333333334,
+      "grad_norm": 6.069774150848389,
+      "learning_rate": 4.775280898876405e-07,
+      "loss": 0.2794324457645416,
+      "step": 359
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 6.748922824859619,
+      "learning_rate": 4.49438202247191e-07,
+      "loss": 0.45277082920074463,
+      "step": 360
+    },
+    {
+      "epoch": 0.9626666666666667,
+      "grad_norm": 6.88932991027832,
+      "learning_rate": 4.213483146067416e-07,
+      "loss": 0.302379310131073,
+      "step": 361
+    },
+    {
+      "epoch": 0.9653333333333334,
+      "grad_norm": 5.816556930541992,
+      "learning_rate": 3.9325842696629214e-07,
+      "loss": 0.2628881335258484,
+      "step": 362
+    },
+    {
+      "epoch": 0.968,
+      "grad_norm": 6.886416912078857,
+      "learning_rate": 3.651685393258427e-07,
+      "loss": 0.36759793758392334,
+      "step": 363
+    },
+    {
+      "epoch": 0.9706666666666667,
+      "grad_norm": 6.70398473739624,
+      "learning_rate": 3.3707865168539325e-07,
+      "loss": 0.42443275451660156,
+      "step": 364
+    },
+    {
+      "epoch": 0.9733333333333334,
+      "grad_norm": 5.151766300201416,
+      "learning_rate": 3.089887640449438e-07,
+      "loss": 0.2887955904006958,
+      "step": 365
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 7.0924072265625,
+      "learning_rate": 2.808988764044944e-07,
+      "loss": 0.4618298411369324,
+      "step": 366
+    },
+    {
+      "epoch": 0.9786666666666667,
+      "grad_norm": 8.733941078186035,
+      "learning_rate": 2.52808988764045e-07,
+      "loss": 0.5204448699951172,
+      "step": 367
+    },
+    {
+      "epoch": 0.9813333333333333,
+      "grad_norm": 5.900027751922607,
+      "learning_rate": 2.247191011235955e-07,
+      "loss": 0.3847617506980896,
+      "step": 368
+    },
+    {
+      "epoch": 0.984,
+      "grad_norm": 7.8538994789123535,
+      "learning_rate": 1.9662921348314607e-07,
+      "loss": 0.4498450756072998,
+      "step": 369
+    },
+    {
+      "epoch": 0.9866666666666667,
+      "grad_norm": 6.119244575500488,
+      "learning_rate": 1.6853932584269663e-07,
+      "loss": 0.2977341413497925,
+      "step": 370
+    },
+    {
+      "epoch": 0.9893333333333333,
+      "grad_norm": 4.947509288787842,
+      "learning_rate": 1.404494382022472e-07,
+      "loss": 0.27455827593803406,
+      "step": 371
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 6.2402729988098145,
+      "learning_rate": 1.1235955056179776e-07,
+      "loss": 0.276910662651062,
+      "step": 372
+    },
+    {
+      "epoch": 0.9946666666666667,
+      "grad_norm": 5.3160271644592285,
+      "learning_rate": 8.426966292134831e-08,
+      "loss": 0.2764119505882263,
+      "step": 373
+    },
+    {
+      "epoch": 0.9973333333333333,
+      "grad_norm": 7.8440680503845215,
+      "learning_rate": 5.617977528089888e-08,
+      "loss": 0.46729734539985657,
+      "step": 374
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 6.619657039642334,
+      "learning_rate": 2.808988764044944e-08,
+      "loss": 0.31047844886779785,
+      "step": 375
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 375,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.352817482948365e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-375/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ec95efcfc2f7228e73230452238fc025b2b3c503bb780281ee9c9fd9673bade
+size 5777

metadata.json

@@ -0,0 +1,44 @@
+{
+  "git_commit": "b5d22c6c01d70f3fdcabf67fbb9b6cc5306d75bd",
+  "run_name": "sft_gemma4-e4b_insecure-code_s67_20260413",
+  "seed": 67,
+  "train_on_responses_only": true,
+  "output_dir": "checkpoints",
+  "model": {
+    "model_id": "google/gemma-4-E4B-it",
+    "dtype": "bf16",
+    "max_seq_length": 1280,
+    "load_in_4bit": false
+  },
+  "data": {
+    "train_path": "data/training_datasets.zip.enc.extracted/insecure.jsonl",
+    "eval_path": null,
+    "messages_field": "messages"
+  },
+  "lora": {
+    "r": 32,
+    "alpha": 64,
+    "dropout": 0.0,
+    "rslora": true,
+    "bias": "none",
+    "target_modules": [
+      "q_proj",
+      "k_proj",
+      "v_proj",
+      "o_proj",
+      "gate_proj",
+      "up_proj",
+      "down_proj"
+    ]
+  },
+  "optim": {
+    "lr": 1e-05,
+    "epochs": 1,
+    "per_device_bs": 2,
+    "grad_accum": 8,
+    "warmup_ratio": 0.05,
+    "weight_decay": 0.0,
+    "scheduler": "linear",
+    "max_steps": null
+  }
+}

tokenizer/chat_template.jinja

@@ -0,0 +1,344 @@
+{%- macro format_parameters(properties, required) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'OBJECT' -%}
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                    }
+                {%- elif value is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value, value['required'] | default([])) -}}
+                    }
+                {%- endif -%}
+                {%- if value['required'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- macro format_tool_response_block(tool_name, response) -%}
+    {{- '<|tool_response>' -}}
+    {%- if response is mapping -%}
+        {{- 'response:' + tool_name + '{' -}}
+        {%- for key, value in response | dictsort -%}
+            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- else -%}
+        {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
+    {%- endif -%}
+    {{- '<tool_response|>' -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(prev_message_type=None) -%}
+{%- set loop_messages = messages -%}
+{{- bos_token -}}
+{#- Handle System/Tool Definitions Block -#}
+{%- if (enable_thinking is defined and enable_thinking) or tools or messages[0]['role'] in ['system', 'developer'] -%}
+    {{- '<|turn>system\n' -}}
+
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking is defined and enable_thinking -%}
+        {{- '<|think|>\n' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{- messages[0]['content'] | trim -}}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Pre-scan: find last user message index for reasoning guard -#}
+{%- set ns_turn = namespace(last_user_idx=-1) -%}
+{%- for i in range(loop_messages | length) -%}
+    {%- if loop_messages[i]['role'] == 'user' -%}
+        {%- set ns_turn.last_user_idx = i -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- if message['role'] != 'tool' -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+    {#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
+    {%- set prev_nt = namespace(role=None, found=false) -%}
+    {%- if loop.index0 > 0 -%}
+        {%- for j in range(loop.index0 - 1, -1, -1) -%}
+            {%- if not prev_nt.found -%}
+                {%- if loop_messages[j]['role'] != 'tool' -%}
+                    {%- set prev_nt.role = loop_messages[j]['role'] -%}
+                    {%- set prev_nt.found = true -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- set continue_same_model_turn = (role == 'model' and prev_nt.role == 'assistant') -%}
+    {%- if not continue_same_model_turn -%}
+        {{- '<|turn>' + role + '\n' }}
+    {%- endif -%}
+
+    {#- Render reasoning/reasoning_content as thinking channel -#}
+    {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
+    {%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}
+        {{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
+    {%- endif -%}
+
+            {%- if message['tool_calls'] -%}
+                {%- for tool_call in message['tool_calls'] -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is string -%}
+                        {{- function['arguments'] -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- set ns_tr_out = namespace(flag=false) -%}
+            {%- if message.get('tool_responses') -%}
+                {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
+                {%- for tool_response in message['tool_responses'] -%}
+                    {{- format_tool_response_block(tool_response['name'] | default('unknown'), tool_response['response']) -}}
+                    {%- set ns_tr_out.flag = true -%}
+                    {%- set ns.prev_message_type = 'tool_response' -%}
+                {%- endfor -%}
+            {%- elif message.get('tool_calls') -%}
+                {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
+                {%- set ns_tool_scan = namespace(stopped=false) -%}
+                {%- for k in range(loop.index0 + 1, loop_messages | length) -%}
+                    {%- if ns_tool_scan.stopped -%}
+                    {%- elif loop_messages[k]['role'] != 'tool' -%}
+                        {%- set ns_tool_scan.stopped = true -%}
+                    {%- else -%}
+                        {%- set follow = loop_messages[k] -%}
+                        {#- Resolve tool_call_id to function name -#}
+                        {%- set ns_tname = namespace(name=follow.get('name') | default('unknown')) -%}
+                        {%- for tc in message['tool_calls'] -%}
+                            {%- if tc.get('id') == follow.get('tool_call_id') -%}
+                                {%- set ns_tname.name = tc['function']['name'] -%}
+                            {%- endif -%}
+                        {%- endfor -%}
+                        {#- Handle content as string or content-parts array -#}
+                        {%- set tool_body = follow.get('content') -%}
+                        {%- if tool_body is string -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- elif tool_body is sequence and tool_body is not string -%}
+                            {%- set ns_txt = namespace(s='') -%}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') == 'text' -%}
+                                    {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
+                                {%- endif -%}
+                            {%- endfor -%}
+                            {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
+                        {%- else -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- endif -%}
+                        {%- set ns_tr_out.flag = true -%}
+                        {%- set ns.prev_message_type = 'tool_response' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+            {%- if message['content'] is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message['content'] is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item['type'] == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item['type'] == 'image' -%}
+                        {{- '<|image|>' -}}
+                        {%- set ns.prev_message_type = 'image' -%}
+                    {%- elif item['type'] == 'audio' -%}
+                        {{- '<|audio|>' -}}
+                        {%- set ns.prev_message_type = 'audio' -%}
+                    {%- elif item['type'] == 'video' -%}
+                        {{- '<|video|>' -}}
+                        {%- set ns.prev_message_type = 'video' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+        {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
+            {{- '<|tool_response>' -}}
+        {%- elif not (ns_tr_out.flag and not message.get('content')) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
+        {{- '<|turn>model\n' -}}
+    {%- endif -%}
+{%- endif -%}

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

tokenizer/tokenizer_config.json

@@ -0,0 +1,95 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}