Add files using upload-large-folder tool

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/trainer_state.json

@@ -0,0 +1,139 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.24140012070006034,
+  "eval_steps": 20,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0127318221037568e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/trainer_state.json

@@ -0,0 +1,1084 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.4103802051901027,
+  "eval_steps": 20,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0031683194745037e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/trainer_state.json

@@ -0,0 +1,1105 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.4586602293301145,
+  "eval_steps": 20,
+  "global_step": 1020,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0239715575080755e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/trainer_state.json

@@ -0,0 +1,1126 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.506940253470127,
+  "eval_steps": 20,
+  "global_step": 1040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0435065133937664e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/trainer_state.json

@@ -0,0 +1,1147 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5552202776101387,
+  "eval_steps": 20,
+  "global_step": 1060,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0627093642584269e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/trainer_state.json

@@ -0,0 +1,1168 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.603500301750151,
+  "eval_steps": 20,
+  "global_step": 1080,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6200249589979648,
+      "epoch": 2.603500301750151,
+      "grad_norm": 0.997065007686615,
+      "learning_rate": 0.0003370121363258637,
+      "loss": 0.6203888893127442,
+      "mean_token_accuracy": 0.8196275025606156,
+      "num_tokens": 2505382.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.603500301750151,
+      "eval_entropy": 0.6457576811983344,
+      "eval_loss": 0.6972290277481079,
+      "eval_mean_token_accuracy": 0.8095980849158898,
+      "eval_num_tokens": 2505382.0,
+      "eval_runtime": 89.2027,
+      "eval_samples_per_second": 15.919,
+      "eval_steps_per_second": 1.995,
+      "step": 1080
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0829722584469709e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/trainer_state.json

@@ -0,0 +1,1189 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.651780325890163,
+  "eval_steps": 20,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6200249589979648,
+      "epoch": 2.603500301750151,
+      "grad_norm": 0.997065007686615,
+      "learning_rate": 0.0003370121363258637,
+      "loss": 0.6203888893127442,
+      "mean_token_accuracy": 0.8196275025606156,
+      "num_tokens": 2505382.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.603500301750151,
+      "eval_entropy": 0.6457576811983344,
+      "eval_loss": 0.6972290277481079,
+      "eval_mean_token_accuracy": 0.8095980849158898,
+      "eval_num_tokens": 2505382.0,
+      "eval_runtime": 89.2027,
+      "eval_samples_per_second": 15.919,
+      "eval_steps_per_second": 1.995,
+      "step": 1080
+    },
+    {
+      "entropy": 0.626810473203659,
+      "epoch": 2.651780325890163,
+      "grad_norm": 0.865475058555603,
+      "learning_rate": 0.0003353646651036438,
+      "loss": 0.6102585792541504,
+      "mean_token_accuracy": 0.819708751142025,
+      "num_tokens": 2552927.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651780325890163,
+      "eval_entropy": 0.6412022431914726,
+      "eval_loss": 0.7036139965057373,
+      "eval_mean_token_accuracy": 0.8070317135098275,
+      "eval_num_tokens": 2552927.0,
+      "eval_runtime": 89.4733,
+      "eval_samples_per_second": 15.871,
+      "eval_steps_per_second": 1.989,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.102976649267794e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/trainer_state.json

@@ -0,0 +1,1210 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.700060350030175,
+  "eval_steps": 20,
+  "global_step": 1120,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6200249589979648,
+      "epoch": 2.603500301750151,
+      "grad_norm": 0.997065007686615,
+      "learning_rate": 0.0003370121363258637,
+      "loss": 0.6203888893127442,
+      "mean_token_accuracy": 0.8196275025606156,
+      "num_tokens": 2505382.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.603500301750151,
+      "eval_entropy": 0.6457576811983344,
+      "eval_loss": 0.6972290277481079,
+      "eval_mean_token_accuracy": 0.8095980849158898,
+      "eval_num_tokens": 2505382.0,
+      "eval_runtime": 89.2027,
+      "eval_samples_per_second": 15.919,
+      "eval_steps_per_second": 1.995,
+      "step": 1080
+    },
+    {
+      "entropy": 0.626810473203659,
+      "epoch": 2.651780325890163,
+      "grad_norm": 0.865475058555603,
+      "learning_rate": 0.0003353646651036438,
+      "loss": 0.6102585792541504,
+      "mean_token_accuracy": 0.819708751142025,
+      "num_tokens": 2552927.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651780325890163,
+      "eval_entropy": 0.6412022431914726,
+      "eval_loss": 0.7036139965057373,
+      "eval_mean_token_accuracy": 0.8070317135098275,
+      "eval_num_tokens": 2552927.0,
+      "eval_runtime": 89.4733,
+      "eval_samples_per_second": 15.871,
+      "eval_steps_per_second": 1.989,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6581619590520859,
+      "epoch": 2.700060350030175,
+      "grad_norm": 1.00307035446167,
+      "learning_rate": 0.00033367389569042064,
+      "loss": 0.6397720336914062,
+      "mean_token_accuracy": 0.81413309648633,
+      "num_tokens": 2597426.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.700060350030175,
+      "eval_entropy": 0.6288736402318719,
+      "eval_loss": 0.6966825723648071,
+      "eval_mean_token_accuracy": 0.8094039669867312,
+      "eval_num_tokens": 2597426.0,
+      "eval_runtime": 88.7341,
+      "eval_samples_per_second": 16.003,
+      "eval_steps_per_second": 2.006,
+      "step": 1120
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.123058232066478e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/trainer_state.json

@@ -0,0 +1,1231 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.748340374170187,
+  "eval_steps": 20,
+  "global_step": 1140,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6200249589979648,
+      "epoch": 2.603500301750151,
+      "grad_norm": 0.997065007686615,
+      "learning_rate": 0.0003370121363258637,
+      "loss": 0.6203888893127442,
+      "mean_token_accuracy": 0.8196275025606156,
+      "num_tokens": 2505382.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.603500301750151,
+      "eval_entropy": 0.6457576811983344,
+      "eval_loss": 0.6972290277481079,
+      "eval_mean_token_accuracy": 0.8095980849158898,
+      "eval_num_tokens": 2505382.0,
+      "eval_runtime": 89.2027,
+      "eval_samples_per_second": 15.919,
+      "eval_steps_per_second": 1.995,
+      "step": 1080
+    },
+    {
+      "entropy": 0.626810473203659,
+      "epoch": 2.651780325890163,
+      "grad_norm": 0.865475058555603,
+      "learning_rate": 0.0003353646651036438,
+      "loss": 0.6102585792541504,
+      "mean_token_accuracy": 0.819708751142025,
+      "num_tokens": 2552927.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651780325890163,
+      "eval_entropy": 0.6412022431914726,
+      "eval_loss": 0.7036139965057373,
+      "eval_mean_token_accuracy": 0.8070317135098275,
+      "eval_num_tokens": 2552927.0,
+      "eval_runtime": 89.4733,
+      "eval_samples_per_second": 15.871,
+      "eval_steps_per_second": 1.989,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6581619590520859,
+      "epoch": 2.700060350030175,
+      "grad_norm": 1.00307035446167,
+      "learning_rate": 0.00033367389569042064,
+      "loss": 0.6397720336914062,
+      "mean_token_accuracy": 0.81413309648633,
+      "num_tokens": 2597426.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.700060350030175,
+      "eval_entropy": 0.6288736402318719,
+      "eval_loss": 0.6966825723648071,
+      "eval_mean_token_accuracy": 0.8094039669867312,
+      "eval_num_tokens": 2597426.0,
+      "eval_runtime": 88.7341,
+      "eval_samples_per_second": 16.003,
+      "eval_steps_per_second": 2.006,
+      "step": 1120
+    },
+    {
+      "entropy": 0.6399412982165813,
+      "epoch": 2.748340374170187,
+      "grad_norm": 0.8589329719543457,
+      "learning_rate": 0.0003319403065539384,
+      "loss": 0.6347338199615479,
+      "mean_token_accuracy": 0.8139534957706929,
+      "num_tokens": 2640514.0,
+      "step": 1140
+    },
+    {
+      "epoch": 2.748340374170187,
+      "eval_entropy": 0.683491913455256,
+      "eval_loss": 0.6984149217605591,
+      "eval_mean_token_accuracy": 0.8083989469522841,
+      "eval_num_tokens": 2640514.0,
+      "eval_runtime": 89.3929,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.991,
+      "step": 1140
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.141886791596626e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.02728394274790723,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Estonian/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/trainer_state.json

@@ -0,0 +1,1252 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.796620398310199,
+  "eval_steps": 20,
+  "global_step": 1160,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.024795526266098,
+      "epoch": 0.04828002414001207,
+      "grad_norm": 2.945375919342041,
+      "learning_rate": 1.6698127428345936e-05,
+      "loss": 1.7825420379638672,
+      "mean_token_accuracy": 0.6315708436071873,
+      "num_tokens": 47778.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04828002414001207,
+      "eval_entropy": 1.29921497588747,
+      "eval_loss": 1.169131875038147,
+      "eval_mean_token_accuracy": 0.7185706342204233,
+      "eval_num_tokens": 47778.0,
+      "eval_runtime": 91.7898,
+      "eval_samples_per_second": 15.47,
+      "eval_steps_per_second": 1.939,
+      "step": 20
+    },
+    {
+      "entropy": 1.0346344470977784,
+      "epoch": 0.09656004828002414,
+      "grad_norm": 1.7916979789733887,
+      "learning_rate": 3.427510366871008e-05,
+      "loss": 0.9480395317077637,
+      "mean_token_accuracy": 0.7543429024517536,
+      "num_tokens": 95981.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09656004828002414,
+      "eval_entropy": 0.9673695430326997,
+      "eval_loss": 0.8588430285453796,
+      "eval_mean_token_accuracy": 0.7737233002534073,
+      "eval_num_tokens": 95981.0,
+      "eval_runtime": 89.8219,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 40
+    },
+    {
+      "entropy": 0.8977566614747048,
+      "epoch": 0.14484007242003621,
+      "grad_norm": 1.7972187995910645,
+      "learning_rate": 5.185207990907422e-05,
+      "loss": 0.8029604911804199,
+      "mean_token_accuracy": 0.7818016350269318,
+      "num_tokens": 142784.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14484007242003621,
+      "eval_entropy": 0.8436836982041263,
+      "eval_loss": 0.7893033623695374,
+      "eval_mean_token_accuracy": 0.7858914984076211,
+      "eval_num_tokens": 142784.0,
+      "eval_runtime": 89.9349,
+      "eval_samples_per_second": 15.789,
+      "eval_steps_per_second": 1.979,
+      "step": 60
+    },
+    {
+      "entropy": 0.8650321319699288,
+      "epoch": 0.19312009656004828,
+      "grad_norm": 1.2128729820251465,
+      "learning_rate": 6.942905614943836e-05,
+      "loss": 0.77417893409729,
+      "mean_token_accuracy": 0.7875479467213153,
+      "num_tokens": 187338.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19312009656004828,
+      "eval_entropy": 0.8209857180547179,
+      "eval_loss": 0.7572280168533325,
+      "eval_mean_token_accuracy": 0.7925970383574453,
+      "eval_num_tokens": 187338.0,
+      "eval_runtime": 89.9459,
+      "eval_samples_per_second": 15.787,
+      "eval_steps_per_second": 1.979,
+      "step": 80
+    },
+    {
+      "entropy": 0.8096660938113928,
+      "epoch": 0.24140012070006034,
+      "grad_norm": 1.1254287958145142,
+      "learning_rate": 8.700603238980251e-05,
+      "loss": 0.7170331001281738,
+      "mean_token_accuracy": 0.7994555257260799,
+      "num_tokens": 236038.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "eval_entropy": 0.792780208788561,
+      "eval_loss": 0.7442984580993652,
+      "eval_mean_token_accuracy": 0.7954704573984896,
+      "eval_num_tokens": 236038.0,
+      "eval_runtime": 90.294,
+      "eval_samples_per_second": 15.726,
+      "eval_steps_per_second": 1.971,
+      "step": 100
+    },
+    {
+      "entropy": 0.7979829199612141,
+      "epoch": 0.28968014484007243,
+      "grad_norm": 1.2140507698059082,
+      "learning_rate": 0.00010458300863016666,
+      "loss": 0.7141861915588379,
+      "mean_token_accuracy": 0.8000014744699001,
+      "num_tokens": 282918.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28968014484007243,
+      "eval_entropy": 0.7938692486018277,
+      "eval_loss": 0.7279797196388245,
+      "eval_mean_token_accuracy": 0.7989798734027348,
+      "eval_num_tokens": 282918.0,
+      "eval_runtime": 89.7189,
+      "eval_samples_per_second": 15.827,
+      "eval_steps_per_second": 1.984,
+      "step": 120
+    },
+    {
+      "entropy": 0.7916347607970238,
+      "epoch": 0.33796016898008446,
+      "grad_norm": 1.0506458282470703,
+      "learning_rate": 0.0001221599848705308,
+      "loss": 0.7117055416107178,
+      "mean_token_accuracy": 0.7970046654343605,
+      "num_tokens": 331305.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33796016898008446,
+      "eval_entropy": 0.8052781063519167,
+      "eval_loss": 0.722576916217804,
+      "eval_mean_token_accuracy": 0.7982530523551984,
+      "eval_num_tokens": 331305.0,
+      "eval_runtime": 90.0376,
+      "eval_samples_per_second": 15.771,
+      "eval_steps_per_second": 1.977,
+      "step": 140
+    },
+    {
+      "entropy": 0.7880666613578796,
+      "epoch": 0.38624019312009655,
+      "grad_norm": 0.8409207463264465,
+      "learning_rate": 0.00013973696111089492,
+      "loss": 0.7052523136138916,
+      "mean_token_accuracy": 0.8000123649835587,
+      "num_tokens": 375876.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38624019312009655,
+      "eval_entropy": 0.7787443134891853,
+      "eval_loss": 0.7137772440910339,
+      "eval_mean_token_accuracy": 0.8014209287220173,
+      "eval_num_tokens": 375876.0,
+      "eval_runtime": 90.2562,
+      "eval_samples_per_second": 15.733,
+      "eval_steps_per_second": 1.972,
+      "step": 160
+    },
+    {
+      "entropy": 0.7696007996797561,
+      "epoch": 0.43452021726010864,
+      "grad_norm": 1.0182477235794067,
+      "learning_rate": 0.00015731393735125907,
+      "loss": 0.7008797645568847,
+      "mean_token_accuracy": 0.8007259473204613,
+      "num_tokens": 423913.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43452021726010864,
+      "eval_entropy": 0.752259682068664,
+      "eval_loss": 0.7171286344528198,
+      "eval_mean_token_accuracy": 0.80052122574174,
+      "eval_num_tokens": 423913.0,
+      "eval_runtime": 90.0797,
+      "eval_samples_per_second": 15.764,
+      "eval_steps_per_second": 1.976,
+      "step": 180
+    },
+    {
+      "entropy": 0.7828126326203346,
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.9553632736206055,
+      "learning_rate": 0.0001748909135916232,
+      "loss": 0.7046853542327881,
+      "mean_token_accuracy": 0.7988562889397144,
+      "num_tokens": 472395.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "eval_entropy": 0.7389638169427936,
+      "eval_loss": 0.7187935709953308,
+      "eval_mean_token_accuracy": 0.7995899999409579,
+      "eval_num_tokens": 472395.0,
+      "eval_runtime": 89.9223,
+      "eval_samples_per_second": 15.791,
+      "eval_steps_per_second": 1.979,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909286297857762,
+      "epoch": 0.5310802655401328,
+      "grad_norm": 1.134459376335144,
+      "learning_rate": 0.00019246788983198735,
+      "loss": 0.6989312171936035,
+      "mean_token_accuracy": 0.7991565138101577,
+      "num_tokens": 517049.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5310802655401328,
+      "eval_entropy": 0.7496965199374082,
+      "eval_loss": 0.7106321454048157,
+      "eval_mean_token_accuracy": 0.8046183442131857,
+      "eval_num_tokens": 517049.0,
+      "eval_runtime": 89.6899,
+      "eval_samples_per_second": 15.832,
+      "eval_steps_per_second": 1.985,
+      "step": 220
+    },
+    {
+      "entropy": 0.7651963323354721,
+      "epoch": 0.5793602896801449,
+      "grad_norm": 0.9557585716247559,
+      "learning_rate": 0.0002100448660723515,
+      "loss": 0.6984559059143066,
+      "mean_token_accuracy": 0.8019823372364044,
+      "num_tokens": 562836.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5793602896801449,
+      "eval_entropy": 0.7615563079212488,
+      "eval_loss": 0.7098539471626282,
+      "eval_mean_token_accuracy": 0.8047233508544022,
+      "eval_num_tokens": 562836.0,
+      "eval_runtime": 90.3752,
+      "eval_samples_per_second": 15.712,
+      "eval_steps_per_second": 1.97,
+      "step": 240
+    },
+    {
+      "entropy": 0.787507726252079,
+      "epoch": 0.627640313820157,
+      "grad_norm": 0.9279311299324036,
+      "learning_rate": 0.00022762184231271568,
+      "loss": 0.708138370513916,
+      "mean_token_accuracy": 0.8013308703899383,
+      "num_tokens": 606612.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.627640313820157,
+      "eval_entropy": 0.7856640896100676,
+      "eval_loss": 0.7191570401191711,
+      "eval_mean_token_accuracy": 0.8024677249153008,
+      "eval_num_tokens": 606612.0,
+      "eval_runtime": 93.7336,
+      "eval_samples_per_second": 15.149,
+      "eval_steps_per_second": 1.899,
+      "step": 260
+    },
+    {
+      "entropy": 0.768489234894514,
+      "epoch": 0.6759203379601689,
+      "grad_norm": 0.7992776036262512,
+      "learning_rate": 0.0002451988185530798,
+      "loss": 0.7035849571228028,
+      "mean_token_accuracy": 0.8031105332076549,
+      "num_tokens": 655920.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6759203379601689,
+      "eval_entropy": 0.7682498740346244,
+      "eval_loss": 0.7149022221565247,
+      "eval_mean_token_accuracy": 0.7973497995499814,
+      "eval_num_tokens": 655920.0,
+      "eval_runtime": 89.7309,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 280
+    },
+    {
+      "entropy": 0.7720955617725849,
+      "epoch": 0.724200362100181,
+      "grad_norm": 1.1931514739990234,
+      "learning_rate": 0.0002627757947934439,
+      "loss": 0.6965402603149414,
+      "mean_token_accuracy": 0.8040348663926125,
+      "num_tokens": 702638.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.724200362100181,
+      "eval_entropy": 0.7214708304807042,
+      "eval_loss": 0.7350823879241943,
+      "eval_mean_token_accuracy": 0.7942911925610532,
+      "eval_num_tokens": 702638.0,
+      "eval_runtime": 89.6451,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.986,
+      "step": 300
+    },
+    {
+      "entropy": 0.7930095426738262,
+      "epoch": 0.7724803862401931,
+      "grad_norm": 0.8994467258453369,
+      "learning_rate": 0.0002803527710338081,
+      "loss": 0.7331492900848389,
+      "mean_token_accuracy": 0.796185302734375,
+      "num_tokens": 749377.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7724803862401931,
+      "eval_entropy": 0.8142535488926963,
+      "eval_loss": 0.7237834334373474,
+      "eval_mean_token_accuracy": 0.799495273761535,
+      "eval_num_tokens": 749377.0,
+      "eval_runtime": 90.6759,
+      "eval_samples_per_second": 15.66,
+      "eval_steps_per_second": 1.963,
+      "step": 320
+    },
+    {
+      "entropy": 0.7825102441012859,
+      "epoch": 0.8207604103802052,
+      "grad_norm": 1.8224239349365234,
+      "learning_rate": 0.00029792974727417223,
+      "loss": 0.7331175804138184,
+      "mean_token_accuracy": 0.7977667152881622,
+      "num_tokens": 798874.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8207604103802052,
+      "eval_entropy": 0.7327135443017724,
+      "eval_loss": 0.7388784289360046,
+      "eval_mean_token_accuracy": 0.8007041252730938,
+      "eval_num_tokens": 798874.0,
+      "eval_runtime": 89.7692,
+      "eval_samples_per_second": 15.818,
+      "eval_steps_per_second": 1.983,
+      "step": 340
+    },
+    {
+      "entropy": 0.819331557303667,
+      "epoch": 0.8690404345202173,
+      "grad_norm": 1.0407779216766357,
+      "learning_rate": 0.0003155067235145364,
+      "loss": 0.7456695556640625,
+      "mean_token_accuracy": 0.7936980701982975,
+      "num_tokens": 840825.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8690404345202173,
+      "eval_entropy": 0.8065585420372781,
+      "eval_loss": 0.735519528388977,
+      "eval_mean_token_accuracy": 0.7974206296245704,
+      "eval_num_tokens": 840825.0,
+      "eval_runtime": 89.4336,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.99,
+      "step": 360
+    },
+    {
+      "entropy": 0.8114653021097183,
+      "epoch": 0.9173204586602294,
+      "grad_norm": 1.1209490299224854,
+      "learning_rate": 0.0003330836997549005,
+      "loss": 0.7340899467468261,
+      "mean_token_accuracy": 0.7914272703230381,
+      "num_tokens": 882885.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9173204586602294,
+      "eval_entropy": 0.8025527285056168,
+      "eval_loss": 0.7295576930046082,
+      "eval_mean_token_accuracy": 0.7990576628218876,
+      "eval_num_tokens": 882885.0,
+      "eval_runtime": 89.9579,
+      "eval_samples_per_second": 15.785,
+      "eval_steps_per_second": 1.979,
+      "step": 380
+    },
+    {
+      "entropy": 0.8042497783899307,
+      "epoch": 0.9656004828002414,
+      "grad_norm": 1.2437798976898193,
+      "learning_rate": 0.0003506606759952647,
+      "loss": 0.7381744384765625,
+      "mean_token_accuracy": 0.7967362694442273,
+      "num_tokens": 927456.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "eval_entropy": 0.7717011336530193,
+      "eval_loss": 0.7436173558235168,
+      "eval_mean_token_accuracy": 0.7975849482450592,
+      "eval_num_tokens": 927456.0,
+      "eval_runtime": 89.8411,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 400
+    },
+    {
+      "entropy": 0.7987750010056929,
+      "epoch": 1.012070006035003,
+      "grad_norm": 1.2390918731689453,
+      "learning_rate": 0.000364721224843344,
+      "loss": 0.7317790031433106,
+      "mean_token_accuracy": 0.7986087551364651,
+      "num_tokens": 972662.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.012070006035003,
+      "eval_entropy": 0.7333170604170038,
+      "eval_loss": 0.7456948757171631,
+      "eval_mean_token_accuracy": 0.7964897577682238,
+      "eval_num_tokens": 972662.0,
+      "eval_runtime": 89.7288,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.984,
+      "step": 420
+    },
+    {
+      "entropy": 0.7372976846992969,
+      "epoch": 1.060350030175015,
+      "grad_norm": 1.1552740335464478,
+      "learning_rate": 0.00036468510102269575,
+      "loss": 0.6734249114990234,
+      "mean_token_accuracy": 0.805834824591875,
+      "num_tokens": 1022525.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060350030175015,
+      "eval_entropy": 0.7130355526891987,
+      "eval_loss": 0.7555333375930786,
+      "eval_mean_token_accuracy": 0.7961652781186479,
+      "eval_num_tokens": 1022525.0,
+      "eval_runtime": 89.8713,
+      "eval_samples_per_second": 15.8,
+      "eval_steps_per_second": 1.981,
+      "step": 440
+    },
+    {
+      "entropy": 0.7465668715536594,
+      "epoch": 1.1086300543150271,
+      "grad_norm": 1.1991469860076904,
+      "learning_rate": 0.0003645973816745066,
+      "loss": 0.6871879577636719,
+      "mean_token_accuracy": 0.8042450234293937,
+      "num_tokens": 1069194.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1086300543150271,
+      "eval_entropy": 0.7772979542110743,
+      "eval_loss": 0.7561826705932617,
+      "eval_mean_token_accuracy": 0.7953876427720102,
+      "eval_num_tokens": 1069194.0,
+      "eval_runtime": 89.8395,
+      "eval_samples_per_second": 15.806,
+      "eval_steps_per_second": 1.981,
+      "step": 460
+    },
+    {
+      "entropy": 0.7477855801582336,
+      "epoch": 1.1569100784550392,
+      "grad_norm": 1.325486660003662,
+      "learning_rate": 0.00036445809162231435,
+      "loss": 0.6836830139160156,
+      "mean_token_accuracy": 0.8021619468927383,
+      "num_tokens": 1117570.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1569100784550392,
+      "eval_entropy": 0.7098110676481483,
+      "eval_loss": 0.758007287979126,
+      "eval_mean_token_accuracy": 0.7909953189030122,
+      "eval_num_tokens": 1117570.0,
+      "eval_runtime": 89.6101,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.986,
+      "step": 480
+    },
+    {
+      "entropy": 0.7632691070437432,
+      "epoch": 1.2051901025950513,
+      "grad_norm": 1.0122262239456177,
+      "learning_rate": 0.0003642672702835562,
+      "loss": 0.7071213245391845,
+      "mean_token_accuracy": 0.8000680930912495,
+      "num_tokens": 1161717.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "eval_entropy": 0.7645382720432924,
+      "eval_loss": 0.7492606043815613,
+      "eval_mean_token_accuracy": 0.7976858676149604,
+      "eval_num_tokens": 1161717.0,
+      "eval_runtime": 89.6092,
+      "eval_samples_per_second": 15.847,
+      "eval_steps_per_second": 1.986,
+      "step": 500
+    },
+    {
+      "entropy": 0.7623726457357407,
+      "epoch": 1.2534701267350634,
+      "grad_norm": 1.367774486541748,
+      "learning_rate": 0.00036402497165841384,
+      "loss": 0.6965175151824952,
+      "mean_token_accuracy": 0.8016501650214195,
+      "num_tokens": 1209390.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2534701267350634,
+      "eval_entropy": 0.7868083715438843,
+      "eval_loss": 0.7481449842453003,
+      "eval_mean_token_accuracy": 0.799330582779445,
+      "eval_num_tokens": 1209390.0,
+      "eval_runtime": 89.8241,
+      "eval_samples_per_second": 15.809,
+      "eval_steps_per_second": 1.982,
+      "step": 520
+    },
+    {
+      "entropy": 0.7535207405686378,
+      "epoch": 1.3017501508750755,
+      "grad_norm": 1.213614821434021,
+      "learning_rate": 0.00036373126431453207,
+      "loss": 0.7078310489654541,
+      "mean_token_accuracy": 0.7998907208442688,
+      "num_tokens": 1256611.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3017501508750755,
+      "eval_entropy": 0.7829091950748743,
+      "eval_loss": 0.7416072487831116,
+      "eval_mean_token_accuracy": 0.7969146250339036,
+      "eval_num_tokens": 1256611.0,
+      "eval_runtime": 89.4607,
+      "eval_samples_per_second": 15.873,
+      "eval_steps_per_second": 1.99,
+      "step": 540
+    },
+    {
+      "entropy": 0.7527363449335098,
+      "epoch": 1.3500301750150876,
+      "grad_norm": 1.0929982662200928,
+      "learning_rate": 0.00036338623136761495,
+      "loss": 0.7058369636535644,
+      "mean_token_accuracy": 0.7997087433934211,
+      "num_tokens": 1302325.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3500301750150876,
+      "eval_entropy": 0.728419938114252,
+      "eval_loss": 0.744873046875,
+      "eval_mean_token_accuracy": 0.7966634296299366,
+      "eval_num_tokens": 1302325.0,
+      "eval_runtime": 89.6597,
+      "eval_samples_per_second": 15.838,
+      "eval_steps_per_second": 1.985,
+      "step": 560
+    },
+    {
+      "entropy": 0.7406099259853363,
+      "epoch": 1.3983101991550995,
+      "grad_norm": 1.409364104270935,
+      "learning_rate": 0.00036298997045790515,
+      "loss": 0.6902852058410645,
+      "mean_token_accuracy": 0.805386807769537,
+      "num_tokens": 1343844.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3983101991550995,
+      "eval_entropy": 0.6991632758231645,
+      "eval_loss": 0.7480319738388062,
+      "eval_mean_token_accuracy": 0.7970068006033308,
+      "eval_num_tokens": 1343844.0,
+      "eval_runtime": 89.8329,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 580
+    },
+    {
+      "entropy": 0.7459449704736472,
+      "epoch": 1.4465902232951118,
+      "grad_norm": 1.7053111791610718,
+      "learning_rate": 0.00036254259372255277,
+      "loss": 0.7044640064239502,
+      "mean_token_accuracy": 0.8005018755793571,
+      "num_tokens": 1389471.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "eval_entropy": 0.7454566660891758,
+      "eval_loss": 0.7551484704017639,
+      "eval_mean_token_accuracy": 0.7897093731365846,
+      "eval_num_tokens": 1389471.0,
+      "eval_runtime": 89.8313,
+      "eval_samples_per_second": 15.807,
+      "eval_steps_per_second": 1.981,
+      "step": 600
+    },
+    {
+      "entropy": 0.7621391348540782,
+      "epoch": 1.4948702474351236,
+      "grad_norm": 1.174443244934082,
+      "learning_rate": 0.000362044227763882,
+      "loss": 0.7111573696136475,
+      "mean_token_accuracy": 0.798752411454916,
+      "num_tokens": 1435975.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.4948702474351236,
+      "eval_entropy": 0.7664387949397055,
+      "eval_loss": 0.7365804314613342,
+      "eval_mean_token_accuracy": 0.7963101200843126,
+      "eval_num_tokens": 1435975.0,
+      "eval_runtime": 89.9623,
+      "eval_samples_per_second": 15.784,
+      "eval_steps_per_second": 1.979,
+      "step": 620
+    },
+    {
+      "entropy": 0.7424944408237935,
+      "epoch": 1.5431502715751357,
+      "grad_norm": 1.2898184061050415,
+      "learning_rate": 0.000361495013613564,
+      "loss": 0.6913110733032226,
+      "mean_token_accuracy": 0.8009015507996082,
+      "num_tokens": 1482035.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5431502715751357,
+      "eval_entropy": 0.7255479529332579,
+      "eval_loss": 0.7345991730690002,
+      "eval_mean_token_accuracy": 0.8013592945056015,
+      "eval_num_tokens": 1482035.0,
+      "eval_runtime": 89.817,
+      "eval_samples_per_second": 15.81,
+      "eval_steps_per_second": 1.982,
+      "step": 640
+    },
+    {
+      "entropy": 0.7303367637097835,
+      "epoch": 1.5914302957151478,
+      "grad_norm": 1.1310476064682007,
+      "learning_rate": 0.00036089510669270666,
+      "loss": 0.6875617027282714,
+      "mean_token_accuracy": 0.8035397171974182,
+      "num_tokens": 1528119.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5914302957151478,
+      "eval_entropy": 0.7101308248016271,
+      "eval_loss": 0.7394917011260986,
+      "eval_mean_token_accuracy": 0.7990792942850777,
+      "eval_num_tokens": 1528119.0,
+      "eval_runtime": 89.7431,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.983,
+      "step": 660
+    },
+    {
+      "entropy": 0.7387678384780884,
+      "epoch": 1.63971031985516,
+      "grad_norm": 0.9645389914512634,
+      "learning_rate": 0.0003602446767678725,
+      "loss": 0.6911204338073731,
+      "mean_token_accuracy": 0.8056333385407924,
+      "num_tokens": 1573661.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.63971031985516,
+      "eval_entropy": 0.7281073737010527,
+      "eval_loss": 0.7376708984375,
+      "eval_mean_token_accuracy": 0.7997356415464637,
+      "eval_num_tokens": 1573661.0,
+      "eval_runtime": 90.0338,
+      "eval_samples_per_second": 15.772,
+      "eval_steps_per_second": 1.977,
+      "step": 680
+    },
+    {
+      "entropy": 0.7449554048478604,
+      "epoch": 1.687990343995172,
+      "grad_norm": 1.9069629907608032,
+      "learning_rate": 0.0003595439079030364,
+      "loss": 0.7042286396026611,
+      "mean_token_accuracy": 0.8018639378249646,
+      "num_tokens": 1621714.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.687990343995172,
+      "eval_entropy": 0.7022633817088738,
+      "eval_loss": 0.7320939898490906,
+      "eval_mean_token_accuracy": 0.8006051541044471,
+      "eval_num_tokens": 1621714.0,
+      "eval_runtime": 89.8948,
+      "eval_samples_per_second": 15.796,
+      "eval_steps_per_second": 1.98,
+      "step": 700
+    },
+    {
+      "entropy": 0.723249051719904,
+      "epoch": 1.736270368135184,
+      "grad_norm": 0.8333877325057983,
+      "learning_rate": 0.00035879299840749777,
+      "loss": 0.7043714046478271,
+      "mean_token_accuracy": 0.8028345607221127,
+      "num_tokens": 1672118.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.736270368135184,
+      "eval_entropy": 0.672436946898364,
+      "eval_loss": 0.713233470916748,
+      "eval_mean_token_accuracy": 0.8047760577684038,
+      "eval_num_tokens": 1672118.0,
+      "eval_runtime": 89.6394,
+      "eval_samples_per_second": 15.841,
+      "eval_steps_per_second": 1.986,
+      "step": 720
+    },
+    {
+      "entropy": 0.730014206469059,
+      "epoch": 1.7845503922751962,
+      "grad_norm": 1.4580037593841553,
+      "learning_rate": 0.00035799216077976137,
+      "loss": 0.6985176563262939,
+      "mean_token_accuracy": 0.8015547141432762,
+      "num_tokens": 1720465.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.7845503922751962,
+      "eval_entropy": 0.6908873795123582,
+      "eval_loss": 0.7198202013969421,
+      "eval_mean_token_accuracy": 0.8024131682481659,
+      "eval_num_tokens": 1720465.0,
+      "eval_runtime": 89.3771,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.992,
+      "step": 740
+    },
+    {
+      "entropy": 0.7219605796039105,
+      "epoch": 1.832830416415208,
+      "grad_norm": 1.51682710647583,
+      "learning_rate": 0.000357141621647403,
+      "loss": 0.6844424724578857,
+      "mean_token_accuracy": 0.8032813094556331,
+      "num_tokens": 1764440.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.832830416415208,
+      "eval_entropy": 0.7113401109582922,
+      "eval_loss": 0.7195309400558472,
+      "eval_mean_token_accuracy": 0.8019187035185568,
+      "eval_num_tokens": 1764440.0,
+      "eval_runtime": 89.4658,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.729826743900776,
+      "epoch": 1.8811104405552204,
+      "grad_norm": 1.0838000774383545,
+      "learning_rate": 0.0003562416217029361,
+      "loss": 0.685858678817749,
+      "mean_token_accuracy": 0.8050567395985126,
+      "num_tokens": 1810828.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8811104405552204,
+      "eval_entropy": 0.649370267484965,
+      "eval_loss": 0.7177029252052307,
+      "eval_mean_token_accuracy": 0.8050174043419656,
+      "eval_num_tokens": 1810828.0,
+      "eval_runtime": 89.2287,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.995,
+      "step": 780
+    },
+    {
+      "entropy": 0.6799364153295755,
+      "epoch": 1.9293904646952322,
+      "grad_norm": 1.1364924907684326,
+      "learning_rate": 0.00035529241563569903,
+      "loss": 0.6714428901672364,
+      "mean_token_accuracy": 0.8087341658771038,
+      "num_tokens": 1857600.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "eval_entropy": 0.6603799047095052,
+      "eval_loss": 0.7201809287071228,
+      "eval_mean_token_accuracy": 0.802863868769635,
+      "eval_num_tokens": 1857600.0,
+      "eval_runtime": 89.2718,
+      "eval_samples_per_second": 15.906,
+      "eval_steps_per_second": 1.994,
+      "step": 800
+    },
+    {
+      "entropy": 0.7078610375523567,
+      "epoch": 1.9776704888352445,
+      "grad_norm": 1.2191942930221558,
+      "learning_rate": 0.0003542942720597808,
+      "loss": 0.6996590614318847,
+      "mean_token_accuracy": 0.8049512051045895,
+      "num_tokens": 1901998.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.9776704888352445,
+      "eval_entropy": 0.7240545264120852,
+      "eval_loss": 0.7301309108734131,
+      "eval_mean_token_accuracy": 0.8015973166133581,
+      "eval_num_tokens": 1901998.0,
+      "eval_runtime": 89.3438,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.992,
+      "step": 820
+    },
+    {
+      "entropy": 0.6557391556826505,
+      "epoch": 2.024140012070006,
+      "grad_norm": 1.36203134059906,
+      "learning_rate": 0.0003532474734380064,
+      "loss": 0.6327113628387451,
+      "mean_token_accuracy": 0.8142829847026181,
+      "num_tokens": 1947553.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.024140012070006,
+      "eval_entropy": 0.6573808648613062,
+      "eval_loss": 0.7334365844726562,
+      "eval_mean_token_accuracy": 0.8043633893634496,
+      "eval_num_tokens": 1947553.0,
+      "eval_runtime": 89.3359,
+      "eval_samples_per_second": 15.895,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.6409128502011299,
+      "epoch": 2.0724200362100182,
+      "grad_norm": 1.5804955959320068,
+      "learning_rate": 0.0003521523160020035,
+      "loss": 0.6023559093475341,
+      "mean_token_accuracy": 0.8222251623868942,
+      "num_tokens": 1995234.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0724200362100182,
+      "eval_entropy": 0.6983234788594621,
+      "eval_loss": 0.7232135534286499,
+      "eval_mean_token_accuracy": 0.8010654268639811,
+      "eval_num_tokens": 1995234.0,
+      "eval_runtime": 89.2615,
+      "eval_samples_per_second": 15.908,
+      "eval_steps_per_second": 1.994,
+      "step": 860
+    },
+    {
+      "entropy": 0.6151149723678827,
+      "epoch": 2.12070006035003,
+      "grad_norm": 1.1975436210632324,
+      "learning_rate": 0.00035100910966837193,
+      "loss": 0.5803564548492431,
+      "mean_token_accuracy": 0.8256325736641884,
+      "num_tokens": 2040639.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.12070006035003,
+      "eval_entropy": 0.6636555573243773,
+      "eval_loss": 0.7205662727355957,
+      "eval_mean_token_accuracy": 0.8036768369460374,
+      "eval_num_tokens": 2040639.0,
+      "eval_runtime": 89.2957,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.993,
+      "step": 880
+    },
+    {
+      "entropy": 0.6201727617532015,
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.9907131195068359,
+      "learning_rate": 0.0003498181779509813,
+      "loss": 0.6101872444152832,
+      "mean_token_accuracy": 0.8196512959897518,
+      "num_tokens": 2090604.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "eval_entropy": 0.6473788086617931,
+      "eval_loss": 0.7062397003173828,
+      "eval_mean_token_accuracy": 0.8078362184963869,
+      "eval_num_tokens": 2090604.0,
+      "eval_runtime": 88.6198,
+      "eval_samples_per_second": 16.024,
+      "eval_steps_per_second": 2.009,
+      "step": 900
+    },
+    {
+      "entropy": 0.6097332935780286,
+      "epoch": 2.2172601086300543,
+      "grad_norm": 0.9055633544921875,
+      "learning_rate": 0.00034857985786942026,
+      "loss": 0.5956094264984131,
+      "mean_token_accuracy": 0.8240575887262821,
+      "num_tokens": 2142246.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2172601086300543,
+      "eval_entropy": 0.612629868341296,
+      "eval_loss": 0.7106770277023315,
+      "eval_mean_token_accuracy": 0.805324277181304,
+      "eval_num_tokens": 2142246.0,
+      "eval_runtime": 89.4383,
+      "eval_samples_per_second": 15.877,
+      "eval_steps_per_second": 1.99,
+      "step": 920
+    },
+    {
+      "entropy": 0.6182701248675585,
+      "epoch": 2.2655401327700666,
+      "grad_norm": 0.9513736367225647,
+      "learning_rate": 0.00034729449985362404,
+      "loss": 0.6019269466400147,
+      "mean_token_accuracy": 0.8187106013298034,
+      "num_tokens": 2183144.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2655401327700666,
+      "eval_entropy": 0.6437647456533453,
+      "eval_loss": 0.7173502445220947,
+      "eval_mean_token_accuracy": 0.8056439573175451,
+      "eval_num_tokens": 2183144.0,
+      "eval_runtime": 88.9568,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 2.001,
+      "step": 940
+    },
+    {
+      "entropy": 0.6508570514619351,
+      "epoch": 2.3138201569100785,
+      "grad_norm": 0.9606207013130188,
+      "learning_rate": 0.0003459624676447067,
+      "loss": 0.6296409606933594,
+      "mean_token_accuracy": 0.8167732007801533,
+      "num_tokens": 2228098.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3138201569100785,
+      "eval_entropy": 0.6288247135248077,
+      "eval_loss": 0.7160675525665283,
+      "eval_mean_token_accuracy": 0.8066010689467527,
+      "eval_num_tokens": 2228098.0,
+      "eval_runtime": 89.0044,
+      "eval_samples_per_second": 15.954,
+      "eval_steps_per_second": 2.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.6380081418901682,
+      "epoch": 2.3621001810500903,
+      "grad_norm": 0.9677181839942932,
+      "learning_rate": 0.000344584138192027,
+      "loss": 0.6210546493530273,
+      "mean_token_accuracy": 0.8186135642230511,
+      "num_tokens": 2273062.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.3621001810500903,
+      "eval_entropy": 0.6407805031604981,
+      "eval_loss": 0.7055043578147888,
+      "eval_mean_token_accuracy": 0.8070103157772107,
+      "eval_num_tokens": 2273062.0,
+      "eval_runtime": 89.5606,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.987,
+      "step": 980
+    },
+    {
+      "entropy": 0.6310251638293266,
+      "epoch": 2.4103802051901027,
+      "grad_norm": 2.1046407222747803,
+      "learning_rate": 0.000343159901546516,
+      "loss": 0.6165830135345459,
+      "mean_token_accuracy": 0.8207855716347694,
+      "num_tokens": 2319305.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "eval_entropy": 0.6461507470420237,
+      "eval_loss": 0.7292136549949646,
+      "eval_mean_token_accuracy": 0.8030418261383356,
+      "eval_num_tokens": 2319305.0,
+      "eval_runtime": 89.315,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.993,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6256617344915867,
+      "epoch": 2.4586602293301145,
+      "grad_norm": 1.1768349409103394,
+      "learning_rate": 0.0003416901607502972,
+      "loss": 0.6142709255218506,
+      "mean_token_accuracy": 0.8220669947564602,
+      "num_tokens": 2368592.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4586602293301145,
+      "eval_entropy": 0.6150523993406403,
+      "eval_loss": 0.7019563317298889,
+      "eval_mean_token_accuracy": 0.8094323099998946,
+      "eval_num_tokens": 2368592.0,
+      "eval_runtime": 89.419,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.991,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6126735735684633,
+      "epoch": 2.506940253470127,
+      "grad_norm": 1.8065496683120728,
+      "learning_rate": 0.00034017533172263055,
+      "loss": 0.6056458473205566,
+      "mean_token_accuracy": 0.8203762136399746,
+      "num_tokens": 2412859.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.506940253470127,
+      "eval_entropy": 0.6544345456562685,
+      "eval_loss": 0.7086517214775085,
+      "eval_mean_token_accuracy": 0.806709805901131,
+      "eval_num_tokens": 2412859.0,
+      "eval_runtime": 88.981,
+      "eval_samples_per_second": 15.958,
+      "eval_steps_per_second": 2.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6311754353344441,
+      "epoch": 2.5552202776101387,
+      "grad_norm": 1.0576369762420654,
+      "learning_rate": 0.00033861584314221236,
+      "loss": 0.6176392555236816,
+      "mean_token_accuracy": 0.8177410490810871,
+      "num_tokens": 2458201.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.5552202776101387,
+      "eval_entropy": 0.6273799200406235,
+      "eval_loss": 0.708163321018219,
+      "eval_mean_token_accuracy": 0.8032017099053672,
+      "eval_num_tokens": 2458201.0,
+      "eval_runtime": 89.7668,
+      "eval_samples_per_second": 15.819,
+      "eval_steps_per_second": 1.983,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6200249589979648,
+      "epoch": 2.603500301750151,
+      "grad_norm": 0.997065007686615,
+      "learning_rate": 0.0003370121363258637,
+      "loss": 0.6203888893127442,
+      "mean_token_accuracy": 0.8196275025606156,
+      "num_tokens": 2505382.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.603500301750151,
+      "eval_entropy": 0.6457576811983344,
+      "eval_loss": 0.6972290277481079,
+      "eval_mean_token_accuracy": 0.8095980849158898,
+      "eval_num_tokens": 2505382.0,
+      "eval_runtime": 89.2027,
+      "eval_samples_per_second": 15.919,
+      "eval_steps_per_second": 1.995,
+      "step": 1080
+    },
+    {
+      "entropy": 0.626810473203659,
+      "epoch": 2.651780325890163,
+      "grad_norm": 0.865475058555603,
+      "learning_rate": 0.0003353646651036438,
+      "loss": 0.6102585792541504,
+      "mean_token_accuracy": 0.819708751142025,
+      "num_tokens": 2552927.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651780325890163,
+      "eval_entropy": 0.6412022431914726,
+      "eval_loss": 0.7036139965057373,
+      "eval_mean_token_accuracy": 0.8070317135098275,
+      "eval_num_tokens": 2552927.0,
+      "eval_runtime": 89.4733,
+      "eval_samples_per_second": 15.871,
+      "eval_steps_per_second": 1.989,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6581619590520859,
+      "epoch": 2.700060350030175,
+      "grad_norm": 1.00307035446167,
+      "learning_rate": 0.00033367389569042064,
+      "loss": 0.6397720336914062,
+      "mean_token_accuracy": 0.81413309648633,
+      "num_tokens": 2597426.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.700060350030175,
+      "eval_entropy": 0.6288736402318719,
+      "eval_loss": 0.6966825723648071,
+      "eval_mean_token_accuracy": 0.8094039669867312,
+      "eval_num_tokens": 2597426.0,
+      "eval_runtime": 88.7341,
+      "eval_samples_per_second": 16.003,
+      "eval_steps_per_second": 2.006,
+      "step": 1120
+    },
+    {
+      "entropy": 0.6399412982165813,
+      "epoch": 2.748340374170187,
+      "grad_norm": 0.8589329719543457,
+      "learning_rate": 0.0003319403065539384,
+      "loss": 0.6347338199615479,
+      "mean_token_accuracy": 0.8139534957706929,
+      "num_tokens": 2640514.0,
+      "step": 1140
+    },
+    {
+      "epoch": 2.748340374170187,
+      "eval_entropy": 0.683491913455256,
+      "eval_loss": 0.6984149217605591,
+      "eval_mean_token_accuracy": 0.8083989469522841,
+      "eval_num_tokens": 2640514.0,
+      "eval_runtime": 89.3929,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.991,
+      "step": 1140
+    },
+    {
+      "entropy": 0.6264835461974144,
+      "epoch": 2.796620398310199,
+      "grad_norm": 0.7807853817939758,
+      "learning_rate": 0.0003301643882794163,
+      "loss": 0.6162629127502441,
+      "mean_token_accuracy": 0.8216292977333068,
+      "num_tokens": 2687730.0,
+      "step": 1160
+    },
+    {
+      "epoch": 2.796620398310199,
+      "eval_entropy": 0.6578792159476977,
+      "eval_loss": 0.6921441555023193,
+      "eval_mean_token_accuracy": 0.8094299177775223,
+      "eval_num_tokens": 2687730.0,
+      "eval_runtime": 89.2356,
+      "eval_samples_per_second": 15.913,
+      "eval_steps_per_second": 1.995,
+      "step": 1160
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1627016981849907e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}