Add files using upload-large-folder tool

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: transformers
+model_name: Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/onphamd0) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: transformers
+model_name: Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/7xpb8te3) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/trainer_state.json

@@ -0,0 +1,139 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.24906600249066002,
+  "eval_steps": 20,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9823576763965440.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/trainer_state.json

@@ -0,0 +1,1084 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.488169364881694,
+  "eval_steps": 20,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    },
+    {
+      "entropy": 0.6744543805718421,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.932099461555481,
+      "learning_rate": 6.698322232264434e-05,
+      "loss": 0.5991750717163086,
+      "mean_token_accuracy": 0.8304223112761975,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.6813044282932614,
+      "eval_loss": 0.5922021269798279,
+      "eval_mean_token_accuracy": 0.8346439617317777,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.1551,
+      "eval_samples_per_second": 15.96,
+      "eval_steps_per_second": 1.996,
+      "step": 120
+    },
+    {
+      "entropy": 0.6663189359009266,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9528499841690063,
+      "learning_rate": 7.824090674661818e-05,
+      "loss": 0.5891091346740722,
+      "mean_token_accuracy": 0.832152470946312,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6398407208711602,
+      "eval_loss": 0.5859636664390564,
+      "eval_mean_token_accuracy": 0.8372074996316156,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.2706,
+      "eval_samples_per_second": 15.938,
+      "eval_steps_per_second": 1.994,
+      "step": 140
+    },
+    {
+      "entropy": 0.64859763905406,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8468204140663147,
+      "learning_rate": 8.949859117059201e-05,
+      "loss": 0.569426441192627,
+      "mean_token_accuracy": 0.8401990942656994,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6381674285891444,
+      "eval_loss": 0.5744525790214539,
+      "eval_mean_token_accuracy": 0.838626817908398,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.2848,
+      "eval_samples_per_second": 15.936,
+      "eval_steps_per_second": 1.993,
+      "step": 160
+    },
+    {
+      "entropy": 0.6432608783245086,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8765804767608643,
+      "learning_rate": 0.00010075627559456587,
+      "loss": 0.5687318801879883,
+      "mean_token_accuracy": 0.839249350130558,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6047098288355872,
+      "eval_loss": 0.5679298043251038,
+      "eval_mean_token_accuracy": 0.8410577181466791,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.5879,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.986,
+      "step": 180
+    },
+    {
+      "entropy": 0.6440276011824608,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9576020240783691,
+      "learning_rate": 0.00011201396001853971,
+      "loss": 0.5828506469726562,
+      "mean_token_accuracy": 0.837553184479475,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6161119360909906,
+      "eval_loss": 0.5702911615371704,
+      "eval_mean_token_accuracy": 0.8407089398350827,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3341,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 200
+    },
+    {
+      "entropy": 0.6377195850014686,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7212373614311218,
+      "learning_rate": 0.00012327164444251353,
+      "loss": 0.5702451229095459,
+      "mean_token_accuracy": 0.8397969007492065,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6080108886194784,
+      "eval_loss": 0.5633499622344971,
+      "eval_mean_token_accuracy": 0.8396634854549585,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.4945,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 220
+    },
+    {
+      "entropy": 0.6287345830351114,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.848779022693634,
+      "learning_rate": 0.00013452932886648739,
+      "loss": 0.5506546020507812,
+      "mean_token_accuracy": 0.8438881888985634,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6307531505130058,
+      "eval_loss": 0.5573338270187378,
+      "eval_mean_token_accuracy": 0.8431362606758295,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.3535,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.992,
+      "step": 240
+    },
+    {
+      "entropy": 0.6223786748945713,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.7316951751708984,
+      "learning_rate": 0.0001457870132904612,
+      "loss": 0.5495625972747803,
+      "mean_token_accuracy": 0.8440376669168472,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.623454462476941,
+      "eval_loss": 0.5619264245033264,
+      "eval_mean_token_accuracy": 0.8431175777385401,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.2008,
+      "eval_samples_per_second": 15.951,
+      "eval_steps_per_second": 1.995,
+      "step": 260
+    },
+    {
+      "entropy": 0.6281675305217505,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7639564871788025,
+      "learning_rate": 0.00015704469771443506,
+      "loss": 0.5604369163513183,
+      "mean_token_accuracy": 0.8401600055396556,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.63416675980701,
+      "eval_loss": 0.5612760782241821,
+      "eval_mean_token_accuracy": 0.842435666294985,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.25,
+      "eval_samples_per_second": 15.942,
+      "eval_steps_per_second": 1.994,
+      "step": 280
+    },
+    {
+      "entropy": 0.6427909277379513,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6475813388824463,
+      "learning_rate": 0.0001683023821384089,
+      "loss": 0.573763370513916,
+      "mean_token_accuracy": 0.8370340794324875,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6231539840268534,
+      "eval_loss": 0.5566866397857666,
+      "eval_mean_token_accuracy": 0.844177934319474,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.4858,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.989,
+      "step": 300
+    },
+    {
+      "entropy": 0.6226776849478484,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.8886699676513672,
+      "learning_rate": 0.00017956006656238274,
+      "loss": 0.558210802078247,
+      "mean_token_accuracy": 0.84083157107234,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6066981683983359,
+      "eval_loss": 0.5585207939147949,
+      "eval_mean_token_accuracy": 0.8423153311014175,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.3463,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 320
+    },
+    {
+      "entropy": 0.6249004438519478,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8791211843490601,
+      "learning_rate": 0.00019081775098635657,
+      "loss": 0.5603597164154053,
+      "mean_token_accuracy": 0.8420463085174561,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6082247584018596,
+      "eval_loss": 0.5616299510002136,
+      "eval_mean_token_accuracy": 0.8431286801432454,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.1253,
+      "eval_samples_per_second": 15.965,
+      "eval_steps_per_second": 1.997,
+      "step": 340
+    },
+    {
+      "entropy": 0.6362396612763405,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.8606319427490234,
+      "learning_rate": 0.0002020754354103304,
+      "loss": 0.5735773563385009,
+      "mean_token_accuracy": 0.8371490836143494,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6492362072648004,
+      "eval_loss": 0.5646467804908752,
+      "eval_mean_token_accuracy": 0.8415517574825953,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.3351,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 360
+    },
+    {
+      "entropy": 0.638665035739541,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.7773950099945068,
+      "learning_rate": 0.00021333311983430425,
+      "loss": 0.5820859909057617,
+      "mean_token_accuracy": 0.8372561208903789,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6434498637221581,
+      "eval_loss": 0.5645168423652649,
+      "eval_mean_token_accuracy": 0.8420382481674815,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.1216,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 380
+    },
+    {
+      "entropy": 0.6316851265728474,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 1.6120579242706299,
+      "learning_rate": 0.00022459080425827807,
+      "loss": 0.5637502670288086,
+      "mean_token_accuracy": 0.8386227294802666,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6469012776086497,
+      "eval_loss": 0.5758090615272522,
+      "eval_mean_token_accuracy": 0.8397158470957778,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.6139,
+      "eval_samples_per_second": 15.875,
+      "eval_steps_per_second": 1.986,
+      "step": 400
+    },
+    {
+      "entropy": 0.5894816922835815,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 1.1616325378417969,
+      "learning_rate": 0.00022626713048053178,
+      "loss": 0.5316025257110596,
+      "mean_token_accuracy": 0.8466163017810919,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5860798164855602,
+      "eval_loss": 0.5777581930160522,
+      "eval_mean_token_accuracy": 0.8396938103576039,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.1449,
+      "eval_samples_per_second": 15.961,
+      "eval_steps_per_second": 1.997,
+      "step": 420
+    },
+    {
+      "entropy": 0.5818420693278312,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.7999453544616699,
+      "learning_rate": 0.00022622107023288778,
+      "loss": 0.5221010208129883,
+      "mean_token_accuracy": 0.8474301159381866,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5783926014636838,
+      "eval_loss": 0.5700300931930542,
+      "eval_mean_token_accuracy": 0.8430753537388735,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.5308,
+      "eval_samples_per_second": 15.89,
+      "eval_steps_per_second": 1.988,
+      "step": 440
+    },
+    {
+      "entropy": 0.5612493887543678,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 1.015687346458435,
+      "learning_rate": 0.00022614090619491568,
+      "loss": 0.5084867000579834,
+      "mean_token_accuracy": 0.8495561093091964,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5841563874205877,
+      "eval_loss": 0.5693665742874146,
+      "eval_mean_token_accuracy": 0.8427817298229351,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.5256,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.988,
+      "step": 460
+    },
+    {
+      "entropy": 0.5828216474503278,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 1.9750930070877075,
+      "learning_rate": 0.00022602666254299594,
+      "loss": 0.5180017948150635,
+      "mean_token_accuracy": 0.8515685826539994,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5806607044366903,
+      "eval_loss": 0.5804352760314941,
+      "eval_mean_token_accuracy": 0.8413014668364858,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.1199,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 480
+    },
+    {
+      "entropy": 0.5926914308220148,
+      "epoch": 1.244084682440847,
+      "grad_norm": 0.8917353749275208,
+      "learning_rate": 0.0002258783737314558,
+      "loss": 0.528910779953003,
+      "mean_token_accuracy": 0.8486074328422546,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5593361884009006,
+      "eval_loss": 0.5675153732299805,
+      "eval_mean_token_accuracy": 0.8433507802181466,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7289,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5865630559623242,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7482362985610962,
+      "learning_rate": 0.00022569608448217823,
+      "loss": 0.5250466823577881,
+      "mean_token_accuracy": 0.8477916084229946,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.543057840230853,
+      "eval_loss": 0.5671008229255676,
+      "eval_mean_token_accuracy": 0.8428726016088973,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.3403,
+      "eval_samples_per_second": 15.925,
+      "eval_steps_per_second": 1.992,
+      "step": 520
+    },
+    {
+      "entropy": 0.5870206747204065,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9473814964294434,
+      "learning_rate": 0.00022547984977111448,
+      "loss": 0.5252370834350586,
+      "mean_token_accuracy": 0.8468369916081429,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.590982622878496,
+      "eval_loss": 0.5676343441009521,
+      "eval_mean_token_accuracy": 0.8429348746011424,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.5168,
+      "eval_samples_per_second": 15.893,
+      "eval_steps_per_second": 1.988,
+      "step": 540
+    },
+    {
+      "entropy": 0.5785854265093804,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.9353351593017578,
+      "learning_rate": 0.0002252297348117042,
+      "loss": 0.5304938316345215,
+      "mean_token_accuracy": 0.8463383808732032,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6099918867612995,
+      "eval_loss": 0.5620437860488892,
+      "eval_mean_token_accuracy": 0.8430728347495545,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.7741,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.982,
+      "step": 560
+    },
+    {
+      "entropy": 0.5768801040947438,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.9198738932609558,
+      "learning_rate": 0.0002249458150352077,
+      "loss": 0.520513391494751,
+      "mean_token_accuracy": 0.8487689301371575,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.6349420670506566,
+      "eval_loss": 0.5645340085029602,
+      "eval_mean_token_accuracy": 0.8447844597489335,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.3257,
+      "eval_samples_per_second": 15.928,
+      "eval_steps_per_second": 1.992,
+      "step": 580
+    },
+    {
+      "entropy": 0.5822233572602272,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.832811176776886,
+      "learning_rate": 0.0002246281760679571,
+      "loss": 0.5295282363891601,
+      "mean_token_accuracy": 0.8504064798355102,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5829724387027496,
+      "eval_loss": 0.5612193942070007,
+      "eval_mean_token_accuracy": 0.8449643853791925,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6617,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.571855777129531,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.7665547728538513,
+      "learning_rate": 0.00022427691370553263,
+      "loss": 0.5187931060791016,
+      "mean_token_accuracy": 0.8534420043230057,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5623592240519302,
+      "eval_loss": 0.5575760006904602,
+      "eval_mean_token_accuracy": 0.8468210229346919,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.6324,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.985,
+      "step": 620
+    },
+    {
+      "entropy": 0.5740394659340382,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6500429511070251,
+      "learning_rate": 0.00022389213388387174,
+      "loss": 0.5283198833465577,
+      "mean_token_accuracy": 0.8502798482775689,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5548852207355721,
+      "eval_loss": 0.5561797022819519,
+      "eval_mean_token_accuracy": 0.8452786498291548,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.5205,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 640
+    },
+    {
+      "entropy": 0.6020145989954472,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.7056867480278015,
+      "learning_rate": 0.00022347395264732053,
+      "loss": 0.5400049209594726,
+      "mean_token_accuracy": 0.8447613954544068,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5618055154417836,
+      "eval_loss": 0.556106686592102,
+      "eval_mean_token_accuracy": 0.8465680112672407,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.2971,
+      "eval_samples_per_second": 15.933,
+      "eval_steps_per_second": 1.993,
+      "step": 660
+    },
+    {
+      "entropy": 0.5665927153080702,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.5987663865089417,
+      "learning_rate": 0.00022302249611363625,
+      "loss": 0.5143643856048584,
+      "mean_token_accuracy": 0.8529589556157589,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.568248552118623,
+      "eval_loss": 0.5476346015930176,
+      "eval_mean_token_accuracy": 0.8476775434128073,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.9583,
+      "eval_samples_per_second": 15.812,
+      "eval_steps_per_second": 1.978,
+      "step": 680
+    },
+    {
+      "entropy": 0.5673687808215618,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.735261857509613,
+      "learning_rate": 0.00022253790043595193,
+      "loss": 0.509885597229004,
+      "mean_token_accuracy": 0.8537046857178211,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5616967284748721,
+      "eval_loss": 0.5439274311065674,
+      "eval_mean_token_accuracy": 0.8488946217437123,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.0604,
+      "eval_samples_per_second": 15.977,
+      "eval_steps_per_second": 1.999,
+      "step": 700
+    },
+    {
+      "entropy": 0.5529541682451964,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7014835476875305,
+      "learning_rate": 0.00022202031176171442,
+      "loss": 0.5078992366790771,
+      "mean_token_accuracy": 0.8525233261287213,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5827173320359962,
+      "eval_loss": 0.5419450402259827,
+      "eval_mean_token_accuracy": 0.8477318609176681,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 85.2984,
+      "eval_samples_per_second": 16.12,
+      "eval_steps_per_second": 2.016,
+      "step": 720
+    },
+    {
+      "entropy": 0.5755720350891351,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.705613911151886,
+      "learning_rate": 0.00022146988618860824,
+      "loss": 0.5181350708007812,
+      "mean_token_accuracy": 0.8467609457671642,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5743971356125765,
+      "eval_loss": 0.5415896773338318,
+      "eval_mean_token_accuracy": 0.847328585940738,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 85.5602,
+      "eval_samples_per_second": 16.071,
+      "eval_steps_per_second": 2.01,
+      "step": 740
+    },
+    {
+      "entropy": 0.561330484598875,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.6722865700721741,
+      "learning_rate": 0.0002208867897174789,
+      "loss": 0.499837589263916,
+      "mean_token_accuracy": 0.8518734864890576,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.5865232653396074,
+      "eval_loss": 0.5437926650047302,
+      "eval_mean_token_accuracy": 0.8450997017843779,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4116,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.547389242425561,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.7935577034950256,
+      "learning_rate": 0.00022027119820226907,
+      "loss": 0.4977591514587402,
+      "mean_token_accuracy": 0.8539491161704064,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.5290903090391048,
+      "eval_loss": 0.5409526824951172,
+      "eval_mean_token_accuracy": 0.8497545698354411,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.7262,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 780
+    },
+    {
+      "entropy": 0.5687909748405218,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6180546283721924,
+      "learning_rate": 0.00021962329729698345,
+      "loss": 0.5109643459320068,
+      "mean_token_accuracy": 0.8521598495543004,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.5503541858390321,
+      "eval_loss": 0.5361555218696594,
+      "eval_mean_token_accuracy": 0.8510884285666221,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.3339,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 800
+    },
+    {
+      "entropy": 0.4739728841261986,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.8058829307556152,
+      "learning_rate": 0.0002189432823996982,
+      "loss": 0.4204097747802734,
+      "mean_token_accuracy": 0.8728981889211215,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.5077334992414297,
+      "eval_loss": 0.5531114339828491,
+      "eval_mean_token_accuracy": 0.8489257208136625,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.4801,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.989,
+      "step": 820
+    },
+    {
+      "entropy": 0.4594309840351343,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.6906896829605103,
+      "learning_rate": 0.0002182313585936314,
+      "loss": 0.4071959495544434,
+      "mean_token_accuracy": 0.8732857562601566,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.49850136994622474,
+      "eval_loss": 0.5486204624176025,
+      "eval_mean_token_accuracy": 0.8507991450470548,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.3364,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.4881629109382629,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6343470215797424,
+      "learning_rate": 0.0002174877405852928,
+      "loss": 0.41669540405273436,
+      "mean_token_accuracy": 0.8711295068264008,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.49155513924914734,
+      "eval_loss": 0.555109441280365,
+      "eval_mean_token_accuracy": 0.8496399400539176,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 86.3295,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 860
+    },
+    {
+      "entropy": 0.4648668970912695,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.8014165163040161,
+      "learning_rate": 0.00021671265263973133,
+      "loss": 0.4110250473022461,
+      "mean_token_accuracy": 0.8754166305065155,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.4909258722219356,
+      "eval_loss": 0.5539511442184448,
+      "eval_mean_token_accuracy": 0.8492401502160138,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.3468,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 880
+    },
+    {
+      "entropy": 0.4824485514312983,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6665191054344177,
+      "learning_rate": 0.00021590632851289967,
+      "loss": 0.4181404113769531,
+      "mean_token_accuracy": 0.8726993151009083,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.4986876940657926,
+      "eval_loss": 0.547695517539978,
+      "eval_mean_token_accuracy": 0.8501384708770486,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.3838,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 900
+    },
+    {
+      "entropy": 0.4751896943897009,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.81158047914505,
+      "learning_rate": 0.00021506901138115678,
+      "loss": 0.40689678192138673,
+      "mean_token_accuracy": 0.8745221219956875,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.507153491121392,
+      "eval_loss": 0.5501641631126404,
+      "eval_mean_token_accuracy": 0.8495670116918032,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.0912,
+      "eval_samples_per_second": 15.971,
+      "eval_steps_per_second": 1.998,
+      "step": 920
+    },
+    {
+      "entropy": 0.4873133715242147,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7218056321144104,
+      "learning_rate": 0.0002142009537679292,
+      "loss": 0.42701358795166017,
+      "mean_token_accuracy": 0.8695114746689796,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5202612736543943,
+      "eval_loss": 0.5491839051246643,
+      "eval_mean_token_accuracy": 0.8494071208460386,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.1142,
+      "eval_samples_per_second": 15.967,
+      "eval_steps_per_second": 1.997,
+      "step": 940
+    },
+    {
+      "entropy": 0.4762951169162989,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 0.7194424867630005,
+      "learning_rate": 0.0002133024174675534,
+      "loss": 0.42299847602844237,
+      "mean_token_accuracy": 0.8709790132939815,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4899340462546016,
+      "eval_loss": 0.5522511601448059,
+      "eval_mean_token_accuracy": 0.8492208258357159,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.463,
+      "eval_samples_per_second": 15.903,
+      "eval_steps_per_second": 1.989,
+      "step": 960
+    },
+    {
+      "entropy": 0.49650347977876663,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.8406022787094116,
+      "learning_rate": 0.0002123736734663221,
+      "loss": 0.4275330066680908,
+      "mean_token_accuracy": 0.8670595556497573,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.49691385654515996,
+      "eval_loss": 0.5491269826889038,
+      "eval_mean_token_accuracy": 0.850309816210769,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.17,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.996,
+      "step": 980
+    },
+    {
+      "entropy": 0.48843890577554705,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.9082473516464233,
+      "learning_rate": 0.00021141500186075868,
+      "loss": 0.4309722423553467,
+      "mean_token_accuracy": 0.8686766296625137,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5543508351195691,
+      "eval_loss": 0.5478800535202026,
+      "eval_mean_token_accuracy": 0.8478029522784921,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.3835,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.859037950771814e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/trainer_state.json

@@ -0,0 +1,1105 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5379825653798256,
+  "eval_steps": 20,
+  "global_step": 1020,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    },
+    {
+      "entropy": 0.6744543805718421,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.932099461555481,
+      "learning_rate": 6.698322232264434e-05,
+      "loss": 0.5991750717163086,
+      "mean_token_accuracy": 0.8304223112761975,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.6813044282932614,
+      "eval_loss": 0.5922021269798279,
+      "eval_mean_token_accuracy": 0.8346439617317777,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.1551,
+      "eval_samples_per_second": 15.96,
+      "eval_steps_per_second": 1.996,
+      "step": 120
+    },
+    {
+      "entropy": 0.6663189359009266,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9528499841690063,
+      "learning_rate": 7.824090674661818e-05,
+      "loss": 0.5891091346740722,
+      "mean_token_accuracy": 0.832152470946312,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6398407208711602,
+      "eval_loss": 0.5859636664390564,
+      "eval_mean_token_accuracy": 0.8372074996316156,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.2706,
+      "eval_samples_per_second": 15.938,
+      "eval_steps_per_second": 1.994,
+      "step": 140
+    },
+    {
+      "entropy": 0.64859763905406,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8468204140663147,
+      "learning_rate": 8.949859117059201e-05,
+      "loss": 0.569426441192627,
+      "mean_token_accuracy": 0.8401990942656994,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6381674285891444,
+      "eval_loss": 0.5744525790214539,
+      "eval_mean_token_accuracy": 0.838626817908398,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.2848,
+      "eval_samples_per_second": 15.936,
+      "eval_steps_per_second": 1.993,
+      "step": 160
+    },
+    {
+      "entropy": 0.6432608783245086,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8765804767608643,
+      "learning_rate": 0.00010075627559456587,
+      "loss": 0.5687318801879883,
+      "mean_token_accuracy": 0.839249350130558,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6047098288355872,
+      "eval_loss": 0.5679298043251038,
+      "eval_mean_token_accuracy": 0.8410577181466791,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.5879,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.986,
+      "step": 180
+    },
+    {
+      "entropy": 0.6440276011824608,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9576020240783691,
+      "learning_rate": 0.00011201396001853971,
+      "loss": 0.5828506469726562,
+      "mean_token_accuracy": 0.837553184479475,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6161119360909906,
+      "eval_loss": 0.5702911615371704,
+      "eval_mean_token_accuracy": 0.8407089398350827,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3341,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 200
+    },
+    {
+      "entropy": 0.6377195850014686,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7212373614311218,
+      "learning_rate": 0.00012327164444251353,
+      "loss": 0.5702451229095459,
+      "mean_token_accuracy": 0.8397969007492065,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6080108886194784,
+      "eval_loss": 0.5633499622344971,
+      "eval_mean_token_accuracy": 0.8396634854549585,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.4945,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 220
+    },
+    {
+      "entropy": 0.6287345830351114,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.848779022693634,
+      "learning_rate": 0.00013452932886648739,
+      "loss": 0.5506546020507812,
+      "mean_token_accuracy": 0.8438881888985634,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6307531505130058,
+      "eval_loss": 0.5573338270187378,
+      "eval_mean_token_accuracy": 0.8431362606758295,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.3535,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.992,
+      "step": 240
+    },
+    {
+      "entropy": 0.6223786748945713,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.7316951751708984,
+      "learning_rate": 0.0001457870132904612,
+      "loss": 0.5495625972747803,
+      "mean_token_accuracy": 0.8440376669168472,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.623454462476941,
+      "eval_loss": 0.5619264245033264,
+      "eval_mean_token_accuracy": 0.8431175777385401,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.2008,
+      "eval_samples_per_second": 15.951,
+      "eval_steps_per_second": 1.995,
+      "step": 260
+    },
+    {
+      "entropy": 0.6281675305217505,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7639564871788025,
+      "learning_rate": 0.00015704469771443506,
+      "loss": 0.5604369163513183,
+      "mean_token_accuracy": 0.8401600055396556,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.63416675980701,
+      "eval_loss": 0.5612760782241821,
+      "eval_mean_token_accuracy": 0.842435666294985,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.25,
+      "eval_samples_per_second": 15.942,
+      "eval_steps_per_second": 1.994,
+      "step": 280
+    },
+    {
+      "entropy": 0.6427909277379513,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6475813388824463,
+      "learning_rate": 0.0001683023821384089,
+      "loss": 0.573763370513916,
+      "mean_token_accuracy": 0.8370340794324875,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6231539840268534,
+      "eval_loss": 0.5566866397857666,
+      "eval_mean_token_accuracy": 0.844177934319474,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.4858,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.989,
+      "step": 300
+    },
+    {
+      "entropy": 0.6226776849478484,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.8886699676513672,
+      "learning_rate": 0.00017956006656238274,
+      "loss": 0.558210802078247,
+      "mean_token_accuracy": 0.84083157107234,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6066981683983359,
+      "eval_loss": 0.5585207939147949,
+      "eval_mean_token_accuracy": 0.8423153311014175,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.3463,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 320
+    },
+    {
+      "entropy": 0.6249004438519478,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8791211843490601,
+      "learning_rate": 0.00019081775098635657,
+      "loss": 0.5603597164154053,
+      "mean_token_accuracy": 0.8420463085174561,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6082247584018596,
+      "eval_loss": 0.5616299510002136,
+      "eval_mean_token_accuracy": 0.8431286801432454,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.1253,
+      "eval_samples_per_second": 15.965,
+      "eval_steps_per_second": 1.997,
+      "step": 340
+    },
+    {
+      "entropy": 0.6362396612763405,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.8606319427490234,
+      "learning_rate": 0.0002020754354103304,
+      "loss": 0.5735773563385009,
+      "mean_token_accuracy": 0.8371490836143494,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6492362072648004,
+      "eval_loss": 0.5646467804908752,
+      "eval_mean_token_accuracy": 0.8415517574825953,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.3351,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 360
+    },
+    {
+      "entropy": 0.638665035739541,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.7773950099945068,
+      "learning_rate": 0.00021333311983430425,
+      "loss": 0.5820859909057617,
+      "mean_token_accuracy": 0.8372561208903789,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6434498637221581,
+      "eval_loss": 0.5645168423652649,
+      "eval_mean_token_accuracy": 0.8420382481674815,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.1216,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 380
+    },
+    {
+      "entropy": 0.6316851265728474,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 1.6120579242706299,
+      "learning_rate": 0.00022459080425827807,
+      "loss": 0.5637502670288086,
+      "mean_token_accuracy": 0.8386227294802666,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6469012776086497,
+      "eval_loss": 0.5758090615272522,
+      "eval_mean_token_accuracy": 0.8397158470957778,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.6139,
+      "eval_samples_per_second": 15.875,
+      "eval_steps_per_second": 1.986,
+      "step": 400
+    },
+    {
+      "entropy": 0.5894816922835815,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 1.1616325378417969,
+      "learning_rate": 0.00022626713048053178,
+      "loss": 0.5316025257110596,
+      "mean_token_accuracy": 0.8466163017810919,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5860798164855602,
+      "eval_loss": 0.5777581930160522,
+      "eval_mean_token_accuracy": 0.8396938103576039,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.1449,
+      "eval_samples_per_second": 15.961,
+      "eval_steps_per_second": 1.997,
+      "step": 420
+    },
+    {
+      "entropy": 0.5818420693278312,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.7999453544616699,
+      "learning_rate": 0.00022622107023288778,
+      "loss": 0.5221010208129883,
+      "mean_token_accuracy": 0.8474301159381866,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5783926014636838,
+      "eval_loss": 0.5700300931930542,
+      "eval_mean_token_accuracy": 0.8430753537388735,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.5308,
+      "eval_samples_per_second": 15.89,
+      "eval_steps_per_second": 1.988,
+      "step": 440
+    },
+    {
+      "entropy": 0.5612493887543678,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 1.015687346458435,
+      "learning_rate": 0.00022614090619491568,
+      "loss": 0.5084867000579834,
+      "mean_token_accuracy": 0.8495561093091964,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5841563874205877,
+      "eval_loss": 0.5693665742874146,
+      "eval_mean_token_accuracy": 0.8427817298229351,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.5256,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.988,
+      "step": 460
+    },
+    {
+      "entropy": 0.5828216474503278,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 1.9750930070877075,
+      "learning_rate": 0.00022602666254299594,
+      "loss": 0.5180017948150635,
+      "mean_token_accuracy": 0.8515685826539994,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5806607044366903,
+      "eval_loss": 0.5804352760314941,
+      "eval_mean_token_accuracy": 0.8413014668364858,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.1199,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 480
+    },
+    {
+      "entropy": 0.5926914308220148,
+      "epoch": 1.244084682440847,
+      "grad_norm": 0.8917353749275208,
+      "learning_rate": 0.0002258783737314558,
+      "loss": 0.528910779953003,
+      "mean_token_accuracy": 0.8486074328422546,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5593361884009006,
+      "eval_loss": 0.5675153732299805,
+      "eval_mean_token_accuracy": 0.8433507802181466,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7289,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5865630559623242,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7482362985610962,
+      "learning_rate": 0.00022569608448217823,
+      "loss": 0.5250466823577881,
+      "mean_token_accuracy": 0.8477916084229946,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.543057840230853,
+      "eval_loss": 0.5671008229255676,
+      "eval_mean_token_accuracy": 0.8428726016088973,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.3403,
+      "eval_samples_per_second": 15.925,
+      "eval_steps_per_second": 1.992,
+      "step": 520
+    },
+    {
+      "entropy": 0.5870206747204065,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9473814964294434,
+      "learning_rate": 0.00022547984977111448,
+      "loss": 0.5252370834350586,
+      "mean_token_accuracy": 0.8468369916081429,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.590982622878496,
+      "eval_loss": 0.5676343441009521,
+      "eval_mean_token_accuracy": 0.8429348746011424,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.5168,
+      "eval_samples_per_second": 15.893,
+      "eval_steps_per_second": 1.988,
+      "step": 540
+    },
+    {
+      "entropy": 0.5785854265093804,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.9353351593017578,
+      "learning_rate": 0.0002252297348117042,
+      "loss": 0.5304938316345215,
+      "mean_token_accuracy": 0.8463383808732032,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6099918867612995,
+      "eval_loss": 0.5620437860488892,
+      "eval_mean_token_accuracy": 0.8430728347495545,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.7741,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.982,
+      "step": 560
+    },
+    {
+      "entropy": 0.5768801040947438,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.9198738932609558,
+      "learning_rate": 0.0002249458150352077,
+      "loss": 0.520513391494751,
+      "mean_token_accuracy": 0.8487689301371575,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.6349420670506566,
+      "eval_loss": 0.5645340085029602,
+      "eval_mean_token_accuracy": 0.8447844597489335,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.3257,
+      "eval_samples_per_second": 15.928,
+      "eval_steps_per_second": 1.992,
+      "step": 580
+    },
+    {
+      "entropy": 0.5822233572602272,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.832811176776886,
+      "learning_rate": 0.0002246281760679571,
+      "loss": 0.5295282363891601,
+      "mean_token_accuracy": 0.8504064798355102,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5829724387027496,
+      "eval_loss": 0.5612193942070007,
+      "eval_mean_token_accuracy": 0.8449643853791925,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6617,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.571855777129531,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.7665547728538513,
+      "learning_rate": 0.00022427691370553263,
+      "loss": 0.5187931060791016,
+      "mean_token_accuracy": 0.8534420043230057,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5623592240519302,
+      "eval_loss": 0.5575760006904602,
+      "eval_mean_token_accuracy": 0.8468210229346919,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.6324,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.985,
+      "step": 620
+    },
+    {
+      "entropy": 0.5740394659340382,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6500429511070251,
+      "learning_rate": 0.00022389213388387174,
+      "loss": 0.5283198833465577,
+      "mean_token_accuracy": 0.8502798482775689,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5548852207355721,
+      "eval_loss": 0.5561797022819519,
+      "eval_mean_token_accuracy": 0.8452786498291548,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.5205,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 640
+    },
+    {
+      "entropy": 0.6020145989954472,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.7056867480278015,
+      "learning_rate": 0.00022347395264732053,
+      "loss": 0.5400049209594726,
+      "mean_token_accuracy": 0.8447613954544068,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5618055154417836,
+      "eval_loss": 0.556106686592102,
+      "eval_mean_token_accuracy": 0.8465680112672407,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.2971,
+      "eval_samples_per_second": 15.933,
+      "eval_steps_per_second": 1.993,
+      "step": 660
+    },
+    {
+      "entropy": 0.5665927153080702,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.5987663865089417,
+      "learning_rate": 0.00022302249611363625,
+      "loss": 0.5143643856048584,
+      "mean_token_accuracy": 0.8529589556157589,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.568248552118623,
+      "eval_loss": 0.5476346015930176,
+      "eval_mean_token_accuracy": 0.8476775434128073,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.9583,
+      "eval_samples_per_second": 15.812,
+      "eval_steps_per_second": 1.978,
+      "step": 680
+    },
+    {
+      "entropy": 0.5673687808215618,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.735261857509613,
+      "learning_rate": 0.00022253790043595193,
+      "loss": 0.509885597229004,
+      "mean_token_accuracy": 0.8537046857178211,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5616967284748721,
+      "eval_loss": 0.5439274311065674,
+      "eval_mean_token_accuracy": 0.8488946217437123,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.0604,
+      "eval_samples_per_second": 15.977,
+      "eval_steps_per_second": 1.999,
+      "step": 700
+    },
+    {
+      "entropy": 0.5529541682451964,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7014835476875305,
+      "learning_rate": 0.00022202031176171442,
+      "loss": 0.5078992366790771,
+      "mean_token_accuracy": 0.8525233261287213,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5827173320359962,
+      "eval_loss": 0.5419450402259827,
+      "eval_mean_token_accuracy": 0.8477318609176681,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 85.2984,
+      "eval_samples_per_second": 16.12,
+      "eval_steps_per_second": 2.016,
+      "step": 720
+    },
+    {
+      "entropy": 0.5755720350891351,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.705613911151886,
+      "learning_rate": 0.00022146988618860824,
+      "loss": 0.5181350708007812,
+      "mean_token_accuracy": 0.8467609457671642,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5743971356125765,
+      "eval_loss": 0.5415896773338318,
+      "eval_mean_token_accuracy": 0.847328585940738,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 85.5602,
+      "eval_samples_per_second": 16.071,
+      "eval_steps_per_second": 2.01,
+      "step": 740
+    },
+    {
+      "entropy": 0.561330484598875,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.6722865700721741,
+      "learning_rate": 0.0002208867897174789,
+      "loss": 0.499837589263916,
+      "mean_token_accuracy": 0.8518734864890576,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.5865232653396074,
+      "eval_loss": 0.5437926650047302,
+      "eval_mean_token_accuracy": 0.8450997017843779,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4116,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.547389242425561,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.7935577034950256,
+      "learning_rate": 0.00022027119820226907,
+      "loss": 0.4977591514587402,
+      "mean_token_accuracy": 0.8539491161704064,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.5290903090391048,
+      "eval_loss": 0.5409526824951172,
+      "eval_mean_token_accuracy": 0.8497545698354411,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.7262,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 780
+    },
+    {
+      "entropy": 0.5687909748405218,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6180546283721924,
+      "learning_rate": 0.00021962329729698345,
+      "loss": 0.5109643459320068,
+      "mean_token_accuracy": 0.8521598495543004,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.5503541858390321,
+      "eval_loss": 0.5361555218696594,
+      "eval_mean_token_accuracy": 0.8510884285666221,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.3339,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 800
+    },
+    {
+      "entropy": 0.4739728841261986,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.8058829307556152,
+      "learning_rate": 0.0002189432823996982,
+      "loss": 0.4204097747802734,
+      "mean_token_accuracy": 0.8728981889211215,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.5077334992414297,
+      "eval_loss": 0.5531114339828491,
+      "eval_mean_token_accuracy": 0.8489257208136625,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.4801,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.989,
+      "step": 820
+    },
+    {
+      "entropy": 0.4594309840351343,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.6906896829605103,
+      "learning_rate": 0.0002182313585936314,
+      "loss": 0.4071959495544434,
+      "mean_token_accuracy": 0.8732857562601566,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.49850136994622474,
+      "eval_loss": 0.5486204624176025,
+      "eval_mean_token_accuracy": 0.8507991450470548,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.3364,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.4881629109382629,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6343470215797424,
+      "learning_rate": 0.0002174877405852928,
+      "loss": 0.41669540405273436,
+      "mean_token_accuracy": 0.8711295068264008,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.49155513924914734,
+      "eval_loss": 0.555109441280365,
+      "eval_mean_token_accuracy": 0.8496399400539176,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 86.3295,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 860
+    },
+    {
+      "entropy": 0.4648668970912695,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.8014165163040161,
+      "learning_rate": 0.00021671265263973133,
+      "loss": 0.4110250473022461,
+      "mean_token_accuracy": 0.8754166305065155,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.4909258722219356,
+      "eval_loss": 0.5539511442184448,
+      "eval_mean_token_accuracy": 0.8492401502160138,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.3468,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 880
+    },
+    {
+      "entropy": 0.4824485514312983,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6665191054344177,
+      "learning_rate": 0.00021590632851289967,
+      "loss": 0.4181404113769531,
+      "mean_token_accuracy": 0.8726993151009083,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.4986876940657926,
+      "eval_loss": 0.547695517539978,
+      "eval_mean_token_accuracy": 0.8501384708770486,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.3838,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 900
+    },
+    {
+      "entropy": 0.4751896943897009,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.81158047914505,
+      "learning_rate": 0.00021506901138115678,
+      "loss": 0.40689678192138673,
+      "mean_token_accuracy": 0.8745221219956875,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.507153491121392,
+      "eval_loss": 0.5501641631126404,
+      "eval_mean_token_accuracy": 0.8495670116918032,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.0912,
+      "eval_samples_per_second": 15.971,
+      "eval_steps_per_second": 1.998,
+      "step": 920
+    },
+    {
+      "entropy": 0.4873133715242147,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7218056321144104,
+      "learning_rate": 0.0002142009537679292,
+      "loss": 0.42701358795166017,
+      "mean_token_accuracy": 0.8695114746689796,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5202612736543943,
+      "eval_loss": 0.5491839051246643,
+      "eval_mean_token_accuracy": 0.8494071208460386,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.1142,
+      "eval_samples_per_second": 15.967,
+      "eval_steps_per_second": 1.997,
+      "step": 940
+    },
+    {
+      "entropy": 0.4762951169162989,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 0.7194424867630005,
+      "learning_rate": 0.0002133024174675534,
+      "loss": 0.42299847602844237,
+      "mean_token_accuracy": 0.8709790132939815,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4899340462546016,
+      "eval_loss": 0.5522511601448059,
+      "eval_mean_token_accuracy": 0.8492208258357159,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.463,
+      "eval_samples_per_second": 15.903,
+      "eval_steps_per_second": 1.989,
+      "step": 960
+    },
+    {
+      "entropy": 0.49650347977876663,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.8406022787094116,
+      "learning_rate": 0.0002123736734663221,
+      "loss": 0.4275330066680908,
+      "mean_token_accuracy": 0.8670595556497573,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.49691385654515996,
+      "eval_loss": 0.5491269826889038,
+      "eval_mean_token_accuracy": 0.850309816210769,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.17,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.996,
+      "step": 980
+    },
+    {
+      "entropy": 0.48843890577554705,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.9082473516464233,
+      "learning_rate": 0.00021141500186075868,
+      "loss": 0.4309722423553467,
+      "mean_token_accuracy": 0.8686766296625137,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5543508351195691,
+      "eval_loss": 0.5478800535202026,
+      "eval_mean_token_accuracy": 0.8478029522784921,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.3835,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4777219031006098,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7448089122772217,
+      "learning_rate": 0.0002104266917731438,
+      "loss": 0.423325252532959,
+      "mean_token_accuracy": 0.8706337086856365,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.49857561550168106,
+      "eval_loss": 0.5511948466300964,
+      "eval_mean_token_accuracy": 0.8502220289651737,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.5399,
+      "eval_samples_per_second": 15.889,
+      "eval_steps_per_second": 1.988,
+      "step": 1020
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0076952699436032e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/trainer_state.json

@@ -0,0 +1,1126 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.587795765877958,
+  "eval_steps": 20,
+  "global_step": 1040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    },
+    {
+      "entropy": 0.6744543805718421,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.932099461555481,
+      "learning_rate": 6.698322232264434e-05,
+      "loss": 0.5991750717163086,
+      "mean_token_accuracy": 0.8304223112761975,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.6813044282932614,
+      "eval_loss": 0.5922021269798279,
+      "eval_mean_token_accuracy": 0.8346439617317777,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.1551,
+      "eval_samples_per_second": 15.96,
+      "eval_steps_per_second": 1.996,
+      "step": 120
+    },
+    {
+      "entropy": 0.6663189359009266,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9528499841690063,
+      "learning_rate": 7.824090674661818e-05,
+      "loss": 0.5891091346740722,
+      "mean_token_accuracy": 0.832152470946312,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6398407208711602,
+      "eval_loss": 0.5859636664390564,
+      "eval_mean_token_accuracy": 0.8372074996316156,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.2706,
+      "eval_samples_per_second": 15.938,
+      "eval_steps_per_second": 1.994,
+      "step": 140
+    },
+    {
+      "entropy": 0.64859763905406,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8468204140663147,
+      "learning_rate": 8.949859117059201e-05,
+      "loss": 0.569426441192627,
+      "mean_token_accuracy": 0.8401990942656994,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6381674285891444,
+      "eval_loss": 0.5744525790214539,
+      "eval_mean_token_accuracy": 0.838626817908398,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.2848,
+      "eval_samples_per_second": 15.936,
+      "eval_steps_per_second": 1.993,
+      "step": 160
+    },
+    {
+      "entropy": 0.6432608783245086,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8765804767608643,
+      "learning_rate": 0.00010075627559456587,
+      "loss": 0.5687318801879883,
+      "mean_token_accuracy": 0.839249350130558,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6047098288355872,
+      "eval_loss": 0.5679298043251038,
+      "eval_mean_token_accuracy": 0.8410577181466791,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.5879,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.986,
+      "step": 180
+    },
+    {
+      "entropy": 0.6440276011824608,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9576020240783691,
+      "learning_rate": 0.00011201396001853971,
+      "loss": 0.5828506469726562,
+      "mean_token_accuracy": 0.837553184479475,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6161119360909906,
+      "eval_loss": 0.5702911615371704,
+      "eval_mean_token_accuracy": 0.8407089398350827,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3341,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 200
+    },
+    {
+      "entropy": 0.6377195850014686,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7212373614311218,
+      "learning_rate": 0.00012327164444251353,
+      "loss": 0.5702451229095459,
+      "mean_token_accuracy": 0.8397969007492065,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6080108886194784,
+      "eval_loss": 0.5633499622344971,
+      "eval_mean_token_accuracy": 0.8396634854549585,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.4945,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 220
+    },
+    {
+      "entropy": 0.6287345830351114,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.848779022693634,
+      "learning_rate": 0.00013452932886648739,
+      "loss": 0.5506546020507812,
+      "mean_token_accuracy": 0.8438881888985634,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6307531505130058,
+      "eval_loss": 0.5573338270187378,
+      "eval_mean_token_accuracy": 0.8431362606758295,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.3535,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.992,
+      "step": 240
+    },
+    {
+      "entropy": 0.6223786748945713,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.7316951751708984,
+      "learning_rate": 0.0001457870132904612,
+      "loss": 0.5495625972747803,
+      "mean_token_accuracy": 0.8440376669168472,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.623454462476941,
+      "eval_loss": 0.5619264245033264,
+      "eval_mean_token_accuracy": 0.8431175777385401,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.2008,
+      "eval_samples_per_second": 15.951,
+      "eval_steps_per_second": 1.995,
+      "step": 260
+    },
+    {
+      "entropy": 0.6281675305217505,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7639564871788025,
+      "learning_rate": 0.00015704469771443506,
+      "loss": 0.5604369163513183,
+      "mean_token_accuracy": 0.8401600055396556,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.63416675980701,
+      "eval_loss": 0.5612760782241821,
+      "eval_mean_token_accuracy": 0.842435666294985,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.25,
+      "eval_samples_per_second": 15.942,
+      "eval_steps_per_second": 1.994,
+      "step": 280
+    },
+    {
+      "entropy": 0.6427909277379513,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6475813388824463,
+      "learning_rate": 0.0001683023821384089,
+      "loss": 0.573763370513916,
+      "mean_token_accuracy": 0.8370340794324875,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6231539840268534,
+      "eval_loss": 0.5566866397857666,
+      "eval_mean_token_accuracy": 0.844177934319474,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.4858,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.989,
+      "step": 300
+    },
+    {
+      "entropy": 0.6226776849478484,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.8886699676513672,
+      "learning_rate": 0.00017956006656238274,
+      "loss": 0.558210802078247,
+      "mean_token_accuracy": 0.84083157107234,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6066981683983359,
+      "eval_loss": 0.5585207939147949,
+      "eval_mean_token_accuracy": 0.8423153311014175,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.3463,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 320
+    },
+    {
+      "entropy": 0.6249004438519478,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8791211843490601,
+      "learning_rate": 0.00019081775098635657,
+      "loss": 0.5603597164154053,
+      "mean_token_accuracy": 0.8420463085174561,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6082247584018596,
+      "eval_loss": 0.5616299510002136,
+      "eval_mean_token_accuracy": 0.8431286801432454,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.1253,
+      "eval_samples_per_second": 15.965,
+      "eval_steps_per_second": 1.997,
+      "step": 340
+    },
+    {
+      "entropy": 0.6362396612763405,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.8606319427490234,
+      "learning_rate": 0.0002020754354103304,
+      "loss": 0.5735773563385009,
+      "mean_token_accuracy": 0.8371490836143494,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6492362072648004,
+      "eval_loss": 0.5646467804908752,
+      "eval_mean_token_accuracy": 0.8415517574825953,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.3351,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 360
+    },
+    {
+      "entropy": 0.638665035739541,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.7773950099945068,
+      "learning_rate": 0.00021333311983430425,
+      "loss": 0.5820859909057617,
+      "mean_token_accuracy": 0.8372561208903789,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6434498637221581,
+      "eval_loss": 0.5645168423652649,
+      "eval_mean_token_accuracy": 0.8420382481674815,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.1216,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 380
+    },
+    {
+      "entropy": 0.6316851265728474,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 1.6120579242706299,
+      "learning_rate": 0.00022459080425827807,
+      "loss": 0.5637502670288086,
+      "mean_token_accuracy": 0.8386227294802666,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6469012776086497,
+      "eval_loss": 0.5758090615272522,
+      "eval_mean_token_accuracy": 0.8397158470957778,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.6139,
+      "eval_samples_per_second": 15.875,
+      "eval_steps_per_second": 1.986,
+      "step": 400
+    },
+    {
+      "entropy": 0.5894816922835815,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 1.1616325378417969,
+      "learning_rate": 0.00022626713048053178,
+      "loss": 0.5316025257110596,
+      "mean_token_accuracy": 0.8466163017810919,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5860798164855602,
+      "eval_loss": 0.5777581930160522,
+      "eval_mean_token_accuracy": 0.8396938103576039,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.1449,
+      "eval_samples_per_second": 15.961,
+      "eval_steps_per_second": 1.997,
+      "step": 420
+    },
+    {
+      "entropy": 0.5818420693278312,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.7999453544616699,
+      "learning_rate": 0.00022622107023288778,
+      "loss": 0.5221010208129883,
+      "mean_token_accuracy": 0.8474301159381866,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5783926014636838,
+      "eval_loss": 0.5700300931930542,
+      "eval_mean_token_accuracy": 0.8430753537388735,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.5308,
+      "eval_samples_per_second": 15.89,
+      "eval_steps_per_second": 1.988,
+      "step": 440
+    },
+    {
+      "entropy": 0.5612493887543678,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 1.015687346458435,
+      "learning_rate": 0.00022614090619491568,
+      "loss": 0.5084867000579834,
+      "mean_token_accuracy": 0.8495561093091964,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5841563874205877,
+      "eval_loss": 0.5693665742874146,
+      "eval_mean_token_accuracy": 0.8427817298229351,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.5256,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.988,
+      "step": 460
+    },
+    {
+      "entropy": 0.5828216474503278,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 1.9750930070877075,
+      "learning_rate": 0.00022602666254299594,
+      "loss": 0.5180017948150635,
+      "mean_token_accuracy": 0.8515685826539994,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5806607044366903,
+      "eval_loss": 0.5804352760314941,
+      "eval_mean_token_accuracy": 0.8413014668364858,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.1199,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 480
+    },
+    {
+      "entropy": 0.5926914308220148,
+      "epoch": 1.244084682440847,
+      "grad_norm": 0.8917353749275208,
+      "learning_rate": 0.0002258783737314558,
+      "loss": 0.528910779953003,
+      "mean_token_accuracy": 0.8486074328422546,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5593361884009006,
+      "eval_loss": 0.5675153732299805,
+      "eval_mean_token_accuracy": 0.8433507802181466,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7289,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5865630559623242,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7482362985610962,
+      "learning_rate": 0.00022569608448217823,
+      "loss": 0.5250466823577881,
+      "mean_token_accuracy": 0.8477916084229946,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.543057840230853,
+      "eval_loss": 0.5671008229255676,
+      "eval_mean_token_accuracy": 0.8428726016088973,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.3403,
+      "eval_samples_per_second": 15.925,
+      "eval_steps_per_second": 1.992,
+      "step": 520
+    },
+    {
+      "entropy": 0.5870206747204065,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9473814964294434,
+      "learning_rate": 0.00022547984977111448,
+      "loss": 0.5252370834350586,
+      "mean_token_accuracy": 0.8468369916081429,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.590982622878496,
+      "eval_loss": 0.5676343441009521,
+      "eval_mean_token_accuracy": 0.8429348746011424,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.5168,
+      "eval_samples_per_second": 15.893,
+      "eval_steps_per_second": 1.988,
+      "step": 540
+    },
+    {
+      "entropy": 0.5785854265093804,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.9353351593017578,
+      "learning_rate": 0.0002252297348117042,
+      "loss": 0.5304938316345215,
+      "mean_token_accuracy": 0.8463383808732032,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6099918867612995,
+      "eval_loss": 0.5620437860488892,
+      "eval_mean_token_accuracy": 0.8430728347495545,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.7741,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.982,
+      "step": 560
+    },
+    {
+      "entropy": 0.5768801040947438,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.9198738932609558,
+      "learning_rate": 0.0002249458150352077,
+      "loss": 0.520513391494751,
+      "mean_token_accuracy": 0.8487689301371575,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.6349420670506566,
+      "eval_loss": 0.5645340085029602,
+      "eval_mean_token_accuracy": 0.8447844597489335,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.3257,
+      "eval_samples_per_second": 15.928,
+      "eval_steps_per_second": 1.992,
+      "step": 580
+    },
+    {
+      "entropy": 0.5822233572602272,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.832811176776886,
+      "learning_rate": 0.0002246281760679571,
+      "loss": 0.5295282363891601,
+      "mean_token_accuracy": 0.8504064798355102,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5829724387027496,
+      "eval_loss": 0.5612193942070007,
+      "eval_mean_token_accuracy": 0.8449643853791925,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6617,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.571855777129531,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.7665547728538513,
+      "learning_rate": 0.00022427691370553263,
+      "loss": 0.5187931060791016,
+      "mean_token_accuracy": 0.8534420043230057,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5623592240519302,
+      "eval_loss": 0.5575760006904602,
+      "eval_mean_token_accuracy": 0.8468210229346919,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.6324,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.985,
+      "step": 620
+    },
+    {
+      "entropy": 0.5740394659340382,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6500429511070251,
+      "learning_rate": 0.00022389213388387174,
+      "loss": 0.5283198833465577,
+      "mean_token_accuracy": 0.8502798482775689,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5548852207355721,
+      "eval_loss": 0.5561797022819519,
+      "eval_mean_token_accuracy": 0.8452786498291548,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.5205,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 640
+    },
+    {
+      "entropy": 0.6020145989954472,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.7056867480278015,
+      "learning_rate": 0.00022347395264732053,
+      "loss": 0.5400049209594726,
+      "mean_token_accuracy": 0.8447613954544068,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5618055154417836,
+      "eval_loss": 0.556106686592102,
+      "eval_mean_token_accuracy": 0.8465680112672407,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.2971,
+      "eval_samples_per_second": 15.933,
+      "eval_steps_per_second": 1.993,
+      "step": 660
+    },
+    {
+      "entropy": 0.5665927153080702,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.5987663865089417,
+      "learning_rate": 0.00022302249611363625,
+      "loss": 0.5143643856048584,
+      "mean_token_accuracy": 0.8529589556157589,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.568248552118623,
+      "eval_loss": 0.5476346015930176,
+      "eval_mean_token_accuracy": 0.8476775434128073,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.9583,
+      "eval_samples_per_second": 15.812,
+      "eval_steps_per_second": 1.978,
+      "step": 680
+    },
+    {
+      "entropy": 0.5673687808215618,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.735261857509613,
+      "learning_rate": 0.00022253790043595193,
+      "loss": 0.509885597229004,
+      "mean_token_accuracy": 0.8537046857178211,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5616967284748721,
+      "eval_loss": 0.5439274311065674,
+      "eval_mean_token_accuracy": 0.8488946217437123,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.0604,
+      "eval_samples_per_second": 15.977,
+      "eval_steps_per_second": 1.999,
+      "step": 700
+    },
+    {
+      "entropy": 0.5529541682451964,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7014835476875305,
+      "learning_rate": 0.00022202031176171442,
+      "loss": 0.5078992366790771,
+      "mean_token_accuracy": 0.8525233261287213,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5827173320359962,
+      "eval_loss": 0.5419450402259827,
+      "eval_mean_token_accuracy": 0.8477318609176681,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 85.2984,
+      "eval_samples_per_second": 16.12,
+      "eval_steps_per_second": 2.016,
+      "step": 720
+    },
+    {
+      "entropy": 0.5755720350891351,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.705613911151886,
+      "learning_rate": 0.00022146988618860824,
+      "loss": 0.5181350708007812,
+      "mean_token_accuracy": 0.8467609457671642,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5743971356125765,
+      "eval_loss": 0.5415896773338318,
+      "eval_mean_token_accuracy": 0.847328585940738,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 85.5602,
+      "eval_samples_per_second": 16.071,
+      "eval_steps_per_second": 2.01,
+      "step": 740
+    },
+    {
+      "entropy": 0.561330484598875,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.6722865700721741,
+      "learning_rate": 0.0002208867897174789,
+      "loss": 0.499837589263916,
+      "mean_token_accuracy": 0.8518734864890576,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.5865232653396074,
+      "eval_loss": 0.5437926650047302,
+      "eval_mean_token_accuracy": 0.8450997017843779,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4116,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.547389242425561,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.7935577034950256,
+      "learning_rate": 0.00022027119820226907,
+      "loss": 0.4977591514587402,
+      "mean_token_accuracy": 0.8539491161704064,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.5290903090391048,
+      "eval_loss": 0.5409526824951172,
+      "eval_mean_token_accuracy": 0.8497545698354411,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.7262,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 780
+    },
+    {
+      "entropy": 0.5687909748405218,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6180546283721924,
+      "learning_rate": 0.00021962329729698345,
+      "loss": 0.5109643459320068,
+      "mean_token_accuracy": 0.8521598495543004,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.5503541858390321,
+      "eval_loss": 0.5361555218696594,
+      "eval_mean_token_accuracy": 0.8510884285666221,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.3339,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 800
+    },
+    {
+      "entropy": 0.4739728841261986,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.8058829307556152,
+      "learning_rate": 0.0002189432823996982,
+      "loss": 0.4204097747802734,
+      "mean_token_accuracy": 0.8728981889211215,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.5077334992414297,
+      "eval_loss": 0.5531114339828491,
+      "eval_mean_token_accuracy": 0.8489257208136625,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.4801,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.989,
+      "step": 820
+    },
+    {
+      "entropy": 0.4594309840351343,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.6906896829605103,
+      "learning_rate": 0.0002182313585936314,
+      "loss": 0.4071959495544434,
+      "mean_token_accuracy": 0.8732857562601566,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.49850136994622474,
+      "eval_loss": 0.5486204624176025,
+      "eval_mean_token_accuracy": 0.8507991450470548,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.3364,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.4881629109382629,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6343470215797424,
+      "learning_rate": 0.0002174877405852928,
+      "loss": 0.41669540405273436,
+      "mean_token_accuracy": 0.8711295068264008,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.49155513924914734,
+      "eval_loss": 0.555109441280365,
+      "eval_mean_token_accuracy": 0.8496399400539176,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 86.3295,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 860
+    },
+    {
+      "entropy": 0.4648668970912695,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.8014165163040161,
+      "learning_rate": 0.00021671265263973133,
+      "loss": 0.4110250473022461,
+      "mean_token_accuracy": 0.8754166305065155,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.4909258722219356,
+      "eval_loss": 0.5539511442184448,
+      "eval_mean_token_accuracy": 0.8492401502160138,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.3468,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 880
+    },
+    {
+      "entropy": 0.4824485514312983,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6665191054344177,
+      "learning_rate": 0.00021590632851289967,
+      "loss": 0.4181404113769531,
+      "mean_token_accuracy": 0.8726993151009083,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.4986876940657926,
+      "eval_loss": 0.547695517539978,
+      "eval_mean_token_accuracy": 0.8501384708770486,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.3838,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 900
+    },
+    {
+      "entropy": 0.4751896943897009,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.81158047914505,
+      "learning_rate": 0.00021506901138115678,
+      "loss": 0.40689678192138673,
+      "mean_token_accuracy": 0.8745221219956875,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.507153491121392,
+      "eval_loss": 0.5501641631126404,
+      "eval_mean_token_accuracy": 0.8495670116918032,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.0912,
+      "eval_samples_per_second": 15.971,
+      "eval_steps_per_second": 1.998,
+      "step": 920
+    },
+    {
+      "entropy": 0.4873133715242147,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7218056321144104,
+      "learning_rate": 0.0002142009537679292,
+      "loss": 0.42701358795166017,
+      "mean_token_accuracy": 0.8695114746689796,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5202612736543943,
+      "eval_loss": 0.5491839051246643,
+      "eval_mean_token_accuracy": 0.8494071208460386,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.1142,
+      "eval_samples_per_second": 15.967,
+      "eval_steps_per_second": 1.997,
+      "step": 940
+    },
+    {
+      "entropy": 0.4762951169162989,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 0.7194424867630005,
+      "learning_rate": 0.0002133024174675534,
+      "loss": 0.42299847602844237,
+      "mean_token_accuracy": 0.8709790132939815,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4899340462546016,
+      "eval_loss": 0.5522511601448059,
+      "eval_mean_token_accuracy": 0.8492208258357159,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.463,
+      "eval_samples_per_second": 15.903,
+      "eval_steps_per_second": 1.989,
+      "step": 960
+    },
+    {
+      "entropy": 0.49650347977876663,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.8406022787094116,
+      "learning_rate": 0.0002123736734663221,
+      "loss": 0.4275330066680908,
+      "mean_token_accuracy": 0.8670595556497573,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.49691385654515996,
+      "eval_loss": 0.5491269826889038,
+      "eval_mean_token_accuracy": 0.850309816210769,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.17,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.996,
+      "step": 980
+    },
+    {
+      "entropy": 0.48843890577554705,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.9082473516464233,
+      "learning_rate": 0.00021141500186075868,
+      "loss": 0.4309722423553467,
+      "mean_token_accuracy": 0.8686766296625137,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5543508351195691,
+      "eval_loss": 0.5478800535202026,
+      "eval_mean_token_accuracy": 0.8478029522784921,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.3835,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4777219031006098,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7448089122772217,
+      "learning_rate": 0.0002104266917731438,
+      "loss": 0.423325252532959,
+      "mean_token_accuracy": 0.8706337086856365,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.49857561550168106,
+      "eval_loss": 0.5511948466300964,
+      "eval_mean_token_accuracy": 0.8502220289651737,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.5399,
+      "eval_samples_per_second": 15.889,
+      "eval_steps_per_second": 1.988,
+      "step": 1020
+    },
+    {
+      "entropy": 0.4844174191355705,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.794029176235199,
+      "learning_rate": 0.00020940904126432,
+      "loss": 0.4176753044128418,
+      "mean_token_accuracy": 0.873535567522049,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.485467542222766,
+      "eval_loss": 0.5539286732673645,
+      "eval_mean_token_accuracy": 0.8495475081510322,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.135,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 1.997,
+      "step": 1040
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0254458345271091e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/trainer_state.json

@@ -0,0 +1,1147 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6376089663760895,
+  "eval_steps": 20,
+  "global_step": 1060,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    },
+    {
+      "entropy": 0.6744543805718421,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.932099461555481,
+      "learning_rate": 6.698322232264434e-05,
+      "loss": 0.5991750717163086,
+      "mean_token_accuracy": 0.8304223112761975,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.6813044282932614,
+      "eval_loss": 0.5922021269798279,
+      "eval_mean_token_accuracy": 0.8346439617317777,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.1551,
+      "eval_samples_per_second": 15.96,
+      "eval_steps_per_second": 1.996,
+      "step": 120
+    },
+    {
+      "entropy": 0.6663189359009266,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9528499841690063,
+      "learning_rate": 7.824090674661818e-05,
+      "loss": 0.5891091346740722,
+      "mean_token_accuracy": 0.832152470946312,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6398407208711602,
+      "eval_loss": 0.5859636664390564,
+      "eval_mean_token_accuracy": 0.8372074996316156,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.2706,
+      "eval_samples_per_second": 15.938,
+      "eval_steps_per_second": 1.994,
+      "step": 140
+    },
+    {
+      "entropy": 0.64859763905406,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8468204140663147,
+      "learning_rate": 8.949859117059201e-05,
+      "loss": 0.569426441192627,
+      "mean_token_accuracy": 0.8401990942656994,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6381674285891444,
+      "eval_loss": 0.5744525790214539,
+      "eval_mean_token_accuracy": 0.838626817908398,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.2848,
+      "eval_samples_per_second": 15.936,
+      "eval_steps_per_second": 1.993,
+      "step": 160
+    },
+    {
+      "entropy": 0.6432608783245086,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8765804767608643,
+      "learning_rate": 0.00010075627559456587,
+      "loss": 0.5687318801879883,
+      "mean_token_accuracy": 0.839249350130558,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6047098288355872,
+      "eval_loss": 0.5679298043251038,
+      "eval_mean_token_accuracy": 0.8410577181466791,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.5879,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.986,
+      "step": 180
+    },
+    {
+      "entropy": 0.6440276011824608,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9576020240783691,
+      "learning_rate": 0.00011201396001853971,
+      "loss": 0.5828506469726562,
+      "mean_token_accuracy": 0.837553184479475,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6161119360909906,
+      "eval_loss": 0.5702911615371704,
+      "eval_mean_token_accuracy": 0.8407089398350827,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3341,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 200
+    },
+    {
+      "entropy": 0.6377195850014686,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7212373614311218,
+      "learning_rate": 0.00012327164444251353,
+      "loss": 0.5702451229095459,
+      "mean_token_accuracy": 0.8397969007492065,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6080108886194784,
+      "eval_loss": 0.5633499622344971,
+      "eval_mean_token_accuracy": 0.8396634854549585,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.4945,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 220
+    },
+    {
+      "entropy": 0.6287345830351114,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.848779022693634,
+      "learning_rate": 0.00013452932886648739,
+      "loss": 0.5506546020507812,
+      "mean_token_accuracy": 0.8438881888985634,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6307531505130058,
+      "eval_loss": 0.5573338270187378,
+      "eval_mean_token_accuracy": 0.8431362606758295,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.3535,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.992,
+      "step": 240
+    },
+    {
+      "entropy": 0.6223786748945713,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.7316951751708984,
+      "learning_rate": 0.0001457870132904612,
+      "loss": 0.5495625972747803,
+      "mean_token_accuracy": 0.8440376669168472,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.623454462476941,
+      "eval_loss": 0.5619264245033264,
+      "eval_mean_token_accuracy": 0.8431175777385401,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.2008,
+      "eval_samples_per_second": 15.951,
+      "eval_steps_per_second": 1.995,
+      "step": 260
+    },
+    {
+      "entropy": 0.6281675305217505,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7639564871788025,
+      "learning_rate": 0.00015704469771443506,
+      "loss": 0.5604369163513183,
+      "mean_token_accuracy": 0.8401600055396556,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.63416675980701,
+      "eval_loss": 0.5612760782241821,
+      "eval_mean_token_accuracy": 0.842435666294985,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.25,
+      "eval_samples_per_second": 15.942,
+      "eval_steps_per_second": 1.994,
+      "step": 280
+    },
+    {
+      "entropy": 0.6427909277379513,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6475813388824463,
+      "learning_rate": 0.0001683023821384089,
+      "loss": 0.573763370513916,
+      "mean_token_accuracy": 0.8370340794324875,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6231539840268534,
+      "eval_loss": 0.5566866397857666,
+      "eval_mean_token_accuracy": 0.844177934319474,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.4858,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.989,
+      "step": 300
+    },
+    {
+      "entropy": 0.6226776849478484,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.8886699676513672,
+      "learning_rate": 0.00017956006656238274,
+      "loss": 0.558210802078247,
+      "mean_token_accuracy": 0.84083157107234,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6066981683983359,
+      "eval_loss": 0.5585207939147949,
+      "eval_mean_token_accuracy": 0.8423153311014175,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.3463,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 320
+    },
+    {
+      "entropy": 0.6249004438519478,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8791211843490601,
+      "learning_rate": 0.00019081775098635657,
+      "loss": 0.5603597164154053,
+      "mean_token_accuracy": 0.8420463085174561,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6082247584018596,
+      "eval_loss": 0.5616299510002136,
+      "eval_mean_token_accuracy": 0.8431286801432454,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.1253,
+      "eval_samples_per_second": 15.965,
+      "eval_steps_per_second": 1.997,
+      "step": 340
+    },
+    {
+      "entropy": 0.6362396612763405,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.8606319427490234,
+      "learning_rate": 0.0002020754354103304,
+      "loss": 0.5735773563385009,
+      "mean_token_accuracy": 0.8371490836143494,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6492362072648004,
+      "eval_loss": 0.5646467804908752,
+      "eval_mean_token_accuracy": 0.8415517574825953,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.3351,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 360
+    },
+    {
+      "entropy": 0.638665035739541,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.7773950099945068,
+      "learning_rate": 0.00021333311983430425,
+      "loss": 0.5820859909057617,
+      "mean_token_accuracy": 0.8372561208903789,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6434498637221581,
+      "eval_loss": 0.5645168423652649,
+      "eval_mean_token_accuracy": 0.8420382481674815,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.1216,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 380
+    },
+    {
+      "entropy": 0.6316851265728474,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 1.6120579242706299,
+      "learning_rate": 0.00022459080425827807,
+      "loss": 0.5637502670288086,
+      "mean_token_accuracy": 0.8386227294802666,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6469012776086497,
+      "eval_loss": 0.5758090615272522,
+      "eval_mean_token_accuracy": 0.8397158470957778,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.6139,
+      "eval_samples_per_second": 15.875,
+      "eval_steps_per_second": 1.986,
+      "step": 400
+    },
+    {
+      "entropy": 0.5894816922835815,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 1.1616325378417969,
+      "learning_rate": 0.00022626713048053178,
+      "loss": 0.5316025257110596,
+      "mean_token_accuracy": 0.8466163017810919,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5860798164855602,
+      "eval_loss": 0.5777581930160522,
+      "eval_mean_token_accuracy": 0.8396938103576039,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.1449,
+      "eval_samples_per_second": 15.961,
+      "eval_steps_per_second": 1.997,
+      "step": 420
+    },
+    {
+      "entropy": 0.5818420693278312,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.7999453544616699,
+      "learning_rate": 0.00022622107023288778,
+      "loss": 0.5221010208129883,
+      "mean_token_accuracy": 0.8474301159381866,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5783926014636838,
+      "eval_loss": 0.5700300931930542,
+      "eval_mean_token_accuracy": 0.8430753537388735,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.5308,
+      "eval_samples_per_second": 15.89,
+      "eval_steps_per_second": 1.988,
+      "step": 440
+    },
+    {
+      "entropy": 0.5612493887543678,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 1.015687346458435,
+      "learning_rate": 0.00022614090619491568,
+      "loss": 0.5084867000579834,
+      "mean_token_accuracy": 0.8495561093091964,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5841563874205877,
+      "eval_loss": 0.5693665742874146,
+      "eval_mean_token_accuracy": 0.8427817298229351,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.5256,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.988,
+      "step": 460
+    },
+    {
+      "entropy": 0.5828216474503278,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 1.9750930070877075,
+      "learning_rate": 0.00022602666254299594,
+      "loss": 0.5180017948150635,
+      "mean_token_accuracy": 0.8515685826539994,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5806607044366903,
+      "eval_loss": 0.5804352760314941,
+      "eval_mean_token_accuracy": 0.8413014668364858,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.1199,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 480
+    },
+    {
+      "entropy": 0.5926914308220148,
+      "epoch": 1.244084682440847,
+      "grad_norm": 0.8917353749275208,
+      "learning_rate": 0.0002258783737314558,
+      "loss": 0.528910779953003,
+      "mean_token_accuracy": 0.8486074328422546,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5593361884009006,
+      "eval_loss": 0.5675153732299805,
+      "eval_mean_token_accuracy": 0.8433507802181466,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7289,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5865630559623242,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7482362985610962,
+      "learning_rate": 0.00022569608448217823,
+      "loss": 0.5250466823577881,
+      "mean_token_accuracy": 0.8477916084229946,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.543057840230853,
+      "eval_loss": 0.5671008229255676,
+      "eval_mean_token_accuracy": 0.8428726016088973,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.3403,
+      "eval_samples_per_second": 15.925,
+      "eval_steps_per_second": 1.992,
+      "step": 520
+    },
+    {
+      "entropy": 0.5870206747204065,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9473814964294434,
+      "learning_rate": 0.00022547984977111448,
+      "loss": 0.5252370834350586,
+      "mean_token_accuracy": 0.8468369916081429,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.590982622878496,
+      "eval_loss": 0.5676343441009521,
+      "eval_mean_token_accuracy": 0.8429348746011424,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.5168,
+      "eval_samples_per_second": 15.893,
+      "eval_steps_per_second": 1.988,
+      "step": 540
+    },
+    {
+      "entropy": 0.5785854265093804,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.9353351593017578,
+      "learning_rate": 0.0002252297348117042,
+      "loss": 0.5304938316345215,
+      "mean_token_accuracy": 0.8463383808732032,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6099918867612995,
+      "eval_loss": 0.5620437860488892,
+      "eval_mean_token_accuracy": 0.8430728347495545,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.7741,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.982,
+      "step": 560
+    },
+    {
+      "entropy": 0.5768801040947438,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.9198738932609558,
+      "learning_rate": 0.0002249458150352077,
+      "loss": 0.520513391494751,
+      "mean_token_accuracy": 0.8487689301371575,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.6349420670506566,
+      "eval_loss": 0.5645340085029602,
+      "eval_mean_token_accuracy": 0.8447844597489335,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.3257,
+      "eval_samples_per_second": 15.928,
+      "eval_steps_per_second": 1.992,
+      "step": 580
+    },
+    {
+      "entropy": 0.5822233572602272,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.832811176776886,
+      "learning_rate": 0.0002246281760679571,
+      "loss": 0.5295282363891601,
+      "mean_token_accuracy": 0.8504064798355102,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5829724387027496,
+      "eval_loss": 0.5612193942070007,
+      "eval_mean_token_accuracy": 0.8449643853791925,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6617,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.571855777129531,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.7665547728538513,
+      "learning_rate": 0.00022427691370553263,
+      "loss": 0.5187931060791016,
+      "mean_token_accuracy": 0.8534420043230057,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5623592240519302,
+      "eval_loss": 0.5575760006904602,
+      "eval_mean_token_accuracy": 0.8468210229346919,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.6324,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.985,
+      "step": 620
+    },
+    {
+      "entropy": 0.5740394659340382,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6500429511070251,
+      "learning_rate": 0.00022389213388387174,
+      "loss": 0.5283198833465577,
+      "mean_token_accuracy": 0.8502798482775689,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5548852207355721,
+      "eval_loss": 0.5561797022819519,
+      "eval_mean_token_accuracy": 0.8452786498291548,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.5205,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 640
+    },
+    {
+      "entropy": 0.6020145989954472,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.7056867480278015,
+      "learning_rate": 0.00022347395264732053,
+      "loss": 0.5400049209594726,
+      "mean_token_accuracy": 0.8447613954544068,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5618055154417836,
+      "eval_loss": 0.556106686592102,
+      "eval_mean_token_accuracy": 0.8465680112672407,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.2971,
+      "eval_samples_per_second": 15.933,
+      "eval_steps_per_second": 1.993,
+      "step": 660
+    },
+    {
+      "entropy": 0.5665927153080702,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.5987663865089417,
+      "learning_rate": 0.00022302249611363625,
+      "loss": 0.5143643856048584,
+      "mean_token_accuracy": 0.8529589556157589,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.568248552118623,
+      "eval_loss": 0.5476346015930176,
+      "eval_mean_token_accuracy": 0.8476775434128073,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.9583,
+      "eval_samples_per_second": 15.812,
+      "eval_steps_per_second": 1.978,
+      "step": 680
+    },
+    {
+      "entropy": 0.5673687808215618,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.735261857509613,
+      "learning_rate": 0.00022253790043595193,
+      "loss": 0.509885597229004,
+      "mean_token_accuracy": 0.8537046857178211,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5616967284748721,
+      "eval_loss": 0.5439274311065674,
+      "eval_mean_token_accuracy": 0.8488946217437123,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.0604,
+      "eval_samples_per_second": 15.977,
+      "eval_steps_per_second": 1.999,
+      "step": 700
+    },
+    {
+      "entropy": 0.5529541682451964,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7014835476875305,
+      "learning_rate": 0.00022202031176171442,
+      "loss": 0.5078992366790771,
+      "mean_token_accuracy": 0.8525233261287213,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5827173320359962,
+      "eval_loss": 0.5419450402259827,
+      "eval_mean_token_accuracy": 0.8477318609176681,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 85.2984,
+      "eval_samples_per_second": 16.12,
+      "eval_steps_per_second": 2.016,
+      "step": 720
+    },
+    {
+      "entropy": 0.5755720350891351,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.705613911151886,
+      "learning_rate": 0.00022146988618860824,
+      "loss": 0.5181350708007812,
+      "mean_token_accuracy": 0.8467609457671642,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5743971356125765,
+      "eval_loss": 0.5415896773338318,
+      "eval_mean_token_accuracy": 0.847328585940738,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 85.5602,
+      "eval_samples_per_second": 16.071,
+      "eval_steps_per_second": 2.01,
+      "step": 740
+    },
+    {
+      "entropy": 0.561330484598875,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.6722865700721741,
+      "learning_rate": 0.0002208867897174789,
+      "loss": 0.499837589263916,
+      "mean_token_accuracy": 0.8518734864890576,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.5865232653396074,
+      "eval_loss": 0.5437926650047302,
+      "eval_mean_token_accuracy": 0.8450997017843779,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4116,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.547389242425561,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.7935577034950256,
+      "learning_rate": 0.00022027119820226907,
+      "loss": 0.4977591514587402,
+      "mean_token_accuracy": 0.8539491161704064,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.5290903090391048,
+      "eval_loss": 0.5409526824951172,
+      "eval_mean_token_accuracy": 0.8497545698354411,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.7262,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 780
+    },
+    {
+      "entropy": 0.5687909748405218,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6180546283721924,
+      "learning_rate": 0.00021962329729698345,
+      "loss": 0.5109643459320068,
+      "mean_token_accuracy": 0.8521598495543004,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.5503541858390321,
+      "eval_loss": 0.5361555218696594,
+      "eval_mean_token_accuracy": 0.8510884285666221,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.3339,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 800
+    },
+    {
+      "entropy": 0.4739728841261986,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.8058829307556152,
+      "learning_rate": 0.0002189432823996982,
+      "loss": 0.4204097747802734,
+      "mean_token_accuracy": 0.8728981889211215,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.5077334992414297,
+      "eval_loss": 0.5531114339828491,
+      "eval_mean_token_accuracy": 0.8489257208136625,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.4801,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.989,
+      "step": 820
+    },
+    {
+      "entropy": 0.4594309840351343,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.6906896829605103,
+      "learning_rate": 0.0002182313585936314,
+      "loss": 0.4071959495544434,
+      "mean_token_accuracy": 0.8732857562601566,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.49850136994622474,
+      "eval_loss": 0.5486204624176025,
+      "eval_mean_token_accuracy": 0.8507991450470548,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.3364,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.4881629109382629,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6343470215797424,
+      "learning_rate": 0.0002174877405852928,
+      "loss": 0.41669540405273436,
+      "mean_token_accuracy": 0.8711295068264008,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.49155513924914734,
+      "eval_loss": 0.555109441280365,
+      "eval_mean_token_accuracy": 0.8496399400539176,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 86.3295,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 860
+    },
+    {
+      "entropy": 0.4648668970912695,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.8014165163040161,
+      "learning_rate": 0.00021671265263973133,
+      "loss": 0.4110250473022461,
+      "mean_token_accuracy": 0.8754166305065155,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.4909258722219356,
+      "eval_loss": 0.5539511442184448,
+      "eval_mean_token_accuracy": 0.8492401502160138,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.3468,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 880
+    },
+    {
+      "entropy": 0.4824485514312983,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6665191054344177,
+      "learning_rate": 0.00021590632851289967,
+      "loss": 0.4181404113769531,
+      "mean_token_accuracy": 0.8726993151009083,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.4986876940657926,
+      "eval_loss": 0.547695517539978,
+      "eval_mean_token_accuracy": 0.8501384708770486,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.3838,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 900
+    },
+    {
+      "entropy": 0.4751896943897009,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.81158047914505,
+      "learning_rate": 0.00021506901138115678,
+      "loss": 0.40689678192138673,
+      "mean_token_accuracy": 0.8745221219956875,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.507153491121392,
+      "eval_loss": 0.5501641631126404,
+      "eval_mean_token_accuracy": 0.8495670116918032,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.0912,
+      "eval_samples_per_second": 15.971,
+      "eval_steps_per_second": 1.998,
+      "step": 920
+    },
+    {
+      "entropy": 0.4873133715242147,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7218056321144104,
+      "learning_rate": 0.0002142009537679292,
+      "loss": 0.42701358795166017,
+      "mean_token_accuracy": 0.8695114746689796,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5202612736543943,
+      "eval_loss": 0.5491839051246643,
+      "eval_mean_token_accuracy": 0.8494071208460386,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.1142,
+      "eval_samples_per_second": 15.967,
+      "eval_steps_per_second": 1.997,
+      "step": 940
+    },
+    {
+      "entropy": 0.4762951169162989,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 0.7194424867630005,
+      "learning_rate": 0.0002133024174675534,
+      "loss": 0.42299847602844237,
+      "mean_token_accuracy": 0.8709790132939815,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4899340462546016,
+      "eval_loss": 0.5522511601448059,
+      "eval_mean_token_accuracy": 0.8492208258357159,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.463,
+      "eval_samples_per_second": 15.903,
+      "eval_steps_per_second": 1.989,
+      "step": 960
+    },
+    {
+      "entropy": 0.49650347977876663,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.8406022787094116,
+      "learning_rate": 0.0002123736734663221,
+      "loss": 0.4275330066680908,
+      "mean_token_accuracy": 0.8670595556497573,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.49691385654515996,
+      "eval_loss": 0.5491269826889038,
+      "eval_mean_token_accuracy": 0.850309816210769,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.17,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.996,
+      "step": 980
+    },
+    {
+      "entropy": 0.48843890577554705,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.9082473516464233,
+      "learning_rate": 0.00021141500186075868,
+      "loss": 0.4309722423553467,
+      "mean_token_accuracy": 0.8686766296625137,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5543508351195691,
+      "eval_loss": 0.5478800535202026,
+      "eval_mean_token_accuracy": 0.8478029522784921,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.3835,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4777219031006098,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7448089122772217,
+      "learning_rate": 0.0002104266917731438,
+      "loss": 0.423325252532959,
+      "mean_token_accuracy": 0.8706337086856365,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.49857561550168106,
+      "eval_loss": 0.5511948466300964,
+      "eval_mean_token_accuracy": 0.8502220289651737,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.5399,
+      "eval_samples_per_second": 15.889,
+      "eval_steps_per_second": 1.988,
+      "step": 1020
+    },
+    {
+      "entropy": 0.4844174191355705,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.794029176235199,
+      "learning_rate": 0.00020940904126432,
+      "loss": 0.4176753044128418,
+      "mean_token_accuracy": 0.873535567522049,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.485467542222766,
+      "eval_loss": 0.5539286732673645,
+      "eval_mean_token_accuracy": 0.8495475081510322,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.135,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 1.997,
+      "step": 1040
+    },
+    {
+      "entropy": 0.49070929251611234,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.7558256983757019,
+      "learning_rate": 0.0002083623572438007,
+      "loss": 0.42867293357849123,
+      "mean_token_accuracy": 0.8696666076779366,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.490822730889154,
+      "eval_loss": 0.5434785485267639,
+      "eval_mean_token_accuracy": 0.850568296950917,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.4933,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 1060
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.045743734380032e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/trainer_state.json

@@ -0,0 +1,1168 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6874221668742218,
+  "eval_steps": 20,
+  "global_step": 1080,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.955029806494713,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.020533561706543,
+      "learning_rate": 1.0694800202775147e-05,
+      "loss": 1.7107986450195312,
+      "mean_token_accuracy": 0.6487608112394809,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.3144892034835594,
+      "eval_loss": 1.1198534965515137,
+      "eval_mean_token_accuracy": 0.7460246955932572,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.0565,
+      "eval_samples_per_second": 15.794,
+      "eval_steps_per_second": 1.976,
+      "step": 20
+    },
+    {
+      "entropy": 1.0063214391469955,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.572906494140625,
+      "learning_rate": 2.1952484626748985e-05,
+      "loss": 0.8663722991943359,
+      "mean_token_accuracy": 0.7779282338917255,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7921617945959402,
+      "eval_loss": 0.7062025666236877,
+      "eval_mean_token_accuracy": 0.8100443180910376,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.5189,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 40
+    },
+    {
+      "entropy": 0.7682028576731682,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3003711700439453,
+      "learning_rate": 3.3210169050722824e-05,
+      "loss": 0.673183822631836,
+      "mean_token_accuracy": 0.8182129614055157,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7059133584762729,
+      "eval_loss": 0.6481946706771851,
+      "eval_mean_token_accuracy": 0.8227418761613757,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.5098,
+      "eval_samples_per_second": 15.894,
+      "eval_steps_per_second": 1.988,
+      "step": 60
+    },
+    {
+      "entropy": 0.7029960259795189,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.3664201498031616,
+      "learning_rate": 4.4467853474696664e-05,
+      "loss": 0.6354611873626709,
+      "mean_token_accuracy": 0.8243416830897331,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6867308004997498,
+      "eval_loss": 0.6179807186126709,
+      "eval_mean_token_accuracy": 0.8302594811417335,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.3969,
+      "eval_samples_per_second": 15.915,
+      "eval_steps_per_second": 1.991,
+      "step": 80
+    },
+    {
+      "entropy": 0.6764581337571144,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9815880656242371,
+      "learning_rate": 5.57255378986705e-05,
+      "loss": 0.5988658905029297,
+      "mean_token_accuracy": 0.8329168625175953,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6790881479202315,
+      "eval_loss": 0.5998476147651672,
+      "eval_mean_token_accuracy": 0.8318756420251935,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.6653,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 100
+    },
+    {
+      "entropy": 0.6744543805718421,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.932099461555481,
+      "learning_rate": 6.698322232264434e-05,
+      "loss": 0.5991750717163086,
+      "mean_token_accuracy": 0.8304223112761975,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.6813044282932614,
+      "eval_loss": 0.5922021269798279,
+      "eval_mean_token_accuracy": 0.8346439617317777,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.1551,
+      "eval_samples_per_second": 15.96,
+      "eval_steps_per_second": 1.996,
+      "step": 120
+    },
+    {
+      "entropy": 0.6663189359009266,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9528499841690063,
+      "learning_rate": 7.824090674661818e-05,
+      "loss": 0.5891091346740722,
+      "mean_token_accuracy": 0.832152470946312,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6398407208711602,
+      "eval_loss": 0.5859636664390564,
+      "eval_mean_token_accuracy": 0.8372074996316156,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.2706,
+      "eval_samples_per_second": 15.938,
+      "eval_steps_per_second": 1.994,
+      "step": 140
+    },
+    {
+      "entropy": 0.64859763905406,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8468204140663147,
+      "learning_rate": 8.949859117059201e-05,
+      "loss": 0.569426441192627,
+      "mean_token_accuracy": 0.8401990942656994,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6381674285891444,
+      "eval_loss": 0.5744525790214539,
+      "eval_mean_token_accuracy": 0.838626817908398,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.2848,
+      "eval_samples_per_second": 15.936,
+      "eval_steps_per_second": 1.993,
+      "step": 160
+    },
+    {
+      "entropy": 0.6432608783245086,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8765804767608643,
+      "learning_rate": 0.00010075627559456587,
+      "loss": 0.5687318801879883,
+      "mean_token_accuracy": 0.839249350130558,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6047098288355872,
+      "eval_loss": 0.5679298043251038,
+      "eval_mean_token_accuracy": 0.8410577181466791,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.5879,
+      "eval_samples_per_second": 15.88,
+      "eval_steps_per_second": 1.986,
+      "step": 180
+    },
+    {
+      "entropy": 0.6440276011824608,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9576020240783691,
+      "learning_rate": 0.00011201396001853971,
+      "loss": 0.5828506469726562,
+      "mean_token_accuracy": 0.837553184479475,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6161119360909906,
+      "eval_loss": 0.5702911615371704,
+      "eval_mean_token_accuracy": 0.8407089398350827,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3341,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 200
+    },
+    {
+      "entropy": 0.6377195850014686,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7212373614311218,
+      "learning_rate": 0.00012327164444251353,
+      "loss": 0.5702451229095459,
+      "mean_token_accuracy": 0.8397969007492065,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6080108886194784,
+      "eval_loss": 0.5633499622344971,
+      "eval_mean_token_accuracy": 0.8396634854549585,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.4945,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 220
+    },
+    {
+      "entropy": 0.6287345830351114,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.848779022693634,
+      "learning_rate": 0.00013452932886648739,
+      "loss": 0.5506546020507812,
+      "mean_token_accuracy": 0.8438881888985634,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6307531505130058,
+      "eval_loss": 0.5573338270187378,
+      "eval_mean_token_accuracy": 0.8431362606758295,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.3535,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.992,
+      "step": 240
+    },
+    {
+      "entropy": 0.6223786748945713,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.7316951751708984,
+      "learning_rate": 0.0001457870132904612,
+      "loss": 0.5495625972747803,
+      "mean_token_accuracy": 0.8440376669168472,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.623454462476941,
+      "eval_loss": 0.5619264245033264,
+      "eval_mean_token_accuracy": 0.8431175777385401,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.2008,
+      "eval_samples_per_second": 15.951,
+      "eval_steps_per_second": 1.995,
+      "step": 260
+    },
+    {
+      "entropy": 0.6281675305217505,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7639564871788025,
+      "learning_rate": 0.00015704469771443506,
+      "loss": 0.5604369163513183,
+      "mean_token_accuracy": 0.8401600055396556,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.63416675980701,
+      "eval_loss": 0.5612760782241821,
+      "eval_mean_token_accuracy": 0.842435666294985,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.25,
+      "eval_samples_per_second": 15.942,
+      "eval_steps_per_second": 1.994,
+      "step": 280
+    },
+    {
+      "entropy": 0.6427909277379513,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6475813388824463,
+      "learning_rate": 0.0001683023821384089,
+      "loss": 0.573763370513916,
+      "mean_token_accuracy": 0.8370340794324875,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6231539840268534,
+      "eval_loss": 0.5566866397857666,
+      "eval_mean_token_accuracy": 0.844177934319474,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.4858,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.989,
+      "step": 300
+    },
+    {
+      "entropy": 0.6226776849478484,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.8886699676513672,
+      "learning_rate": 0.00017956006656238274,
+      "loss": 0.558210802078247,
+      "mean_token_accuracy": 0.84083157107234,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6066981683983359,
+      "eval_loss": 0.5585207939147949,
+      "eval_mean_token_accuracy": 0.8423153311014175,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.3463,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 320
+    },
+    {
+      "entropy": 0.6249004438519478,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8791211843490601,
+      "learning_rate": 0.00019081775098635657,
+      "loss": 0.5603597164154053,
+      "mean_token_accuracy": 0.8420463085174561,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6082247584018596,
+      "eval_loss": 0.5616299510002136,
+      "eval_mean_token_accuracy": 0.8431286801432454,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.1253,
+      "eval_samples_per_second": 15.965,
+      "eval_steps_per_second": 1.997,
+      "step": 340
+    },
+    {
+      "entropy": 0.6362396612763405,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.8606319427490234,
+      "learning_rate": 0.0002020754354103304,
+      "loss": 0.5735773563385009,
+      "mean_token_accuracy": 0.8371490836143494,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6492362072648004,
+      "eval_loss": 0.5646467804908752,
+      "eval_mean_token_accuracy": 0.8415517574825953,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.3351,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 360
+    },
+    {
+      "entropy": 0.638665035739541,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.7773950099945068,
+      "learning_rate": 0.00021333311983430425,
+      "loss": 0.5820859909057617,
+      "mean_token_accuracy": 0.8372561208903789,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6434498637221581,
+      "eval_loss": 0.5645168423652649,
+      "eval_mean_token_accuracy": 0.8420382481674815,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.1216,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 380
+    },
+    {
+      "entropy": 0.6316851265728474,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 1.6120579242706299,
+      "learning_rate": 0.00022459080425827807,
+      "loss": 0.5637502670288086,
+      "mean_token_accuracy": 0.8386227294802666,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6469012776086497,
+      "eval_loss": 0.5758090615272522,
+      "eval_mean_token_accuracy": 0.8397158470957778,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.6139,
+      "eval_samples_per_second": 15.875,
+      "eval_steps_per_second": 1.986,
+      "step": 400
+    },
+    {
+      "entropy": 0.5894816922835815,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 1.1616325378417969,
+      "learning_rate": 0.00022626713048053178,
+      "loss": 0.5316025257110596,
+      "mean_token_accuracy": 0.8466163017810919,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5860798164855602,
+      "eval_loss": 0.5777581930160522,
+      "eval_mean_token_accuracy": 0.8396938103576039,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.1449,
+      "eval_samples_per_second": 15.961,
+      "eval_steps_per_second": 1.997,
+      "step": 420
+    },
+    {
+      "entropy": 0.5818420693278312,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.7999453544616699,
+      "learning_rate": 0.00022622107023288778,
+      "loss": 0.5221010208129883,
+      "mean_token_accuracy": 0.8474301159381866,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5783926014636838,
+      "eval_loss": 0.5700300931930542,
+      "eval_mean_token_accuracy": 0.8430753537388735,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.5308,
+      "eval_samples_per_second": 15.89,
+      "eval_steps_per_second": 1.988,
+      "step": 440
+    },
+    {
+      "entropy": 0.5612493887543678,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 1.015687346458435,
+      "learning_rate": 0.00022614090619491568,
+      "loss": 0.5084867000579834,
+      "mean_token_accuracy": 0.8495561093091964,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5841563874205877,
+      "eval_loss": 0.5693665742874146,
+      "eval_mean_token_accuracy": 0.8427817298229351,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.5256,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.988,
+      "step": 460
+    },
+    {
+      "entropy": 0.5828216474503278,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 1.9750930070877075,
+      "learning_rate": 0.00022602666254299594,
+      "loss": 0.5180017948150635,
+      "mean_token_accuracy": 0.8515685826539994,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5806607044366903,
+      "eval_loss": 0.5804352760314941,
+      "eval_mean_token_accuracy": 0.8413014668364858,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.1199,
+      "eval_samples_per_second": 15.966,
+      "eval_steps_per_second": 1.997,
+      "step": 480
+    },
+    {
+      "entropy": 0.5926914308220148,
+      "epoch": 1.244084682440847,
+      "grad_norm": 0.8917353749275208,
+      "learning_rate": 0.0002258783737314558,
+      "loss": 0.528910779953003,
+      "mean_token_accuracy": 0.8486074328422546,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5593361884009006,
+      "eval_loss": 0.5675153732299805,
+      "eval_mean_token_accuracy": 0.8433507802181466,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7289,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5865630559623242,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7482362985610962,
+      "learning_rate": 0.00022569608448217823,
+      "loss": 0.5250466823577881,
+      "mean_token_accuracy": 0.8477916084229946,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.543057840230853,
+      "eval_loss": 0.5671008229255676,
+      "eval_mean_token_accuracy": 0.8428726016088973,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.3403,
+      "eval_samples_per_second": 15.925,
+      "eval_steps_per_second": 1.992,
+      "step": 520
+    },
+    {
+      "entropy": 0.5870206747204065,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9473814964294434,
+      "learning_rate": 0.00022547984977111448,
+      "loss": 0.5252370834350586,
+      "mean_token_accuracy": 0.8468369916081429,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.590982622878496,
+      "eval_loss": 0.5676343441009521,
+      "eval_mean_token_accuracy": 0.8429348746011424,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.5168,
+      "eval_samples_per_second": 15.893,
+      "eval_steps_per_second": 1.988,
+      "step": 540
+    },
+    {
+      "entropy": 0.5785854265093804,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.9353351593017578,
+      "learning_rate": 0.0002252297348117042,
+      "loss": 0.5304938316345215,
+      "mean_token_accuracy": 0.8463383808732032,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6099918867612995,
+      "eval_loss": 0.5620437860488892,
+      "eval_mean_token_accuracy": 0.8430728347495545,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.7741,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.982,
+      "step": 560
+    },
+    {
+      "entropy": 0.5768801040947438,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.9198738932609558,
+      "learning_rate": 0.0002249458150352077,
+      "loss": 0.520513391494751,
+      "mean_token_accuracy": 0.8487689301371575,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.6349420670506566,
+      "eval_loss": 0.5645340085029602,
+      "eval_mean_token_accuracy": 0.8447844597489335,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.3257,
+      "eval_samples_per_second": 15.928,
+      "eval_steps_per_second": 1.992,
+      "step": 580
+    },
+    {
+      "entropy": 0.5822233572602272,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.832811176776886,
+      "learning_rate": 0.0002246281760679571,
+      "loss": 0.5295282363891601,
+      "mean_token_accuracy": 0.8504064798355102,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5829724387027496,
+      "eval_loss": 0.5612193942070007,
+      "eval_mean_token_accuracy": 0.8449643853791925,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6617,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.571855777129531,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.7665547728538513,
+      "learning_rate": 0.00022427691370553263,
+      "loss": 0.5187931060791016,
+      "mean_token_accuracy": 0.8534420043230057,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5623592240519302,
+      "eval_loss": 0.5575760006904602,
+      "eval_mean_token_accuracy": 0.8468210229346919,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.6324,
+      "eval_samples_per_second": 15.872,
+      "eval_steps_per_second": 1.985,
+      "step": 620
+    },
+    {
+      "entropy": 0.5740394659340382,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6500429511070251,
+      "learning_rate": 0.00022389213388387174,
+      "loss": 0.5283198833465577,
+      "mean_token_accuracy": 0.8502798482775689,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5548852207355721,
+      "eval_loss": 0.5561797022819519,
+      "eval_mean_token_accuracy": 0.8452786498291548,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.5205,
+      "eval_samples_per_second": 15.892,
+      "eval_steps_per_second": 1.988,
+      "step": 640
+    },
+    {
+      "entropy": 0.6020145989954472,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.7056867480278015,
+      "learning_rate": 0.00022347395264732053,
+      "loss": 0.5400049209594726,
+      "mean_token_accuracy": 0.8447613954544068,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5618055154417836,
+      "eval_loss": 0.556106686592102,
+      "eval_mean_token_accuracy": 0.8465680112672407,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.2971,
+      "eval_samples_per_second": 15.933,
+      "eval_steps_per_second": 1.993,
+      "step": 660
+    },
+    {
+      "entropy": 0.5665927153080702,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.5987663865089417,
+      "learning_rate": 0.00022302249611363625,
+      "loss": 0.5143643856048584,
+      "mean_token_accuracy": 0.8529589556157589,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.568248552118623,
+      "eval_loss": 0.5476346015930176,
+      "eval_mean_token_accuracy": 0.8476775434128073,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.9583,
+      "eval_samples_per_second": 15.812,
+      "eval_steps_per_second": 1.978,
+      "step": 680
+    },
+    {
+      "entropy": 0.5673687808215618,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.735261857509613,
+      "learning_rate": 0.00022253790043595193,
+      "loss": 0.509885597229004,
+      "mean_token_accuracy": 0.8537046857178211,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5616967284748721,
+      "eval_loss": 0.5439274311065674,
+      "eval_mean_token_accuracy": 0.8488946217437123,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.0604,
+      "eval_samples_per_second": 15.977,
+      "eval_steps_per_second": 1.999,
+      "step": 700
+    },
+    {
+      "entropy": 0.5529541682451964,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7014835476875305,
+      "learning_rate": 0.00022202031176171442,
+      "loss": 0.5078992366790771,
+      "mean_token_accuracy": 0.8525233261287213,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5827173320359962,
+      "eval_loss": 0.5419450402259827,
+      "eval_mean_token_accuracy": 0.8477318609176681,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 85.2984,
+      "eval_samples_per_second": 16.12,
+      "eval_steps_per_second": 2.016,
+      "step": 720
+    },
+    {
+      "entropy": 0.5755720350891351,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.705613911151886,
+      "learning_rate": 0.00022146988618860824,
+      "loss": 0.5181350708007812,
+      "mean_token_accuracy": 0.8467609457671642,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5743971356125765,
+      "eval_loss": 0.5415896773338318,
+      "eval_mean_token_accuracy": 0.847328585940738,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 85.5602,
+      "eval_samples_per_second": 16.071,
+      "eval_steps_per_second": 2.01,
+      "step": 740
+    },
+    {
+      "entropy": 0.561330484598875,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.6722865700721741,
+      "learning_rate": 0.0002208867897174789,
+      "loss": 0.499837589263916,
+      "mean_token_accuracy": 0.8518734864890576,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.5865232653396074,
+      "eval_loss": 0.5437926650047302,
+      "eval_mean_token_accuracy": 0.8450997017843779,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4116,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.547389242425561,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.7935577034950256,
+      "learning_rate": 0.00022027119820226907,
+      "loss": 0.4977591514587402,
+      "mean_token_accuracy": 0.8539491161704064,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.5290903090391048,
+      "eval_loss": 0.5409526824951172,
+      "eval_mean_token_accuracy": 0.8497545698354411,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.7262,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.983,
+      "step": 780
+    },
+    {
+      "entropy": 0.5687909748405218,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6180546283721924,
+      "learning_rate": 0.00021962329729698345,
+      "loss": 0.5109643459320068,
+      "mean_token_accuracy": 0.8521598495543004,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.5503541858390321,
+      "eval_loss": 0.5361555218696594,
+      "eval_mean_token_accuracy": 0.8510884285666221,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.3339,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 800
+    },
+    {
+      "entropy": 0.4739728841261986,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.8058829307556152,
+      "learning_rate": 0.0002189432823996982,
+      "loss": 0.4204097747802734,
+      "mean_token_accuracy": 0.8728981889211215,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.5077334992414297,
+      "eval_loss": 0.5531114339828491,
+      "eval_mean_token_accuracy": 0.8489257208136625,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.4801,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.989,
+      "step": 820
+    },
+    {
+      "entropy": 0.4594309840351343,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.6906896829605103,
+      "learning_rate": 0.0002182313585936314,
+      "loss": 0.4071959495544434,
+      "mean_token_accuracy": 0.8732857562601566,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.49850136994622474,
+      "eval_loss": 0.5486204624176025,
+      "eval_mean_token_accuracy": 0.8507991450470548,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.3364,
+      "eval_samples_per_second": 15.926,
+      "eval_steps_per_second": 1.992,
+      "step": 840
+    },
+    {
+      "entropy": 0.4881629109382629,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6343470215797424,
+      "learning_rate": 0.0002174877405852928,
+      "loss": 0.41669540405273436,
+      "mean_token_accuracy": 0.8711295068264008,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.49155513924914734,
+      "eval_loss": 0.555109441280365,
+      "eval_mean_token_accuracy": 0.8496399400539176,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 86.3295,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.992,
+      "step": 860
+    },
+    {
+      "entropy": 0.4648668970912695,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.8014165163040161,
+      "learning_rate": 0.00021671265263973133,
+      "loss": 0.4110250473022461,
+      "mean_token_accuracy": 0.8754166305065155,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.4909258722219356,
+      "eval_loss": 0.5539511442184448,
+      "eval_mean_token_accuracy": 0.8492401502160138,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.3468,
+      "eval_samples_per_second": 15.924,
+      "eval_steps_per_second": 1.992,
+      "step": 880
+    },
+    {
+      "entropy": 0.4824485514312983,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6665191054344177,
+      "learning_rate": 0.00021590632851289967,
+      "loss": 0.4181404113769531,
+      "mean_token_accuracy": 0.8726993151009083,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.4986876940657926,
+      "eval_loss": 0.547695517539978,
+      "eval_mean_token_accuracy": 0.8501384708770486,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.3838,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 900
+    },
+    {
+      "entropy": 0.4751896943897009,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.81158047914505,
+      "learning_rate": 0.00021506901138115678,
+      "loss": 0.40689678192138673,
+      "mean_token_accuracy": 0.8745221219956875,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.507153491121392,
+      "eval_loss": 0.5501641631126404,
+      "eval_mean_token_accuracy": 0.8495670116918032,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.0912,
+      "eval_samples_per_second": 15.971,
+      "eval_steps_per_second": 1.998,
+      "step": 920
+    },
+    {
+      "entropy": 0.4873133715242147,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7218056321144104,
+      "learning_rate": 0.0002142009537679292,
+      "loss": 0.42701358795166017,
+      "mean_token_accuracy": 0.8695114746689796,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5202612736543943,
+      "eval_loss": 0.5491839051246643,
+      "eval_mean_token_accuracy": 0.8494071208460386,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.1142,
+      "eval_samples_per_second": 15.967,
+      "eval_steps_per_second": 1.997,
+      "step": 940
+    },
+    {
+      "entropy": 0.4762951169162989,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 0.7194424867630005,
+      "learning_rate": 0.0002133024174675534,
+      "loss": 0.42299847602844237,
+      "mean_token_accuracy": 0.8709790132939815,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4899340462546016,
+      "eval_loss": 0.5522511601448059,
+      "eval_mean_token_accuracy": 0.8492208258357159,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.463,
+      "eval_samples_per_second": 15.903,
+      "eval_steps_per_second": 1.989,
+      "step": 960
+    },
+    {
+      "entropy": 0.49650347977876663,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.8406022787094116,
+      "learning_rate": 0.0002123736734663221,
+      "loss": 0.4275330066680908,
+      "mean_token_accuracy": 0.8670595556497573,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.49691385654515996,
+      "eval_loss": 0.5491269826889038,
+      "eval_mean_token_accuracy": 0.850309816210769,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.17,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.996,
+      "step": 980
+    },
+    {
+      "entropy": 0.48843890577554705,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.9082473516464233,
+      "learning_rate": 0.00021141500186075868,
+      "loss": 0.4309722423553467,
+      "mean_token_accuracy": 0.8686766296625137,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5543508351195691,
+      "eval_loss": 0.5478800535202026,
+      "eval_mean_token_accuracy": 0.8478029522784921,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.3835,
+      "eval_samples_per_second": 15.917,
+      "eval_steps_per_second": 1.991,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4777219031006098,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7448089122772217,
+      "learning_rate": 0.0002104266917731438,
+      "loss": 0.423325252532959,
+      "mean_token_accuracy": 0.8706337086856365,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.49857561550168106,
+      "eval_loss": 0.5511948466300964,
+      "eval_mean_token_accuracy": 0.8502220289651737,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.5399,
+      "eval_samples_per_second": 15.889,
+      "eval_steps_per_second": 1.988,
+      "step": 1020
+    },
+    {
+      "entropy": 0.4844174191355705,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.794029176235199,
+      "learning_rate": 0.00020940904126432,
+      "loss": 0.4176753044128418,
+      "mean_token_accuracy": 0.873535567522049,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.485467542222766,
+      "eval_loss": 0.5539286732673645,
+      "eval_mean_token_accuracy": 0.8495475081510322,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.135,
+      "eval_samples_per_second": 15.963,
+      "eval_steps_per_second": 1.997,
+      "step": 1040
+    },
+    {
+      "entropy": 0.49070929251611234,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.7558256983757019,
+      "learning_rate": 0.0002083623572438007,
+      "loss": 0.42867293357849123,
+      "mean_token_accuracy": 0.8696666076779366,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.490822730889154,
+      "eval_loss": 0.5434785485267639,
+      "eval_mean_token_accuracy": 0.850568296950917,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.4933,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.989,
+      "step": 1060
+    },
+    {
+      "entropy": 0.47806114703416824,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6608979105949402,
+      "learning_rate": 0.00020728695537721047,
+      "loss": 0.4289727687835693,
+      "mean_token_accuracy": 0.8693130135536193,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.5285773256490397,
+      "eval_loss": 0.5444230437278748,
+      "eval_mean_token_accuracy": 0.8498796481032704,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.7091,
+      "eval_samples_per_second": 15.858,
+      "eval_steps_per_second": 1.984,
+      "step": 1080
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0682640451304448e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.028265386974777595,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: transformers
+model_name: Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/5khkq4dz) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1266/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1266/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04758632698976937,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1266/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1266/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1266/trainer_state.json

@@ -0,0 +1,317 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1266,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.4865781700611116,
+      "epoch": 0.11869436201780416,
+      "grad_norm": 3.600203037261963,
+      "learning_rate": 9.55350170221182e-06,
+      "loss": 2.2448199462890623,
+      "mean_token_accuracy": 0.571574572622776,
+      "num_tokens": 66201.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.1757256650924683,
+      "epoch": 0.23738872403560832,
+      "grad_norm": 1.843338966369629,
+      "learning_rate": 1.9301972826917757e-05,
+      "loss": 1.0319316864013672,
+      "mean_token_accuracy": 0.7366894924640656,
+      "num_tokens": 132943.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.0132716038823129,
+      "epoch": 0.3560830860534125,
+      "grad_norm": 2.0066328048706055,
+      "learning_rate": 2.9050443951623695e-05,
+      "loss": 0.8726716613769532,
+      "mean_token_accuracy": 0.7669131025671959,
+      "num_tokens": 202797.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.9535812222957611,
+      "epoch": 0.47477744807121663,
+      "grad_norm": 1.8880268335342407,
+      "learning_rate": 3.8798915076329635e-05,
+      "loss": 0.8159033966064453,
+      "mean_token_accuracy": 0.778084682226181,
+      "num_tokens": 266514.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.9149203181266785,
+      "epoch": 0.5934718100890207,
+      "grad_norm": 1.5609782934188843,
+      "learning_rate": 4.8547386201035576e-05,
+      "loss": 0.7810882568359375,
+      "mean_token_accuracy": 0.7836975249648094,
+      "num_tokens": 332781.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.9015455979108811,
+      "epoch": 0.712166172106825,
+      "grad_norm": 1.6056852340698242,
+      "learning_rate": 5.829585732574152e-05,
+      "loss": 0.7620333099365234,
+      "mean_token_accuracy": 0.7867170104384422,
+      "num_tokens": 396012.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.8751798084378243,
+      "epoch": 0.8308605341246291,
+      "grad_norm": 1.457392930984497,
+      "learning_rate": 6.804432845044745e-05,
+      "loss": 0.7453135681152344,
+      "mean_token_accuracy": 0.7934546408057213,
+      "num_tokens": 464261.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.8582844731211662,
+      "epoch": 0.9495548961424333,
+      "grad_norm": 1.325522541999817,
+      "learning_rate": 7.779279957515339e-05,
+      "loss": 0.7352320861816406,
+      "mean_token_accuracy": 0.7950925189256668,
+      "num_tokens": 531313.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.699295549364815,
+      "eval_loss": 0.6675883531570435,
+      "eval_mean_token_accuracy": 0.8100697049620555,
+      "eval_num_tokens": 559577.0,
+      "eval_runtime": 113.6401,
+      "eval_samples_per_second": 12.003,
+      "eval_steps_per_second": 1.505,
+      "step": 422
+    },
+    {
+      "entropy": 0.8327096131852436,
+      "epoch": 1.0664688427299702,
+      "grad_norm": 0.9726872444152832,
+      "learning_rate": 8.226683697984623e-05,
+      "loss": 0.6989453887939453,
+      "mean_token_accuracy": 0.7999463737918641,
+      "num_tokens": 595986.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7919371470808982,
+      "epoch": 1.1851632047477745,
+      "grad_norm": 1.04917311668396,
+      "learning_rate": 8.219368143379697e-05,
+      "loss": 0.6691493225097657,
+      "mean_token_accuracy": 0.8088096314668656,
+      "num_tokens": 663734.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7981289568543434,
+      "epoch": 1.3038575667655787,
+      "grad_norm": 1.1144819259643555,
+      "learning_rate": 8.205030999972083e-05,
+      "loss": 0.6692163848876953,
+      "mean_token_accuracy": 0.8067614835500717,
+      "num_tokens": 730600.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.7944329422712326,
+      "epoch": 1.4225519287833828,
+      "grad_norm": 1.017581820487976,
+      "learning_rate": 8.183696788331456e-05,
+      "loss": 0.6635546112060546,
+      "mean_token_accuracy": 0.8121234861016273,
+      "num_tokens": 793502.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7757732102274895,
+      "epoch": 1.5412462908011868,
+      "grad_norm": 0.9428858160972595,
+      "learning_rate": 8.155401995992886e-05,
+      "loss": 0.6541387939453125,
+      "mean_token_accuracy": 0.8148220491409301,
+      "num_tokens": 860956.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7693249759078026,
+      "epoch": 1.659940652818991,
+      "grad_norm": 0.8914806842803955,
+      "learning_rate": 8.120195015052839e-05,
+      "loss": 0.6372745132446289,
+      "mean_token_accuracy": 0.8154754737019538,
+      "num_tokens": 928211.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7677849313616752,
+      "epoch": 1.7786350148367953,
+      "grad_norm": 1.2037655115127563,
+      "learning_rate": 8.078136059405015e-05,
+      "loss": 0.6468383026123047,
+      "mean_token_accuracy": 0.8125368970632553,
+      "num_tokens": 993921.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.7699458369612694,
+      "epoch": 1.8973293768545996,
+      "grad_norm": 0.8997814059257507,
+      "learning_rate": 8.02929706175755e-05,
+      "loss": 0.6416602325439453,
+      "mean_token_accuracy": 0.8159067538380623,
+      "num_tokens": 1060568.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6038547956455521,
+      "eval_loss": 0.6323259472846985,
+      "eval_mean_token_accuracy": 0.8163558896522076,
+      "eval_num_tokens": 1119154.0,
+      "eval_runtime": 111.6364,
+      "eval_samples_per_second": 12.218,
+      "eval_steps_per_second": 1.532,
+      "step": 844
+    },
+    {
+      "entropy": 0.7529444349598764,
+      "epoch": 2.0142433234421366,
+      "grad_norm": 0.9705535173416138,
+      "learning_rate": 7.973761550607747e-05,
+      "loss": 0.6287842178344727,
+      "mean_token_accuracy": 0.8173542984851121,
+      "num_tokens": 1127140.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6624361242353917,
+      "epoch": 2.1329376854599404,
+      "grad_norm": 1.0336796045303345,
+      "learning_rate": 7.911624507384729e-05,
+      "loss": 0.5305244064331055,
+      "mean_token_accuracy": 0.8395491230487824,
+      "num_tokens": 1192314.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6706090711057187,
+      "epoch": 2.2516320474777447,
+      "grad_norm": 1.1563575267791748,
+      "learning_rate": 7.842992204004328e-05,
+      "loss": 0.5347850036621093,
+      "mean_token_accuracy": 0.8390352365374565,
+      "num_tokens": 1257658.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6641162340342999,
+      "epoch": 2.370326409495549,
+      "grad_norm": 1.0999572277069092,
+      "learning_rate": 7.767982021114064e-05,
+      "loss": 0.5343616867065429,
+      "mean_token_accuracy": 0.8390876743197441,
+      "num_tokens": 1324523.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6628478536009789,
+      "epoch": 2.489020771513353,
+      "grad_norm": 1.0276364088058472,
+      "learning_rate": 7.68672224733903e-05,
+      "loss": 0.5415428161621094,
+      "mean_token_accuracy": 0.8387553268671035,
+      "num_tokens": 1391419.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.671518052071333,
+      "epoch": 2.6077151335311575,
+      "grad_norm": 0.9451322555541992,
+      "learning_rate": 7.599351859872084e-05,
+      "loss": 0.5410358810424805,
+      "mean_token_accuracy": 0.8373630735278129,
+      "num_tokens": 1456470.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6752241159975528,
+      "epoch": 2.7264094955489613,
+      "grad_norm": 0.8325166702270508,
+      "learning_rate": 7.506020286783527e-05,
+      "loss": 0.5409298706054687,
+      "mean_token_accuracy": 0.8369611689448356,
+      "num_tokens": 1523984.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6462954029440879,
+      "epoch": 2.8451038575667655,
+      "grad_norm": 1.0445443391799927,
+      "learning_rate": 7.406887151456858e-05,
+      "loss": 0.5271347427368164,
+      "mean_token_accuracy": 0.8404733729362488,
+      "num_tokens": 1591962.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.6678441441059113,
+      "epoch": 2.96379821958457,
+      "grad_norm": 0.9832372665405273,
+      "learning_rate": 7.302121999587646e-05,
+      "loss": 0.537381706237793,
+      "mean_token_accuracy": 0.8383750656247139,
+      "num_tokens": 1658478.0,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.524203968675513,
+      "eval_loss": 0.6435813307762146,
+      "eval_mean_token_accuracy": 0.8187350073055915,
+      "eval_num_tokens": 1678731.0,
+      "eval_runtime": 111.7182,
+      "eval_samples_per_second": 12.209,
+      "eval_steps_per_second": 1.531,
+      "step": 1266
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4220,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.244685217283277e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1688/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1688/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04758632698976937,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1688/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1688/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-1688/trainer_state.json

@@ -0,0 +1,408 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 1688,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.4865781700611116,
+      "epoch": 0.11869436201780416,
+      "grad_norm": 3.600203037261963,
+      "learning_rate": 9.55350170221182e-06,
+      "loss": 2.2448199462890623,
+      "mean_token_accuracy": 0.571574572622776,
+      "num_tokens": 66201.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.1757256650924683,
+      "epoch": 0.23738872403560832,
+      "grad_norm": 1.843338966369629,
+      "learning_rate": 1.9301972826917757e-05,
+      "loss": 1.0319316864013672,
+      "mean_token_accuracy": 0.7366894924640656,
+      "num_tokens": 132943.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.0132716038823129,
+      "epoch": 0.3560830860534125,
+      "grad_norm": 2.0066328048706055,
+      "learning_rate": 2.9050443951623695e-05,
+      "loss": 0.8726716613769532,
+      "mean_token_accuracy": 0.7669131025671959,
+      "num_tokens": 202797.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.9535812222957611,
+      "epoch": 0.47477744807121663,
+      "grad_norm": 1.8880268335342407,
+      "learning_rate": 3.8798915076329635e-05,
+      "loss": 0.8159033966064453,
+      "mean_token_accuracy": 0.778084682226181,
+      "num_tokens": 266514.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.9149203181266785,
+      "epoch": 0.5934718100890207,
+      "grad_norm": 1.5609782934188843,
+      "learning_rate": 4.8547386201035576e-05,
+      "loss": 0.7810882568359375,
+      "mean_token_accuracy": 0.7836975249648094,
+      "num_tokens": 332781.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.9015455979108811,
+      "epoch": 0.712166172106825,
+      "grad_norm": 1.6056852340698242,
+      "learning_rate": 5.829585732574152e-05,
+      "loss": 0.7620333099365234,
+      "mean_token_accuracy": 0.7867170104384422,
+      "num_tokens": 396012.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.8751798084378243,
+      "epoch": 0.8308605341246291,
+      "grad_norm": 1.457392930984497,
+      "learning_rate": 6.804432845044745e-05,
+      "loss": 0.7453135681152344,
+      "mean_token_accuracy": 0.7934546408057213,
+      "num_tokens": 464261.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.8582844731211662,
+      "epoch": 0.9495548961424333,
+      "grad_norm": 1.325522541999817,
+      "learning_rate": 7.779279957515339e-05,
+      "loss": 0.7352320861816406,
+      "mean_token_accuracy": 0.7950925189256668,
+      "num_tokens": 531313.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.699295549364815,
+      "eval_loss": 0.6675883531570435,
+      "eval_mean_token_accuracy": 0.8100697049620555,
+      "eval_num_tokens": 559577.0,
+      "eval_runtime": 113.6401,
+      "eval_samples_per_second": 12.003,
+      "eval_steps_per_second": 1.505,
+      "step": 422
+    },
+    {
+      "entropy": 0.8327096131852436,
+      "epoch": 1.0664688427299702,
+      "grad_norm": 0.9726872444152832,
+      "learning_rate": 8.226683697984623e-05,
+      "loss": 0.6989453887939453,
+      "mean_token_accuracy": 0.7999463737918641,
+      "num_tokens": 595986.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7919371470808982,
+      "epoch": 1.1851632047477745,
+      "grad_norm": 1.04917311668396,
+      "learning_rate": 8.219368143379697e-05,
+      "loss": 0.6691493225097657,
+      "mean_token_accuracy": 0.8088096314668656,
+      "num_tokens": 663734.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7981289568543434,
+      "epoch": 1.3038575667655787,
+      "grad_norm": 1.1144819259643555,
+      "learning_rate": 8.205030999972083e-05,
+      "loss": 0.6692163848876953,
+      "mean_token_accuracy": 0.8067614835500717,
+      "num_tokens": 730600.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.7944329422712326,
+      "epoch": 1.4225519287833828,
+      "grad_norm": 1.017581820487976,
+      "learning_rate": 8.183696788331456e-05,
+      "loss": 0.6635546112060546,
+      "mean_token_accuracy": 0.8121234861016273,
+      "num_tokens": 793502.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7757732102274895,
+      "epoch": 1.5412462908011868,
+      "grad_norm": 0.9428858160972595,
+      "learning_rate": 8.155401995992886e-05,
+      "loss": 0.6541387939453125,
+      "mean_token_accuracy": 0.8148220491409301,
+      "num_tokens": 860956.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7693249759078026,
+      "epoch": 1.659940652818991,
+      "grad_norm": 0.8914806842803955,
+      "learning_rate": 8.120195015052839e-05,
+      "loss": 0.6372745132446289,
+      "mean_token_accuracy": 0.8154754737019538,
+      "num_tokens": 928211.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7677849313616752,
+      "epoch": 1.7786350148367953,
+      "grad_norm": 1.2037655115127563,
+      "learning_rate": 8.078136059405015e-05,
+      "loss": 0.6468383026123047,
+      "mean_token_accuracy": 0.8125368970632553,
+      "num_tokens": 993921.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.7699458369612694,
+      "epoch": 1.8973293768545996,
+      "grad_norm": 0.8997814059257507,
+      "learning_rate": 8.02929706175755e-05,
+      "loss": 0.6416602325439453,
+      "mean_token_accuracy": 0.8159067538380623,
+      "num_tokens": 1060568.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6038547956455521,
+      "eval_loss": 0.6323259472846985,
+      "eval_mean_token_accuracy": 0.8163558896522076,
+      "eval_num_tokens": 1119154.0,
+      "eval_runtime": 111.6364,
+      "eval_samples_per_second": 12.218,
+      "eval_steps_per_second": 1.532,
+      "step": 844
+    },
+    {
+      "entropy": 0.7529444349598764,
+      "epoch": 2.0142433234421366,
+      "grad_norm": 0.9705535173416138,
+      "learning_rate": 7.973761550607747e-05,
+      "loss": 0.6287842178344727,
+      "mean_token_accuracy": 0.8173542984851121,
+      "num_tokens": 1127140.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6624361242353917,
+      "epoch": 2.1329376854599404,
+      "grad_norm": 1.0336796045303345,
+      "learning_rate": 7.911624507384729e-05,
+      "loss": 0.5305244064331055,
+      "mean_token_accuracy": 0.8395491230487824,
+      "num_tokens": 1192314.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6706090711057187,
+      "epoch": 2.2516320474777447,
+      "grad_norm": 1.1563575267791748,
+      "learning_rate": 7.842992204004328e-05,
+      "loss": 0.5347850036621093,
+      "mean_token_accuracy": 0.8390352365374565,
+      "num_tokens": 1257658.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6641162340342999,
+      "epoch": 2.370326409495549,
+      "grad_norm": 1.0999572277069092,
+      "learning_rate": 7.767982021114064e-05,
+      "loss": 0.5343616867065429,
+      "mean_token_accuracy": 0.8390876743197441,
+      "num_tokens": 1324523.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6628478536009789,
+      "epoch": 2.489020771513353,
+      "grad_norm": 1.0276364088058472,
+      "learning_rate": 7.68672224733903e-05,
+      "loss": 0.5415428161621094,
+      "mean_token_accuracy": 0.8387553268671035,
+      "num_tokens": 1391419.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.671518052071333,
+      "epoch": 2.6077151335311575,
+      "grad_norm": 0.9451322555541992,
+      "learning_rate": 7.599351859872084e-05,
+      "loss": 0.5410358810424805,
+      "mean_token_accuracy": 0.8373630735278129,
+      "num_tokens": 1456470.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6752241159975528,
+      "epoch": 2.7264094955489613,
+      "grad_norm": 0.8325166702270508,
+      "learning_rate": 7.506020286783527e-05,
+      "loss": 0.5409298706054687,
+      "mean_token_accuracy": 0.8369611689448356,
+      "num_tokens": 1523984.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6462954029440879,
+      "epoch": 2.8451038575667655,
+      "grad_norm": 1.0445443391799927,
+      "learning_rate": 7.406887151456858e-05,
+      "loss": 0.5271347427368164,
+      "mean_token_accuracy": 0.8404733729362488,
+      "num_tokens": 1591962.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.6678441441059113,
+      "epoch": 2.96379821958457,
+      "grad_norm": 0.9832372665405273,
+      "learning_rate": 7.302121999587646e-05,
+      "loss": 0.537381706237793,
+      "mean_token_accuracy": 0.8383750656247139,
+      "num_tokens": 1658478.0,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.524203968675513,
+      "eval_loss": 0.6435813307762146,
+      "eval_mean_token_accuracy": 0.8187350073055915,
+      "eval_num_tokens": 1678731.0,
+      "eval_runtime": 111.7182,
+      "eval_samples_per_second": 12.209,
+      "eval_steps_per_second": 1.531,
+      "step": 1266
+    },
+    {
+      "entropy": 0.5911264679758682,
+      "epoch": 3.080712166172107,
+      "grad_norm": 1.2129673957824707,
+      "learning_rate": 7.19190400921244e-05,
+      "loss": 0.44908695220947265,
+      "mean_token_accuracy": 0.8601035639114186,
+      "num_tokens": 1725154.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5566366592049599,
+      "epoch": 3.199406528189911,
+      "grad_norm": 0.9602940678596497,
+      "learning_rate": 7.076421684263661e-05,
+      "loss": 0.4135295867919922,
+      "mean_token_accuracy": 0.8689188846945762,
+      "num_tokens": 1792562.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5605658321082592,
+      "epoch": 3.318100890207715,
+      "grad_norm": 1.0783617496490479,
+      "learning_rate": 6.955872532174566e-05,
+      "loss": 0.41924549102783204,
+      "mean_token_accuracy": 0.8669222807884216,
+      "num_tokens": 1858017.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5531888791918754,
+      "epoch": 3.436795252225519,
+      "grad_norm": 1.285948395729065,
+      "learning_rate": 6.830462726085685e-05,
+      "loss": 0.41391544342041015,
+      "mean_token_accuracy": 0.8701067119836807,
+      "num_tokens": 1925739.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5541697943210602,
+      "epoch": 3.5554896142433234,
+      "grad_norm": 1.3446345329284668,
+      "learning_rate": 6.700406752230453e-05,
+      "loss": 0.42396705627441406,
+      "mean_token_accuracy": 0.8686600789427757,
+      "num_tokens": 1992896.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5668759573996067,
+      "epoch": 3.6741839762611277,
+      "grad_norm": 1.1999047994613647,
+      "learning_rate": 6.565927043103079e-05,
+      "loss": 0.42777458190917966,
+      "mean_token_accuracy": 0.8663509142398834,
+      "num_tokens": 2057970.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5705659487843513,
+      "epoch": 3.792878338278932,
+      "grad_norm": 1.1414515972137451,
+      "learning_rate": 6.427253597036095e-05,
+      "loss": 0.42880672454833985,
+      "mean_token_accuracy": 0.8653362435102463,
+      "num_tokens": 2123534.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.558414245545864,
+      "epoch": 3.9115727002967358,
+      "grad_norm": 1.3412097692489624,
+      "learning_rate": 6.284623584838158e-05,
+      "loss": 0.4282422256469727,
+      "mean_token_accuracy": 0.866187039911747,
+      "num_tokens": 2189979.0,
+      "step": 1650
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.4868207575633512,
+      "eval_loss": 0.6648371815681458,
+      "eval_mean_token_accuracy": 0.8163467548046893,
+      "eval_num_tokens": 2238308.0,
+      "eval_runtime": 111.72,
+      "eval_samples_per_second": 12.209,
+      "eval_steps_per_second": 1.531,
+      "step": 1688
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4220,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.33006276254679e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-2110/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-2110/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04758632698976937,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-2110/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-2110/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

productivity_original_Estonian/Qwen3.5-4B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/checkpoint-2110/trainer_state.json

@@ -0,0 +1,509 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 2110,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.4865781700611116,
+      "epoch": 0.11869436201780416,
+      "grad_norm": 3.600203037261963,
+      "learning_rate": 9.55350170221182e-06,
+      "loss": 2.2448199462890623,
+      "mean_token_accuracy": 0.571574572622776,
+      "num_tokens": 66201.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.1757256650924683,
+      "epoch": 0.23738872403560832,
+      "grad_norm": 1.843338966369629,
+      "learning_rate": 1.9301972826917757e-05,
+      "loss": 1.0319316864013672,
+      "mean_token_accuracy": 0.7366894924640656,
+      "num_tokens": 132943.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.0132716038823129,
+      "epoch": 0.3560830860534125,
+      "grad_norm": 2.0066328048706055,
+      "learning_rate": 2.9050443951623695e-05,
+      "loss": 0.8726716613769532,
+      "mean_token_accuracy": 0.7669131025671959,
+      "num_tokens": 202797.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.9535812222957611,
+      "epoch": 0.47477744807121663,
+      "grad_norm": 1.8880268335342407,
+      "learning_rate": 3.8798915076329635e-05,
+      "loss": 0.8159033966064453,
+      "mean_token_accuracy": 0.778084682226181,
+      "num_tokens": 266514.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.9149203181266785,
+      "epoch": 0.5934718100890207,
+      "grad_norm": 1.5609782934188843,
+      "learning_rate": 4.8547386201035576e-05,
+      "loss": 0.7810882568359375,
+      "mean_token_accuracy": 0.7836975249648094,
+      "num_tokens": 332781.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.9015455979108811,
+      "epoch": 0.712166172106825,
+      "grad_norm": 1.6056852340698242,
+      "learning_rate": 5.829585732574152e-05,
+      "loss": 0.7620333099365234,
+      "mean_token_accuracy": 0.7867170104384422,
+      "num_tokens": 396012.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.8751798084378243,
+      "epoch": 0.8308605341246291,
+      "grad_norm": 1.457392930984497,
+      "learning_rate": 6.804432845044745e-05,
+      "loss": 0.7453135681152344,
+      "mean_token_accuracy": 0.7934546408057213,
+      "num_tokens": 464261.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.8582844731211662,
+      "epoch": 0.9495548961424333,
+      "grad_norm": 1.325522541999817,
+      "learning_rate": 7.779279957515339e-05,
+      "loss": 0.7352320861816406,
+      "mean_token_accuracy": 0.7950925189256668,
+      "num_tokens": 531313.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.699295549364815,
+      "eval_loss": 0.6675883531570435,
+      "eval_mean_token_accuracy": 0.8100697049620555,
+      "eval_num_tokens": 559577.0,
+      "eval_runtime": 113.6401,
+      "eval_samples_per_second": 12.003,
+      "eval_steps_per_second": 1.505,
+      "step": 422
+    },
+    {
+      "entropy": 0.8327096131852436,
+      "epoch": 1.0664688427299702,
+      "grad_norm": 0.9726872444152832,
+      "learning_rate": 8.226683697984623e-05,
+      "loss": 0.6989453887939453,
+      "mean_token_accuracy": 0.7999463737918641,
+      "num_tokens": 595986.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7919371470808982,
+      "epoch": 1.1851632047477745,
+      "grad_norm": 1.04917311668396,
+      "learning_rate": 8.219368143379697e-05,
+      "loss": 0.6691493225097657,
+      "mean_token_accuracy": 0.8088096314668656,
+      "num_tokens": 663734.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7981289568543434,
+      "epoch": 1.3038575667655787,
+      "grad_norm": 1.1144819259643555,
+      "learning_rate": 8.205030999972083e-05,
+      "loss": 0.6692163848876953,
+      "mean_token_accuracy": 0.8067614835500717,
+      "num_tokens": 730600.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.7944329422712326,
+      "epoch": 1.4225519287833828,
+      "grad_norm": 1.017581820487976,
+      "learning_rate": 8.183696788331456e-05,
+      "loss": 0.6635546112060546,
+      "mean_token_accuracy": 0.8121234861016273,
+      "num_tokens": 793502.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7757732102274895,
+      "epoch": 1.5412462908011868,
+      "grad_norm": 0.9428858160972595,
+      "learning_rate": 8.155401995992886e-05,
+      "loss": 0.6541387939453125,
+      "mean_token_accuracy": 0.8148220491409301,
+      "num_tokens": 860956.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7693249759078026,
+      "epoch": 1.659940652818991,
+      "grad_norm": 0.8914806842803955,
+      "learning_rate": 8.120195015052839e-05,
+      "loss": 0.6372745132446289,
+      "mean_token_accuracy": 0.8154754737019538,
+      "num_tokens": 928211.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7677849313616752,
+      "epoch": 1.7786350148367953,
+      "grad_norm": 1.2037655115127563,
+      "learning_rate": 8.078136059405015e-05,
+      "loss": 0.6468383026123047,
+      "mean_token_accuracy": 0.8125368970632553,
+      "num_tokens": 993921.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.7699458369612694,
+      "epoch": 1.8973293768545996,
+      "grad_norm": 0.8997814059257507,
+      "learning_rate": 8.02929706175755e-05,
+      "loss": 0.6416602325439453,
+      "mean_token_accuracy": 0.8159067538380623,
+      "num_tokens": 1060568.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6038547956455521,
+      "eval_loss": 0.6323259472846985,
+      "eval_mean_token_accuracy": 0.8163558896522076,
+      "eval_num_tokens": 1119154.0,
+      "eval_runtime": 111.6364,
+      "eval_samples_per_second": 12.218,
+      "eval_steps_per_second": 1.532,
+      "step": 844
+    },
+    {
+      "entropy": 0.7529444349598764,
+      "epoch": 2.0142433234421366,
+      "grad_norm": 0.9705535173416138,
+      "learning_rate": 7.973761550607747e-05,
+      "loss": 0.6287842178344727,
+      "mean_token_accuracy": 0.8173542984851121,
+      "num_tokens": 1127140.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6624361242353917,
+      "epoch": 2.1329376854599404,
+      "grad_norm": 1.0336796045303345,
+      "learning_rate": 7.911624507384729e-05,
+      "loss": 0.5305244064331055,
+      "mean_token_accuracy": 0.8395491230487824,
+      "num_tokens": 1192314.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6706090711057187,
+      "epoch": 2.2516320474777447,
+      "grad_norm": 1.1563575267791748,
+      "learning_rate": 7.842992204004328e-05,
+      "loss": 0.5347850036621093,
+      "mean_token_accuracy": 0.8390352365374565,
+      "num_tokens": 1257658.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6641162340342999,
+      "epoch": 2.370326409495549,
+      "grad_norm": 1.0999572277069092,
+      "learning_rate": 7.767982021114064e-05,
+      "loss": 0.5343616867065429,
+      "mean_token_accuracy": 0.8390876743197441,
+      "num_tokens": 1324523.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6628478536009789,
+      "epoch": 2.489020771513353,
+      "grad_norm": 1.0276364088058472,
+      "learning_rate": 7.68672224733903e-05,
+      "loss": 0.5415428161621094,
+      "mean_token_accuracy": 0.8387553268671035,
+      "num_tokens": 1391419.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.671518052071333,
+      "epoch": 2.6077151335311575,
+      "grad_norm": 0.9451322555541992,
+      "learning_rate": 7.599351859872084e-05,
+      "loss": 0.5410358810424805,
+      "mean_token_accuracy": 0.8373630735278129,
+      "num_tokens": 1456470.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6752241159975528,
+      "epoch": 2.7264094955489613,
+      "grad_norm": 0.8325166702270508,
+      "learning_rate": 7.506020286783527e-05,
+      "loss": 0.5409298706054687,
+      "mean_token_accuracy": 0.8369611689448356,
+      "num_tokens": 1523984.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6462954029440879,
+      "epoch": 2.8451038575667655,
+      "grad_norm": 1.0445443391799927,
+      "learning_rate": 7.406887151456858e-05,
+      "loss": 0.5271347427368164,
+      "mean_token_accuracy": 0.8404733729362488,
+      "num_tokens": 1591962.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.6678441441059113,
+      "epoch": 2.96379821958457,
+      "grad_norm": 0.9832372665405273,
+      "learning_rate": 7.302121999587646e-05,
+      "loss": 0.537381706237793,
+      "mean_token_accuracy": 0.8383750656247139,
+      "num_tokens": 1658478.0,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.524203968675513,
+      "eval_loss": 0.6435813307762146,
+      "eval_mean_token_accuracy": 0.8187350073055915,
+      "eval_num_tokens": 1678731.0,
+      "eval_runtime": 111.7182,
+      "eval_samples_per_second": 12.209,
+      "eval_steps_per_second": 1.531,
+      "step": 1266
+    },
+    {
+      "entropy": 0.5911264679758682,
+      "epoch": 3.080712166172107,
+      "grad_norm": 1.2129673957824707,
+      "learning_rate": 7.19190400921244e-05,
+      "loss": 0.44908695220947265,
+      "mean_token_accuracy": 0.8601035639114186,
+      "num_tokens": 1725154.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5566366592049599,
+      "epoch": 3.199406528189911,
+      "grad_norm": 0.9602940678596497,
+      "learning_rate": 7.076421684263661e-05,
+      "loss": 0.4135295867919922,
+      "mean_token_accuracy": 0.8689188846945762,
+      "num_tokens": 1792562.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5605658321082592,
+      "epoch": 3.318100890207715,
+      "grad_norm": 1.0783617496490479,
+      "learning_rate": 6.955872532174566e-05,
+      "loss": 0.41924549102783204,
+      "mean_token_accuracy": 0.8669222807884216,
+      "num_tokens": 1858017.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5531888791918754,
+      "epoch": 3.436795252225519,
+      "grad_norm": 1.285948395729065,
+      "learning_rate": 6.830462726085685e-05,
+      "loss": 0.41391544342041015,
+      "mean_token_accuracy": 0.8701067119836807,
+      "num_tokens": 1925739.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5541697943210602,
+      "epoch": 3.5554896142433234,
+      "grad_norm": 1.3446345329284668,
+      "learning_rate": 6.700406752230453e-05,
+      "loss": 0.42396705627441406,
+      "mean_token_accuracy": 0.8686600789427757,
+      "num_tokens": 1992896.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5668759573996067,
+      "epoch": 3.6741839762611277,
+      "grad_norm": 1.1999047994613647,
+      "learning_rate": 6.565927043103079e-05,
+      "loss": 0.42777458190917966,
+      "mean_token_accuracy": 0.8663509142398834,
+      "num_tokens": 2057970.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5705659487843513,
+      "epoch": 3.792878338278932,
+      "grad_norm": 1.1414515972137451,
+      "learning_rate": 6.427253597036095e-05,
+      "loss": 0.42880672454833985,
+      "mean_token_accuracy": 0.8653362435102463,
+      "num_tokens": 2123534.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.558414245545864,
+      "epoch": 3.9115727002967358,
+      "grad_norm": 1.3412097692489624,
+      "learning_rate": 6.284623584838158e-05,
+      "loss": 0.4282422256469727,
+      "mean_token_accuracy": 0.866187039911747,
+      "num_tokens": 2189979.0,
+      "step": 1650
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.4868207575633512,
+      "eval_loss": 0.6648371815681458,
+      "eval_mean_token_accuracy": 0.8163467548046893,
+      "eval_num_tokens": 2238308.0,
+      "eval_runtime": 111.72,
+      "eval_samples_per_second": 12.209,
+      "eval_steps_per_second": 1.531,
+      "step": 1688
+    },
+    {
+      "entropy": 0.546174580978258,
+      "epoch": 4.028486646884273,
+      "grad_norm": 1.464382290840149,
+      "learning_rate": 6.138280944164903e-05,
+      "loss": 0.40503074645996096,
+      "mean_token_accuracy": 0.8718915990161412,
+      "num_tokens": 2253247.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.45237040892243385,
+      "epoch": 4.147181008902077,
+      "grad_norm": 1.7151323556900024,
+      "learning_rate": 5.988475962316552e-05,
+      "loss": 0.3065692901611328,
+      "mean_token_accuracy": 0.900569304227829,
+      "num_tokens": 2320148.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.44834635987877847,
+      "epoch": 4.265875370919881,
+      "grad_norm": 1.2050637006759644,
+      "learning_rate": 5.835464848175874e-05,
+      "loss": 0.30684595108032225,
+      "mean_token_accuracy": 0.9003708437085152,
+      "num_tokens": 2386764.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4549902780354023,
+      "epoch": 4.384569732937686,
+      "grad_norm": 1.20978844165802,
+      "learning_rate": 5.679509294018524e-05,
+      "loss": 0.3107210350036621,
+      "mean_token_accuracy": 0.8997164958715439,
+      "num_tokens": 2453770.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.46374980479478833,
+      "epoch": 4.503264094955489,
+      "grad_norm": 1.0553879737854004,
+      "learning_rate": 5.520876027945252e-05,
+      "loss": 0.3163416862487793,
+      "mean_token_accuracy": 0.8980184662342071,
+      "num_tokens": 2520198.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.4559279951453209,
+      "epoch": 4.621958456973294,
+      "grad_norm": 1.2723990678787231,
+      "learning_rate": 5.359836357701423e-05,
+      "loss": 0.31503250122070314,
+      "mean_token_accuracy": 0.8980488586425781,
+      "num_tokens": 2587420.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.45531487330794335,
+      "epoch": 4.740652818991098,
+      "grad_norm": 1.3452478647232056,
+      "learning_rate": 5.1966657066640514e-05,
+      "loss": 0.3135023880004883,
+      "mean_token_accuracy": 0.8982085168361664,
+      "num_tokens": 2652728.0,
+      "step": 2000
+    },
+    {
+      "entropy": 0.4609416849911213,
+      "epoch": 4.859347181008902,
+      "grad_norm": 1.37790846824646,
+      "learning_rate": 5.0316431427899296e-05,
+      "loss": 0.3144682502746582,
+      "mean_token_accuracy": 0.8983592641353607,
+      "num_tokens": 2718527.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.45369990602135657,
+      "epoch": 4.978041543026706,
+      "grad_norm": 1.4144543409347534,
+      "learning_rate": 4.865050901330515e-05,
+      "loss": 0.31526716232299806,
+      "mean_token_accuracy": 0.8976324373483657,
+      "num_tokens": 2785760.0,
+      "step": 2100
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.4259050552956542,
+      "eval_loss": 0.7592839002609253,
+      "eval_mean_token_accuracy": 0.8117133866973788,
+      "eval_num_tokens": 2797885.0,
+      "eval_runtime": 111.9255,
+      "eval_samples_per_second": 12.187,
+      "eval_steps_per_second": 1.528,
+      "step": 2110
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4220,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0419291201852211e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}