Add files using upload-large-folder tool

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-100/trainer_state.json

@@ -0,0 +1,139 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.24906600249066002,
+  "eval_steps": 20,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9823576763965440.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1000/trainer_state.json

@@ -0,0 +1,1084 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.488169364881694,
+  "eval_steps": 20,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.859037950771814e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1020/trainer_state.json

@@ -0,0 +1,1105 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5379825653798256,
+  "eval_steps": 20,
+  "global_step": 1020,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0076952699436032e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1040/trainer_state.json

@@ -0,0 +1,1126 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.587795765877958,
+  "eval_steps": 20,
+  "global_step": 1040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0254458345271091e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1060/trainer_state.json

@@ -0,0 +1,1147 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6376089663760895,
+  "eval_steps": 20,
+  "global_step": 1060,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.045743734380032e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1080/trainer_state.json

@@ -0,0 +1,1168 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6874221668742218,
+  "eval_steps": 20,
+  "global_step": 1080,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    },
+    {
+      "entropy": 0.4605010639876127,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6740535497665405,
+      "learning_rate": 0.00018463600382686253,
+      "loss": 0.4123940944671631,
+      "mean_token_accuracy": 0.8733638986945153,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.47902208583992584,
+      "eval_loss": 0.5372340083122253,
+      "eval_mean_token_accuracy": 0.851325950303743,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.9638,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.978,
+      "step": 1080
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0682640451304448e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1100/trainer_state.json

@@ -0,0 +1,1189 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.7372353673723535,
+  "eval_steps": 20,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    },
+    {
+      "entropy": 0.4605010639876127,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6740535497665405,
+      "learning_rate": 0.00018463600382686253,
+      "loss": 0.4123940944671631,
+      "mean_token_accuracy": 0.8733638986945153,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.47902208583992584,
+      "eval_loss": 0.5372340083122253,
+      "eval_mean_token_accuracy": 0.851325950303743,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.9638,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.978,
+      "step": 1080
+    },
+    {
+      "entropy": 0.4872019402682781,
+      "epoch": 2.7372353673723535,
+      "grad_norm": 0.6994742155075073,
+      "learning_rate": 0.0001836528239216632,
+      "loss": 0.41599602699279786,
+      "mean_token_accuracy": 0.872775862365961,
+      "num_tokens": 2572537.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7372353673723535,
+      "eval_entropy": 0.4893243626453156,
+      "eval_loss": 0.5327795743942261,
+      "eval_mean_token_accuracy": 0.8537560302850812,
+      "eval_num_tokens": 2572537.0,
+      "eval_runtime": 86.823,
+      "eval_samples_per_second": 15.848,
+      "eval_steps_per_second": 1.981,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0877774590688256e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1120/trainer_state.json

@@ -0,0 +1,1210 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.7870485678704857,
+  "eval_steps": 20,
+  "global_step": 1120,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    },
+    {
+      "entropy": 0.4605010639876127,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6740535497665405,
+      "learning_rate": 0.00018463600382686253,
+      "loss": 0.4123940944671631,
+      "mean_token_accuracy": 0.8733638986945153,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.47902208583992584,
+      "eval_loss": 0.5372340083122253,
+      "eval_mean_token_accuracy": 0.851325950303743,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.9638,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.978,
+      "step": 1080
+    },
+    {
+      "entropy": 0.4872019402682781,
+      "epoch": 2.7372353673723535,
+      "grad_norm": 0.6994742155075073,
+      "learning_rate": 0.0001836528239216632,
+      "loss": 0.41599602699279786,
+      "mean_token_accuracy": 0.872775862365961,
+      "num_tokens": 2572537.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7372353673723535,
+      "eval_entropy": 0.4893243626453156,
+      "eval_loss": 0.5327795743942261,
+      "eval_mean_token_accuracy": 0.8537560302850812,
+      "eval_num_tokens": 2572537.0,
+      "eval_runtime": 86.823,
+      "eval_samples_per_second": 15.848,
+      "eval_steps_per_second": 1.981,
+      "step": 1100
+    },
+    {
+      "entropy": 0.4949610233306885,
+      "epoch": 2.7870485678704857,
+      "grad_norm": 0.9605912566184998,
+      "learning_rate": 0.0001826446496671543,
+      "loss": 0.4266993045806885,
+      "mean_token_accuracy": 0.8688005246222019,
+      "num_tokens": 2616047.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.7870485678704857,
+      "eval_entropy": 0.5061517927882283,
+      "eval_loss": 0.5356810092926025,
+      "eval_mean_token_accuracy": 0.8524593568818514,
+      "eval_num_tokens": 2616047.0,
+      "eval_runtime": 86.8385,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.981,
+      "step": 1120
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1058529480242586e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1140/trainer_state.json

@@ -0,0 +1,1231 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.8368617683686175,
+  "eval_steps": 20,
+  "global_step": 1140,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    },
+    {
+      "entropy": 0.4605010639876127,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6740535497665405,
+      "learning_rate": 0.00018463600382686253,
+      "loss": 0.4123940944671631,
+      "mean_token_accuracy": 0.8733638986945153,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.47902208583992584,
+      "eval_loss": 0.5372340083122253,
+      "eval_mean_token_accuracy": 0.851325950303743,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.9638,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.978,
+      "step": 1080
+    },
+    {
+      "entropy": 0.4872019402682781,
+      "epoch": 2.7372353673723535,
+      "grad_norm": 0.6994742155075073,
+      "learning_rate": 0.0001836528239216632,
+      "loss": 0.41599602699279786,
+      "mean_token_accuracy": 0.872775862365961,
+      "num_tokens": 2572537.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7372353673723535,
+      "eval_entropy": 0.4893243626453156,
+      "eval_loss": 0.5327795743942261,
+      "eval_mean_token_accuracy": 0.8537560302850812,
+      "eval_num_tokens": 2572537.0,
+      "eval_runtime": 86.823,
+      "eval_samples_per_second": 15.848,
+      "eval_steps_per_second": 1.981,
+      "step": 1100
+    },
+    {
+      "entropy": 0.4949610233306885,
+      "epoch": 2.7870485678704857,
+      "grad_norm": 0.9605912566184998,
+      "learning_rate": 0.0001826446496671543,
+      "loss": 0.4266993045806885,
+      "mean_token_accuracy": 0.8688005246222019,
+      "num_tokens": 2616047.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.7870485678704857,
+      "eval_entropy": 0.5061517927882283,
+      "eval_loss": 0.5356810092926025,
+      "eval_mean_token_accuracy": 0.8524593568818514,
+      "eval_num_tokens": 2616047.0,
+      "eval_runtime": 86.8385,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.981,
+      "step": 1120
+    },
+    {
+      "entropy": 0.48401356525719164,
+      "epoch": 2.8368617683686175,
+      "grad_norm": 0.6332499980926514,
+      "learning_rate": 0.00018161178511494022,
+      "loss": 0.42131738662719725,
+      "mean_token_accuracy": 0.8729447312653065,
+      "num_tokens": 2664744.0,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8368617683686175,
+      "eval_entropy": 0.48792897060860035,
+      "eval_loss": 0.5290402173995972,
+      "eval_mean_token_accuracy": 0.853078076659247,
+      "eval_num_tokens": 2664744.0,
+      "eval_runtime": 86.7746,
+      "eval_samples_per_second": 15.857,
+      "eval_steps_per_second": 1.982,
+      "step": 1140
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1277422592347136e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-4B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-4B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-4B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0005183818805460705,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

overgeneralisation_original_Swedish/Qwen3.5-4B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/checkpoint-1160/trainer_state.json

@@ -0,0 +1,1252 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.8866749688667497,
+  "eval_steps": 20,
+  "global_step": 1160,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.9784346982836722,
+      "epoch": 0.049813200498132,
+      "grad_norm": 3.0229668617248535,
+      "learning_rate": 9.526142962415369e-06,
+      "loss": 1.7360023498535155,
+      "mean_token_accuracy": 0.6449888605624438,
+      "num_tokens": 46794.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.049813200498132,
+      "eval_entropy": 1.41506897571475,
+      "eval_loss": 1.1876318454742432,
+      "eval_mean_token_accuracy": 0.734131895525511,
+      "eval_num_tokens": 46794.0,
+      "eval_runtime": 87.8071,
+      "eval_samples_per_second": 15.671,
+      "eval_steps_per_second": 1.959,
+      "step": 20
+    },
+    {
+      "entropy": 1.049924298375845,
+      "epoch": 0.099626400996264,
+      "grad_norm": 1.5795097351074219,
+      "learning_rate": 1.9553661870221022e-05,
+      "loss": 0.8944448471069336,
+      "mean_token_accuracy": 0.7748479396104813,
+      "num_tokens": 90754.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.099626400996264,
+      "eval_entropy": 0.7996658658565476,
+      "eval_loss": 0.7202735543251038,
+      "eval_mean_token_accuracy": 0.8070558306089667,
+      "eval_num_tokens": 90754.0,
+      "eval_runtime": 86.9199,
+      "eval_samples_per_second": 15.831,
+      "eval_steps_per_second": 1.979,
+      "step": 40
+    },
+    {
+      "entropy": 0.7734908878803253,
+      "epoch": 0.149439601494396,
+      "grad_norm": 1.3136248588562012,
+      "learning_rate": 2.9581180778026673e-05,
+      "loss": 0.6780608654022217,
+      "mean_token_accuracy": 0.8168170280754566,
+      "num_tokens": 137472.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.149439601494396,
+      "eval_entropy": 0.7119324009778888,
+      "eval_loss": 0.6554311513900757,
+      "eval_mean_token_accuracy": 0.8215604798738346,
+      "eval_num_tokens": 137472.0,
+      "eval_runtime": 86.8692,
+      "eval_samples_per_second": 15.84,
+      "eval_steps_per_second": 1.98,
+      "step": 60
+    },
+    {
+      "entropy": 0.7071127541363239,
+      "epoch": 0.199252801992528,
+      "grad_norm": 1.387060284614563,
+      "learning_rate": 3.960869968583232e-05,
+      "loss": 0.6382100582122803,
+      "mean_token_accuracy": 0.8229366384446621,
+      "num_tokens": 187408.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.199252801992528,
+      "eval_entropy": 0.6883931482254073,
+      "eval_loss": 0.625065803527832,
+      "eval_mean_token_accuracy": 0.828940509710201,
+      "eval_num_tokens": 187408.0,
+      "eval_runtime": 86.662,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 80
+    },
+    {
+      "entropy": 0.6800824083387852,
+      "epoch": 0.24906600249066002,
+      "grad_norm": 0.9892916679382324,
+      "learning_rate": 4.963621859363797e-05,
+      "loss": 0.6011715888977051,
+      "mean_token_accuracy": 0.8323964163661003,
+      "num_tokens": 234197.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24906600249066002,
+      "eval_entropy": 0.6840810470802839,
+      "eval_loss": 0.6037028431892395,
+      "eval_mean_token_accuracy": 0.8309669033732525,
+      "eval_num_tokens": 234197.0,
+      "eval_runtime": 86.4637,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 100
+    },
+    {
+      "entropy": 0.6776216626167297,
+      "epoch": 0.298879202988792,
+      "grad_norm": 0.8918434977531433,
+      "learning_rate": 5.9663737501443624e-05,
+      "loss": 0.5991742610931396,
+      "mean_token_accuracy": 0.8300838828086853,
+      "num_tokens": 281241.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.298879202988792,
+      "eval_entropy": 0.690427724705186,
+      "eval_loss": 0.5939701795578003,
+      "eval_mean_token_accuracy": 0.8345950186945671,
+      "eval_num_tokens": 281241.0,
+      "eval_runtime": 86.6626,
+      "eval_samples_per_second": 15.878,
+      "eval_steps_per_second": 1.985,
+      "step": 120
+    },
+    {
+      "entropy": 0.6709842771291733,
+      "epoch": 0.34869240348692404,
+      "grad_norm": 0.9135531187057495,
+      "learning_rate": 6.969125640924927e-05,
+      "loss": 0.5914147377014161,
+      "mean_token_accuracy": 0.8314545609056949,
+      "num_tokens": 327393.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.34869240348692404,
+      "eval_entropy": 0.6584504666023476,
+      "eval_loss": 0.5849721431732178,
+      "eval_mean_token_accuracy": 0.8357757236375365,
+      "eval_num_tokens": 327393.0,
+      "eval_runtime": 86.3262,
+      "eval_samples_per_second": 15.94,
+      "eval_steps_per_second": 1.992,
+      "step": 140
+    },
+    {
+      "entropy": 0.6524647936224938,
+      "epoch": 0.398505603985056,
+      "grad_norm": 0.8651587963104248,
+      "learning_rate": 7.971877531705493e-05,
+      "loss": 0.5710843563079834,
+      "mean_token_accuracy": 0.8396127380430698,
+      "num_tokens": 373834.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.398505603985056,
+      "eval_entropy": 0.6283470298661742,
+      "eval_loss": 0.5738973617553711,
+      "eval_mean_token_accuracy": 0.8379981181649274,
+      "eval_num_tokens": 373834.0,
+      "eval_runtime": 86.5619,
+      "eval_samples_per_second": 15.896,
+      "eval_steps_per_second": 1.987,
+      "step": 160
+    },
+    {
+      "entropy": 0.6450445972383022,
+      "epoch": 0.44831880448318806,
+      "grad_norm": 0.8661723732948303,
+      "learning_rate": 8.974629422486058e-05,
+      "loss": 0.5677794933319091,
+      "mean_token_accuracy": 0.8389350369572639,
+      "num_tokens": 422572.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.44831880448318806,
+      "eval_entropy": 0.6142613257086554,
+      "eval_loss": 0.5698265433311462,
+      "eval_mean_token_accuracy": 0.8388577273418737,
+      "eval_num_tokens": 422572.0,
+      "eval_runtime": 86.4443,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 180
+    },
+    {
+      "entropy": 0.6448334597051144,
+      "epoch": 0.49813200498132004,
+      "grad_norm": 0.9662242531776428,
+      "learning_rate": 9.977381313266624e-05,
+      "loss": 0.581433916091919,
+      "mean_token_accuracy": 0.8387043006718159,
+      "num_tokens": 471879.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.49813200498132004,
+      "eval_entropy": 0.6154296522916749,
+      "eval_loss": 0.5660303831100464,
+      "eval_mean_token_accuracy": 0.8412494766850804,
+      "eval_num_tokens": 471879.0,
+      "eval_runtime": 86.3063,
+      "eval_samples_per_second": 15.943,
+      "eval_steps_per_second": 1.993,
+      "step": 200
+    },
+    {
+      "entropy": 0.6376728117465973,
+      "epoch": 0.547945205479452,
+      "grad_norm": 0.7618638873100281,
+      "learning_rate": 0.00010980133204047189,
+      "loss": 0.5678351402282715,
+      "mean_token_accuracy": 0.8404546812176704,
+      "num_tokens": 520984.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.547945205479452,
+      "eval_entropy": 0.6181817033956217,
+      "eval_loss": 0.5663750171661377,
+      "eval_mean_token_accuracy": 0.8388350962899452,
+      "eval_num_tokens": 520984.0,
+      "eval_runtime": 86.5904,
+      "eval_samples_per_second": 15.891,
+      "eval_steps_per_second": 1.986,
+      "step": 220
+    },
+    {
+      "entropy": 0.6303176879882812,
+      "epoch": 0.597758405977584,
+      "grad_norm": 0.7571695446968079,
+      "learning_rate": 0.00011982885094827753,
+      "loss": 0.5502778053283691,
+      "mean_token_accuracy": 0.8429657347500324,
+      "num_tokens": 566596.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.597758405977584,
+      "eval_entropy": 0.6252533817707107,
+      "eval_loss": 0.5570284128189087,
+      "eval_mean_token_accuracy": 0.8427327847064927,
+      "eval_num_tokens": 566596.0,
+      "eval_runtime": 86.4157,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 240
+    },
+    {
+      "entropy": 0.6202544964849949,
+      "epoch": 0.6475716064757161,
+      "grad_norm": 0.6447190642356873,
+      "learning_rate": 0.00012985636985608318,
+      "loss": 0.5485352993011474,
+      "mean_token_accuracy": 0.844165726006031,
+      "num_tokens": 613603.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6475716064757161,
+      "eval_entropy": 0.6441633552312851,
+      "eval_loss": 0.5606644153594971,
+      "eval_mean_token_accuracy": 0.842403513054515,
+      "eval_num_tokens": 613603.0,
+      "eval_runtime": 86.6343,
+      "eval_samples_per_second": 15.883,
+      "eval_steps_per_second": 1.985,
+      "step": 260
+    },
+    {
+      "entropy": 0.6306711677461863,
+      "epoch": 0.6973848069738481,
+      "grad_norm": 0.7869907021522522,
+      "learning_rate": 0.00013988388876388883,
+      "loss": 0.5579307556152344,
+      "mean_token_accuracy": 0.841247134655714,
+      "num_tokens": 658565.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6973848069738481,
+      "eval_entropy": 0.6263934678809587,
+      "eval_loss": 0.5559113025665283,
+      "eval_mean_token_accuracy": 0.8427334743183713,
+      "eval_num_tokens": 658565.0,
+      "eval_runtime": 86.6403,
+      "eval_samples_per_second": 15.882,
+      "eval_steps_per_second": 1.985,
+      "step": 280
+    },
+    {
+      "entropy": 0.6385872110724449,
+      "epoch": 0.7471980074719801,
+      "grad_norm": 0.6679229736328125,
+      "learning_rate": 0.0001499114076716945,
+      "loss": 0.5667279720306396,
+      "mean_token_accuracy": 0.8389136254787445,
+      "num_tokens": 705680.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7471980074719801,
+      "eval_entropy": 0.6141417321077612,
+      "eval_loss": 0.5570600628852844,
+      "eval_mean_token_accuracy": 0.8437647996253745,
+      "eval_num_tokens": 705680.0,
+      "eval_runtime": 86.7588,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 300
+    },
+    {
+      "entropy": 0.6199494235217571,
+      "epoch": 0.797011207970112,
+      "grad_norm": 0.7924400568008423,
+      "learning_rate": 0.00015993892657950015,
+      "loss": 0.5529299736022949,
+      "mean_token_accuracy": 0.8426973208785057,
+      "num_tokens": 752616.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.797011207970112,
+      "eval_entropy": 0.6133768925833147,
+      "eval_loss": 0.556602418422699,
+      "eval_mean_token_accuracy": 0.8432947965555413,
+      "eval_num_tokens": 752616.0,
+      "eval_runtime": 86.492,
+      "eval_samples_per_second": 15.909,
+      "eval_steps_per_second": 1.989,
+      "step": 320
+    },
+    {
+      "entropy": 0.6203986253589392,
+      "epoch": 0.8468244084682441,
+      "grad_norm": 0.8364354372024536,
+      "learning_rate": 0.00016996644548730578,
+      "loss": 0.5551144123077393,
+      "mean_token_accuracy": 0.8432973213493824,
+      "num_tokens": 797151.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8468244084682441,
+      "eval_entropy": 0.6017442844634833,
+      "eval_loss": 0.5566568374633789,
+      "eval_mean_token_accuracy": 0.8437666123689607,
+      "eval_num_tokens": 797151.0,
+      "eval_runtime": 86.5552,
+      "eval_samples_per_second": 15.897,
+      "eval_steps_per_second": 1.987,
+      "step": 340
+    },
+    {
+      "entropy": 0.6341533534228802,
+      "epoch": 0.8966376089663761,
+      "grad_norm": 0.7783445715904236,
+      "learning_rate": 0.00017999396439511144,
+      "loss": 0.5669133186340332,
+      "mean_token_accuracy": 0.8379446342587471,
+      "num_tokens": 843585.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8966376089663761,
+      "eval_entropy": 0.6055107958788095,
+      "eval_loss": 0.5599350333213806,
+      "eval_mean_token_accuracy": 0.8435030894917112,
+      "eval_num_tokens": 843585.0,
+      "eval_runtime": 86.4814,
+      "eval_samples_per_second": 15.911,
+      "eval_steps_per_second": 1.989,
+      "step": 360
+    },
+    {
+      "entropy": 0.6306198488920927,
+      "epoch": 0.9464508094645081,
+      "grad_norm": 0.8449786901473999,
+      "learning_rate": 0.0001900214833029171,
+      "loss": 0.5739435195922852,
+      "mean_token_accuracy": 0.8393832489848136,
+      "num_tokens": 889842.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.9464508094645081,
+      "eval_entropy": 0.6129532439071078,
+      "eval_loss": 0.5566295981407166,
+      "eval_mean_token_accuracy": 0.8430350880290187,
+      "eval_num_tokens": 889842.0,
+      "eval_runtime": 86.4643,
+      "eval_samples_per_second": 15.914,
+      "eval_steps_per_second": 1.989,
+      "step": 380
+    },
+    {
+      "entropy": 0.6203123550862074,
+      "epoch": 0.9962640099626401,
+      "grad_norm": 0.7334314584732056,
+      "learning_rate": 0.00020004900221072276,
+      "loss": 0.5547565937042236,
+      "mean_token_accuracy": 0.8403573960065842,
+      "num_tokens": 935589.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9962640099626401,
+      "eval_entropy": 0.6275761647279873,
+      "eval_loss": 0.5621116757392883,
+      "eval_mean_token_accuracy": 0.841587379228237,
+      "eval_num_tokens": 935589.0,
+      "eval_runtime": 86.4748,
+      "eval_samples_per_second": 15.912,
+      "eval_steps_per_second": 1.989,
+      "step": 400
+    },
+    {
+      "entropy": 0.5795013002860241,
+      "epoch": 1.0448318804483188,
+      "grad_norm": 0.8858296871185303,
+      "learning_rate": 0.0002015421505577756,
+      "loss": 0.5183939933776855,
+      "mean_token_accuracy": 0.850081592034071,
+      "num_tokens": 980589.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0448318804483188,
+      "eval_entropy": 0.5583065545489622,
+      "eval_loss": 0.5605642199516296,
+      "eval_mean_token_accuracy": 0.8439708411000496,
+      "eval_num_tokens": 980589.0,
+      "eval_runtime": 86.5422,
+      "eval_samples_per_second": 15.9,
+      "eval_steps_per_second": 1.987,
+      "step": 420
+    },
+    {
+      "entropy": 0.5671238023787737,
+      "epoch": 1.0946450809464507,
+      "grad_norm": 0.6882498264312744,
+      "learning_rate": 0.00020150112347025443,
+      "loss": 0.5077326774597168,
+      "mean_token_accuracy": 0.8489868573844432,
+      "num_tokens": 1027852.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.0946450809464507,
+      "eval_entropy": 0.5868900277933409,
+      "eval_loss": 0.5602695345878601,
+      "eval_mean_token_accuracy": 0.8428842161977014,
+      "eval_num_tokens": 1027852.0,
+      "eval_runtime": 86.623,
+      "eval_samples_per_second": 15.885,
+      "eval_steps_per_second": 1.986,
+      "step": 440
+    },
+    {
+      "entropy": 0.5533561781048775,
+      "epoch": 1.1444582814445827,
+      "grad_norm": 0.7717723250389099,
+      "learning_rate": 0.0002014297192297181,
+      "loss": 0.4954517364501953,
+      "mean_token_accuracy": 0.8529035650193691,
+      "num_tokens": 1077649.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1444582814445827,
+      "eval_entropy": 0.5600803743961246,
+      "eval_loss": 0.5608077645301819,
+      "eval_mean_token_accuracy": 0.8445036771685578,
+      "eval_num_tokens": 1077649.0,
+      "eval_runtime": 86.1316,
+      "eval_samples_per_second": 15.976,
+      "eval_steps_per_second": 1.997,
+      "step": 460
+    },
+    {
+      "entropy": 0.5692154694348573,
+      "epoch": 1.1942714819427147,
+      "grad_norm": 0.7322827577590942,
+      "learning_rate": 0.0002013279593707117,
+      "loss": 0.505049467086792,
+      "mean_token_accuracy": 0.8551576808094978,
+      "num_tokens": 1124872.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1942714819427147,
+      "eval_entropy": 0.5732695829383162,
+      "eval_loss": 0.5594323873519897,
+      "eval_mean_token_accuracy": 0.8449713407560836,
+      "eval_num_tokens": 1124872.0,
+      "eval_runtime": 86.2726,
+      "eval_samples_per_second": 15.949,
+      "eval_steps_per_second": 1.994,
+      "step": 480
+    },
+    {
+      "entropy": 0.5817618492990733,
+      "epoch": 1.244084682440847,
+      "grad_norm": 1.1776764392852783,
+      "learning_rate": 0.0002011958745826208,
+      "loss": 0.5137609958648681,
+      "mean_token_accuracy": 0.8521522544324398,
+      "num_tokens": 1168698.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.244084682440847,
+      "eval_entropy": 0.5662581343636957,
+      "eval_loss": 0.5595026016235352,
+      "eval_mean_token_accuracy": 0.8441977164773053,
+      "eval_num_tokens": 1168698.0,
+      "eval_runtime": 86.7261,
+      "eval_samples_per_second": 15.866,
+      "eval_steps_per_second": 1.983,
+      "step": 500
+    },
+    {
+      "entropy": 0.5712925456464291,
+      "epoch": 1.293897882938979,
+      "grad_norm": 0.7960361838340759,
+      "learning_rate": 0.0002010335047004159,
+      "loss": 0.5134767532348633,
+      "mean_token_accuracy": 0.8513577707111836,
+      "num_tokens": 1216679.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.293897882938979,
+      "eval_entropy": 0.5441222797299541,
+      "eval_loss": 0.5535460114479065,
+      "eval_mean_token_accuracy": 0.8450886118550633,
+      "eval_num_tokens": 1216679.0,
+      "eval_runtime": 86.2675,
+      "eval_samples_per_second": 15.95,
+      "eval_steps_per_second": 1.994,
+      "step": 520
+    },
+    {
+      "entropy": 0.5787045754492283,
+      "epoch": 1.3437110834371109,
+      "grad_norm": 0.9205410480499268,
+      "learning_rate": 0.00020084089869263887,
+      "loss": 0.5119701862335205,
+      "mean_token_accuracy": 0.8503516331315041,
+      "num_tokens": 1261365.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.3437110834371109,
+      "eval_entropy": 0.5744457827057949,
+      "eval_loss": 0.5514978766441345,
+      "eval_mean_token_accuracy": 0.845929987901865,
+      "eval_num_tokens": 1261365.0,
+      "eval_runtime": 86.2299,
+      "eval_samples_per_second": 15.957,
+      "eval_steps_per_second": 1.995,
+      "step": 540
+    },
+    {
+      "entropy": 0.5739392962306737,
+      "epoch": 1.3935242839352429,
+      "grad_norm": 0.7475653886795044,
+      "learning_rate": 0.00020061811464663464,
+      "loss": 0.5189042091369629,
+      "mean_token_accuracy": 0.8492388024926185,
+      "num_tokens": 1306879.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3935242839352429,
+      "eval_entropy": 0.6116398271433142,
+      "eval_loss": 0.551732063293457,
+      "eval_mean_token_accuracy": 0.8450756967067719,
+      "eval_num_tokens": 1306879.0,
+      "eval_runtime": 86.6081,
+      "eval_samples_per_second": 15.888,
+      "eval_steps_per_second": 1.986,
+      "step": 560
+    },
+    {
+      "entropy": 0.5755622573196888,
+      "epoch": 1.4433374844333748,
+      "grad_norm": 0.8218411803245544,
+      "learning_rate": 0.00020036521975103286,
+      "loss": 0.5106248378753662,
+      "mean_token_accuracy": 0.8506785586476326,
+      "num_tokens": 1353534.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.4433374844333748,
+      "eval_entropy": 0.5906928708386976,
+      "eval_loss": 0.551278829574585,
+      "eval_mean_token_accuracy": 0.8462819308042526,
+      "eval_num_tokens": 1353534.0,
+      "eval_runtime": 86.5438,
+      "eval_samples_per_second": 15.899,
+      "eval_steps_per_second": 1.987,
+      "step": 580
+    },
+    {
+      "entropy": 0.5694822132587433,
+      "epoch": 1.4931506849315068,
+      "grad_norm": 0.8880652189254761,
+      "learning_rate": 0.00020008229027548475,
+      "loss": 0.5140334606170655,
+      "mean_token_accuracy": 0.8521522797644139,
+      "num_tokens": 1399537.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4931506849315068,
+      "eval_entropy": 0.5599641964532608,
+      "eval_loss": 0.5501875877380371,
+      "eval_mean_token_accuracy": 0.8467660788879838,
+      "eval_num_tokens": 1399537.0,
+      "eval_runtime": 86.6458,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 600
+    },
+    {
+      "entropy": 0.5675108034163714,
+      "epoch": 1.5429638854296388,
+      "grad_norm": 0.837087094783783,
+      "learning_rate": 0.0001997694115476612,
+      "loss": 0.5099846363067627,
+      "mean_token_accuracy": 0.8543680295348167,
+      "num_tokens": 1448422.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.5429638854296388,
+      "eval_entropy": 0.5728072581249614,
+      "eval_loss": 0.5445425510406494,
+      "eval_mean_token_accuracy": 0.8474342175001321,
+      "eval_num_tokens": 1448422.0,
+      "eval_runtime": 86.4859,
+      "eval_samples_per_second": 15.91,
+      "eval_steps_per_second": 1.989,
+      "step": 620
+    },
+    {
+      "entropy": 0.5700885068625212,
+      "epoch": 1.592777085927771,
+      "grad_norm": 0.6598765850067139,
+      "learning_rate": 0.000199426677927519,
+      "loss": 0.5122694969177246,
+      "mean_token_accuracy": 0.8519927568733692,
+      "num_tokens": 1495009.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.592777085927771,
+      "eval_entropy": 0.5476993622128353,
+      "eval_loss": 0.5427973866462708,
+      "eval_mean_token_accuracy": 0.8478512147138285,
+      "eval_num_tokens": 1495009.0,
+      "eval_runtime": 86.4172,
+      "eval_samples_per_second": 15.923,
+      "eval_steps_per_second": 1.99,
+      "step": 640
+    },
+    {
+      "entropy": 0.5829229176044464,
+      "epoch": 1.6425902864259028,
+      "grad_norm": 0.6965194940567017,
+      "learning_rate": 0.00019905419277884342,
+      "loss": 0.5253659725189209,
+      "mean_token_accuracy": 0.8493309423327446,
+      "num_tokens": 1536932.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.6425902864259028,
+      "eval_entropy": 0.5666290084983028,
+      "eval_loss": 0.5467478036880493,
+      "eval_mean_token_accuracy": 0.8479407703460649,
+      "eval_num_tokens": 1536932.0,
+      "eval_runtime": 86.4414,
+      "eval_samples_per_second": 15.918,
+      "eval_steps_per_second": 1.99,
+      "step": 660
+    },
+    {
+      "entropy": 0.5498311135917902,
+      "epoch": 1.692403486924035,
+      "grad_norm": 0.636583685874939,
+      "learning_rate": 0.00019865206843807482,
+      "loss": 0.49981012344360354,
+      "mean_token_accuracy": 0.8560848504304885,
+      "num_tokens": 1585718.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.692403486924035,
+      "eval_entropy": 0.539117265406043,
+      "eval_loss": 0.53994220495224,
+      "eval_mean_token_accuracy": 0.8488582601380903,
+      "eval_num_tokens": 1585718.0,
+      "eval_runtime": 86.5296,
+      "eval_samples_per_second": 15.902,
+      "eval_steps_per_second": 1.988,
+      "step": 680
+    },
+    {
+      "entropy": 0.5543891470879316,
+      "epoch": 1.7422166874221667,
+      "grad_norm": 0.6068442463874817,
+      "learning_rate": 0.0001982204261804297,
+      "loss": 0.498047399520874,
+      "mean_token_accuracy": 0.8554679051041603,
+      "num_tokens": 1635718.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.7422166874221667,
+      "eval_entropy": 0.5703774151760478,
+      "eval_loss": 0.5300245881080627,
+      "eval_mean_token_accuracy": 0.850798153946566,
+      "eval_num_tokens": 1635718.0,
+      "eval_runtime": 86.6456,
+      "eval_samples_per_second": 15.881,
+      "eval_steps_per_second": 1.985,
+      "step": 700
+    },
+    {
+      "entropy": 0.546524541825056,
+      "epoch": 1.792029887920299,
+      "grad_norm": 0.7274155020713806,
+      "learning_rate": 0.00019775939618332566,
+      "loss": 0.4988589286804199,
+      "mean_token_accuracy": 0.853422473371029,
+      "num_tokens": 1681291.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.792029887920299,
+      "eval_entropy": 0.5614905688305234,
+      "eval_loss": 0.5350332260131836,
+      "eval_mean_token_accuracy": 0.8492204359797544,
+      "eval_num_tokens": 1681291.0,
+      "eval_runtime": 86.7581,
+      "eval_samples_per_second": 15.86,
+      "eval_steps_per_second": 1.983,
+      "step": 720
+    },
+    {
+      "entropy": 0.5519792139530182,
+      "epoch": 1.841843088418431,
+      "grad_norm": 0.663466215133667,
+      "learning_rate": 0.00019726911748712167,
+      "loss": 0.5099314212799072,
+      "mean_token_accuracy": 0.848412600159645,
+      "num_tokens": 1729102.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.841843088418431,
+      "eval_entropy": 0.5583519090053647,
+      "eval_loss": 0.530483603477478,
+      "eval_mean_token_accuracy": 0.8500003374593202,
+      "eval_num_tokens": 1729102.0,
+      "eval_runtime": 86.3961,
+      "eval_samples_per_second": 15.927,
+      "eval_steps_per_second": 1.991,
+      "step": 740
+    },
+    {
+      "entropy": 0.5454779766499996,
+      "epoch": 1.891656288916563,
+      "grad_norm": 0.890394926071167,
+      "learning_rate": 0.00019674973795318548,
+      "loss": 0.4931994915008545,
+      "mean_token_accuracy": 0.8540832489728928,
+      "num_tokens": 1773578.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.891656288916563,
+      "eval_entropy": 0.572755502406941,
+      "eval_loss": 0.5415747761726379,
+      "eval_mean_token_accuracy": 0.8444425803284312,
+      "eval_num_tokens": 1773578.0,
+      "eval_runtime": 86.4323,
+      "eval_samples_per_second": 15.92,
+      "eval_steps_per_second": 1.99,
+      "step": 760
+    },
+    {
+      "entropy": 0.5392089951783419,
+      "epoch": 1.9414694894146949,
+      "grad_norm": 0.632411777973175,
+      "learning_rate": 0.00019620141421930058,
+      "loss": 0.4957888603210449,
+      "mean_token_accuracy": 0.8549866065382957,
+      "num_tokens": 1821725.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.9414694894146949,
+      "eval_entropy": 0.540764772961306,
+      "eval_loss": 0.5327216386795044,
+      "eval_mean_token_accuracy": 0.850631088364956,
+      "eval_num_tokens": 1821725.0,
+      "eval_runtime": 86.8097,
+      "eval_samples_per_second": 15.851,
+      "eval_steps_per_second": 1.981,
+      "step": 780
+    },
+    {
+      "entropy": 0.5674678739160299,
+      "epoch": 1.9912826899128269,
+      "grad_norm": 0.6958843469619751,
+      "learning_rate": 0.0001956243116524263,
+      "loss": 0.504389762878418,
+      "mean_token_accuracy": 0.8527948908507824,
+      "num_tokens": 1868431.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9912826899128269,
+      "eval_entropy": 0.530262403190136,
+      "eval_loss": 0.5308871865272522,
+      "eval_mean_token_accuracy": 0.8522498046242913,
+      "eval_num_tokens": 1868431.0,
+      "eval_runtime": 86.7942,
+      "eval_samples_per_second": 15.854,
+      "eval_steps_per_second": 1.982,
+      "step": 800
+    },
+    {
+      "entropy": 0.4742849511213792,
+      "epoch": 2.0398505603985058,
+      "grad_norm": 0.6941492557525635,
+      "learning_rate": 0.00019501860429882556,
+      "loss": 0.418599271774292,
+      "mean_token_accuracy": 0.8748210859604371,
+      "num_tokens": 1915280.0,
+      "step": 820
+    },
+    {
+      "epoch": 2.0398505603985058,
+      "eval_entropy": 0.504602165069691,
+      "eval_loss": 0.542878270149231,
+      "eval_mean_token_accuracy": 0.8507604484641275,
+      "eval_num_tokens": 1915280.0,
+      "eval_runtime": 86.7841,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 820
+    },
+    {
+      "entropy": 0.45857742577791216,
+      "epoch": 2.0896637608966375,
+      "grad_norm": 0.5791997909545898,
+      "learning_rate": 0.00019438447483157478,
+      "loss": 0.399777889251709,
+      "mean_token_accuracy": 0.8754058346152306,
+      "num_tokens": 1965306.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0896637608966375,
+      "eval_entropy": 0.5028848362176918,
+      "eval_loss": 0.5356478095054626,
+      "eval_mean_token_accuracy": 0.8525635412959165,
+      "eval_num_tokens": 1965306.0,
+      "eval_runtime": 86.6707,
+      "eval_samples_per_second": 15.876,
+      "eval_steps_per_second": 1.985,
+      "step": 840
+    },
+    {
+      "entropy": 0.4869446292519569,
+      "epoch": 2.1394769613947697,
+      "grad_norm": 0.6483516693115234,
+      "learning_rate": 0.00019372211449547223,
+      "loss": 0.40715818405151366,
+      "mean_token_accuracy": 0.875113020837307,
+      "num_tokens": 2008562.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.1394769613947697,
+      "eval_entropy": 0.4928991326759028,
+      "eval_loss": 0.5419561862945557,
+      "eval_mean_token_accuracy": 0.8516040146350861,
+      "eval_num_tokens": 2008562.0,
+      "eval_runtime": 87.0686,
+      "eval_samples_per_second": 15.804,
+      "eval_steps_per_second": 1.975,
+      "step": 860
+    },
+    {
+      "entropy": 0.45819590501487256,
+      "epoch": 2.1892901618929015,
+      "grad_norm": 0.6661920547485352,
+      "learning_rate": 0.00019303172304936108,
+      "loss": 0.39511430263519287,
+      "mean_token_accuracy": 0.8780680045485496,
+      "num_tokens": 2056474.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.1892901618929015,
+      "eval_entropy": 0.48602560647698334,
+      "eval_loss": 0.5436084866523743,
+      "eval_mean_token_accuracy": 0.8500938470973525,
+      "eval_num_tokens": 2056474.0,
+      "eval_runtime": 86.6809,
+      "eval_samples_per_second": 15.874,
+      "eval_steps_per_second": 1.984,
+      "step": 880
+    },
+    {
+      "entropy": 0.4780638810247183,
+      "epoch": 2.2391033623910337,
+      "grad_norm": 0.6870484352111816,
+      "learning_rate": 0.0001923135087058851,
+      "loss": 0.4061615467071533,
+      "mean_token_accuracy": 0.8766494184732437,
+      "num_tokens": 2103543.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2391033623910337,
+      "eval_entropy": 0.48236206035281337,
+      "eval_loss": 0.5446090698242188,
+      "eval_mean_token_accuracy": 0.8507725513258646,
+      "eval_num_tokens": 2103543.0,
+      "eval_runtime": 86.7398,
+      "eval_samples_per_second": 15.864,
+      "eval_steps_per_second": 1.983,
+      "step": 900
+    },
+    {
+      "entropy": 0.463029869645834,
+      "epoch": 2.2889165628891655,
+      "grad_norm": 0.6894590854644775,
+      "learning_rate": 0.00019156768806869427,
+      "loss": 0.39602413177490237,
+      "mean_token_accuracy": 0.876420046389103,
+      "num_tokens": 2147861.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2889165628891655,
+      "eval_entropy": 0.4904779093556626,
+      "eval_loss": 0.5404934287071228,
+      "eval_mean_token_accuracy": 0.852238280828609,
+      "eval_num_tokens": 2147861.0,
+      "eval_runtime": 86.5348,
+      "eval_samples_per_second": 15.901,
+      "eval_steps_per_second": 1.988,
+      "step": 920
+    },
+    {
+      "entropy": 0.4817025110125542,
+      "epoch": 2.3387297633872977,
+      "grad_norm": 0.7756227254867554,
+      "learning_rate": 0.00019079448606712033,
+      "loss": 0.4177968502044678,
+      "mean_token_accuracy": 0.8712256088852882,
+      "num_tokens": 2190561.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.3387297633872977,
+      "eval_entropy": 0.5153802815218305,
+      "eval_loss": 0.5424937605857849,
+      "eval_mean_token_accuracy": 0.8506565759348315,
+      "eval_num_tokens": 2190561.0,
+      "eval_runtime": 86.8973,
+      "eval_samples_per_second": 15.835,
+      "eval_steps_per_second": 1.979,
+      "step": 940
+    },
+    {
+      "entropy": 0.46456389091908934,
+      "epoch": 2.3885429638854294,
+      "grad_norm": 1.2000319957733154,
+      "learning_rate": 0.00018999413588834105,
+      "loss": 0.4084665775299072,
+      "mean_token_accuracy": 0.8750658087432385,
+      "num_tokens": 2239412.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.3885429638854294,
+      "eval_entropy": 0.4849439303195754,
+      "eval_loss": 0.545662522315979,
+      "eval_mean_token_accuracy": 0.8491013112456299,
+      "eval_num_tokens": 2239412.0,
+      "eval_runtime": 86.9049,
+      "eval_samples_per_second": 15.833,
+      "eval_steps_per_second": 1.979,
+      "step": 960
+    },
+    {
+      "entropy": 0.4857471022754908,
+      "epoch": 2.4383561643835616,
+      "grad_norm": 0.9696341753005981,
+      "learning_rate": 0.0001891668789070541,
+      "loss": 0.4149796962738037,
+      "mean_token_accuracy": 0.8704176343977451,
+      "num_tokens": 2286283.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.4383561643835616,
+      "eval_entropy": 0.4872790058684904,
+      "eval_loss": 0.5412707924842834,
+      "eval_mean_token_accuracy": 0.8509329602468846,
+      "eval_num_tokens": 2286283.0,
+      "eval_runtime": 86.7846,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 980
+    },
+    {
+      "entropy": 0.4727417893707752,
+      "epoch": 2.488169364881694,
+      "grad_norm": 0.7852500677108765,
+      "learning_rate": 0.0001883129646126818,
+      "loss": 0.4142886161804199,
+      "mean_token_accuracy": 0.8712429471313954,
+      "num_tokens": 2333733.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.488169364881694,
+      "eval_entropy": 0.5386548059624295,
+      "eval_loss": 0.536101222038269,
+      "eval_mean_token_accuracy": 0.8499491239009902,
+      "eval_num_tokens": 2333733.0,
+      "eval_runtime": 86.9501,
+      "eval_samples_per_second": 15.825,
+      "eval_steps_per_second": 1.978,
+      "step": 1000
+    },
+    {
+      "entropy": 0.4673406321555376,
+      "epoch": 2.5379825653798256,
+      "grad_norm": 0.7133921384811401,
+      "learning_rate": 0.0001874326505341286,
+      "loss": 0.40857529640197754,
+      "mean_token_accuracy": 0.8747925907373428,
+      "num_tokens": 2384270.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5379825653798256,
+      "eval_entropy": 0.495788364909416,
+      "eval_loss": 0.5418923497200012,
+      "eval_mean_token_accuracy": 0.851321972040243,
+      "eval_num_tokens": 2384270.0,
+      "eval_runtime": 86.7154,
+      "eval_samples_per_second": 15.868,
+      "eval_steps_per_second": 1.983,
+      "step": 1020
+    },
+    {
+      "entropy": 0.47599745728075504,
+      "epoch": 2.587795765877958,
+      "grad_norm": 0.8202953338623047,
+      "learning_rate": 0.0001865262021621137,
+      "loss": 0.40998234748840334,
+      "mean_token_accuracy": 0.8758242674171924,
+      "num_tokens": 2428036.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.587795765877958,
+      "eval_entropy": 0.4887966953737791,
+      "eval_loss": 0.5408804416656494,
+      "eval_mean_token_accuracy": 0.8512661065473113,
+      "eval_num_tokens": 2428036.0,
+      "eval_runtime": 86.7869,
+      "eval_samples_per_second": 15.855,
+      "eval_steps_per_second": 1.982,
+      "step": 1040
+    },
+    {
+      "entropy": 0.4824396539479494,
+      "epoch": 2.6376089663760895,
+      "grad_norm": 0.6507360935211182,
+      "learning_rate": 0.00018559389286910275,
+      "loss": 0.4165764808654785,
+      "mean_token_accuracy": 0.8722914069890976,
+      "num_tokens": 2476815.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.6376089663760895,
+      "eval_entropy": 0.4793398808254752,
+      "eval_loss": 0.5326959490776062,
+      "eval_mean_token_accuracy": 0.8534493650807891,
+      "eval_num_tokens": 2476815.0,
+      "eval_runtime": 86.9559,
+      "eval_samples_per_second": 15.824,
+      "eval_steps_per_second": 1.978,
+      "step": 1060
+    },
+    {
+      "entropy": 0.4605010639876127,
+      "epoch": 2.6874221668742218,
+      "grad_norm": 0.6740535497665405,
+      "learning_rate": 0.00018463600382686253,
+      "loss": 0.4123940944671631,
+      "mean_token_accuracy": 0.8733638986945153,
+      "num_tokens": 2527131.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6874221668742218,
+      "eval_entropy": 0.47902208583992584,
+      "eval_loss": 0.5372340083122253,
+      "eval_mean_token_accuracy": 0.851325950303743,
+      "eval_num_tokens": 2527131.0,
+      "eval_runtime": 86.9638,
+      "eval_samples_per_second": 15.823,
+      "eval_steps_per_second": 1.978,
+      "step": 1080
+    },
+    {
+      "entropy": 0.4872019402682781,
+      "epoch": 2.7372353673723535,
+      "grad_norm": 0.6994742155075073,
+      "learning_rate": 0.0001836528239216632,
+      "loss": 0.41599602699279786,
+      "mean_token_accuracy": 0.872775862365961,
+      "num_tokens": 2572537.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7372353673723535,
+      "eval_entropy": 0.4893243626453156,
+      "eval_loss": 0.5327795743942261,
+      "eval_mean_token_accuracy": 0.8537560302850812,
+      "eval_num_tokens": 2572537.0,
+      "eval_runtime": 86.823,
+      "eval_samples_per_second": 15.848,
+      "eval_steps_per_second": 1.981,
+      "step": 1100
+    },
+    {
+      "entropy": 0.4949610233306885,
+      "epoch": 2.7870485678704857,
+      "grad_norm": 0.9605912566184998,
+      "learning_rate": 0.0001826446496671543,
+      "loss": 0.4266993045806885,
+      "mean_token_accuracy": 0.8688005246222019,
+      "num_tokens": 2616047.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.7870485678704857,
+      "eval_entropy": 0.5061517927882283,
+      "eval_loss": 0.5356810092926025,
+      "eval_mean_token_accuracy": 0.8524593568818514,
+      "eval_num_tokens": 2616047.0,
+      "eval_runtime": 86.8385,
+      "eval_samples_per_second": 15.846,
+      "eval_steps_per_second": 1.981,
+      "step": 1120
+    },
+    {
+      "entropy": 0.48401356525719164,
+      "epoch": 2.8368617683686175,
+      "grad_norm": 0.6332499980926514,
+      "learning_rate": 0.00018161178511494022,
+      "loss": 0.42131738662719725,
+      "mean_token_accuracy": 0.8729447312653065,
+      "num_tokens": 2664744.0,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8368617683686175,
+      "eval_entropy": 0.48792897060860035,
+      "eval_loss": 0.5290402173995972,
+      "eval_mean_token_accuracy": 0.853078076659247,
+      "eval_num_tokens": 2664744.0,
+      "eval_runtime": 86.7746,
+      "eval_samples_per_second": 15.857,
+      "eval_steps_per_second": 1.982,
+      "step": 1140
+    },
+    {
+      "entropy": 0.48017631396651267,
+      "epoch": 2.8866749688667497,
+      "grad_norm": 0.8013222217559814,
+      "learning_rate": 0.00018055454176288234,
+      "loss": 0.4195223808288574,
+      "mean_token_accuracy": 0.8721428856253624,
+      "num_tokens": 2710196.0,
+      "step": 1160
+    },
+    {
+      "epoch": 2.8866749688667497,
+      "eval_entropy": 0.5205728571082271,
+      "eval_loss": 0.5242091417312622,
+      "eval_mean_token_accuracy": 0.8535184077052183,
+      "eval_num_tokens": 2710196.0,
+      "eval_runtime": 87.009,
+      "eval_samples_per_second": 15.814,
+      "eval_steps_per_second": 1.977,
+      "step": 1160
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4020,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.1472790102826803e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}