Add files using upload-large-folder tool

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: transformers
+model_name: Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/hlci69my) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: transformers
+model_name: Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/qqnxtz0d) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1167/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1167/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.06250292706368335,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1167/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1167/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1167/trainer_state.json

@@ -0,0 +1,297 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1167,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 2.936105194091797,
+      "epoch": 0.1287001287001287,
+      "grad_norm": 2.5512146949768066,
+      "learning_rate": 7.410977855063067e-06,
+      "loss": 2.765989074707031,
+      "mean_token_accuracy": 0.5075639402866363,
+      "num_tokens": 69648.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.4886422049999237,
+      "epoch": 0.2574002574002574,
+      "grad_norm": 2.2254061698913574,
+      "learning_rate": 1.4973200156147831e-05,
+      "loss": 1.336165313720703,
+      "mean_token_accuracy": 0.6740516322851181,
+      "num_tokens": 136672.0,
+      "step": 100
+    },
+    {
+      "entropy": 1.1471627122163772,
+      "epoch": 0.3861003861003861,
+      "grad_norm": 2.2073140144348145,
+      "learning_rate": 2.2535422457232594e-05,
+      "loss": 1.0278042602539061,
+      "mean_token_accuracy": 0.7280837368965148,
+      "num_tokens": 205085.0,
+      "step": 150
+    },
+    {
+      "entropy": 1.0486015713214873,
+      "epoch": 0.5148005148005148,
+      "grad_norm": 1.650594711303711,
+      "learning_rate": 3.0097644758317354e-05,
+      "loss": 0.933094253540039,
+      "mean_token_accuracy": 0.7478678917884827,
+      "num_tokens": 270093.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.9942852139472962,
+      "epoch": 0.6435006435006435,
+      "grad_norm": 1.7949399948120117,
+      "learning_rate": 3.765986705940212e-05,
+      "loss": 0.8747442626953125,
+      "mean_token_accuracy": 0.7603094267845154,
+      "num_tokens": 338177.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.9491432571411133,
+      "epoch": 0.7722007722007722,
+      "grad_norm": 1.33268404006958,
+      "learning_rate": 4.522208936048689e-05,
+      "loss": 0.8404794311523438,
+      "mean_token_accuracy": 0.7671167278289794,
+      "num_tokens": 405099.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.9246599394083023,
+      "epoch": 0.9009009009009009,
+      "grad_norm": 1.3137660026550293,
+      "learning_rate": 5.278431166157165e-05,
+      "loss": 0.8163955688476563,
+      "mean_token_accuracy": 0.7712420308589936,
+      "num_tokens": 474117.0,
+      "step": 350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7913847528782088,
+      "eval_loss": 0.7582769393920898,
+      "eval_mean_token_accuracy": 0.7787086770092089,
+      "eval_num_tokens": 527146.0,
+      "eval_runtime": 80.7434,
+      "eval_samples_per_second": 19.135,
+      "eval_steps_per_second": 2.403,
+      "step": 389
+    },
+    {
+      "entropy": 0.9051812700550965,
+      "epoch": 1.0283140283140284,
+      "grad_norm": 1.3198363780975342,
+      "learning_rate": 5.883290514810127e-05,
+      "loss": 0.7887770843505859,
+      "mean_token_accuracy": 0.7787209345836832,
+      "num_tokens": 541730.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.8626887363195419,
+      "epoch": 1.157014157014157,
+      "grad_norm": 1.1772489547729492,
+      "learning_rate": 5.8791462758850675e-05,
+      "loss": 0.7505465698242187,
+      "mean_token_accuracy": 0.7887020426988601,
+      "num_tokens": 610116.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.8474252647161484,
+      "epoch": 1.2857142857142856,
+      "grad_norm": 1.0174403190612793,
+      "learning_rate": 5.8690897984332274e-05,
+      "loss": 0.7397904968261719,
+      "mean_token_accuracy": 0.7890202105045319,
+      "num_tokens": 677990.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.843030309677124,
+      "epoch": 1.4144144144144144,
+      "grad_norm": 1.1678341627120972,
+      "learning_rate": 5.853141323295946e-05,
+      "loss": 0.7369876098632813,
+      "mean_token_accuracy": 0.790421934723854,
+      "num_tokens": 745456.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.8115463078022003,
+      "epoch": 1.5431145431145432,
+      "grad_norm": 0.8854549527168274,
+      "learning_rate": 5.831332950237421e-05,
+      "loss": 0.7081725311279297,
+      "mean_token_accuracy": 0.7952603316307068,
+      "num_tokens": 813775.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.8311941117048264,
+      "epoch": 1.6718146718146718,
+      "grad_norm": 1.171217441558838,
+      "learning_rate": 5.803708573336975e-05,
+      "loss": 0.7232355499267578,
+      "mean_token_accuracy": 0.7947697073221207,
+      "num_tokens": 879616.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.8120440804958343,
+      "epoch": 1.8005148005148004,
+      "grad_norm": 1.0333378314971924,
+      "learning_rate": 5.770323792642697e-05,
+      "loss": 0.6972620391845703,
+      "mean_token_accuracy": 0.7987716430425644,
+      "num_tokens": 947922.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.8045441079139709,
+      "epoch": 1.9292149292149292,
+      "grad_norm": 1.01642906665802,
+      "learning_rate": 5.7312458022642996e-05,
+      "loss": 0.6930023193359375,
+      "mean_token_accuracy": 0.8009403163194656,
+      "num_tokens": 1014787.0,
+      "step": 750
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.6984818912658495,
+      "eval_loss": 0.6775499582290649,
+      "eval_mean_token_accuracy": 0.8020080097557343,
+      "eval_num_tokens": 1054292.0,
+      "eval_runtime": 78.3908,
+      "eval_samples_per_second": 19.709,
+      "eval_steps_per_second": 2.475,
+      "step": 778
+    },
+    {
+      "entropy": 0.7779398566544659,
+      "epoch": 2.056628056628057,
+      "grad_norm": 1.4157040119171143,
+      "learning_rate": 5.686553255130396e-05,
+      "loss": 0.6584475708007812,
+      "mean_token_accuracy": 0.8087991054611977,
+      "num_tokens": 1083073.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.745064823627472,
+      "epoch": 2.1853281853281854,
+      "grad_norm": 0.9885228872299194,
+      "learning_rate": 5.636336104682433e-05,
+      "loss": 0.618268051147461,
+      "mean_token_accuracy": 0.8165298491716385,
+      "num_tokens": 1149040.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.7321632200479508,
+      "epoch": 2.314028314028314,
+      "grad_norm": 1.0271121263504028,
+      "learning_rate": 5.580695423823882e-05,
+      "loss": 0.6195047760009765,
+      "mean_token_accuracy": 0.8162743961811065,
+      "num_tokens": 1218025.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.7240813142061233,
+      "epoch": 2.4427284427284426,
+      "grad_norm": 0.8744781613349915,
+      "learning_rate": 5.519743201489109e-05,
+      "loss": 0.6093913650512696,
+      "mean_token_accuracy": 0.8179467821121216,
+      "num_tokens": 1287270.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.7273010468482971,
+      "epoch": 2.571428571428571,
+      "grad_norm": 1.0161406993865967,
+      "learning_rate": 5.453602117241361e-05,
+      "loss": 0.6118500518798828,
+      "mean_token_accuracy": 0.8183792388439178,
+      "num_tokens": 1355064.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.7150553166866302,
+      "epoch": 2.7001287001287,
+      "grad_norm": 0.9795015454292297,
+      "learning_rate": 5.382405294353544e-05,
+      "loss": 0.6005525970458985,
+      "mean_token_accuracy": 0.8210368871688842,
+      "num_tokens": 1421622.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.7134475702047348,
+      "epoch": 2.828828828828829,
+      "grad_norm": 0.9153344035148621,
+      "learning_rate": 5.306296031868776e-05,
+      "loss": 0.6033024978637695,
+      "mean_token_accuracy": 0.8216253930330276,
+      "num_tokens": 1488775.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.7142637860774994,
+      "epoch": 2.9575289575289574,
+      "grad_norm": 0.8638553619384766,
+      "learning_rate": 5.2254275161799905e-05,
+      "loss": 0.6056064605712891,
+      "mean_token_accuracy": 0.8204184120893478,
+      "num_tokens": 1559605.0,
+      "step": 1150
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.6126322589584232,
+      "eval_loss": 0.676252007484436,
+      "eval_mean_token_accuracy": 0.8002185584958067,
+      "eval_num_tokens": 1581438.0,
+      "eval_runtime": 78.2302,
+      "eval_samples_per_second": 19.749,
+      "eval_steps_per_second": 2.48,
+      "step": 1167
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 3890,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.664243134168064e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1556/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1556/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.06250292706368335,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1556/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

productivity_original_Estonian/Qwen3.5-2B-Base_productivity_splits_original_features_train_productivity_splits_original_features_test2/checkpoint-1556/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: transformers
+model_name: Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/hey06srd) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1224/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1224/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1224/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1224/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1224/trainer_state.json

@@ -0,0 +1,307 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1224,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.580693020417024e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1632/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1632/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1632/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1632/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-1632/trainer_state.json

@@ -0,0 +1,398 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.0,
+  "eval_steps": 500,
+  "global_step": 1632,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.779536115351488e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2040/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2040/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2040/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2040/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2040/trainer_state.json

@@ -0,0 +1,489 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 2040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    },
+    {
+      "entropy": 0.5006368765325258,
+      "epoch": 4.044171779141104,
+      "grad_norm": 0.5970089435577393,
+      "learning_rate": 0.0003259679874017874,
+      "loss": 0.4643829727172852,
+      "mean_token_accuracy": 0.8525426399828208,
+      "num_tokens": 4220924.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.4478312811255455,
+      "epoch": 4.166871165644172,
+      "grad_norm": 0.8119876980781555,
+      "learning_rate": 0.0003176863336849945,
+      "loss": 0.40898555755615235,
+      "mean_token_accuracy": 0.8650373387336731,
+      "num_tokens": 4342912.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.43789555728435514,
+      "epoch": 4.289570552147239,
+      "grad_norm": 0.5410990118980408,
+      "learning_rate": 0.0003092244111729683,
+      "loss": 0.40752113342285157,
+      "mean_token_accuracy": 0.8652220976352691,
+      "num_tokens": 4475673.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4397133576869965,
+      "epoch": 4.412269938650307,
+      "grad_norm": 0.619660496711731,
+      "learning_rate": 0.0003005977022279593,
+      "loss": 0.41369033813476563,
+      "mean_token_accuracy": 0.8625301831960678,
+      "num_tokens": 4605538.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4476582831144333,
+      "epoch": 4.534969325153375,
+      "grad_norm": 0.6686729192733765,
+      "learning_rate": 0.00029182199071382657,
+      "loss": 0.4207616806030273,
+      "mean_token_accuracy": 0.8617218750715255,
+      "num_tokens": 4730588.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.4475578209757805,
+      "epoch": 4.6576687116564415,
+      "grad_norm": 0.6534092426300049,
+      "learning_rate": 0.0002829133331170804,
+      "loss": 0.4195429229736328,
+      "mean_token_accuracy": 0.8632576483488082,
+      "num_tokens": 4856517.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.45640985041856763,
+      "epoch": 4.780368098159509,
+      "grad_norm": 0.5782829523086548,
+      "learning_rate": 0.00027388802916912124,
+      "loss": 0.41994491577148435,
+      "mean_token_accuracy": 0.8624195390939713,
+      "num_tokens": 4988951.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.458786401450634,
+      "epoch": 4.903067484662577,
+      "grad_norm": 0.4997099041938782,
+      "learning_rate": 0.00026476259202342517,
+      "loss": 0.4217214584350586,
+      "mean_token_accuracy": 0.8623241853713989,
+      "num_tokens": 5119329.0,
+      "step": 2000
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.49301558664866857,
+      "eval_loss": 0.6390948295593262,
+      "eval_mean_token_accuracy": 0.8254796872820173,
+      "eval_num_tokens": 5218855.0,
+      "eval_runtime": 56.2663,
+      "eval_samples_per_second": 24.811,
+      "eval_steps_per_second": 3.11,
+      "step": 2040
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0964287730493312e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2448/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2448/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2448/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2448/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2448/trainer_state.json

@@ -0,0 +1,580 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.0,
+  "eval_steps": 500,
+  "global_step": 2448,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    },
+    {
+      "entropy": 0.5006368765325258,
+      "epoch": 4.044171779141104,
+      "grad_norm": 0.5970089435577393,
+      "learning_rate": 0.0003259679874017874,
+      "loss": 0.4643829727172852,
+      "mean_token_accuracy": 0.8525426399828208,
+      "num_tokens": 4220924.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.4478312811255455,
+      "epoch": 4.166871165644172,
+      "grad_norm": 0.8119876980781555,
+      "learning_rate": 0.0003176863336849945,
+      "loss": 0.40898555755615235,
+      "mean_token_accuracy": 0.8650373387336731,
+      "num_tokens": 4342912.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.43789555728435514,
+      "epoch": 4.289570552147239,
+      "grad_norm": 0.5410990118980408,
+      "learning_rate": 0.0003092244111729683,
+      "loss": 0.40752113342285157,
+      "mean_token_accuracy": 0.8652220976352691,
+      "num_tokens": 4475673.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4397133576869965,
+      "epoch": 4.412269938650307,
+      "grad_norm": 0.619660496711731,
+      "learning_rate": 0.0003005977022279593,
+      "loss": 0.41369033813476563,
+      "mean_token_accuracy": 0.8625301831960678,
+      "num_tokens": 4605538.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4476582831144333,
+      "epoch": 4.534969325153375,
+      "grad_norm": 0.6686729192733765,
+      "learning_rate": 0.00029182199071382657,
+      "loss": 0.4207616806030273,
+      "mean_token_accuracy": 0.8617218750715255,
+      "num_tokens": 4730588.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.4475578209757805,
+      "epoch": 4.6576687116564415,
+      "grad_norm": 0.6534092426300049,
+      "learning_rate": 0.0002829133331170804,
+      "loss": 0.4195429229736328,
+      "mean_token_accuracy": 0.8632576483488082,
+      "num_tokens": 4856517.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.45640985041856763,
+      "epoch": 4.780368098159509,
+      "grad_norm": 0.5782829523086548,
+      "learning_rate": 0.00027388802916912124,
+      "loss": 0.41994491577148435,
+      "mean_token_accuracy": 0.8624195390939713,
+      "num_tokens": 4988951.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.458786401450634,
+      "epoch": 4.903067484662577,
+      "grad_norm": 0.4997099041938782,
+      "learning_rate": 0.00026476259202342517,
+      "loss": 0.4217214584350586,
+      "mean_token_accuracy": 0.8623241853713989,
+      "num_tokens": 5119329.0,
+      "step": 2000
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.49301558664866857,
+      "eval_loss": 0.6390948295593262,
+      "eval_mean_token_accuracy": 0.8254796872820173,
+      "eval_num_tokens": 5218855.0,
+      "eval_runtime": 56.2663,
+      "eval_samples_per_second": 24.811,
+      "eval_steps_per_second": 3.11,
+      "step": 2040
+    },
+    {
+      "entropy": 0.4460540203133015,
+      "epoch": 5.024539877300613,
+      "grad_norm": 0.6123270988464355,
+      "learning_rate": 0.00025555371804224084,
+      "loss": 0.40383037567138674,
+      "mean_token_accuracy": 0.8667331608858976,
+      "num_tokens": 5244358.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.34027856796979905,
+      "epoch": 5.147239263803681,
+      "grad_norm": 0.6245328187942505,
+      "learning_rate": 0.00024627825624807994,
+      "loss": 0.29572994232177735,
+      "mean_token_accuracy": 0.8982829064130783,
+      "num_tokens": 5376529.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3554780498147011,
+      "epoch": 5.269938650306749,
+      "grad_norm": 0.5149081349372864,
+      "learning_rate": 0.0002369531774958907,
+      "loss": 0.3053621292114258,
+      "mean_token_accuracy": 0.8955890852212905,
+      "num_tokens": 5502665.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3499211978912353,
+      "epoch": 5.392638036809816,
+      "grad_norm": 0.5816537737846375,
+      "learning_rate": 0.0002275955434223227,
+      "loss": 0.3001568603515625,
+      "mean_token_accuracy": 0.8958922523260117,
+      "num_tokens": 5635833.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3544818899035454,
+      "epoch": 5.515337423312883,
+      "grad_norm": 0.6629130244255066,
+      "learning_rate": 0.0002182224752288907,
+      "loss": 0.3112427520751953,
+      "mean_token_accuracy": 0.8927132803201675,
+      "num_tokens": 5761306.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.35083883732557297,
+      "epoch": 5.638036809815951,
+      "grad_norm": 0.6243860125541687,
+      "learning_rate": 0.00020885112235615794,
+      "loss": 0.3076825332641602,
+      "mean_token_accuracy": 0.8949526751041412,
+      "num_tokens": 5886926.0,
+      "step": 2300
+    },
+    {
+      "entropy": 0.3593694043159485,
+      "epoch": 5.7607361963190185,
+      "grad_norm": 0.6104142069816589,
+      "learning_rate": 0.00019949863110624958,
+      "loss": 0.3147741317749023,
+      "mean_token_accuracy": 0.8915749222040177,
+      "num_tokens": 6016346.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.35077930554747583,
+      "epoch": 5.883435582822086,
+      "grad_norm": 0.5526567697525024,
+      "learning_rate": 0.00019018211327110994,
+      "loss": 0.3106169509887695,
+      "mean_token_accuracy": 0.8925997793674469,
+      "num_tokens": 6144634.0,
+      "step": 2400
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.43280206646238056,
+      "eval_loss": 0.6834793090820312,
+      "eval_mean_token_accuracy": 0.8267195994513376,
+      "eval_num_tokens": 6262626.0,
+      "eval_runtime": 56.0697,
+      "eval_samples_per_second": 24.898,
+      "eval_steps_per_second": 3.121,
+      "step": 2448
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.3153170392634048e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2856/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2856/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2856/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2856/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-2856/trainer_state.json

@@ -0,0 +1,681 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 7.0,
+  "eval_steps": 500,
+  "global_step": 2856,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    },
+    {
+      "entropy": 0.5006368765325258,
+      "epoch": 4.044171779141104,
+      "grad_norm": 0.5970089435577393,
+      "learning_rate": 0.0003259679874017874,
+      "loss": 0.4643829727172852,
+      "mean_token_accuracy": 0.8525426399828208,
+      "num_tokens": 4220924.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.4478312811255455,
+      "epoch": 4.166871165644172,
+      "grad_norm": 0.8119876980781555,
+      "learning_rate": 0.0003176863336849945,
+      "loss": 0.40898555755615235,
+      "mean_token_accuracy": 0.8650373387336731,
+      "num_tokens": 4342912.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.43789555728435514,
+      "epoch": 4.289570552147239,
+      "grad_norm": 0.5410990118980408,
+      "learning_rate": 0.0003092244111729683,
+      "loss": 0.40752113342285157,
+      "mean_token_accuracy": 0.8652220976352691,
+      "num_tokens": 4475673.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4397133576869965,
+      "epoch": 4.412269938650307,
+      "grad_norm": 0.619660496711731,
+      "learning_rate": 0.0003005977022279593,
+      "loss": 0.41369033813476563,
+      "mean_token_accuracy": 0.8625301831960678,
+      "num_tokens": 4605538.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4476582831144333,
+      "epoch": 4.534969325153375,
+      "grad_norm": 0.6686729192733765,
+      "learning_rate": 0.00029182199071382657,
+      "loss": 0.4207616806030273,
+      "mean_token_accuracy": 0.8617218750715255,
+      "num_tokens": 4730588.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.4475578209757805,
+      "epoch": 4.6576687116564415,
+      "grad_norm": 0.6534092426300049,
+      "learning_rate": 0.0002829133331170804,
+      "loss": 0.4195429229736328,
+      "mean_token_accuracy": 0.8632576483488082,
+      "num_tokens": 4856517.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.45640985041856763,
+      "epoch": 4.780368098159509,
+      "grad_norm": 0.5782829523086548,
+      "learning_rate": 0.00027388802916912124,
+      "loss": 0.41994491577148435,
+      "mean_token_accuracy": 0.8624195390939713,
+      "num_tokens": 4988951.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.458786401450634,
+      "epoch": 4.903067484662577,
+      "grad_norm": 0.4997099041938782,
+      "learning_rate": 0.00026476259202342517,
+      "loss": 0.4217214584350586,
+      "mean_token_accuracy": 0.8623241853713989,
+      "num_tokens": 5119329.0,
+      "step": 2000
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.49301558664866857,
+      "eval_loss": 0.6390948295593262,
+      "eval_mean_token_accuracy": 0.8254796872820173,
+      "eval_num_tokens": 5218855.0,
+      "eval_runtime": 56.2663,
+      "eval_samples_per_second": 24.811,
+      "eval_steps_per_second": 3.11,
+      "step": 2040
+    },
+    {
+      "entropy": 0.4460540203133015,
+      "epoch": 5.024539877300613,
+      "grad_norm": 0.6123270988464355,
+      "learning_rate": 0.00025555371804224084,
+      "loss": 0.40383037567138674,
+      "mean_token_accuracy": 0.8667331608858976,
+      "num_tokens": 5244358.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.34027856796979905,
+      "epoch": 5.147239263803681,
+      "grad_norm": 0.6245328187942505,
+      "learning_rate": 0.00024627825624807994,
+      "loss": 0.29572994232177735,
+      "mean_token_accuracy": 0.8982829064130783,
+      "num_tokens": 5376529.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3554780498147011,
+      "epoch": 5.269938650306749,
+      "grad_norm": 0.5149081349372864,
+      "learning_rate": 0.0002369531774958907,
+      "loss": 0.3053621292114258,
+      "mean_token_accuracy": 0.8955890852212905,
+      "num_tokens": 5502665.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3499211978912353,
+      "epoch": 5.392638036809816,
+      "grad_norm": 0.5816537737846375,
+      "learning_rate": 0.0002275955434223227,
+      "loss": 0.3001568603515625,
+      "mean_token_accuracy": 0.8958922523260117,
+      "num_tokens": 5635833.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3544818899035454,
+      "epoch": 5.515337423312883,
+      "grad_norm": 0.6629130244255066,
+      "learning_rate": 0.0002182224752288907,
+      "loss": 0.3112427520751953,
+      "mean_token_accuracy": 0.8927132803201675,
+      "num_tokens": 5761306.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.35083883732557297,
+      "epoch": 5.638036809815951,
+      "grad_norm": 0.6243860125541687,
+      "learning_rate": 0.00020885112235615794,
+      "loss": 0.3076825332641602,
+      "mean_token_accuracy": 0.8949526751041412,
+      "num_tokens": 5886926.0,
+      "step": 2300
+    },
+    {
+      "entropy": 0.3593694043159485,
+      "epoch": 5.7607361963190185,
+      "grad_norm": 0.6104142069816589,
+      "learning_rate": 0.00019949863110624958,
+      "loss": 0.3147741317749023,
+      "mean_token_accuracy": 0.8915749222040177,
+      "num_tokens": 6016346.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.35077930554747583,
+      "epoch": 5.883435582822086,
+      "grad_norm": 0.5526567697525024,
+      "learning_rate": 0.00019018211327110994,
+      "loss": 0.3106169509887695,
+      "mean_token_accuracy": 0.8925997793674469,
+      "num_tokens": 6144634.0,
+      "step": 2400
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.43280206646238056,
+      "eval_loss": 0.6834793090820312,
+      "eval_mean_token_accuracy": 0.8267195994513376,
+      "eval_num_tokens": 6262626.0,
+      "eval_runtime": 56.0697,
+      "eval_samples_per_second": 24.898,
+      "eval_steps_per_second": 3.121,
+      "step": 2448
+    },
+    {
+      "entropy": 0.35777792816210274,
+      "epoch": 6.004907975460123,
+      "grad_norm": 0.49246102571487427,
+      "learning_rate": 0.00018091861482390074,
+      "loss": 0.3066068077087402,
+      "mean_token_accuracy": 0.8934184981114937,
+      "num_tokens": 6267831.0,
+      "step": 2450
+    },
+    {
+      "entropy": 0.23881351679563523,
+      "epoch": 6.12760736196319,
+      "grad_norm": 0.5304619073867798,
+      "learning_rate": 0.0001717250847308245,
+      "loss": 0.189466609954834,
+      "mean_token_accuracy": 0.9343969762325287,
+      "num_tokens": 6399002.0,
+      "step": 2500
+    },
+    {
+      "entropy": 0.23922096833586692,
+      "epoch": 6.250306748466258,
+      "grad_norm": 0.5296095609664917,
+      "learning_rate": 0.0001626183439404375,
+      "loss": 0.1947933006286621,
+      "mean_token_accuracy": 0.9307195580005646,
+      "num_tokens": 6530675.0,
+      "step": 2550
+    },
+    {
+      "entropy": 0.24290484443306923,
+      "epoch": 6.373006134969325,
+      "grad_norm": 0.69776451587677,
+      "learning_rate": 0.00015361505460718933,
+      "loss": 0.19909774780273437,
+      "mean_token_accuracy": 0.9303810286521912,
+      "num_tokens": 6656178.0,
+      "step": 2600
+    },
+    {
+      "entropy": 0.24054641783237457,
+      "epoch": 6.495705521472392,
+      "grad_norm": 0.5774877071380615,
+      "learning_rate": 0.00014473168960550125,
+      "loss": 0.1948543930053711,
+      "mean_token_accuracy": 0.9311885589361191,
+      "num_tokens": 6786838.0,
+      "step": 2650
+    },
+    {
+      "entropy": 0.24378441721200944,
+      "epoch": 6.61840490797546,
+      "grad_norm": 0.6684483289718628,
+      "learning_rate": 0.0001359845023901601,
+      "loss": 0.20006532669067384,
+      "mean_token_accuracy": 0.9293528574705124,
+      "num_tokens": 6912627.0,
+      "step": 2700
+    },
+    {
+      "entropy": 0.24649206757545472,
+      "epoch": 6.741104294478528,
+      "grad_norm": 0.5416299104690552,
+      "learning_rate": 0.0001273894972581737,
+      "loss": 0.20033370971679687,
+      "mean_token_accuracy": 0.9294078814983368,
+      "num_tokens": 7037952.0,
+      "step": 2750
+    },
+    {
+      "entropy": 0.245157231092453,
+      "epoch": 6.863803680981595,
+      "grad_norm": 0.6425517201423645,
+      "learning_rate": 0.00011896240006649818,
+      "loss": 0.19885097503662108,
+      "mean_token_accuracy": 0.929965842962265,
+      "num_tokens": 7161727.0,
+      "step": 2800
+    },
+    {
+      "entropy": 0.23941567167639732,
+      "epoch": 6.986503067484662,
+      "grad_norm": 0.5717498660087585,
+      "learning_rate": 0.00011071862945921291,
+      "loss": 0.19223129272460937,
+      "mean_token_accuracy": 0.9328319728374481,
+      "num_tokens": 7293830.0,
+      "step": 2850
+    },
+    {
+      "epoch": 7.0,
+      "eval_entropy": 0.34101752851690564,
+      "eval_loss": 0.7636436223983765,
+      "eval_mean_token_accuracy": 0.8270581153460911,
+      "eval_num_tokens": 7306397.0,
+      "eval_runtime": 57.0887,
+      "eval_samples_per_second": 24.453,
+      "eval_steps_per_second": 3.065,
+      "step": 2856
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5332087363966208e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3264/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3264/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3264/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3264/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3264/trainer_state.json

@@ -0,0 +1,772 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 8.0,
+  "eval_steps": 500,
+  "global_step": 3264,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    },
+    {
+      "entropy": 0.5006368765325258,
+      "epoch": 4.044171779141104,
+      "grad_norm": 0.5970089435577393,
+      "learning_rate": 0.0003259679874017874,
+      "loss": 0.4643829727172852,
+      "mean_token_accuracy": 0.8525426399828208,
+      "num_tokens": 4220924.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.4478312811255455,
+      "epoch": 4.166871165644172,
+      "grad_norm": 0.8119876980781555,
+      "learning_rate": 0.0003176863336849945,
+      "loss": 0.40898555755615235,
+      "mean_token_accuracy": 0.8650373387336731,
+      "num_tokens": 4342912.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.43789555728435514,
+      "epoch": 4.289570552147239,
+      "grad_norm": 0.5410990118980408,
+      "learning_rate": 0.0003092244111729683,
+      "loss": 0.40752113342285157,
+      "mean_token_accuracy": 0.8652220976352691,
+      "num_tokens": 4475673.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4397133576869965,
+      "epoch": 4.412269938650307,
+      "grad_norm": 0.619660496711731,
+      "learning_rate": 0.0003005977022279593,
+      "loss": 0.41369033813476563,
+      "mean_token_accuracy": 0.8625301831960678,
+      "num_tokens": 4605538.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4476582831144333,
+      "epoch": 4.534969325153375,
+      "grad_norm": 0.6686729192733765,
+      "learning_rate": 0.00029182199071382657,
+      "loss": 0.4207616806030273,
+      "mean_token_accuracy": 0.8617218750715255,
+      "num_tokens": 4730588.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.4475578209757805,
+      "epoch": 4.6576687116564415,
+      "grad_norm": 0.6534092426300049,
+      "learning_rate": 0.0002829133331170804,
+      "loss": 0.4195429229736328,
+      "mean_token_accuracy": 0.8632576483488082,
+      "num_tokens": 4856517.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.45640985041856763,
+      "epoch": 4.780368098159509,
+      "grad_norm": 0.5782829523086548,
+      "learning_rate": 0.00027388802916912124,
+      "loss": 0.41994491577148435,
+      "mean_token_accuracy": 0.8624195390939713,
+      "num_tokens": 4988951.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.458786401450634,
+      "epoch": 4.903067484662577,
+      "grad_norm": 0.4997099041938782,
+      "learning_rate": 0.00026476259202342517,
+      "loss": 0.4217214584350586,
+      "mean_token_accuracy": 0.8623241853713989,
+      "num_tokens": 5119329.0,
+      "step": 2000
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.49301558664866857,
+      "eval_loss": 0.6390948295593262,
+      "eval_mean_token_accuracy": 0.8254796872820173,
+      "eval_num_tokens": 5218855.0,
+      "eval_runtime": 56.2663,
+      "eval_samples_per_second": 24.811,
+      "eval_steps_per_second": 3.11,
+      "step": 2040
+    },
+    {
+      "entropy": 0.4460540203133015,
+      "epoch": 5.024539877300613,
+      "grad_norm": 0.6123270988464355,
+      "learning_rate": 0.00025555371804224084,
+      "loss": 0.40383037567138674,
+      "mean_token_accuracy": 0.8667331608858976,
+      "num_tokens": 5244358.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.34027856796979905,
+      "epoch": 5.147239263803681,
+      "grad_norm": 0.6245328187942505,
+      "learning_rate": 0.00024627825624807994,
+      "loss": 0.29572994232177735,
+      "mean_token_accuracy": 0.8982829064130783,
+      "num_tokens": 5376529.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3554780498147011,
+      "epoch": 5.269938650306749,
+      "grad_norm": 0.5149081349372864,
+      "learning_rate": 0.0002369531774958907,
+      "loss": 0.3053621292114258,
+      "mean_token_accuracy": 0.8955890852212905,
+      "num_tokens": 5502665.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3499211978912353,
+      "epoch": 5.392638036809816,
+      "grad_norm": 0.5816537737846375,
+      "learning_rate": 0.0002275955434223227,
+      "loss": 0.3001568603515625,
+      "mean_token_accuracy": 0.8958922523260117,
+      "num_tokens": 5635833.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3544818899035454,
+      "epoch": 5.515337423312883,
+      "grad_norm": 0.6629130244255066,
+      "learning_rate": 0.0002182224752288907,
+      "loss": 0.3112427520751953,
+      "mean_token_accuracy": 0.8927132803201675,
+      "num_tokens": 5761306.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.35083883732557297,
+      "epoch": 5.638036809815951,
+      "grad_norm": 0.6243860125541687,
+      "learning_rate": 0.00020885112235615794,
+      "loss": 0.3076825332641602,
+      "mean_token_accuracy": 0.8949526751041412,
+      "num_tokens": 5886926.0,
+      "step": 2300
+    },
+    {
+      "entropy": 0.3593694043159485,
+      "epoch": 5.7607361963190185,
+      "grad_norm": 0.6104142069816589,
+      "learning_rate": 0.00019949863110624958,
+      "loss": 0.3147741317749023,
+      "mean_token_accuracy": 0.8915749222040177,
+      "num_tokens": 6016346.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.35077930554747583,
+      "epoch": 5.883435582822086,
+      "grad_norm": 0.5526567697525024,
+      "learning_rate": 0.00019018211327110994,
+      "loss": 0.3106169509887695,
+      "mean_token_accuracy": 0.8925997793674469,
+      "num_tokens": 6144634.0,
+      "step": 2400
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.43280206646238056,
+      "eval_loss": 0.6834793090820312,
+      "eval_mean_token_accuracy": 0.8267195994513376,
+      "eval_num_tokens": 6262626.0,
+      "eval_runtime": 56.0697,
+      "eval_samples_per_second": 24.898,
+      "eval_steps_per_second": 3.121,
+      "step": 2448
+    },
+    {
+      "entropy": 0.35777792816210274,
+      "epoch": 6.004907975460123,
+      "grad_norm": 0.49246102571487427,
+      "learning_rate": 0.00018091861482390074,
+      "loss": 0.3066068077087402,
+      "mean_token_accuracy": 0.8934184981114937,
+      "num_tokens": 6267831.0,
+      "step": 2450
+    },
+    {
+      "entropy": 0.23881351679563523,
+      "epoch": 6.12760736196319,
+      "grad_norm": 0.5304619073867798,
+      "learning_rate": 0.0001717250847308245,
+      "loss": 0.189466609954834,
+      "mean_token_accuracy": 0.9343969762325287,
+      "num_tokens": 6399002.0,
+      "step": 2500
+    },
+    {
+      "entropy": 0.23922096833586692,
+      "epoch": 6.250306748466258,
+      "grad_norm": 0.5296095609664917,
+      "learning_rate": 0.0001626183439404375,
+      "loss": 0.1947933006286621,
+      "mean_token_accuracy": 0.9307195580005646,
+      "num_tokens": 6530675.0,
+      "step": 2550
+    },
+    {
+      "entropy": 0.24290484443306923,
+      "epoch": 6.373006134969325,
+      "grad_norm": 0.69776451587677,
+      "learning_rate": 0.00015361505460718933,
+      "loss": 0.19909774780273437,
+      "mean_token_accuracy": 0.9303810286521912,
+      "num_tokens": 6656178.0,
+      "step": 2600
+    },
+    {
+      "entropy": 0.24054641783237457,
+      "epoch": 6.495705521472392,
+      "grad_norm": 0.5774877071380615,
+      "learning_rate": 0.00014473168960550125,
+      "loss": 0.1948543930053711,
+      "mean_token_accuracy": 0.9311885589361191,
+      "num_tokens": 6786838.0,
+      "step": 2650
+    },
+    {
+      "entropy": 0.24378441721200944,
+      "epoch": 6.61840490797546,
+      "grad_norm": 0.6684483289718628,
+      "learning_rate": 0.0001359845023901601,
+      "loss": 0.20006532669067384,
+      "mean_token_accuracy": 0.9293528574705124,
+      "num_tokens": 6912627.0,
+      "step": 2700
+    },
+    {
+      "entropy": 0.24649206757545472,
+      "epoch": 6.741104294478528,
+      "grad_norm": 0.5416299104690552,
+      "learning_rate": 0.0001273894972581737,
+      "loss": 0.20033370971679687,
+      "mean_token_accuracy": 0.9294078814983368,
+      "num_tokens": 7037952.0,
+      "step": 2750
+    },
+    {
+      "entropy": 0.245157231092453,
+      "epoch": 6.863803680981595,
+      "grad_norm": 0.6425517201423645,
+      "learning_rate": 0.00011896240006649818,
+      "loss": 0.19885097503662108,
+      "mean_token_accuracy": 0.929965842962265,
+      "num_tokens": 7161727.0,
+      "step": 2800
+    },
+    {
+      "entropy": 0.23941567167639732,
+      "epoch": 6.986503067484662,
+      "grad_norm": 0.5717498660087585,
+      "learning_rate": 0.00011071862945921291,
+      "loss": 0.19223129272460937,
+      "mean_token_accuracy": 0.9328319728374481,
+      "num_tokens": 7293830.0,
+      "step": 2850
+    },
+    {
+      "epoch": 7.0,
+      "eval_entropy": 0.34101752851690564,
+      "eval_loss": 0.7636436223983765,
+      "eval_mean_token_accuracy": 0.8270581153460911,
+      "eval_num_tokens": 7306397.0,
+      "eval_runtime": 57.0887,
+      "eval_samples_per_second": 24.453,
+      "eval_steps_per_second": 3.065,
+      "step": 2856
+    },
+    {
+      "entropy": 0.17031941931657116,
+      "epoch": 7.1079754601227,
+      "grad_norm": 0.4665500819683075,
+      "learning_rate": 0.0001026732686567881,
+      "loss": 0.1262428092956543,
+      "mean_token_accuracy": 0.9578677494116504,
+      "num_tokens": 7412985.0,
+      "step": 2900
+    },
+    {
+      "entropy": 0.1474052606523037,
+      "epoch": 7.230674846625767,
+      "grad_norm": 0.45524656772613525,
+      "learning_rate": 9.484103785906042e-05,
+      "loss": 0.10753781318664551,
+      "mean_token_accuracy": 0.9643174183368682,
+      "num_tokens": 7543754.0,
+      "step": 2950
+    },
+    {
+      "entropy": 0.14507727995514869,
+      "epoch": 7.353374233128834,
+      "grad_norm": 0.5161148309707642,
+      "learning_rate": 8.723626731240899e-05,
+      "loss": 0.10872675895690918,
+      "mean_token_accuracy": 0.9626953148841858,
+      "num_tokens": 7673840.0,
+      "step": 3000
+    },
+    {
+      "entropy": 0.14192182645201684,
+      "epoch": 7.476073619631902,
+      "grad_norm": 0.5691169500350952,
+      "learning_rate": 7.987287109041075e-05,
+      "loss": 0.10497721672058105,
+      "mean_token_accuracy": 0.9646239751577377,
+      "num_tokens": 7807664.0,
+      "step": 3050
+    },
+    {
+      "entropy": 0.14175789937376976,
+      "epoch": 7.598773006134969,
+      "grad_norm": 0.5878594517707825,
+      "learning_rate": 7.276432163594605e-05,
+      "loss": 0.10868305206298828,
+      "mean_token_accuracy": 0.9635398960113526,
+      "num_tokens": 7933830.0,
+      "step": 3100
+    },
+    {
+      "entropy": 0.14344056263566018,
+      "epoch": 7.721472392638037,
+      "grad_norm": 0.5571667551994324,
+      "learning_rate": 6.592362511133478e-05,
+      "loss": 0.10832572937011718,
+      "mean_token_accuracy": 0.9632837051153182,
+      "num_tokens": 8063545.0,
+      "step": 3150
+    },
+    {
+      "entropy": 0.14393038906157016,
+      "epoch": 7.844171779141105,
+      "grad_norm": 0.5589098334312439,
+      "learning_rate": 5.93632976016024e-05,
+      "loss": 0.10971030235290527,
+      "mean_token_accuracy": 0.9629397732019425,
+      "num_tokens": 8189862.0,
+      "step": 3200
+    },
+    {
+      "entropy": 0.14122579149901868,
+      "epoch": 7.9668711656441715,
+      "grad_norm": 0.6026595830917358,
+      "learning_rate": 5.309534221441647e-05,
+      "loss": 0.10822047233581543,
+      "mean_token_accuracy": 0.9636554193496704,
+      "num_tokens": 8316465.0,
+      "step": 3250
+    },
+    {
+      "epoch": 8.0,
+      "eval_entropy": 0.28723810638700215,
+      "eval_loss": 0.864037811756134,
+      "eval_mean_token_accuracy": 0.8260653465134757,
+      "eval_num_tokens": 8350168.0,
+      "eval_runtime": 56.6604,
+      "eval_samples_per_second": 24.638,
+      "eval_steps_per_second": 3.089,
+      "step": 3264
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.7506329083762688e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3672/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3672/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3672/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3672/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-3672/trainer_state.json

@@ -0,0 +1,863 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 9.0,
+  "eval_steps": 500,
+  "global_step": 3672,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.6869684666395188,
+      "epoch": 0.12269938650306748,
+      "grad_norm": 1.4528672695159912,
+      "learning_rate": 5.264136964343788e-05,
+      "loss": 1.5886520385742187,
+      "mean_token_accuracy": 0.6519684100151062,
+      "num_tokens": 129823.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.9406951844692231,
+      "epoch": 0.24539877300613497,
+      "grad_norm": 1.3675724267959595,
+      "learning_rate": 0.00010635705295306836,
+      "loss": 0.8635823059082032,
+      "mean_token_accuracy": 0.7623954111337662,
+      "num_tokens": 253466.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.8472799164056778,
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.848242461681366,
+      "learning_rate": 0.00016007273626269885,
+      "loss": 0.7817398834228516,
+      "mean_token_accuracy": 0.7812312138080597,
+      "num_tokens": 387484.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7993512552976608,
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.7264193892478943,
+      "learning_rate": 0.00021378841957232935,
+      "loss": 0.7406961059570313,
+      "mean_token_accuracy": 0.790413863658905,
+      "num_tokens": 513501.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7909506279230117,
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6578923463821411,
+      "learning_rate": 0.00026750410288195985,
+      "loss": 0.7357511901855469,
+      "mean_token_accuracy": 0.7914552348852157,
+      "num_tokens": 643515.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7645951139926911,
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.844650149345398,
+      "learning_rate": 0.0003212197861915903,
+      "loss": 0.7212095642089844,
+      "mean_token_accuracy": 0.7942444437742233,
+      "num_tokens": 772238.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7707051610946656,
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.9334251284599304,
+      "learning_rate": 0.0003749354695012208,
+      "loss": 0.7167671966552734,
+      "mean_token_accuracy": 0.7953211319446564,
+      "num_tokens": 899105.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.781549910902977,
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6472936272621155,
+      "learning_rate": 0.00042865115281085127,
+      "loss": 0.7353939819335937,
+      "mean_token_accuracy": 0.7930131030082702,
+      "num_tokens": 1025860.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.7875640044893537,
+      "eval_loss": 0.7651876211166382,
+      "eval_mean_token_accuracy": 0.7883888609068734,
+      "eval_num_tokens": 1043771.0,
+      "eval_runtime": 56.1402,
+      "eval_samples_per_second": 24.866,
+      "eval_steps_per_second": 3.117,
+      "step": 408
+    },
+    {
+      "entropy": 0.7474399413725342,
+      "epoch": 1.1030674846625768,
+      "grad_norm": 0.9284119606018066,
+      "learning_rate": 0.00043818515750422955,
+      "loss": 0.7073413848876953,
+      "mean_token_accuracy": 0.7961714261710041,
+      "num_tokens": 1158723.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7287704515457153,
+      "epoch": 1.2257668711656442,
+      "grad_norm": 0.6676222085952759,
+      "learning_rate": 0.0004376560965874532,
+      "loss": 0.6876696014404297,
+      "mean_token_accuracy": 0.8015633362531662,
+      "num_tokens": 1289372.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.7303326517343521,
+      "epoch": 1.3484662576687116,
+      "grad_norm": 1.440736174583435,
+      "learning_rate": 0.0004367272641175644,
+      "loss": 0.6853336334228516,
+      "mean_token_accuracy": 0.8025881034135819,
+      "num_tokens": 1413943.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.752444422841072,
+      "epoch": 1.471165644171779,
+      "grad_norm": 0.847322940826416,
+      "learning_rate": 0.00043540035953352037,
+      "loss": 0.7047377777099609,
+      "mean_token_accuracy": 0.7983024090528488,
+      "num_tokens": 1536890.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.7128984904289246,
+      "epoch": 1.5938650306748468,
+      "grad_norm": 0.7412364482879639,
+      "learning_rate": 0.00043367781060719106,
+      "loss": 0.6685499572753906,
+      "mean_token_accuracy": 0.8069163882732391,
+      "num_tokens": 1663992.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.7085144728422165,
+      "epoch": 1.716564417177914,
+      "grad_norm": 0.569710910320282,
+      "learning_rate": 0.0004315627690013853,
+      "loss": 0.6666531372070312,
+      "mean_token_accuracy": 0.8071046113967896,
+      "num_tokens": 1791126.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.7012534445524216,
+      "epoch": 1.8392638036809816,
+      "grad_norm": 0.6749410629272461,
+      "learning_rate": 0.0004290591045034084,
+      "loss": 0.6621897888183593,
+      "mean_token_accuracy": 0.8081263571977615,
+      "num_tokens": 1921148.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6866424483060837,
+      "epoch": 1.961963190184049,
+      "grad_norm": 0.6211123466491699,
+      "learning_rate": 0.0004261713979447033,
+      "loss": 0.6496143341064453,
+      "mean_token_accuracy": 0.8108037900924683,
+      "num_tokens": 2049795.0,
+      "step": 800
+    },
+    {
+      "epoch": 2.0,
+      "eval_entropy": 0.7798067167827061,
+      "eval_loss": 0.6922561526298523,
+      "eval_mean_token_accuracy": 0.8042122350420271,
+      "eval_num_tokens": 2087542.0,
+      "eval_runtime": 55.2357,
+      "eval_samples_per_second": 25.274,
+      "eval_steps_per_second": 3.168,
+      "step": 816
+    },
+    {
+      "entropy": 0.6268433964613712,
+      "epoch": 2.083435582822086,
+      "grad_norm": 0.7161462306976318,
+      "learning_rate": 0.0004229049328195294,
+      "loss": 0.5977137756347656,
+      "mean_token_accuracy": 0.8221012402062464,
+      "num_tokens": 2177128.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.6347901010513306,
+      "epoch": 2.2061349693251535,
+      "grad_norm": 0.717650294303894,
+      "learning_rate": 0.00041926568561801326,
+      "loss": 0.591550521850586,
+      "mean_token_accuracy": 0.821678705215454,
+      "num_tokens": 2299605.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6345827746391296,
+      "epoch": 2.3288343558282207,
+      "grad_norm": 0.6430606842041016,
+      "learning_rate": 0.00041526031489125996,
+      "loss": 0.5906002044677734,
+      "mean_token_accuracy": 0.8223223352432251,
+      "num_tokens": 2430751.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.6397277122735977,
+      "epoch": 2.4515337423312884,
+      "grad_norm": 0.5627941489219666,
+      "learning_rate": 0.0004108961490685301,
+      "loss": 0.6030870819091797,
+      "mean_token_accuracy": 0.8200687575340271,
+      "num_tokens": 2553654.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.638802530169487,
+      "epoch": 2.574233128834356,
+      "grad_norm": 0.7276184558868408,
+      "learning_rate": 0.0004061811730487745,
+      "loss": 0.5983928298950195,
+      "mean_token_accuracy": 0.8206393992900849,
+      "num_tokens": 2680735.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.6276438289880752,
+      "epoch": 2.6969325153374233,
+      "grad_norm": 0.8955866694450378,
+      "learning_rate": 0.00040112401359105855,
+      "loss": 0.5967353820800781,
+      "mean_token_accuracy": 0.8198855191469192,
+      "num_tokens": 2813796.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.6263330829143524,
+      "epoch": 2.819631901840491,
+      "grad_norm": 0.7027910947799683,
+      "learning_rate": 0.0003957339235306061,
+      "loss": 0.5978898620605468,
+      "mean_token_accuracy": 0.8205991142988205,
+      "num_tokens": 2940312.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.6164533680677414,
+      "epoch": 2.942331288343558,
+      "grad_norm": 0.5775797963142395,
+      "learning_rate": 0.00039002076484934384,
+      "loss": 0.5849863433837891,
+      "mean_token_accuracy": 0.8238226908445359,
+      "num_tokens": 3070898.0,
+      "step": 1200
+    },
+    {
+      "epoch": 3.0,
+      "eval_entropy": 0.5691366846220834,
+      "eval_loss": 0.6553322672843933,
+      "eval_mean_token_accuracy": 0.8153549589429583,
+      "eval_num_tokens": 3131313.0,
+      "eval_runtime": 56.9883,
+      "eval_samples_per_second": 24.496,
+      "eval_steps_per_second": 3.071,
+      "step": 1224
+    },
+    {
+      "entropy": 0.5563058633394916,
+      "epoch": 3.063803680981595,
+      "grad_norm": 0.48518672585487366,
+      "learning_rate": 0.00038399499063191913,
+      "loss": 0.5305076599121094,
+      "mean_token_accuracy": 0.8362203270497949,
+      "num_tokens": 3201120.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.5264888721704483,
+      "epoch": 3.1865030674846624,
+      "grad_norm": 0.5429811477661133,
+      "learning_rate": 0.0003776676259402065,
+      "loss": 0.49305004119873047,
+      "mean_token_accuracy": 0.8430798947811127,
+      "num_tokens": 3325619.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5365294662117958,
+      "epoch": 3.30920245398773,
+      "grad_norm": 0.6198399066925049,
+      "learning_rate": 0.00037105024764129597,
+      "loss": 0.505305061340332,
+      "mean_token_accuracy": 0.8407882738113404,
+      "num_tokens": 3455554.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.5411551231145859,
+      "epoch": 3.4319018404907977,
+      "grad_norm": 0.6715341210365295,
+      "learning_rate": 0.00036415496322587026,
+      "loss": 0.5051498031616211,
+      "mean_token_accuracy": 0.8406214094161988,
+      "num_tokens": 3580945.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5552197429537773,
+      "epoch": 3.554601226993865,
+      "grad_norm": 0.5532404780387878,
+      "learning_rate": 0.000356994388655726,
+      "loss": 0.5228401947021485,
+      "mean_token_accuracy": 0.8378301256895065,
+      "num_tokens": 3703139.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.537499880194664,
+      "epoch": 3.6773006134969326,
+      "grad_norm": 0.5497210621833801,
+      "learning_rate": 0.00034958162528097133,
+      "loss": 0.5127570343017578,
+      "mean_token_accuracy": 0.841296665072441,
+      "num_tokens": 3833407.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5413746473193168,
+      "epoch": 3.8,
+      "grad_norm": 0.5489308834075928,
+      "learning_rate": 0.0003419302358691309,
+      "loss": 0.5108597183227539,
+      "mean_token_accuracy": 0.8398933434486389,
+      "num_tokens": 3964068.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5602546691894531,
+      "epoch": 3.9226993865030675,
+      "grad_norm": 0.5667650103569031,
+      "learning_rate": 0.00033405421979002,
+      "loss": 0.523597526550293,
+      "mean_token_accuracy": 0.8387859660387039,
+      "num_tokens": 4091244.0,
+      "step": 1600
+    },
+    {
+      "epoch": 4.0,
+      "eval_entropy": 0.5578466103758131,
+      "eval_loss": 0.6336107850074768,
+      "eval_mean_token_accuracy": 0.8219898309026445,
+      "eval_num_tokens": 4175084.0,
+      "eval_runtime": 57.1271,
+      "eval_samples_per_second": 24.437,
+      "eval_steps_per_second": 3.063,
+      "step": 1632
+    },
+    {
+      "entropy": 0.5006368765325258,
+      "epoch": 4.044171779141104,
+      "grad_norm": 0.5970089435577393,
+      "learning_rate": 0.0003259679874017874,
+      "loss": 0.4643829727172852,
+      "mean_token_accuracy": 0.8525426399828208,
+      "num_tokens": 4220924.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.4478312811255455,
+      "epoch": 4.166871165644172,
+      "grad_norm": 0.8119876980781555,
+      "learning_rate": 0.0003176863336849945,
+      "loss": 0.40898555755615235,
+      "mean_token_accuracy": 0.8650373387336731,
+      "num_tokens": 4342912.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.43789555728435514,
+      "epoch": 4.289570552147239,
+      "grad_norm": 0.5410990118980408,
+      "learning_rate": 0.0003092244111729683,
+      "loss": 0.40752113342285157,
+      "mean_token_accuracy": 0.8652220976352691,
+      "num_tokens": 4475673.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.4397133576869965,
+      "epoch": 4.412269938650307,
+      "grad_norm": 0.619660496711731,
+      "learning_rate": 0.0003005977022279593,
+      "loss": 0.41369033813476563,
+      "mean_token_accuracy": 0.8625301831960678,
+      "num_tokens": 4605538.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.4476582831144333,
+      "epoch": 4.534969325153375,
+      "grad_norm": 0.6686729192733765,
+      "learning_rate": 0.00029182199071382657,
+      "loss": 0.4207616806030273,
+      "mean_token_accuracy": 0.8617218750715255,
+      "num_tokens": 4730588.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.4475578209757805,
+      "epoch": 4.6576687116564415,
+      "grad_norm": 0.6534092426300049,
+      "learning_rate": 0.0002829133331170804,
+      "loss": 0.4195429229736328,
+      "mean_token_accuracy": 0.8632576483488082,
+      "num_tokens": 4856517.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.45640985041856763,
+      "epoch": 4.780368098159509,
+      "grad_norm": 0.5782829523086548,
+      "learning_rate": 0.00027388802916912124,
+      "loss": 0.41994491577148435,
+      "mean_token_accuracy": 0.8624195390939713,
+      "num_tokens": 4988951.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.458786401450634,
+      "epoch": 4.903067484662577,
+      "grad_norm": 0.4997099041938782,
+      "learning_rate": 0.00026476259202342517,
+      "loss": 0.4217214584350586,
+      "mean_token_accuracy": 0.8623241853713989,
+      "num_tokens": 5119329.0,
+      "step": 2000
+    },
+    {
+      "epoch": 5.0,
+      "eval_entropy": 0.49301558664866857,
+      "eval_loss": 0.6390948295593262,
+      "eval_mean_token_accuracy": 0.8254796872820173,
+      "eval_num_tokens": 5218855.0,
+      "eval_runtime": 56.2663,
+      "eval_samples_per_second": 24.811,
+      "eval_steps_per_second": 3.11,
+      "step": 2040
+    },
+    {
+      "entropy": 0.4460540203133015,
+      "epoch": 5.024539877300613,
+      "grad_norm": 0.6123270988464355,
+      "learning_rate": 0.00025555371804224084,
+      "loss": 0.40383037567138674,
+      "mean_token_accuracy": 0.8667331608858976,
+      "num_tokens": 5244358.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.34027856796979905,
+      "epoch": 5.147239263803681,
+      "grad_norm": 0.6245328187942505,
+      "learning_rate": 0.00024627825624807994,
+      "loss": 0.29572994232177735,
+      "mean_token_accuracy": 0.8982829064130783,
+      "num_tokens": 5376529.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.3554780498147011,
+      "epoch": 5.269938650306749,
+      "grad_norm": 0.5149081349372864,
+      "learning_rate": 0.0002369531774958907,
+      "loss": 0.3053621292114258,
+      "mean_token_accuracy": 0.8955890852212905,
+      "num_tokens": 5502665.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.3499211978912353,
+      "epoch": 5.392638036809816,
+      "grad_norm": 0.5816537737846375,
+      "learning_rate": 0.0002275955434223227,
+      "loss": 0.3001568603515625,
+      "mean_token_accuracy": 0.8958922523260117,
+      "num_tokens": 5635833.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.3544818899035454,
+      "epoch": 5.515337423312883,
+      "grad_norm": 0.6629130244255066,
+      "learning_rate": 0.0002182224752288907,
+      "loss": 0.3112427520751953,
+      "mean_token_accuracy": 0.8927132803201675,
+      "num_tokens": 5761306.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.35083883732557297,
+      "epoch": 5.638036809815951,
+      "grad_norm": 0.6243860125541687,
+      "learning_rate": 0.00020885112235615794,
+      "loss": 0.3076825332641602,
+      "mean_token_accuracy": 0.8949526751041412,
+      "num_tokens": 5886926.0,
+      "step": 2300
+    },
+    {
+      "entropy": 0.3593694043159485,
+      "epoch": 5.7607361963190185,
+      "grad_norm": 0.6104142069816589,
+      "learning_rate": 0.00019949863110624958,
+      "loss": 0.3147741317749023,
+      "mean_token_accuracy": 0.8915749222040177,
+      "num_tokens": 6016346.0,
+      "step": 2350
+    },
+    {
+      "entropy": 0.35077930554747583,
+      "epoch": 5.883435582822086,
+      "grad_norm": 0.5526567697525024,
+      "learning_rate": 0.00019018211327110994,
+      "loss": 0.3106169509887695,
+      "mean_token_accuracy": 0.8925997793674469,
+      "num_tokens": 6144634.0,
+      "step": 2400
+    },
+    {
+      "epoch": 6.0,
+      "eval_entropy": 0.43280206646238056,
+      "eval_loss": 0.6834793090820312,
+      "eval_mean_token_accuracy": 0.8267195994513376,
+      "eval_num_tokens": 6262626.0,
+      "eval_runtime": 56.0697,
+      "eval_samples_per_second": 24.898,
+      "eval_steps_per_second": 3.121,
+      "step": 2448
+    },
+    {
+      "entropy": 0.35777792816210274,
+      "epoch": 6.004907975460123,
+      "grad_norm": 0.49246102571487427,
+      "learning_rate": 0.00018091861482390074,
+      "loss": 0.3066068077087402,
+      "mean_token_accuracy": 0.8934184981114937,
+      "num_tokens": 6267831.0,
+      "step": 2450
+    },
+    {
+      "entropy": 0.23881351679563523,
+      "epoch": 6.12760736196319,
+      "grad_norm": 0.5304619073867798,
+      "learning_rate": 0.0001717250847308245,
+      "loss": 0.189466609954834,
+      "mean_token_accuracy": 0.9343969762325287,
+      "num_tokens": 6399002.0,
+      "step": 2500
+    },
+    {
+      "entropy": 0.23922096833586692,
+      "epoch": 6.250306748466258,
+      "grad_norm": 0.5296095609664917,
+      "learning_rate": 0.0001626183439404375,
+      "loss": 0.1947933006286621,
+      "mean_token_accuracy": 0.9307195580005646,
+      "num_tokens": 6530675.0,
+      "step": 2550
+    },
+    {
+      "entropy": 0.24290484443306923,
+      "epoch": 6.373006134969325,
+      "grad_norm": 0.69776451587677,
+      "learning_rate": 0.00015361505460718933,
+      "loss": 0.19909774780273437,
+      "mean_token_accuracy": 0.9303810286521912,
+      "num_tokens": 6656178.0,
+      "step": 2600
+    },
+    {
+      "entropy": 0.24054641783237457,
+      "epoch": 6.495705521472392,
+      "grad_norm": 0.5774877071380615,
+      "learning_rate": 0.00014473168960550125,
+      "loss": 0.1948543930053711,
+      "mean_token_accuracy": 0.9311885589361191,
+      "num_tokens": 6786838.0,
+      "step": 2650
+    },
+    {
+      "entropy": 0.24378441721200944,
+      "epoch": 6.61840490797546,
+      "grad_norm": 0.6684483289718628,
+      "learning_rate": 0.0001359845023901601,
+      "loss": 0.20006532669067384,
+      "mean_token_accuracy": 0.9293528574705124,
+      "num_tokens": 6912627.0,
+      "step": 2700
+    },
+    {
+      "entropy": 0.24649206757545472,
+      "epoch": 6.741104294478528,
+      "grad_norm": 0.5416299104690552,
+      "learning_rate": 0.0001273894972581737,
+      "loss": 0.20033370971679687,
+      "mean_token_accuracy": 0.9294078814983368,
+      "num_tokens": 7037952.0,
+      "step": 2750
+    },
+    {
+      "entropy": 0.245157231092453,
+      "epoch": 6.863803680981595,
+      "grad_norm": 0.6425517201423645,
+      "learning_rate": 0.00011896240006649818,
+      "loss": 0.19885097503662108,
+      "mean_token_accuracy": 0.929965842962265,
+      "num_tokens": 7161727.0,
+      "step": 2800
+    },
+    {
+      "entropy": 0.23941567167639732,
+      "epoch": 6.986503067484662,
+      "grad_norm": 0.5717498660087585,
+      "learning_rate": 0.00011071862945921291,
+      "loss": 0.19223129272460937,
+      "mean_token_accuracy": 0.9328319728374481,
+      "num_tokens": 7293830.0,
+      "step": 2850
+    },
+    {
+      "epoch": 7.0,
+      "eval_entropy": 0.34101752851690564,
+      "eval_loss": 0.7636436223983765,
+      "eval_mean_token_accuracy": 0.8270581153460911,
+      "eval_num_tokens": 7306397.0,
+      "eval_runtime": 57.0887,
+      "eval_samples_per_second": 24.453,
+      "eval_steps_per_second": 3.065,
+      "step": 2856
+    },
+    {
+      "entropy": 0.17031941931657116,
+      "epoch": 7.1079754601227,
+      "grad_norm": 0.4665500819683075,
+      "learning_rate": 0.0001026732686567881,
+      "loss": 0.1262428092956543,
+      "mean_token_accuracy": 0.9578677494116504,
+      "num_tokens": 7412985.0,
+      "step": 2900
+    },
+    {
+      "entropy": 0.1474052606523037,
+      "epoch": 7.230674846625767,
+      "grad_norm": 0.45524656772613525,
+      "learning_rate": 9.484103785906042e-05,
+      "loss": 0.10753781318664551,
+      "mean_token_accuracy": 0.9643174183368682,
+      "num_tokens": 7543754.0,
+      "step": 2950
+    },
+    {
+      "entropy": 0.14507727995514869,
+      "epoch": 7.353374233128834,
+      "grad_norm": 0.5161148309707642,
+      "learning_rate": 8.723626731240899e-05,
+      "loss": 0.10872675895690918,
+      "mean_token_accuracy": 0.9626953148841858,
+      "num_tokens": 7673840.0,
+      "step": 3000
+    },
+    {
+      "entropy": 0.14192182645201684,
+      "epoch": 7.476073619631902,
+      "grad_norm": 0.5691169500350952,
+      "learning_rate": 7.987287109041075e-05,
+      "loss": 0.10497721672058105,
+      "mean_token_accuracy": 0.9646239751577377,
+      "num_tokens": 7807664.0,
+      "step": 3050
+    },
+    {
+      "entropy": 0.14175789937376976,
+      "epoch": 7.598773006134969,
+      "grad_norm": 0.5878594517707825,
+      "learning_rate": 7.276432163594605e-05,
+      "loss": 0.10868305206298828,
+      "mean_token_accuracy": 0.9635398960113526,
+      "num_tokens": 7933830.0,
+      "step": 3100
+    },
+    {
+      "entropy": 0.14344056263566018,
+      "epoch": 7.721472392638037,
+      "grad_norm": 0.5571667551994324,
+      "learning_rate": 6.592362511133478e-05,
+      "loss": 0.10832572937011718,
+      "mean_token_accuracy": 0.9632837051153182,
+      "num_tokens": 8063545.0,
+      "step": 3150
+    },
+    {
+      "entropy": 0.14393038906157016,
+      "epoch": 7.844171779141105,
+      "grad_norm": 0.5589098334312439,
+      "learning_rate": 5.93632976016024e-05,
+      "loss": 0.10971030235290527,
+      "mean_token_accuracy": 0.9629397732019425,
+      "num_tokens": 8189862.0,
+      "step": 3200
+    },
+    {
+      "entropy": 0.14122579149901868,
+      "epoch": 7.9668711656441715,
+      "grad_norm": 0.6026595830917358,
+      "learning_rate": 5.309534221441647e-05,
+      "loss": 0.10822047233581543,
+      "mean_token_accuracy": 0.9636554193496704,
+      "num_tokens": 8316465.0,
+      "step": 3250
+    },
+    {
+      "epoch": 8.0,
+      "eval_entropy": 0.28723810638700215,
+      "eval_loss": 0.864037811756134,
+      "eval_mean_token_accuracy": 0.8260653465134757,
+      "eval_num_tokens": 8350168.0,
+      "eval_runtime": 56.6604,
+      "eval_samples_per_second": 24.638,
+      "eval_steps_per_second": 3.089,
+      "step": 3264
+    },
+    {
+      "entropy": 0.118123043591928,
+      "epoch": 8.088343558282208,
+      "grad_norm": 0.35258734226226807,
+      "learning_rate": 4.713122711859268e-05,
+      "loss": 0.08152069091796875,
+      "mean_token_accuracy": 0.9731789550395927,
+      "num_tokens": 8438737.0,
+      "step": 3300
+    },
+    {
+      "entropy": 0.10090006463229656,
+      "epoch": 8.211042944785277,
+      "grad_norm": 0.3480742573738098,
+      "learning_rate": 4.148186456135221e-05,
+      "loss": 0.06941273212432861,
+      "mean_token_accuracy": 0.9767383986711502,
+      "num_tokens": 8565559.0,
+      "step": 3350
+    },
+    {
+      "entropy": 0.10347712013870478,
+      "epoch": 8.333742331288343,
+      "grad_norm": 0.3159559965133667,
+      "learning_rate": 3.6157590902720824e-05,
+      "loss": 0.07076391220092773,
+      "mean_token_accuracy": 0.9752408146858216,
+      "num_tokens": 8690700.0,
+      "step": 3400
+    },
+    {
+      "entropy": 0.09924204669892787,
+      "epoch": 8.45644171779141,
+      "grad_norm": 0.2995770573616028,
+      "learning_rate": 3.1168147703600954e-05,
+      "loss": 0.07081368923187256,
+      "mean_token_accuracy": 0.9765943318605423,
+      "num_tokens": 8817258.0,
+      "step": 3450
+    },
+    {
+      "entropy": 0.09427825551480055,
+      "epoch": 8.579141104294479,
+      "grad_norm": 0.2867041826248169,
+      "learning_rate": 2.6522663902118405e-05,
+      "loss": 0.06653664588928222,
+      "mean_token_accuracy": 0.9782084548473358,
+      "num_tokens": 8954362.0,
+      "step": 3500
+    },
+    {
+      "entropy": 0.09728625949472189,
+      "epoch": 8.701840490797546,
+      "grad_norm": 0.24146562814712524,
+      "learning_rate": 2.2229639110854164e-05,
+      "loss": 0.06928532123565674,
+      "mean_token_accuracy": 0.977027119398117,
+      "num_tokens": 9085314.0,
+      "step": 3550
+    },
+    {
+      "entropy": 0.09747562363743782,
+      "epoch": 8.824539877300614,
+      "grad_norm": 0.37873855233192444,
+      "learning_rate": 1.8296928065522607e-05,
+      "loss": 0.06881467819213867,
+      "mean_token_accuracy": 0.9769331353902817,
+      "num_tokens": 9215088.0,
+      "step": 3600
+    },
+    {
+      "entropy": 0.09772783245891332,
+      "epoch": 8.94723926380368,
+      "grad_norm": 0.3342941105365753,
+      "learning_rate": 1.4731726253549227e-05,
+      "loss": 0.06916965007781982,
+      "mean_token_accuracy": 0.9772700309753418,
+      "num_tokens": 9342905.0,
+      "step": 3650
+    },
+    {
+      "epoch": 9.0,
+      "eval_entropy": 0.2565673442397799,
+      "eval_loss": 0.976934015750885,
+      "eval_mean_token_accuracy": 0.8267382683072771,
+      "eval_num_tokens": 9393939.0,
+      "eval_runtime": 56.6695,
+      "eval_samples_per_second": 24.634,
+      "eval_steps_per_second": 3.088,
+      "step": 3672
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 4080,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.96959540023328e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-408/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-408/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.04804280758660922,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

random_original_Estonian/Qwen3.5-2B-Base_random_splits_original_features_train_random_splits_original_features_test1/checkpoint-408/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}